How are the steps of data processing and analysis

Data processing and analysis is divided into five steps:

Step 1: Determine the customer's data needs

The more typical scenario is that we need to analyze the data for the enterprise, for example, the company usually has sales data, user data, operational data, and product production data ......What do you need to get from these data? useful information to guide the development of strategy? Another example is the need to do a market research or industry analysis, then need to know what information to get about the industry.

Step 2: Data Collection According to Customer Needs

Collect data from five data sources: web crawlers, structured data, local data, IoT devices, and manual entry, and provide customized data collection for customers. The purpose is to customize data collection and build a single data source according to the customer's needs.

Step 3: Data Preprocessing

Data in the real world is by and large incomplete, inconsistent and dirty data, which cannot be directly analyzed by data analysis, or the results of the analysis are poor. Data preprocessing has a variety of methods: data cleaning, data integration, data transformation, data normalization, etc.. These affect the analysis of data processing, in order to obtain more accurate analysis results.

Step 4: data analysis and modeling

Data analysis refers to the use of appropriate statistical analysis methods to analyze a large amount of data collected, to extract useful information and form conclusions on the data to be studied in detail and summarize the process. This process also supports the quality management system. In practical terms, data analysis can help people make judgments so that appropriate action can be taken.

A data model is a data description of objective things and their connections in an information system, which is an overall logical structure of complex data relationships. The data model not only provides the basis upon which data is collected throughout the organization, but it also works with other models in the organization to accurately and appropriately document business requirements and to support the information system's ongoing development and refinement to meet changing business needs.

Step 5: Data Visualization and Data Report Writing

The most direct result of analysis is the description and presentation of statistics. The data analysis report is not only a direct presentation of the analysis results, but also a comprehensive understanding of the relevant situation.