Short answer questions design and development of cross-border e-commerce big data full link processing workflow includes what steps?

Data collection, data import and cleaning preprocessing, data statistical analysis and mining, results visualization.

1, first, data collection. Big data collection using ETL tools is responsible for extracting data from distributed, heterogeneous data sources such as relational data, flat data, and other unstructured data into temporary files or databases.

2, followed by data import and cleaning preprocessing. Collecting good data, there must be a lot of repetitive or useless data, at this time you need to carry out simple cleaning and preprocessing of data, so that the data from different sources are integrated into a consistent, suitable for data analysis algorithms and tools to read the data, such as data de-emphasis, anomaly handling and data normalization, etc., and then these data are stored to a large-scale distributed database or distributed storage clusters.

3. Then, data statistical analysis and mining. Statistical analysis requires the use of tools to deal with, such as SPSS tools, some structural algorithm models, classification and summarization to meet a variety of data analysis needs.

4. Finally, the results are visualized. Big data analysis of the users of big data analysis experts, as well as ordinary users, but both of them for big data analysis of the most basic requirements is the visual analysis, because visual analysis can intuitively present the characteristics of big data, at the same time can be very easy to be accepted by the readers, as simple as looking at the map to speak.