What's the difference between big data, data analysis and data mining?

The concept of big data: Big data was put forward in the last two years. It has three important characteristics: large amount of data, complex structure and fast data update. Due to the development of web technology, the data generated by Web users are automatically saved, sensors are constantly collecting data, the speed of automatic data collection and storage is accelerating with the development of mobile Internet, and the amount of data in the world is constantly expanding. The storage and calculation of data are beyond the capacity of a single computer (minicomputer or mainframe), which challenges the realization of data mining technology (generally speaking, the realization of data mining is based on a minicomputer or mainframe, and parallel calculation can also be carried out). ?

The concept of data mining: Data mining is based on database theory, machine learning, artificial intelligence and modern statistics, and has been applied in many fields. It involves many algorithms, such as neural network and decision tree derived from machine learning, support vector machine based on statistical learning theory, classification regression tree, correlation analysis and so on. The definition of data mining is to discover meaningful patterns or knowledge from massive data. ?

Big data needs to be mapped into small units for calculation, and then all the results are integrated, which is the so-called map-reduce algorithm framework. Some data mining techniques are still needed for computing on a single computer. The difference is that some original data mining techniques may not be easily embedded in the map-reduce framework, and some algorithms need to be adjusted. ?

The similarity or correlation between big data and data mining lies in that the future of data mining is no longer aimed at a small amount of accurate data or sampling and randomization, but a large number of mixed big data. Data analysis refers to the process of analyzing a large number of collected data with appropriate statistical analysis methods, extracting useful information and forming conclusions, and then studying and summarizing the data in detail. This process is also the supporting process of quality management system. In practice, data analysis can help people make judgments. ?

Big data refers to a collection of data that cannot be captured, managed and processed by conventional software tools within a certain time range. It is a massive, high-growth and diversified information asset, which needs a new processing mode to have stronger decision-making, insight and discovery, and process optimization ability. ?

In The Age of Big Data, co-authored by Victor Meyer-Schoenberg and Kenneth Cookeye, big data means that all data are used for analysis and processing, and there is no shortcut to random analysis (sampling survey). 5V characteristics of big data (proposed by IBM): volume (mass), speed (high speed), diversity (diversity), value (low value density) and authenticity.