Discussion paper on network security and application of big data technology

Discussion paper on network security and application of big data technology

With the rapid development and popularization of Internet technology, Internet technology has been widely used in people's work and life, bringing unprecedented convenience to people, but at the same time, various network security issues have also emerged. Based on this, this paper mainly introduces the specific application of big data technology in the field of network security, hoping to provide a feasible theoretical reference for the sustainable development of the Internet industry while studying the security of network systems.

Keywords: network security; Big data technology; applied analysis

order

With the continuous development of Internet technology in recent years, network security accidents also occur frequently. Due to the importance of network information security, China established the National Security Council in 20 14, and officially upgraded network security to a national strategic deployment. This also shows that China's network information security situation is not optimistic, and network attacks are in a state of high incidence. The number of cyber attacks such as Trojan bots, malicious ransomware, distributed denial-of-service attacks and stealing sensitive information from users ranks among the top in the world. Frequent network attacks such as mobile phone malicious programs, APT, DDOS and Trojan virus will not only seriously hinder the network bandwidth and slow down the network speed, but also have a certain impact on the corporate reputation of telecom operators. A large number of data show that the new generation of network threats can no longer be dealt with only by traditional network preventive measures, and early warning through accurate detection and analysis has become the key to network security capabilities at this stage.

1 Network Security Analysis

Network security is not only related to citizens' privacy and information security, but also related to national security. For example, Yahoo's information disclosure has caused at least 500 million pieces of user information to be stolen. The incidents such as Prism Gate and Hillary Mail Gate in the United States have further escalated and expanded the network security issue. With the increasing complexity of Internet architecture, the amount of data used for network security analysis is also increasing. In the process of moving from TB level to PB level, not only the data sources are richer and the content is more detailed, but also the dimensions required for data analysis are more extensive. With the growth of network performance, the data source transmission speed is faster, the security information collection speed is higher, and the vulnerabilities such as Odav caused by version update delay are increasing day by day, and the influence scope of network attacks is further expanded; For example, APT, an organized, targeted and long-term latent multi-stage combined attack, is more difficult to prevent. Only by analyzing more kinds of security information and synthesizing various means can we detect and resist it. In the traditional technical architecture, structured database is mostly used to store data. However, due to the high cost of data storage, the system often standardizes the original data before storing it, which easily leads to data loss and distortion, and the difficulty in tracing the source caused by the difficulty in storing historical data. At the same time, the efficiency of analyzing and querying noisy large-scale unstructured data sets is very low, which leads to the difficulty in ensuring the real-time and accuracy of data and the low efficiency of safe operation. Therefore, the traditional network security technology has been difficult to meet the new requirements of network security analysis at this stage. The concept of big data technology was first put forward by Victor Mayer Schoenberg and Kenneth Cookeye in the book "The Age of Big Data" published in 2008. Big data refers to the comprehensive analysis and processing of all data, rather than random analysis. As the first trend of information architecture development at this stage, big data technology has been widely used in many fields of the Internet in recent years because of its unique characteristics of high speed, diversity, diversity and low value density. The strategic significance of big data is that it can master a large amount of data information, so as to realize the storage and analysis of massive original security information. Compared with the traditional database, the storage cost of distributed database is reduced, and the data is easy to expand horizontally on low-cost hardware, which greatly reduces the cost of security investment. With the great improvement of data mining ability, the response speed of security information collection and detection is getting faster and faster. With the support of heterogeneous and massive data storage, the foundation of multi-dimensional and multi-stage correlation analysis is built, which enhances the depth and breadth of analysis. For network security defense, comprehensive management, processing, analysis and optimization of data from different sources can achieve rapid locking of target data in massive data and real-time feedback of analysis results, which is very important for network security defense at this stage.

2 Application of Big Data in Network Security

Applying big data to network security analysis can not only optimize and process data, but also comprehensively process logs and access behaviors, thus improving the efficiency of event processing. The role of big data technology in network security analysis can be specifically analyzed from the following points:

2. 1 data acquisition efficiency

Big data technology can collect data in a distributed way, which can reach the collection speed of 100 megabytes per second, greatly improving the data collection rate, which also lays the foundation for subsequent correlation analysis.

2.2 data storage

In the network security analysis system, the storage of original data is very important. Big data technology can collect different data according to different data types, and can actively use different ways to improve the efficiency of data query. For example, it is suitable to use column storage when querying log information, but it is suitable to use distributed preprocessing when analyzing and processing standardized data, and the results after data processing can be stored in column storage. Or you can set a query module of MapReduce in the system. When querying, you can directly put the instructions on the designated nodes, and then sort out the nodes after processing, which can ensure the query speed and response speed.

2.3 Real-time data analysis and subsequent data processing

In the analysis of real-time data, correlation analysis algorithm or CEP technology can be used to analyze, which can realize the whole process of data collection, analysis and processing, and realize higher speed and higher efficiency processing; For the processing of statistical results and data, because this kind of processing does not require high timeliness, various data processing technologies or off-line processing methods can be adopted, so as to better complete the analysis of system risks and attacks.

2.4 Analysis of complex data

When analyzing different sources and types of complex data, big data technology can better complete data analysis and query, and can effectively deal with complex data, security risks, malicious attacks and so on. When the network system appears malicious destruction and attacks, we can use big data technology to achieve all-round prevention and resistance from the perspective of traffic and DNS.

3 security analysis of building a network system based on big data technology

Introducing big data technology into the network security system mainly involves the following three modules:

3. 1 data source module

With the development of Internet technology, the data and data sources in the network security system will multiply. Distributed collectors can collect information from software and hardware in the system. In addition to firewall, detection system and other software, the requirements for equipment hardware are also improving, such as the inspection and maintenance of servers and memory.

3.2 Data acquisition module

Big data technology can reverse analyze the data, thus building a distributed data foundation, explaining the original data from appearance to deletion, and truly realizing the functions of data access and traceability, especially for today's increasingly large amount of data, distributed data storage can better improve the stability of the database.

3.3 Data analysis module

For the operation of network security system, the user's business system is the ultimate security object. Big data analysis can timely analyze and feedback user data at the beginning of its generation, so that network users can get a more personalized service experience. For users, getting what they want will also generate more trust in the network system and big data technology, and the doubts about the storage of personal safety and privacy information on the system will be greatly reduced. At present, the field of network and information security is facing brand-new challenges. Enterprises, organizations and individual users will generate a large amount of security data every day, and the existing security analysis technology can no longer meet the needs of efficient and accurate security analysis. The unique network security analysis capabilities of big data technology, such as flexibility, mass, speed, low cost and high capacity, have become the trend of the industry at this stage. For Internet companies, the deep processing of data is the key to realize the value-added of data, which is very important for enterprise operation.

4 conclusion

Nowadays, information data has penetrated into various industries and business fields and become an important factor of social production. Because of this, the amount of Internet data is increasing day by day, which brings certain difficulties and pressures to network security analysis, and big data technology can improve this problem. The application of big data technology in network system can not only meet people's requirements for high efficiency and accuracy of data processing, but also build a relatively perfect prevention and early warning system on this basis, which plays a very key role in maintaining the security of network system. I believe that big data technology will be more widely used in the future.

References:

[1] Lu Wansheng. On the application of big data technology in network security analysis [J]. Digital World, 20 17.

[2], Wang, Jin Huamin et al. Application of big data technology in network security analysis [J]. Telecommunication Science, 20 15.

[3] Yu Sun. On the application of big data technology in network security analysis [J]. Network security technology and application, 20 17.

;