What are the career development directions of deep learning?

At present, the development of artificial intelligence has been fully concerned and promoted by the breakthrough of deep learning technology. Governments around the world attach great importance to it, and the capital boom is still increasing. All walks of life have also reached a * * * understanding that it has become a hot spot for development. This paper aims to analyze the current situation of deep learning technology, judge the development trend of deep learning, and put forward development suggestions according to the technical level of our country.

First, the status quo of deep learning technology

Deep learning is the key technology of this round of artificial intelligence explosion. The breakthrough of artificial intelligence technology in computer vision and natural language processing ushered in a new round of explosive development of artificial intelligence. And deep learning is the key technology to achieve these breakthroughs. Among them, the image classification technology based on deep convolution network has exceeded the accuracy of human eyes, the speech recognition technology based on deep neural network has reached 95% accuracy, and the machine translation technology based on deep neural network has approached the average translation level of human beings. With the rapid improvement of accuracy, computer vision and natural language processing have entered the stage of industrialization and brought about the rise of emerging industries.

Deep learning is an algorithmic weapon in the era of big data and has become a research hotspot in recent years. Compared with the traditional machine learning algorithm, deep learning technology has two advantages. First, deep learning technology can continuously improve performance with the increase of data scale, while traditional machine learning algorithms can hardly use massive data to continuously improve performance. Second, deep learning technology can directly extract features from data, reducing the work of designing feature extractors for each problem, while traditional machine learning algorithms need to extract features manually. Therefore, deep learning has become a hot technology in the era of big data, and both academia and industry have done a lot of research and practical work on deep learning.

Various models of deep learning fully empower basic applications. Convolutional neural network and cyclic neural network are two widely used deep neural network models. Computer vision and natural language processing are two basic applications of artificial intelligence. Convolutional neural network is widely used in the field of computer vision, and its performance in image classification, target detection, semantic segmentation and other tasks greatly exceeds that of traditional methods. Cyclic neural network is suitable for solving problems related to sequence information, and has been widely used in the field of natural language processing, such as speech recognition, machine translation, dialogue system and so on.

Deep learning technology is still not perfect and needs to be further improved. First, the model of deep neural network is highly complex, and the huge number of parameters leads to the huge scale of the model, which is difficult to deploy to mobile terminal devices. Second, model training needs a lot of data, and the cost of obtaining and labeling training data samples is high, and it is difficult to obtain some scene samples. Third, the application threshold is high, the process of algorithm modeling and parameter adjustment is complex, the algorithm design cycle is long, and the system implementation and maintenance are difficult. Fourth, lack of causal reasoning ability. Judea Pearl, the Turing Prize winner and the father of Bayesian networks, pointed out that the current deep learning is only "curve fitting". Fifth, there is the question of interpretability. Because of internal parameters and complex feature extraction and combination, it is difficult to explain what the model has learned, but for security reasons and ethical and legal needs, the interpretability of the algorithm is very necessary. So deep learning still needs to solve the above problems.

Second, the development trend of deep learning

Deep neural network presents the development trend of deeper and deeper levels and more complex structure. In order to continuously improve the performance of deep neural network, the industry has been exploring from two aspects: network depth and network structure. The number of layers of neural network has expanded to hundreds or even thousands. With the deepening of network layers, its learning effect is getting better and better. In 20 15 years, the ResNet proposed by Microsoft exceeded the accuracy of image classification task for the first time with a network depth of 152 layers. New network design structures are constantly proposed, which makes the structure of neural network more and more complex. For example, 20 14, Google proposed the initial network structure, 20 15, Microsoft proposed the residual network structure, 20 16, and Huang Gao and others proposed the dense connection network structure. These network structure designs continuously improve the performance of the deep neural network.

The functions of deep neural network nodes are constantly enriched. In order to overcome the limitations of the current neural network, the industry has explored and proposed a new type of neural network node, which makes the functions of the neural network more and more abundant. In 20 17, Jeffrey Hinton put forward the concept of capsule network, which is closer to the behavior of human brain in theory, in order to overcome the limitations of convolutional neural network such as lack of spatial stratification and reasoning ability. In 20 18, scholars from DeepMind, Google Brain and MIT jointly put forward the concept of graph network, and defined a new class of modules with relation induction bias function, aiming at giving deep learning the ability of causal reasoning.

Deep neural network engineering application technology is deepening. Most of the deep neural network models have hundreds of millions of parameters and occupy hundreds of megabytes of space, so it is difficult to deploy them to terminal devices with limited performance and resources such as smartphones, cameras and wearable devices. In order to solve this problem, the industry adopts model compression technology to reduce the parameters and size of the model and reduce the amount of calculation. At present, the model compression methods used include pruning the trained model (such as pruning, weight sharing and quantization). ) and design more elaborate models (such as MobileNet, etc.). ). The modeling and parameter adjustment process of deep learning algorithm is complex and the application threshold is high. In order to lower the application threshold of deep learning, the industry has put forward automatic machine learning (AutoML) technology, which can realize the automatic design of deep neural network and simplify the use process.

Deep learning and a variety of machine learning technologies continue to integrate and develop. The deep reinforcement learning technology, which was born by the combination of deep learning and reinforcement learning, combines the perception ability of deep learning and the decision-making ability of reinforcement learning, overcomes the defect that reinforcement learning is only suitable for discrete and low-dimensional States, and can learn control strategies directly from high-dimensional raw data. In order to reduce the amount of data needed for deep neural network model training, the industry introduced the idea of transfer learning, thus giving birth to deep transfer learning technology. Transfer learning refers to the learning process of applying the model learned in the old field to the new field by using the similarity between data, tasks or models. By migrating the trained model to a similar scene, only a small amount of training data is needed to achieve good results.

Third, suggestions for future development.

Strengthen the research of cutting-edge technologies such as graph network, deep reinforcement learning and generating countermeasure network. Due to the lack of major original research results in the field of deep learning in China, the contribution of basic theoretical research is insufficient. For example, innovative and original concepts such as capsule net and graph net were put forward by American experts, but China's research contribution was insufficient. In the aspect of deep reinforcement learning, the latest research results are mostly put forward by researchers from foreign companies such as DeepMind and OpenAI, and there is no breakthrough research result in China. In recent years, the research hotspot-Generative Countermeasure Network (GAN) was put forward by American researcher Goodfellow, and various improvement and application models were put forward by companies such as Google, facebook, twitter and Apple, which effectively promoted the development of GAN technology. However, there are few research results in China. Therefore, scientific research institutes and enterprises should be encouraged to strengthen the research on cutting-edge technologies such as the combination of deep neural network and causal reasoning model, the generation of antagonistic network and deep reinforcement learning, and put forward more original research results to enhance the influence of global academic research.

Accelerate the research on deep learning application technologies such as automated machine learning and model compression. Relying on the advantages of domestic market and enterprise growth, and aiming at the personalized application demand with China characteristics, we will speed up the research on deep learning application technology. Strengthen the research of automatic machine learning, model compression and other technologies to accelerate the engineering application of deep learning. Strengthen the application research of deep learning in the field of computer vision, and further improve the accuracy of visual tasks such as target recognition and the performance in practical application scenarios. Strengthen the application research of deep learning in the field of natural language processing, propose a better algorithm model, and improve the performance of machine translation, dialogue system and other applications.

Source: industrial intelligence officer

end

For more exciting content, please visit official website.

Selected previous issues ▼

1.2018-2019 China top 100 list of artificial intelligence industry innovation released!

2.20 18-20 19 The list of the top 20 investment institutions in the artificial intelligence industry in China was released!

3.20 18-20 19 China's top 100 data industry innovation list was released!

4.20 18-20 19 China's top 20 data industry investment institutions list released!

5.20 18-20 19 China Internet of Things Industry Top 100 Innovation List released!

6.20 18-20 19 The list of top 20 investment institutions in China's 5G and Internet of Things industries was released!

7.20 18-20 19 China Top 100 Innovative Integrated Circuit Industry List released!

8.20 18-20 19 The list of the top 20 investment institutions in China IC industry was released!

9.20 18-20 19 The list of the top 100 innovative enterprises in China service industry is released!

10. The list of the top 20 investment institutions in China's enterprise service industry was released in Yinlu.com on 2018-2019!