What are the applications of deep learning in natural language processing?

The application of deep learning in natural language processing has been very extensive, and it can be said that it has swept the various applications of natural language processing, from the bottom of the word segmentation, language modeling, syntactic analysis, etc. to the top of the semantic understanding, conversation management, knowledge Q&A, chatting, machine translation and so on, almost all of them have deep learning models, and have achieved good results. You can refer to the list of accepted papers of ACL 2017, Accepted Papers, Demonstrations and TACL Articles for ACL 2017, from which you can see that most of the papers use deep learning models. Then why deep learning has made so much progress in natural language. First, from the data, after the development of the Internet in the previous years, many applications have accumulated a sufficient amount of data. When the amount of data increases, the shallow model represented by SVM, CRF, etc., cannot bring performance improvement because the model is shallow and cannot model the nonlinear relationship in the massive data. On the contrary, the deep model represented by CNN and RNN can model the data more accurately with the increase of model complexity, thus getting better results. Second, algorithmically, deep learning also brings many benefits to the task of natural language processing. First of all, the emergence of word2vec makes it possible to efficiently represent words as low-dimensional dense vectors (distributed representation), compared to one-hot-representation, which on the one hand alleviates the problem of semantic gap brought by one-hot-representation to a certain extent, and on the other hand reduces the dimensionality of the input features. On the other hand, it reduces the dimensionality of the input features, thus reducing the complexity of the input layer. Secondly, due to the flexibility of the deep learning model, it makes it possible to use the end to end method to solve the previously more complex tasks that contain multiple processes. For example, for the machine translation task, if the traditional method is used, multiple modules such as the segmentation module, the alignment module, the translation module, the language modeling module, etc. need to cooperate with each other, and the error generated by each module may have an impact on the other modules, which makes the construction complexity of the original traditional method very large. With the use of encoder-decoder architecture in machine translation, we can map the source language directly to the target language, which can be optimized as a whole, avoiding the problem of error transfer, and greatly reducing the complexity of the system. Although deep learning is a powerful tool, it can't completely solve all the problems in natural language, which is mainly due to the fact that unlike speech and images, which are signals in nature, natural language is an abstract and condensed representation of human knowledge. In the process of human expression, a lot of things will be omitted due to the existence of background knowledge, which makes the expression of natural language more concise, but this also brings great challenges to the processing of natural language. For example, the short text classification problem is more difficult due to the fact that the text is short and the information carried by the text is limited. Problems like these, when the sample size is not enough, how to integrate deep learning methods and knowledge information to improve the performance of the system will be the main problem of research in the field of natural language processing in the coming period.