Deep Learning in NLP

Deep Learning in NLP


In the Deep Learning & Word Embeddings research area, we design deep learning algorithms for solving various NLP tasks.

Deep Learning is a branch of Machine Learning that uses Deep Artificial Neural Networks for modeling problems. While the field is as old as the field of Machine Learning itself, it has experienced a tremendous revival in recent years, with a large portion of top publications devoted to the different facets of Neural Networks and their applications to NLP tasks. Particular interest has been in so-called Word Embeddings, vector representations of words that encode their semantic and syntactic regularities, and that can be efficiently and very successfully learned with Deep Learning architectures. Word Embeddings, also called vector representations, may come in different facets, such as those that represent syntactic, semantic, or even multilingual information of words.

Deep Learning benefits from the fact that the representation of the linguistic input (e.g. words or sentences) is learned fully automatically. In a growing number of applications, NLP methods based on Embeddings and Deep Learning outperform approaches based on machine learning with manually constructed feature representations.

At UKP, we use Deep Learning for NLP problems ranging from sequence labeling tasks such as Named Entity Recognition, Event Detection, Metaphor Detection, to Text Classification and Information Retrieval problems. Recently, we have been applying Deep Learning in the form of word embedding features to the very active field of Argumentation Mining.

Current Projects

  • Automatically structuring story chains: As thousands of news articles are published daily, it is challenging to stay up-to-date on every topic. The goal of the project is to help readers to tackle the information-overload. This is done by extracting and analyzing the causal connections between articles. The results are useful for various tasks, e.g. to get a quicker overview on a certain topic.
  • Metaphor recognition: This project focuses on the detection and exploration of metaphors in English and German texts. The aim to facilitate corpus analysis with regards to metaphors for researchers from Digital Humanities shall be achieved, amongst others, with methods from Deep Learning. As manual metaphor detection or identification is costly and tedious, reliable automatic detection can ease further analysis, e.g. into whether certain metaphors are often used to convey opinions, or if metaphors from specific sources tend to occur more often in some topics than in others.
  • C3 Aiphes: This project is about connecting information from unstructured text to information stored in Knowledge Bases. The goal is to develop methods for embedding information from such different sources in a way that will benefit adaptive language processing. The learned embeddings will be applied to the task of Semantic Role Labeling which in turn is contributing to more high-level tasks as text summarization.