Open Theses

  • Abstractive text summarization aims at distilling the essential information from a text to produce a shorter version. Recent abstractive summarization methods are mainly deep learning-based models, which rely on a large amount of data and computational resources. Such data and resources are not always available. In addition, gathering data for new domains is rather expensive and time-consuming. The goal of this project is to explore and evaluate various domain adaptation approaches for abstractive summarization [1]. We also target finding optimal subnetworks in pre-trained language models for text summarization [2].

    Supervisors: Prof. Dr. Iryna Gurevych, Thy Tran, PhD

    Announcement as PDF

  • Task-oriented dialogue systems are designed to support users achieve predefined goals or tasks such as restaurant reservation or navigation inquiry. They often use a pipeline approach that employs multiple modules to perform natural language understanding, dialogue action decision making and response generation. Conventional task-oriented dialogue systems train these modules independently, which can lead to error propagation when the full dialog context is not provided in the subsequent modules. To address the limitation of the conventional pipeline, recent work has explored large pretrained models in the sequence-to-sequence setting for end-to-end task-oriented dialogue systems [1,2]. Despite of the efforts of recent studies, several challenges still remain, including coherence and consistent response generation, mitigating inappropriate response, better strategies for few-shot learning, learning new knowledge or dialogue skills, and better evaluation metrics. This project aims to investigate different approaches for alleviating these challenges.

    Supervisors: Prof. Dr. Iryna Gurevych, Thy Tran, PhD

    Announcement as PDF

  • Mental health issues are one of the most common illnesses. Thus, in the last years, mental health has become a more prominent problem domain in NLP. In this interdisciplinary research, human language is examined as a tool to better understand emotional and mental states to reduce the emotional suffering. The research directions are manifold. They range from the detection of mental illnesses or suicide risk in social media, the analysis of psychiatric or psycho-therapeutical dialogues, to the development of online therapeutic dialogue systems.

    Supervisors: Prof. Dr. Iryna Gurevych, Tobias Mayer, PhD

    Announcement as PDF

  • The prediction of the outcome of a (medical) treatment can be conducted considering various information. Depending on the treatment, information can come in different data modalities, e.g., audio or video recording, textual records, or bio-markers. To train an automatic prediction model, it can be beneficial to combine data with different modalities. However, it is important to study the effects of different modality combinations. In particular, the role of textual and audio data as additional diagnostic indicators in therapy will be the focus of this investigation.

    Supervisors: Prof. Dr. Iryna Gurevych, Tobias Mayer, PhD

    Announcement as PDF

  • Automatic Speech Recognition (ASR) models transcribe speech to text by a probabilistic prediction of the transcription given the speech audio. Despite recent advances in the field with cross-lingual and current unsupervised approaches, ASR systems for low-resource languages remain an open challenge.

    Supervisors: Prof. Dr. Iryna Gurevych, Tobias Mayer, PhD

    Announcement as PDF

  • A very important step in conducting research is reviewing existing literature. This allows to develop novel ideas, identify current gaps and eventually produce impactful research. However, the number of publications grows rapidly and is far too big for humans to study in detail. In order to overcome this challenge, innovative tools that allow to automate part of the literature review process have to be developed. One such technology is QA, where the answer to a question is automatically produced based on scientific background knowledge. This task is challenging due to scarce data, complex nature of the questions and underlying texts, and long documents.

    Supervisors: Prof. Dr. Iryna Gurevych, Tim Baumgärtner, M.Sc.

    Announcement as PDF

  • Despite the recent advances in natural language processing and human-level performances of state-of- the-art neural models on common benchmarks, recent models lack various reasoning capabilities. For in- stance, they struggle on datasets that require performing coreference or arithmetic reasoning [Wu et. al, 2021, Moosavi et al., 2021]. In this regard, we explore two different directions: (1) developing innovative models that have improved reasoning capabilities to solve the existing reasoning-aware challenge da- tasets, and (2) creating new benchmarks for less-explored reasoning skills.

    Supervisors: Prof. Dr. Iryna Gurevych, Dr. phil. Nafise Sadat Moosavi

    Announcement as PDF

  • The majority of NLP datasets contain spurious patterns that are associated with the target label and are easy to learn. Models tend to focus on learning these dataset-specific spurious patterns instead of learning more generalizable patterns to solve the underlying task. As a result, while models that are trained on such datasets achieve high performances on the same data distribution, they fail on out-of-domain data distributions. In this regard, we explore innovative approaches to improve the robustness of NLP models across various datasets, tasks, and data distributions.

    Supervisors: Prof. Dr. Iryna Gurevych, Dr. phil. Nafise Sadat Moosavi

    Announcement as PDF

  • Master Thesis

    Question answering (QA) models must be capable of obtaining appropriate knowledge and reason over it (e.g., multi-hop reasoning). Typically, knowledge can be implicitly encoded in large pretrained language models (PLMs), or explicitly represented in structured knowledge graphs (KGs), such as DBPedia and ConceptNet, where nodes are entities and edges represent relations between them. PLMs have comprehensive coverage of knowledge, but they do not empirically perform well on structured reasoning. On the other hand, KGs are more suited for structured reasoning. This project aims to develop and research deep learning techniques to answer questions using knowledge from pre-trained language models (PLMs) and knowledge graphs (KGs).

    Supervisors: Prof. Dr. Iryna Gurevych, Leonardo Filipe Rodrigues Ribeiro, M.Sc.

    Announcement as PDF

  • 2021/07/08

    Detecting and combating fake news is a new task to natural language processing (NLP). However, the pandemic increased society’s dependence on the Internet and social media, exacerbating the urgency of dealing with fake news automatically. Additionally, different issues arise depending on the context in which fake news appears; for example, a fake news article looks a lot of different than a fake news tweet. Fake news will continue to evolve and so too must the tools developed to handle this issue. Through this project you will understand the fake news problem, explore various modelling techniques, from traditional NLP to modern deep methods, and examine their robustness in different contexts.

    Supervisors: Prof. Dr. Iryna Gurevych, Luke Bates, M.A.

    Announcement as PDF

  • Crisis events, including hurricanes, earthquakes, and pandemics such as COVID-19, yield massive amounts of data. In order to leverage this data to support relief efforts, provide analysis and mitigate harm, we need fast and robust tools. Critically, we need to be able to quickly adapt models trained on previous events to new emerging crises.

    Supervisors: Prof. Dr. Iryna Gurevych, Kevin Stowe, PhD

    Announcement as PDF

  • Master Thesis

    Creative language is ubiquitous in everyday life, and while deep learning models are adept at handling many types of speech, many aspects of creativity remain challenging. This is largely due to the lack of sufficient data: creative components such as metaphor, irony, humor, and sarcasm are notoriously difficult to annotate, and thus current resources are insufficient. This work aims to explore data collection for creative data, the necessity of good data sources, and methods for overcoming difficulties in creative data collection.

    Supervisors: Prof. Dr. Iryna Gurevych, Kevin Stowe, PhD

    Announcement as PDF

  • Although increasingly complex models like GPT-3 and BERT continue to set new state-of-the-arts in many natural language processing tasks, training such models requires a vast amount of data and resources. Increasing the complexity and data even further poses an essential problem due to the limits of currently available hardware, and more- over, is often only possible for large tech-companies. The goal of this thesis is to explore and evaluate various approaches that specifically opt for efficient model training in low-resource scenarios. By investigating approaches from meta-learning [1], curriculum learning [2] and active learning [3] on a wide range of NLP tasks, our goal is to better understand the mechanisms of efficiently training deep neural networks.

    Supervisors: Prof. Dr. Iryna Gurevych, Ji-Ung Lee, M.Sc.

    Announcement as PDF

  • Master Thesis

    Many tasks require annotations on sentence AND on token level. This can e.g. be overall sentiment and phrase sentiment, overal l argument stance and pro/con argument spans or intent and slots. A large chunk of time during token- or character-level annotation is spent on deciding for the correct span, and inputting that span into an annotation system. Furthermore, reading, understanding, and remembering annotation guidelines which explain the correct annotation of spans imposes considerable mental load on annotators. The goal of this thesis is training machine learning models that help human annotators with making token-level annotations given already made sentence level annotations

    Supervisors: Prof. Dr. Iryna Gurevych, Jan-Christoph Klie, M.Sc.

    Announcement as PDF

  • Bachelor Thesis, Master Thesis

    We have unprecedented access to information stored in digital documents, but not all of it is trustworthy [1]. Online discussions on platforms such as twitter, reddit or hypothes.is can help to navigate the uncertainty by providing background information and critical comments. However, it is often hard to get an overview of the comments, because they are spread over various sources and they can be related to any aspect of a document.

    The goal of this thesis is to develop NLP techniques that link comments from various sources to the exact part of a document they comment on. This would allow aggregating comments by the aspect they discuss, greatly improving their accessibility.

    Supervisors: Prof. Dr. Iryna Gurevych, Jan Buchmann, M.Sc.

    Announcement as PDF

  • Master Thesis

    Annotation technology has been prevalent for quite some time, especially in the context of corpus creation and social reading. Our goal is to extend this technology with natural language processing capacity for fact verification. We will build on top of an existing open source annotation tool called hypothesis.

    Our goal is to extend the basic hypothesis tool with a stable and responsive back-end service. This service will allow us to capture a user-selected text span in place and provide an assessment regarding its face veracity.

    Supervisors: Prof. Dr. Iryna Gurevych, Soumya Sarkar, PhD

    Announcement as PDF

  • Master Thesis

    Text summarization aims at distilling the essential information from a text to produce a shorter version, such as generating headlines for news and subject lines for emails. Recent summarization methods use deep learning models that are trained and evaluated on English and standard corpora. However, to what extent these models can generalize to other languages like German and technical domains with rare words have not been explored. Specifically, summarizing technical texts is challenging as we require expertise and deep understanding in the domain of interest.

    This project aims to explore and develop transfer learning methods to generate a sentence summary from a given textual snippet (a news article or an email written in German on a technical domain).

    Supervisors: Prof. Dr. Iryna Gurevych, Thy Tran, PhD

    Announcement as PDF

  • Bachelor Thesis, Master Thesis

    Social media have been a place for very passionate discussions regarding controversial topics such as nuclear energy or gun control. With NLP techniques for automated estimation of sociodemographic characteristics of users based on their language, we can analyze the average social profiles of groups with different opinion. How diverse are these ‚bubbles‘ and what are their characteristic properties? Which properties are characteristic for users vulnerable to fake news? Can we, in turn, use this information to improve fake news detection and/or opinion classification?

    Supervisor: Prof. Dr. Iryna Gurevych

    Announcement as PDF

  • Bachelor Thesis, Master Thesis

    Recently, large pre-trained language models have shown great success on a variety of applications in NLP. Using these models, it has already been shown, that it is possible to find the correct answer to questions, that require reasoning over multiple sentences of the input text. Preferably, however, we would like to have computer systems, that are able to aggregate different information on their own in a meaningful ways, that help people with their problems. In this thesis we like to investigate, how well models can reason over the (causal) implications of multiple steps of HowTo instructions from WikiHow. Specifically, we want to investigate, how well language models are able to re-construct the order (when required) of multiple step-wise instructions, leading to a specific goal (e.g. “How to build a house?”)

    Supervisors: Prof. Dr. Iryna Gurevych, Max Glockner, M.Sc.

    Announcement as PDF

  • Bachelor Thesis, Master Thesis

    Social media have been a place for very passionate discussions regarding controversial topics. Do these discussions have any effect or are we all stubborn victims of our filter bubbles? Using data from social media conversations of the same users over several months, the aim of this project is to find if social media users ever weaken or strengthen the intensity with which they communicate their opinion to others, and if so, what are the possible interactions causing such a change.

    Supervisor: Prof. Dr. Iryna Gurevych

    Announcement as PDF

  • Master Thesis

    Applications of conversational agents are becoming widespread. However, training such agents to generate high-quality responses is still a big challenge as the quality of responses depends on various factors. One of these factors is coherence. In this thesis, we built upon one of our existing models to measure the coherence of a response to its preceding dialog utterances using BERT-based language models.

    Supervisors: Prof. Dr. Iryna Gurevych, Dr. Mohsen Mesgar

    Announcement as PDF

  • Master Thesis

    Applications of open-domain conversational agents are becoming widespread. However, training such agents to generate high-quality responses is still a big challenge as the quality of responses depends on various factors. Recent methods train agents directly by gold responses from training sets. These methods have been shown generating low-quality responses at evaluation. In this thesis, we propose to train a function that quantifies the quality of the generated responses by a deep preference learning method. Then, we use this function as a reward estimator in a reinforcement learning model to train agents.

    Supervisors: Prof. Dr. Iryna Gurevych, Dr. Mohsen Mesgar

    Announcement as PDF

  • Bachelor Thesis, Master Thesis

    Modern science revolves around publications. The worldwide acceleration of research and the democratization of scientific publishing have led to an unprecedented increase in publication volumes. Today anyone can put a paper on arXiv and get it cited, but how do we know if it actually is good research worth building upon?

    Supervisors: Prof. Dr. Iryna Gurevych, Ilia Kuznetsov, Dipl.-Ling.

    Announcement as PDF

  • Master Thesis

    When we as humans reason about our world, we use information from multiple modalities (vision, sound, text, smell, etc.) to reach conclusions. These conclusions are often based on implicit experiences of our past and can be regarded as common sense reasoning. This thesis has the ambition of understanding what commonsense entails and how we can train an AI that is able to perform true reasoning.

    Supervisors: Prof. Dr. Iryna Gurevych, Jonas Pfeiffer, M.Sc.

    Announcement as PDF

  • Bachelor Thesis, Master Thesis

    Argumentation Mining deals with the automatic extraction of argumentative structures from natural text. Given the recent advances in this field, we are able to identify large numbers of arguments supporting or opposing a controversial topic (e.g. “nuclear energy”). To make this information digestible for users, we would like to aggregate the arguments by grouping similar arguments together and present the most prominent ones to the user.

    Supervisors: Prof. Iryna Gurevych, Tilman Beck, M.Sc.

    Announcement as PDF

  • Bachelor Thesis, Master Thesis

    We would like to be able to explore the area of creative argument generation: can we generate creative, humorous, and insightful arguments automatically? Despite recent advances in natural language processing, natural communication with artificial intelligence remains difficult. This is in part because AI systems generally struggle with using and understanding creative language. Creative language (including metaphors, similes, humor and more) is particularly challenging for natural language generation (NLG), the research area that works to automatically generate natural, fluent text. In order to develop friendly, and convincing AI, we need to be able to incorporate creative language into our models of language generation.

    Supervisors: Prof. Iryna Gurevych, Kevin Stowe, PhD

    Announcement as PDF

  • Master Thesis

    The goal of this thesis therefore is to research and evaluate online learning algorithms, i.e. algorithms that can be used to update models by only training on newly incoming data, reducing the need to retrain from scratch.

    Supervisors: Prof. Iryna Gurevych, Jan-Christoph Klie, M.Sc.

    Announcement as PDF

  • Master Thesis

    Supervisors: Prof. Iryna Gurevych, Dr. Mohsen Mesgar

    Announcement as PDF

  • Master Thesis

    Coreference resolution is the task of finding different expressions that refer to the same entity. The availability of coreference information is beneficial for several applications including question answering. In this project, we want to examine the effect of state of the art supervised and unsupervised coreference resolvers in question answering systems by answering the following questions:

    How good should a coreference resolver be in order to improve the performance of a QA system?

    Due to the limited generalization of coreference resolvers, are unsupervised systems better suited for QA?

    Would it be more beneficial for QA if we customized coreference resolvers for only a subset of noun phrases (e.g. only resolving pronouns)?

    Supervisors: Prof. Dr. Iryna Gurevych, Dr. phil. Nafise Sadat Moosavi