Open Theses

Although increasingly complex models like GPT-3 and BERT continue to set new state-of-the-arts in many natural language processing tasks, training such models requires a vast amount of data and resources. Increasing the complexity and data even further poses an essential problem due to the limits of currently available hardware, and more- over, is often only possible for large tech-companies. The goal of this thesis is to explore and evaluate various approaches that specifically opt for efficient model training in low-resource scenarios. By investigating approaches from meta-learning [1], curriculum learning [2] and active learning [3] on a wide range of NLP tasks, our goal is to better understand the mechanisms of efficiently training deep neural networks.