With the rise of machine learning in Natural Language Processing, more and diverse annotated corpora are needed. Creating these is an expensive, time consuming and difficult undertaking. In order to speed up this process, we investigate interactive annotation in the open-source INCEpTION project (http://tiny.cc/inception). Our annotation editor offers machine learning based recommenders that suggest possible annotations to the user. Recommenders are retrained in the background and thereby improve their suggestion quality. A problem with this approach is the training time, as of right now, recommenders are often retrained from scratch. The goal of this thesis therefore is to research and evaluate online learning algorithms, i.e. algorithms that can be used to update models by only training on newly incoming data, reducing the need to retrain from scratch.