UKPConvArg2 Corpus

Difficulty Prediction for Language Tests

This page provides additional information for our work on difficulty prediction:

Predicting the Difficulty of Language Proficiency Tests

Lisa Beinborn and Torsten Zesch and Iryna Gurevych.

In: Transactions of the Association for Computational Linguistics, Volume 2, Issue 1, pp. 517--529, November 2014.

In a follow-up paper, we extended the prediction task to other languages (French and German) and other test types (prefix deletion, multiple choice cloze).

Candidate Evaluation Strategies for Improved Difficulty Prediction of Language Tests

Lisa Beinborn and Torsten Zesch and Iryna Gurevych

In: Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, to appear, 2015.

We have developed a web demo that visualises the results of our approach. You can test it here.

The data for the cloze tests can be downloaded here.

We received the data for the prefix deletion test from Dirk Scholten-Akoun and Anna Mashkovskaya.

We received the data for the German C-test from the TestDaf Institut.

The results for the human annotation are available here.

The French and the English C-tests are still in use at the language centre of TU Darmstadt and cannot be published. However, we are allowed to share them for research purposes.

The models and the results are available here.

You can contact Dr. Lisa Beinborn if you have any questions.

Inter-item dependencies

We performed calculations using the Q3-model to determine inter-item dependencies. If you are interested, you can find them here.

Difficulty Manipulation

We provide alexical substitution set of noun synonyms extracted from Uby that are enriched with cognateness and spelling difficulty.