Computer Science for Educational Applications

Computer Science for Educational Applications: Automatic Grading of Free Text

Description

This seminar first provides an introduction to the fields of computer science for educational applications and natural language processing (NLP). The latter is crucial in understanding the content, structure, and style in text. This knowledge may be applied to educational areas is computer science such as plagiarism detection, automatic essay grading, or grading short text answers as part of large-scale assessments such as PISA studies.

In particular, this offering of the seminar focuses on free text assessment and methods for grading such content automatically. Here, free text assessment refers to examination settings where the respondent is required to produce a short answer or essay in response to a question. This is in contrast to fixed response assessment, where the respondent must select one or more answers from a predefined list as per multiple choice questions and similar.

Grading fixed response items automatically is nowadays considered a solved problem. Since the demand for grading free text answers is high, it has become a hot topic in the research community in the last 10-15 years. Importantly, the free text format requires respondents to recall knowledge, whereas fixed responses only require the respondent to recognize correct knowledge from alternatives. So the free text format represents a higher level of understanding, and grading these responses automatically provides new possibilities for efficiently and consistently processing large scale assessments such as competency tests and college admissions.

In the first weeks of the seminar program, students will participate in a welcome lecture, an introduction to natural language processing lecture, and an introduction to free text assessment lecture. Then after the assignment of topics, students will return in the second half of the semester to share their topic in a presentation and a term paper. Therefore in the course of the seminar, students will acquire key skills like the fundamentals in academic research and scientific writing, practice English, and they will be encouraged to improve their presentation skills.

Literature

R. Ziai, N. Ott, & D. Meurers. Short Answer Assessment: Establishing Links Between Research Strands. In J. Tetreault, J. Burstein, and C. Leacock, editors, Proceedings of the Seventh Workshop on the Innovative Use of NLP for Building Educational Applications, pages 190-200, Mantreal, Candada, June 2012. [ pdf ]

D. Perez-Marin. Adaptive Computer Assisted Assessment of Free-Text Students' Answers: An Approach to Automatically Generate Students' Conceptual Models. PhD thesis, chapter 4, Department of Computer Science, Autonomous University of Madrid, Madrid, Spain, May 2007. [ pdf ]

S. Valenti, F. Neri, & A. Cucchiarelli. An Overview of Current Research on Automated Essay Grading. Journal of Information Technology Education, 2:319-330, 2002. [ pdf ]

Additional literature is provided in class.

Prerequisites and Preparation

Introductory lectures will be given by the lecturer. It is expected that these overview contents are thoroughly worked through in addition to participation in the seminar.

Expectations

Each student is expected to:

  • Give a 30 minute talk in class + 15 minutes for questions afterwards.
  • Write a term paper.
  • Show active participation in class.

All aspects of this course are conducted in English.

Materials and Forum

The Moodle course management system is used as the primary communication platform for the seminar and also contains course materials. Further details are provided in class.

Timetable

Seminars are on Tuesdays from 9:50am to 11:30am in room S3|13|56.

Week Date Topic
1 Tue Apr 16 No classes.
2 Tue Apr 23 1st Lecture: Welcome and Course Organization.
3 Tue Apr 30 2nd Lecture: Introduction to NLP.
4 Tue May 7 3rd Lecture: Introduction to Free Text Assessment.
5 Tue May 14 First week of student presentations (exact week subject to enrollment numbers).
6 Tue May 21 Student presentations continue.
7 Tue May 28 Student presentations continue.
8 Tue Jun 4 Student presentations continue.
9 Tue Jun 11 Student presentations continue.
10 Tue Jun 18 Student presentations continue.
11 Tue Jun 25 Student presentations continue.
12 Tue Jul 2 Student presentations continue.
13 Tue Jul 9 Last week of student presentations.
14 Tue Jul 16 Last week of student presentations.

Consultation is by appointment.

Lecturers

Dr. Steven Burrows, UKP Lab, German Institute for International Educational Research (DIPF), Frankfurt.

Prof. Dr. Iryna Gurevych, UKP Lab, Computer Science Department, TU Darmstadt and DIPF, Frankfurt.