Three papers on Shared Tasks for NLIDBs, Explainable AI, and Interactive Text Exploration accepted to DESIRES 2021

Nadja Geisler and Benjamin Hättasch will present their work at the second conference on Design of Experimental Search & Information REtrieval Systems in Padova

2021/09/14

The three papers are “Netted?! How to Improve the Usefulness of Spider & Co.”, “Quest: A Query-driven Explanation Framework for Black-Box Classifiers on Tabular Data”, and “WannaDB: Ad-hoc Structured Exploration of Text Collections Using Queries”.

Netted?! How to Improve the Usefulness of Spider & Co.

by Benjamin Hättasch, Nadja Geisler, and Carsten Binnig

Natural language interfaces for databases (NLIDBs) are an intuitive way to access and explore structured data. That makes challenges like Spider (Yale’s semantic parsing and text-to-SQL challenge) valuable, as they produce a series of approaches for NL-to-SQL-translation. However, the resulting contributions leave something to be desired. In this paper, we analyze the usefulness of those submissions to the leaderboard for future research. We also present a prototypical implementation called UniverSQL that makes these approaches easier to use in information access systems. We hope that this lowered barrier encourages (future) participants of these challenges to add support for actual usage of their submissions. Finally, we discuss what could be done to improve future benchmarks and shared tasks for (not only) NLIDBs.

---

Quest: A Query-driven Explanation Framework for Black-Box Classifiers on Tabular Data

by Nadja Geisler

is a framework to generate explanations for black-box classification on tabular data. Its query style explanations representations focus on comprehensibility and have the advantage of explaining not only why a model produced an outcome but also why not.

---

WannaDB: Ad-hoc Structured Exploration of Text Collections Using Queries

by Benjamin Hättasch

outlines a new method for exploring text using SQL-style queries. The proposed method works ad-hoc without rule specification or domain specific training and is based on a short interaction phase.