Hybrid Interactive Surfaces

Interactive tabletop displays are increasingly wide-spread. They provide very natural ways of interacting with digital media. Information is displayed on a horizontal display and people can use touch input to interact with this information, very much the way we interact with physical objects in the real world.

We explore a hybrid setting in which both types of media are used in combination on the same surface. An example scenario of hybrid use is a desk of the future. On this desk, we concurrently work with books, printed articles, and sticky notes as well as with digital media, such as e-mails or web pages displayed on the desk surface. Moreover, the surface might be covered by non-media physical objects such as a keyboard, cups, pens, etc. While a traditional screen might be better suited for intense reading activity or for writing documents, documents on a tabletop screen favor collaborative practices and structuring by physical arrangements.

In this project, we examine the affordances of this hybrid setting with in-depth user studies. One main challenge of hybrid use is physical occlusion of screen contents. Our first results show in which situations occlusion is problematic. However, we also found that users intentionally create occlusion in order to express semantic information in spatial arrangements. Hybrid piles of printed and digital documents are a frequent arrangement.

Based on these findings, we currently design novel interaction techniques for hybrid tabletops. In particular, we provide support for the efficient selection of occluded items as well as for hybrid grouping and sorting activities. Our fields of application are knowledge work, learning, and interaction in command-and-control-centers.