New at the department: Professor Justus Thies
3D Graphics & Vision – at the interface between computer graphics and machine vision.
2023/09/04
On 1 September, Professor Dr Justus Thies joined the Department of Computer Science in the area of 3D Graphics & Vision. Thies, whose academic career has included positions as a research assistant at the Friedrich-Alexander University of Erlangen-Nuremberg, as a postdoctoral researcher at the Technical University of Munich, and most recently as an independent research group leader at the Max Planck Institute for Intelligent Systems in Tübingen, brings a wealth of experience and knowledge to the department. To find out more about his research interests and future plans, we asked Professor Thies a few questions:
Why should students be interested in your topics? What makes your topics exciting?
My research topics deal with the dynamic 3D capture of people, objects and entire spaces and environments. This digitisation is the basis for any human-machine interaction, as the machine needs to understand what and where the human is doing, for example to avoid collisions (e.g. domestic robots, autonomous vehicles).
On the other hand, the digitisation of people, their movements and interactions with objects and the environment is necessary for many applications in virtual and augmented reality (VR and AR). For example, people can meet, talk and work in a virtual world (e.g. metaverse). They can virtually try on new clothes (Virtual Mirror), which can then be personalised.
3D digital images of people (avatars) can be inserted into films and video games, simplifying the use of special effects. Films can be automatically translated and actors' lip movements can be matched to the new audio track.
In addition to these applications, 3D capture has many other uses, such as medical imaging and disease analysis (e.g. the healing process of a cleft palate, analysis of movement sequences, etc.), or the production of spare parts by 3D printing or similar.
To lay the foundations for all these applications, my group is working on topics at the interface between computer graphics and machine vision. We use optimisation-based methods such as analysis-by-synthesis as well as methods from artificial intelligence. The goal is to find 3D/4D representations of an object or a person that allow us to create new photorealistic content.
This reconstruction can be done from single images, videos, depth sensors, audio or just text input. Once you are able to reproduce the surface properties (colour, reflections, geometry and other physical parameters such as elasticity, stiffness, etc.), you have captured the object and can use these properties further. This is the principle of analysis by synthesis.
Working in my field is very varied and allows you to be creative. The results are visual and also tangible when printed. This has always been important to me because I want to see what I am creating and not just chase numbers (benchmarks).
Interdisciplinarity is very important at TU Darmstadt. Where in your area of work are there interfaces with other disciplines?
My field is very broad and allows for numerous collaborations with other disciplines. For example, as mentioned above, motion detection and 3D reconstruction of facial features, body pose and interaction with objects can be used in medicine. Environmental sensing can be used in robotics. Social sciences can analyse human behaviour in AR/VR; interviews or similar can be simulated virtually.
If I were a student today, I would …
If I were a student today, I would study computer graphics and machine vision again. The possibilities offered by artificial intelligence methods are manifold. Algorithms can be learned and adapted directly from data, eliminating the need for the tedious development of heuristics.
You can participate in the development of new methods and influence how and for what the new technologies can be used. It is important to understand to what extent and in what areas the use of artificial intelligence methods is possible and useful – you have the opportunity to shape the future.
