ST 2018: AI for the Transfer of Control from Autonomous Systems to Humans

Autonomous systems, like self-driving cars and collaborative robots, must occasionally ask people around them for help in anomalous situations. They form a new generation of interaction platforms that provide a comprehensive multimodal presentation of the current situation in real-time, so that a smooth Transfer of Control (ToC) to human agents is guaranteed. Several scientific questions are associated with this ToC, including what should cause a ToC, when and how to notify a user, and how to manage many of these situations. In this seminar, we will investigate several methods of Artificial Intelligence (AI) that may be applied to these challenges.

  • Offered by: Chair of Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster and Chair of Prof. Dr. Antonio Krüger
  • Lecturers: Michael Feld, Tim Schwartz, Florian Daiber
  • Topic Supervisors: tbd
  • Time: On Mondays, 14-16h (c.t.)
  • Kick-off: 16.04.18 14:15 h
  • Location: Building E1 3, Seminar Room 016
  • Credit Points: 7

This seminar covers theoretical and practical aspects of the ToC challenges. Attendees will investigate a topic or particular scenario based on scientific publications, as well as experiment with real data or implement a small demonstration.

If you are interested in attending the seminar, please let us know by writing an e-mail to Michael Feld (see above). This is not an official registration. Signup takes place during the first meeting (kick-off).


  • Good English skills (Literature will be English)
  • Basic programming skills (the practical part will involve some programming)
  • For the Machine Learning topics, basic ML knowledge is recommended

Grading Criteria:

  • Presentation of a topic based on a scientific paper
  • Short written summary of the topic
  • Active participation in the discussion of presented topics
  • Realization of a practical assignment, e.g. implementation or model creation (in groups of 2-4 people)



1) Cause of Transfer

Determining cases when a transfer of control should occur and to whom control should be transferred on the example of autonomous robots. Given their sensors, abilities (actuators) and the environment (human positions and skills), a decision is made.

AI Methods: Robot Programming, Context Modeling
Practical Assignments*: Monitoring and Decision Module

2) Time of Transfer

When should an autonomous system initiate transfer of control to a human? On the example of autonomous vehicles, we investigate what is the best time to transfer control from a system to a human based on personal (e.g. reaction time) and situational (e.g. driving speed) factors.

AI Methods: Machine Learning, Deep Learning
Practical Assignments*: Prediction Model

3) Mode of Transfer

The transfer can be communicated using a number of channels and interaction options. We investigate different multimodal concepts, cognitive aspects, and multi-level dialogue.

AI Methods: Multimodal Interaction Design, Dialogue Development
Practical Assignments*: Dialogue System for ToC

4) Management of Transfer

In an environment where multiple human agents can receive control from multiple robots on a regular basis, they need to be able to have an overview of the autonomous agents and their situation. On the example of a retail bot, we create a dashboard that gives human agents an overview, alerts, and a way to return control to the robot.

AI Methods: Situation Summary, Plan Generation, Return of Control
Practical Assignments*: Development of a Management Dashboard

* The current assignment topics are subject to change / extension.




Practical Assignments