TWIRL: Teaching With Interactive Reinforcement Learning

Reinforcement Learning has been a very useful approach, but often works slowly, because of large-scale exploration. A variant of RL, that tries to improve speed of convergence, and that has been rarely used until now is Interactive Reinforcement Learning (IRL), that is, RL is supported by a human trainer who gives some directions on how to tackle the problem.

Date: 2013 - 2017

Persons participating in the project:

PIs: Prof. Dr. Stefan Wermter
Associates: Dr. Francisco Cruz, Dr. Sven Magg, Dr. Cornelius Weber

Research areas

• Artificial Intelligence, Artificial Neural Networks, Machine Learning, Cognitive and Developmental Robotic, Bioinspired Models
• Reinforcement Learning, Contextual Affordances
• Interactive Reinforcement Learning, Human-Robot Interaction, Multimodal Integration

Description Nowadays, robotics research is advancing rapidly across a wide range of fields through the use of diverse learning algorithms. Tasks such as navigation, grasping, vision, speech recognition, and pattern recognition, among others, can be addressed using different machine learning paradigms, including supervised learning, unsupervised learning, and reinforcement learning. These tasks are often performed in domestic or human-centered environments, where active human participation is required to accomplish them collaboratively.

Reinforcement Learning (RL) is based on sequential decision-making, in which an agent interacts with its environment. In this context, the environment is defined as everything outside the agent’s control, rather than merely outside its physical boundaries. At each state, the agent selects an action and subsequently receives either a reward or a penalty. Over time, the agent aims to maximize the cumulative reward obtained through its interactions. Therefore, the problem can be formulated as finding an appropriate policy that maps states to actions in order to maximize future rewards.

Although Reinforcement Learning has proven to be a powerful approach, it often suffers from slow learning due to the extensive exploration required in large state-action spaces. One variant that aims to improve convergence speed, but has received comparatively limited attention, is Interactive Reinforcement Learning (IRL). In IRL, the learning process is supported by either a human or artificial trainer who provides guidance or feedback to help the agent solve the task more efficiently.

Media
Additional images/video.

Selected Publications	Web
Cruz, F., Magg, S., Nagai, Y., & Wermter, S. (2018). Improving interactive reinforcement learning: What makes a good teacher?. Connection Science, 30(3), 306-325.
Cruz, F., Magg, S., Weber, C., & Wermter, S. (2016). Training agents with interactive reinforcement learning and contextual affordances. IEEE Transactions on Cognitive and Developmental Systems, 8(4), 271-284.
Cruz, F., Parisi, G. I., Twiefel, J., & Wermter, S. (2016, October). Multi-modal integration of dynamic audiovisual patterns for an interactive reinforcement learning scenario. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), (pp. 759-766). IEEE.