Developmentally inspired exploration strategies for reinforcement learning
During the first few months of life, human infants acquire a large set of skills with little to no supervision or rewards.
This project investigates how these early stages of development can inspire multiple exploration strategies for reinforcement learning agents,
from intrinsic motivations to temporally correlated noise, and their possible applications for robotics.
Date: 2026
Persons participating in the project:
- PIs: Dr Francisco Cruz
- Associates: Dr Francisco M. López
- Corresponding contact: fmlopez@unsw.edu.au
Research areas:
- Developmental Robotics and AI
- Reinforcement Learning
- Intrinsic Motivation
Description:
Exploration is a fundamental component of reinforcement learning that allows agents to discover new states and acquire skills through interaction with their environment.
While exploration is commonly implemented through random perturbations of an agent's actions, recent research suggests that temporally correlated (colored) noise can produce
more structured exploratory behaviors and improve learning efficiency in continuous control and robotics.
This project investigates how the temporal structure of exploration noise can evolve throughout learning.
Our first studies have demonstrated that progressively increasing temporal correlations, inspired by developmental changes in infants' spontaneous movements,
can improve exploration efficiency compared to fixed exploration strategies.
Our current research aims to extend these approaches by introducing adaptive mechanisms inspired by intrinsic motivations.
In particular, we investigate how learning progress can dynamically regulate the temporal correlations of exploration noise,
allowing agents to adjust their exploratory behavior according to their learning experience rather than following a predefined schedule.
We aim to evaluate these strategies in simulated robotics environments, with the long-term goal of developing more efficient and autonomous learning mechanisms for real-world robotics.
Media:
Additional images/video
| Selected Publications |
Web |
| López, F. M., Ernst, M. R., Cruz, F., Hoffmann, M., & Triesch, J. (2026). Infant Spontaneous Movement Noise Improves Exploration in Deep RL. In 2026 IEEE International Conference on Development and Learning (ICDL), (pp. 1-6). IEEE.
|
|
| López, F. M., & Cruz, F. (2026). Can Hindsight Critics Guide Delayed Actors? In Australasian Joint Conference on Artificial Intelligence, (pp. 1-12). Cham: Springer International Publishing.
|
|