Johannes Heeg

University of Stuttgart

Thursday, April 24, 2025, 11:00 - Friday, April 25, 2025, 11:59

SR 01-012

Reinforcement Learning (RL) has become a cornerstone of robotics research, enabling the development of powerful, end-to-end controllers through large-scale simulation. Neural network policies trained with RL have achieved impressive performance across a variety of robotic tasks by integrating diverse input modalities and directly optimizing task-level objectives. However, this flexibility often comes at the cost of poor sample efficiency, resulting in training times that span hours or even days.

Differentiable simulation offers a promising alternative by enabling the computation of analytical policy gradients, thereby significantly improving sample efficiency. In this talk, I present the application of learning in differentiable simulation to vision-based quadrotor control—a setting that involves unstable dynamics, the need to compute gradients of complex, realistic dynamics, and nonlinear observation mappings. I will discuss the key challenges of this approach and outline the resulting design choices for reward shaping, gradient computation, and controller pre-training. Finally, I will demonstrate how differentiable simulation accelerates training compared to model-free RL, and present results from real-world experiments.