Lectures: Joschka Boedecker and Moritz Diehl
Guest Lectures: Sebastien Gros (NTNU Trondheim) and Sergey Levine (UC Berkeley)
Exercises: Katrin Baumgärtner and Jasper Hoffmann
University of Freiburg, July 26 to August 4, 2021
(online available, all times are Central European Summer Time)
This block course of 8 days duration is intended for master and PhD students from engineering, computer science, mathematics, physics, and other mathematical sciences. The aim is that participants understand the main concepts of model predictive control (MPC) and reinforcement learning (RL) as well the similarities and differences between the two approaches. In handson exercises and project work they learn to apply the methods to practical optimal control problems from science and engineering.
The course consists of lectures, exercises, and project work. The lectures in the first week will be given by Prof. Dr. Joschka Boedecker and Prof. Dr. Moritz Diehl from the University of Freiburg. In the second week of the course, invited guest lectures will be given by Prof. Dr. Sebastien Gros from NTNU Trondheim (Norway) and Prof. Dr. Sergey Levine from UC Berkeley (California, US).
Topics include
 Optimal Control Problem (OCP) formulations  constrained, infinite horizon, discrete time, stochastic, robust
 Markov Decision Processes (MDP)
 From continuous to discrete: discretization in space and time
 Dynamic Programming (DP) concepts and algorithms  value iteration and policy iteration
 Linear Quadratic Regulator (LQR) and Riccati equations
 Convexity considerations in DP for constrained linear systems
 Model predictive control (MPC) formulations and stability guarantees
 MPC algorithms  quadratic programming, direct multiple shooting, GaussNewton, realtime iterations
 Differential Dynamic Programming (DDP) for the solution of unconstrained MPC problems
 Reinforcement Learning (RL) formulations and approaches
 Modelfree RL: Monte Carlo, temporal differences, model learning, direct policy search
 RL with function approximation
 Modelbased RL and combinations of modelbased and modelfree methods
 Similarities and differences between MPC and RL
The course will be conducted as a mixed virtual and real event if the Corona situation in July 2021 allows, and otherwise be fully virtual. All lecture videos and exercises will be made openly available. Each course day starts at 9:00 and ends at 17:00 (5 p.m.) Freiburg time (MET). Lectures are typically followed by computer exercises in Python. A mandatory requirement for officially passing the course is successful participation in an online microexam on Friday July 30, 2021, at 9:00. In the second week, on August 24, 2021, participants will work on application projects which apply at least one of the MPC and RL methods to selfchosen application problems from any area of science or engineering. The results of the projects, that can be performed in teams of either one or two people (preferred), will be presented in a public poster presentation on the last day of the course, and a short report to be submitted two weeks after the course. The report will determine the final grade of the course.
The course can be followed in different levels of participation:
Participation Level  Passing Requirements  Certificate  Max. No of Participants 

Level A  Online Listening     
unlimited 
Level B  First Week with Exam  exercises and microexam on July 30  "Certificate B", without grade 
120 
Level C  Full Time with Project  Level B requirements plus project presentation on Aug. 4  "Certificate C", without grade 
90 
Level D  Full Time with Report  Level C requirements plus report on Aug. 18  "Certificate D", with or without grade, 3 ECTS. 
60 
The maximum number of physical participants in Freiburg is 24. For Level D participation, priority will be given to students of the University of Freiburg.
Registration to the course is closed!
Any questions on the course can be addressed to Katrin Baumgärtner (katrin.baumgaertner@imtek.unifreiburg.de).
Microexam:
Projects: guidelines, teams
Preparation for the exercises:
For the exercises you need:
 An editor that can handle Jupyter Notebooks.
 A way to install python packages like pip/pip3. The packages are listed in the requirements.txt.
To achieve this, it is helpful to make yourself first familiar with how to install python packages. After that, download the requirements file linked above and install the packages as described here. This should also install Jupyter Lab, which is an editor that lives in your browser. To start Jupyter Lab type jupyter lab in your terminal. Further instructions for Jupyter Lab can be found here.
Optional: There is an optional exercise using the opensource software acados
for highperformance embedded NMPC, which is developed within the group of Prof. Diehl. If you would like to try it, please follow the installation instructions here.
Exercises:
Monday: exercise01 (solution), exercise02 (solution)
Tuesday: exercise03 (solution), exercise04 (solution)
Wednesday: exercise05 (solution), exercise06 (solution)
Thursday: exercise07 (solution), exercise08 (solution)
Lecture location:
HS 1015, Kollegiengebäude I, Platz der Universität 3 , D79098 Freiburg, Germany
At the lecture location, there will be basic catering like coffee, water and fruits.
All Lectures and Exercise Sessions are broadcasted via Zoom:
Join Zoom Meeting
https://unifreiburg.zoom.us/j/65439158240?pwd=OHJrb2p2U2xDUEpVQTRBWWlvZGo0UT09
Meeting ID: 654 3915 8240
Passcode: 0uu1tq0ek
Gettoknoweachother session:
There will be a get to know each other session on the first Monday. For the participants that are taking part via Zoom, we will meet in wonder.me under the following link:
https://www.wonder.me/r?id=20f5f8da411c4c79856005fbe4475ebf
For everyone else, we will just meet in the lecture room.
Schedule







































