(START) WORKSHOP – 3rd Edition

Feb. 17 – 21, 2020 | AIMS Ghana

Nicole Bäuerle

Karlsruhe Institute of Technology, Germany


Title: Markov Decision Processes with Applications

The theory of Markov Decision Processes focuses on controlled Markov chains in discrete time.

In these lectures we establish the theory for general state and action spaces and at the same time show its application by means of numerous examples, mostly taken from the fields of finance and operations research.

We cover problems with finite and infinite horizon, as well as stopping problems. The lecture is based on the book “Markov Decision Processes with Applications” jointly written with Ulrich Rieder.



Nicole Bäuerle is a Full Professor in Probability at the Karlsruhe Institute of Technology in Germany. Prior to this, she was Professor for Insurance Mathematics at the University of Hannover and she received her PhD in Mathematics from the University of Ulm. Up to now she has supervised more than 100 Master thesis and 13 PhD thesis.

She is editor of SIAM Journal on Control and Optimization, the Applied Probability Journals and Statistics and Risk Modeling. Her research interests lie in Stochastic processes and optimal control, Markov Decision Processes, stochastic orderings and applications in finance, insurance and energy.

Ralf Wunderlich

BTU Cottbus–Senftenberg, Germany


Title: Continuous-Time Stochastic Optimal Control with Applications to Finance

The aim of this course is to provide an introduction to continuous-time stochastic optimal control problems and to study an application to dynamic portfolio optimization financial markets. First we will review some underlying tools from stochastic calculus such as continuous-time stochastic processes, martingales, stochastic integration, Itô’s formula, stochastic differential equations and Dynkin’s formula. Based on these concepts we will then formulate a general stochastic optimal control problem which we will solve using the dynamic programming approach. Finally, we solve a specific portfolio selection problem for a utility maximizing investor.



Ralf Wunderlich is a Full Professor in Financial Mathematics at the Brandenburg University of Technology in Cottbus, Germany. His scientific interests include Probability Theory, Stochastic Finance, Optimal Stochastic Control and Portfolio Optimization. He holds a Diploma in Mathematics from the Chemnitz University of Technology where he received his PhD in Probability Theory and Mathematical Statistics in 1992 and his Habilitation in Stochastic Analysis in 1999. He held positions as Professor of Mathematics at Zwickau University of Applied Sciences from 2003-2011, as a substitute for a chair of Stochastics at Martin-Luther University in Halle-Wittenberg from 2002-2003 and as a Assistant Professor at Chemnitz University of Technology from 1993-2002.