The theory and applications of mean-field games/control have stimulated a growing interest and generated important literature over the last decade since the seminal paper by Lasry and Lions and the two-volume book of Carmona and Delarue. This talk will address some learning methods for numerically solving continuous time mean-field control problems, also called McKean-Vlasov control (MKV). In a first part, we consider a model-based setting, and present a numerical approximation for the Master Bellman equation in the Wasserstein space of probability measures that characterizes the solution to MKV. The approach relies on particles approximation of the Master equation, for which we prove a rate of convergence, and deep learning algorithms with a class of equivariant neural networks, named Deepsets. The second part of the lecture is devoted to a model-free setting, a.k.a. reinforcement learning. We develop a policy gradient approach under entropy regularization based on a suitable representation of the gradient value function with respect to parametrized randomized policies. This study leads to actor-critic algorithms for learning simultaneously and alternately value function and optimal policies. Numerical examples in a linear-quadratic mean-field setting illustrate our results.
Bio: Huyên PHAM is Distinguished Professor of Mathematics at Université de Paris, where he headed the Mathematical Finance research team of the Laboratoire de Probabilités, Statistique et Modélisation. He leads research in quantitative finance, stochastic analysis and control, machine learning techniques for numerical probabilities, and is the author of more than 100 publications, including the monograph Continuous time Stochastic Control and Optimization with Financial Applications. He serves on the editorial boards of several international journals, and is the co-editor in chief of the journal Applied Mathematics and Optimization. Prof. Pham was appointed member of the Institut Universitaire de France in 2006, awarded the Louis Bachelier prize by the French Academy of Sciences in 2007, and was a plenary speaker at the 9th World congress of the Bachelier Finance Society in 2016.