We consider the stochastic optimal control problem of McKean−Vlasov stochastic differential equation where the coefficients may depend upon the joint law of the state and control. By using feedback controls, we reformulate the problem into a deterministic control problem with only the marginal distribution of the process as controlled state variable, and prove that dynamic programming principle holds in its general form. Then, by relying on the notion of differentiability with respect to probability measures recently introduced by [P.L. Lions, Cours au Collège de France: Théorie des jeux à champ moyens, audio conference 2006−2012], and a special Itô formula for flows of probability measures, we derive the (dynamic programming) Bellman equation for mean-field stochastic control problem, and prove a verification theorem in our McKean−Vlasov framework. We give explicit solutions to the Bellman equation for the linear quadratic mean-field control problem, with applications to the mean-variance portfolio selection and a systemic risk model. We also consider a notion of lifted viscosity solutions for the Bellman equation, and show the viscosity property and uniqueness of the value function to the McKean−Vlasov control problem. Finally, we consider the case of McKean−Vlasov control problem with open-loop controls and discuss the associated dynamic programming equation that we compare with the case of closed-loop controls.

Accepted:

DOI: 10.1051/cocv/2017019

Keywords: McKean−Vlasov SDEs, dynamic programming, Bellman Equation, Wasserstein space, viscosity solutions

^{1}; Wei, Xiaoli

^{1}

@article{COCV_2018__24_1_437_0, author = {Pham, Huy\^en and Wei, Xiaoli}, title = {Bellman equation and viscosity solutions for mean-field stochastic control problem}, journal = {ESAIM: Control, Optimisation and Calculus of Variations}, pages = {437--461}, publisher = {EDP-Sciences}, volume = {24}, number = {1}, year = {2018}, doi = {10.1051/cocv/2017019}, mrnumber = {3843191}, zbl = {1396.93134}, language = {en}, url = {http://archive.numdam.org/articles/10.1051/cocv/2017019/} }

TY - JOUR AU - Pham, Huyên AU - Wei, Xiaoli TI - Bellman equation and viscosity solutions for mean-field stochastic control problem JO - ESAIM: Control, Optimisation and Calculus of Variations PY - 2018 SP - 437 EP - 461 VL - 24 IS - 1 PB - EDP-Sciences UR - http://archive.numdam.org/articles/10.1051/cocv/2017019/ DO - 10.1051/cocv/2017019 LA - en ID - COCV_2018__24_1_437_0 ER -

%0 Journal Article %A Pham, Huyên %A Wei, Xiaoli %T Bellman equation and viscosity solutions for mean-field stochastic control problem %J ESAIM: Control, Optimisation and Calculus of Variations %D 2018 %P 437-461 %V 24 %N 1 %I EDP-Sciences %U http://archive.numdam.org/articles/10.1051/cocv/2017019/ %R 10.1051/cocv/2017019 %G en %F COCV_2018__24_1_437_0

Pham, Huyên; Wei, Xiaoli. Bellman equation and viscosity solutions for mean-field stochastic control problem. ESAIM: Control, Optimisation and Calculus of Variations, Volume 24 (2018) no. 1, pp. 437-461. doi : 10.1051/cocv/2017019. http://archive.numdam.org/articles/10.1051/cocv/2017019/

[1] Controlled McKean−Vlasov equation. Commun. Appl. Anal. 5 (2001) 183–206. | MR | Zbl

and ,[2] Gradient Flows in Metric Spaces and in the Space of Probability Measures. Lect. Math. Birkhäuser Verlag, Basel (2005). | MR | Zbl

, and ,[3] A maximum principle for SDEs of mean-field type. Appl. Math. Optimiz. 63 (2010) 341–356. | DOI | MR | Zbl

and ,[4] Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean−Vlasov dynamics. Trans. Amer. Math. Soc. 370 (2018) 2115–2160. | DOI | MR | Zbl

, and ,[5] The Master equation in mean-field theory. J. Math. Pures Appl. 103 (2015) 1441–1474. | DOI | MR | Zbl

, and ,[6] On the interpretation of the Master equation. Stochastic Processes their Appl. 127 (2017) 2093–2137. | DOI | MR | Zbl

, and ,[7] Linear-quadratic mean field games. J. Optimiz. Theory Appl. 169 (2016) 496–529. | DOI | MR | Zbl

, , and ,[8] On time inconsistent stochastic control in continuous time. Finance Stoch. 21 (2017) 331–360. | DOI | MR | Zbl

, and ,[9] A general maximum principle for SDEs of mean-field type. Appl. Math. Optimiz. 64 (2011) 197–216. | DOI | MR | Zbl

, and ,[10] Mean-field stochastic differential equations and associated PDEs. Ann. probab. 45 (2017) 824–878. | DOI | MR | Zbl

, , and ,[11] Notes on mean field games, Notes from P.L. Lions lectures at Collège de France (2013)

,[12] The Master equation for large population equilibriums, Proceedings in Mathematics and Statistics 100. | MR

and ,[13] Forward-backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics, Ann. Probab. 43 (2015) 2647–2700. | DOI | MR | Zbl

and ,[14] Control of McKean−Vlasov dynamicsversus mean field games. Math. Financial Econ. 7 (2013) 131–166. | DOI | MR | Zbl

, and ,[15] Mean field games and systemic risk. Commun. Math. Sci. 13 (2015) 911–933. | DOI | MR | Zbl

, and ,[16] J.F. Chassagneux, D. Crisan and F. Delarue, A probabilistic approach to classical solutions of the master equation for large population equilibria. Preprint (2015). | arXiv | MR

[17] J.L. Doob, Measure Theory. Graduate texts Math. 143 Springer (1994). | MR | Zbl

[18] Stochastic Optimal Control in Infinite Dimension: Dynamic Programming and HJB Equations with Chapter 6 by and (2015). | MR

, and ,[19] A comparison principle for Hamilton-Jacobi equations related to controlled gradient flows in infinite dimensions. Archive Rat. Mech. Anal. 192 (2009) 275–310. | DOI | MR | Zbl

and ,[20] Continuous time mean-variance portfolio optimization through the mean-field approach. ESAIM: PS 20 (2016) 30–44. | DOI | Numdam | MR | Zbl

and ,[21] Controlled Markov Processes and Viscosity Solutions, 2nd edition, Springer Verlag (2006). | MR | Zbl

and ,[22] Hamilton-Jacobi equations in the Wasserstein space. Methods Appl. Anal. 15 (2008) 155–184. | DOI | MR | Zbl

, and ,[23] Metric viscosity solutions of Hamilton-Jacobi equations depending on local slopes. Calcul. Variat. Partial Differ. Equ. 54 (2015) 1183–1218. | DOI | MR | Zbl

and ,[24] Large population stochastic dynamic games: closed-loop McKean−Vlasov systems and the Nash certainty equivalence principle. Commun. Infor. Syst. 6 (2006) 221–252. | DOI | MR | Zbl

, and ,[25] Nonlinear SDEs driven by Lévy processes and related PDEs. ALEA, Latin Amer. J. Probab. 4 (2008) 1–29. | MR | Zbl

, and ,[26] Foundations of kinetic theory, in Proceedings of the 3rd Berkeley Symposium on Mathematical Statistics and Probability 3 (1956) 171–197. | MR | Zbl

,[27] Mean-field games. Japanese J. Math. 2 (2007) 229–60. | DOI | MR | Zbl

and ,[28] Dynamic programming for mean-field type control. J. Optimiz. Theory Appl. 169 (2016) 902–924. | DOI | MR | Zbl

and ,[29] Continuous-time mean-variance portfolio selection: a stochastic LQ framework. App. Math. Optimiz. 42 (2000) 19–33. | DOI | MR | Zbl

and ,[30] Viscosity solutions of fully nonlinear second-order equations and optimal control in infinite dimension. Part I: the case of bounded stochastic evolution. Acta Math. 161 (1988) 243–278. | DOI | MR | Zbl

,[31] Viscosity solutions of fully nonlinear second-order equations and optimal control in infinite dimension. Part III: Uniqueness of viscosity solutions for general second-order equations. J. Functional Anal. 86 (1989) 1–18. | DOI | MR | Zbl

,[32] Cours au Collège de France: Théorie des jeux à champ moyens, audio conference 2006–2012.

,[33] Propagation of chaos for a class of nonlinear parabolic equations. Lect. Series Differ. Equ. 7 (1967) 41–57. | MR

,[34] Continuous-time stochastic control and applications with financial applications. Series Stochastic Modeling and Applied Probability 61. Springer (2009). | MR | Zbl

,[35] Continuous Martingales and Brownian Motion, 3rd edition. New York, Berlin: Springer (1999). | DOI | MR | Zbl

and ,[36] A.S. Sznitman, Topics in propagation of chaos, in Lect. Notes Math. Springer 1464 (1989) 165–251. | MR | Zbl

[37] Optimal Transport, Old and New. Springer (2009). | DOI | Zbl

,[38] A linear-quadratic optimal control problem for mean-field stochastic differential equations. SIAM J. Control Optimiz. 51 (2013) 2809–2838. | DOI | MR | Zbl

,*Cited by Sources: *