A deterministic affine-quadratic optimal control problem
ESAIM: Control, Optimisation and Calculus of Variations, Volume 20 (2014) no. 3, p. 633-661

A deterministic affine-quadratic optimal control problem is considered. Due to the nature of the problem, optimal controls exist under some very mild conditions. Further, it is shown that under some assumptions, the optimal control is unique which leads to the differentiability of the value function. Therefore, the value function satisfies the corresponding Hamilton-Jacobi-Bellman equation in the classical sense, and the optimal control admits a state feedback representation. Under some additional conditions, it is shown that the value function is actually twice differentiable and the so-called quasi-Riccati equation is derived, whose solution can be used to construct the state feedback representation for the optimal control.

DOI : https://doi.org/10.1051/cocv/2013078
Classification:  49J15,  49K15,  49L20,  49N10
Keywords: affine quadratic optimal control, dynamic programming, Hamilton-Jacobi-Bellman equation, quasi-Riccati equation, state feedback representation
@article{COCV_2014__20_3_633_0,
author = {Wang, Yuanchang and Yong, Jiongmin},
title = {A deterministic affine-quadratic optimal control problem},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
publisher = {EDP-Sciences},
volume = {20},
number = {3},
year = {2014},
pages = {633-661},
doi = {10.1051/cocv/2013078},
zbl = {1293.49004},
mrnumber = {3270127},
language = {en},
url = {http://www.numdam.org/item/COCV_2014__20_3_633_0}
}

Wang, Yuanchang; Yong, Jiongmin. A deterministic affine-quadratic optimal control problem. ESAIM: Control, Optimisation and Calculus of Variations, Volume 20 (2014) no. 3, pp. 633-661. doi : 10.1051/cocv/2013078. http://www.numdam.org/item/COCV_2014__20_3_633_0/

[1] S.P. Banks and T. Cimen, Global optimal feedback control for general nonlinear systems with nonquadratic performance criteria. System Control Lett. 53 (2004) 327-346. | MR 2097793 | Zbl 1157.49313

[2] S.P. Banks and T. Cimen, Optimal control of nonlinear systems, Optimization and Control with Applications. In vol. 96 of Appl. Optim. Springer, New York (2005) 353-367. | MR 2144384 | Zbl 1089.49034

[3] H.T. Banks, B.M. Lewis and H.T. Tran, Nonlinear feedback controllers and compensators: a state-dependent Riccati equation approach. Comput. Optim. Appl. 37 (2007) 177-218. | MR 2325656 | Zbl 1117.49032

[4] M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997). | MR 1484411 | Zbl 0890.49011

[5] M. Bardi and F. Dalio, On the Bellman equation for some unbounded control problems. Nonlinear Differ. Eqs. Appl. 4 (1997) 491-510. | MR 1485734 | Zbl 0894.49017

[6] L.M. Benveniste and J.A. Scheinkman, On the differentiability of the value function in dynamic models of economics. Econometrica 47 (1979) 727-732. | MR 533081 | Zbl 0435.90031

[7] L.D. Berkovitz, Optimal Control Theory. Springer-Verlag, New York (1974). | MR 372707 | Zbl 0295.49001

[8] J.F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems. Springer, New York (2000). | MR 1756264 | Zbl 0966.49001

[9] P. Cannarsa and H. Frankowska, Some characterizatins of optimal trajecotries in control theory. SIAM J. Control Optim. 29 (1991) 1322-1347. | MR 1132185 | Zbl 0744.49011

[10] T. Cimen, State-dependent Riccati equation (SDRE) control: a survey. Proc. 17th World Congress IFAC (2008) 3761-3775.

[11] H. Frankowska, Value Function in Optimal Control, Mathematical Control Theory, Part 1, 2 (2001) 516-653. | MR 1972793 | Zbl 1098.49501

[12] T. Hildebrandt and L. Graves, Implicit functions and their differentials in general analysis. Trans. Amer. Math. Soc. 29 (1927) 127-153. | JFM 53.0234.02 | MR 1501380

[13] Y. Hu and S. Peng, Solution of forward-backward stochastic differential equations. Probab. Theory Rel. Fields 103 (1995) 273-283. | MR 1355060 | Zbl 0831.60065

[14] R.E. Kalman, Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5 (1960) 102-119. | MR 127472 | Zbl 0112.06303

[15] J. Ma and J. Yong, Forward-Backward Stochastic Differential Equations and Their Applications. Vol. 1702 of Lect. Notes Math. Springer-Verlag (1999). | MR 1704232 | Zbl 0927.60004

[16] H. Qiu and J. Yong, Hamilton-Jacobi equations and two-person zero-sum differential games with unbounded controls. ESAIM: COCV 19 (2013) 404-437. | Numdam | MR 3049717 | Zbl 1263.49024

[17] J.P. Rincón-Zapatero and M.S. Santos, Differentiability of the value function in continuous-time economic models. J. Math. Anal. Appl. 394 (2012) 305-323. | MR 2926223 | Zbl 1251.91046

[18] J. Yong, Finding adapted solutions of forward-backward stochastic differential equations - method of continuation, Probab. Theory Rel. Fields 107 (1997) 537-572. | MR 1440146 | Zbl 0883.60053

[19] J. Yong, Stochastic optimal control and forward-backward stochastic differential equations. Comput. Appl. Math. 21 (2002) 369-403. | MR 2009959 | Zbl 1123.60313

[20] J. Yong, Forward backward stochastic differential equations with mixed initial and terminal conditions. Trans. AMS 362 (2010) 1047-1096. | MR 2551515 | Zbl 1185.60067

[21] J. Yong and X.Y. Zhou, Stochastic Control: Hamiltonian Systems and HJB Equations. Springer-Verlag (1999). | MR 1696772 | Zbl 0943.93002

[22] Y. You, A nonquadratic Bolza problem and a quasi-Riccati equation for distributed parameter systems. SIAM J. Control Optim. 25 (1987) 905-920. | MR 893989 | Zbl 0632.49004

[23] Y. You, Synthesis of time-variant optimal control with nonquadratic criteria. J. Math. Anal. Appl. 209 (1997) 662-682. | MR 1474631 | Zbl 0872.49015

[24] E. Zeidler, Nonlinear Functional Analysis and Its Applications, I: Fixed-Point Theorems. Springer-Verlag, New York (1986) 150-151. | MR 816732 | Zbl 0583.47050

[25] E. Zeidler, Nonlinear Functional Analysis and Its Applications, II/B: Nonlinear Monotone Operators. Springer-Verlag, New York (1990). | MR 1033498 | Zbl 0684.47029