Hamilton-Jacobi-Bellman equations for the optimal control of a state equation with memory

Carlier, Guillaume; Tahraoui, Rabah

ESAIM: Control, Optimisation and Calculus of Variations, Tome 16 (2010) no. 3, pp. 744-763.

This article is devoted to the optimal control of state equations with memory of the form: $\dot{x}\left(t\right)=F(x\left(t\right),u\left(t\right),{\int }_0^{+\infty }A\left(s\right)x(t-s)\mathrm{d}s),t>0,$ with initial conditions $x\left(0\right)=x,x(-s)=z\left(s\right),s>0$. Denoting by $y_{x,z,u}$ the solution of the previous Cauchy problem and: $v(x,z):={inf}_{u\in V}\left\{{\int }_0^{+\infty }{\mathrm{e}}^{-\lambda s}L(y_{x,z,u}\left(s\right),u\left(s\right))\mathrm{d}s\right\}$ where V is a class of admissible controls, we prove that v is the only viscosity solution of an Hamilton-Jacobi-Bellman equation of the form: $\lambda v(x,z)+H(x,z,{\nabla }_{x}v(x,z))+\langle D_{z}v(x,z),\dot{z}\rangle =0$ in the sense of the theory of viscosity solutions in infinite-dimensions of Crandall and Lions.

Classification : 49L20, 49L25
Mots clés : dynamic programming, state equations with memory, viscosity solutions, Hamilton-Jacobi-Bellman equations in infinite dimensions

[1] C.T.H. Baker, G.A. Bocharov and F.A. Rihan, A Report on the Use of Delay Differential Equations in Numerical Modelling in the Biosciences. Technical report, Manchester Centre for Computational Mathematics, UK (1999).

[2] A. Bensoussan, G. Da Prato, M. Delfour and S.K. Mitter, Representation and control of infinite dimensional systems. Second Edition, Birkhäuser (2007). | Zbl

[3] M. Bardi and I.C. Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997). | Zbl

[4] G. Barles, Solutions de viscosité des équations de Hamilton-Jacobi, Mathematics and Applications 17. Springer-Verlag, Paris (1994). | Zbl

[5] R. Boucekkine, O. Licandro, L. Puch and F. Del Rio, Vintage capital and the dynamics of the AK model. J. Econ. Theory 120 (2005) 39-72. | Zbl

[6] H. Brezis, Analyse fonctionnelle, théorie et applications. Masson, Paris (1983). | Zbl

[7] G. Carlier and R. Tahraoui, On some optimal control problems governed by a state equation with memory. ESAIM: COCV 14 (2008) 725-743. | Numdam

[8] M. Crandall and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions. I. Uniqueness of viscosity solutions. J. Funct. Anal. 62 (1985) 379-396. | Zbl

[9] M. Crandall and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions. II. Existence of viscosity solutions. J. Funct. Anal. 65 (1986) 368-405. | Zbl

[10] M. Crandall and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions. III. J. Funct. Anal. 68 (1986) 214-247. | Zbl

[11] M. Crandall and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions. IV. Hamiltonians with unbounded linear terms. J. Funct. Anal. 90 (1990) 237-283. | Zbl

[12] M. Crandall and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions. V. Unbounded linear terms and B-continuous solutions. J. Funct. Anal. 97 (1991) 417-465. | Zbl

[13] M. Crandall and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions. VI. Nonlinear A and Tataru's method refined, in Evolution equations, control theory, and biomathematics, Lect. Notes Pure Appl. Math. 155, Dekker, New York (1994) 51-89. | Zbl

[14] I. Elsanosi, B. Øksendal and A. Sulem, Some solvable stochastic control problems with delay. Stochast. Stochast. Rep. 71 (2000) 69-89. | Zbl

[15] G. Fabbri, Viscosity solutions to delay differential equations in demo-economy. Math. Popul. Stud. 15 (2008) 27-54. | Zbl

[16] G. Fabbri, S. Faggian and F. Gozzi, On dynamic programming in economic models governed by DDEs. Math. Popul. Stud. 15 (2008) 267-290. | Zbl

[17] S. Faggian and F. Gozzi, On the dynamic programming approach for optimal control problems of PDE's with age structure. Math. Popul. Stud. 11 (2004) 233-270. | Zbl

[18] F. Gozzi and C. Marinelli, Stochastic optimal control of delay equations arising in advertising models, in Stochastic partial differential equations and applications VII, Chapman & Hall, Boca Raton, Lect. Notes Pure Appl. Math. 245 (2006) 133-148. | Zbl

[19] V.B. Kolmanovskii and L.E. Shaikhet, Control of systems with aftereffect, Translations of Mathematical Monographs. American Mathematical Society, Providence, USA (1996). | Zbl

[20] B. Larssen and N.H. Risebro, When are HJB-equations in stochastic control of delay systems finite dimensional? Stochastic Anal. Appl. 21 (2003) 643-671. | Zbl

[21] L. Samassi and R. Tahraoui, Comment établir des conditions nécessaires d'optimalité dans les problèmes de contrôle dont certains arguments sont déviés ? C. R. Math. Acad. Sci. Paris 338 (2004) 611-616. | Zbl

[22] L. Samassi and R. Tahraoui, How to state necessary optimality conditions for control problems with deviating arguments? ESAIM: COCV 14 (2008) 381-409. | Numdam | Zbl