A general Hamilton-Jacobi framework for non-linear state-constrained control problems

Altarovici, Albert; Bokanowski, Olivier; Zidani, Hasnaa

ESAIM: Control, Optimisation and Calculus of Variations, Tome 19 (2013) no. 2, pp. 337-357.

The paper deals with deterministic optimal control problems with state constraints and non-linear dynamics. It is known for such problems that the value function is in general discontinuous and its characterization by means of a Hamilton-Jacobi equation requires some controllability assumptions involving the dynamics and the set of state constraints. Here, we first adopt the viability point of view and look at the value function as its epigraph. Then, we prove that this epigraph can always be described by an auxiliary optimal control problem free of state constraints, and for which the value function is Lipschitz continuous and can be characterized, without any additional assumptions, as the unique viscosity solution of a Hamilton-Jacobi equation. The idea introduced in this paper bypasses the regularity issues on the value function of the constrained control problem and leads to a constructive way to compute its epigraph by a large panel of numerical schemes. Our approach can be extended to more general control problems. We study in this paper the extension to the infinite horizon problem as well as for the two-player game setting. Finally, an illustrative numerical example is given to show the relevance of the approach.

[1] J.-P. Aubin, Viability theory. Birkäuser, Boston (1991). | MR | Zbl

[2] J.-P. Aubin, Viability solutions to structured Hamilton-Jacobi equations under constraints. SIAM J. Control Optim. 49 (2011) 1881-1915. | MR | Zbl

[3] J.-P. Aubin and A. Cellina, Differential inclusions, Comprehensive Studies in Mathematics. Springer, Berlin, Heidelberg, New York, Tokyo 264 (1984). | MR | Zbl

[4] J.-P. Aubin and H. Frankowska, Set-valued analysis, Birkhäuser Boston Inc., Boston, MA. Systems and Control : Foundations and Applications 2 (1990). | MR | Zbl

[5] J.-P. Aubin and H. Frankowska, The viability kernel algorithm for computing value functions of infinite horizon optimal control problems. J. Math. Anal. Appl. 201 (1996) 555-576. | MR | Zbl

[6] M. Bardi and I. Capuzzo-Dolcetta, Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations, Systems and Control : Foundations and Applications. Birkhäuser, Boston (1997). | MR | Zbl

[7] M. Bardi, S. Koike and P. Soravia, Pursuit-evasion games with state constraints : dynamic programming and discrete-time approximations. Discrete Contin. Dyn. Syst. 6 (2000) 361-380. | MR | Zbl

[8] G. Barles, Solutions de viscosité des équations de Hamilton-Jacobi, Springer, Paris. Math. Appl. 17 (1994). | MR | Zbl

[9] R.C. Barnard and P.R. Wolenski, The minimal time function on stratified domains. Submitted (2011).

[10] E.N. Barron, Viscosity solutions and analysis in L∞, in Proc. of the NATO Advanced Study Institute (1999) 1-60. | MR | Zbl

[11] E.N. Barron and H. Ishii, The bellman equation for minimizing the maximum cost. Nonlinear Anal. 13 (1989) 1067-1090. | Zbl

[12] E.N. Barron and R. Jensen, Semicontinuous viscosity solutions for Hamilton-Jacobi equations with convex Hamiltonians. Commun. Partial Differ. Equ. 15 (1990) 1713-1742. | MR | Zbl

[13] E.N. Barron and R. Jensen, Relaxation of constrained control problems. SIAM J. Control Optim. 34 (1996) 2077-2091. | MR | Zbl

[14] O. Bokanowski, E. Cristiani and H. Zidani, An efficient data structure and accurate scheme to solve front propagation problems. J. Sci. Comput. 42 (2010) 251-273. | MR | Zbl

[15] O. Bokanowski, N. Forcadel and H. Zidani, Reachability and minimal times for state constrained nonlinear problems without any controllability assumption. SIAM J. Control Optim. 48 (2010) 4292-4316. | MR | Zbl

[16] O. Bokanowski, N. Forcadel and H. Zidani, Deterministic state constrained optimal control problems without controllability assumptions. ESAIM : COCV 17 (2011) 995-1015. | Numdam | MR | Zbl

[17] O. Bokanowski, J. Zhao and H. Zidani, Binope-HJ : a d-dimensional C++ parallel HJ solver. http://www.ensta-paristech.fr/~zidani/BiNoPe-HJ/ (2011).

[18] I. Capuzzo-Dolcetta and P.-L. Lions, Hamilton-Jacobi equations with state constraints. Trans. Amer. Math. Soc. 318 (1990) 643-683. | MR | Zbl

[19] P. Cardaliaguet, M. Quincampoix and P. Saint-Pierre, Optimal times for constrained nonlinear control problems without local controllability. Appl. Math. Optim. 36 (1997) 21-42. | MR | Zbl

[20] P. Cardaliaguet, M. Quincampoix and P. Saint-Pierre, Numerical schemes for discontinuous value function of optimal control. Set-Valued Analysis 8 (2000) 111-126. | MR | Zbl

[21] P. Cardaliaguet, M. Quincampoix and P. Saint-Pierre, Pursuit differential games with state constraints. SIAM J. Control Optim. 39 (2000) 1615-1632 (electronic). | MR | Zbl

[22] F. Clarke, Y.S. Ledyaev, R. Stern and P. Wolenski, Nonsmooth analysis and control theory. Springer (1998). | MR | Zbl

[23] M. Crandall and P.-L. Lions, Viscosity solutions of Hamilton Jacobi equations. Bull. Amer. Math. Soc. 277 (1983) 1-42. | MR | Zbl

[24] M. Crandall, L. Evans and P.-L. Lions, Some properties of viscosity solutions of Hamilton-Jacobi equations. Trans. Amer. Math. Soc. 282 (1984) 487-502. | MR | Zbl

[25] R.J. Elliott and N.J. Kalton, The existence of value in differential games, American Mathematical Society, Providence, RI. Memoirs of the American Mathematical Society 126 (1972). | MR | Zbl

[26] H. Frankowska, Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim. 31 (1993) 257-272. | MR | Zbl

[27] H. Frankowska and S. Plaskacz, Semicontinuous solutions of Hamilton-Jacobi-Bellman equations with degenerate state constraints. J. Math. Anal. Appl. 251 (2000) 818-838. | MR | Zbl

[28] H. Frankowska and F. Rampazzo, Relaxation of control systems under state constraints. SIAM J. Control Optim. 37 (1999) 1291-1309. | MR | Zbl

[29] H. Frankowska and R.B. Vinter, Existence of neighboring feasible trajectories : applications to dynamic programming for state-constrained optimal control problems. J. Optim. Theory Appl. 104 (2000) 21-40. | MR | Zbl

[30] H. Ishii, Uniqueness of unbounded viscosity solution of Hamilton-Jacobi equations. Indiana Univ. Math. J. 33 (1984) 721-748. | MR | Zbl

[31] H. Ishii and S. Koike, A new formulation of state constraint problems for first-order PDEs. SIAM J. Control Optim. 34 (1996) 554-571. | MR | Zbl

[32] P. Loreti, Some properties of constrained viscosity solutions of Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim. 25 (1987) 1244-1252. | MR | Zbl

[33] P. Loreti and E. Tessitore, Approximation and regularity results on constrained viscosity solutions of Hamilton-Jacobi-Bellman equations. J. Math. Systems, Estimation Control 4 (1994) 467-483. | MR | Zbl

[34] K. Margellos and J. Lygeros, Hamilton-Jacobi formulation for reach-avoid differential games. IEEE Trans. Automat. Control 56 (2011) 1849-1861. | MR

[35] M. Motta, On nonlinear optimal control problems with state constraints. SIAM J. Control Optim. 33 (1995) 1411-1424. | MR | Zbl

[36] M. Motta and F. Rampazzo, Multivalued dynamics on a closed domain with absorbing boundary. applications to optimal control problems with integral constraints. Nonlinear Anal. 41 (2000) 631-647. | MR | Zbl

[37] D.P. Peng, B. Merriman, S. Osher, H.K. Zhao and M.J. Kang, A PDE-based fast local level set method. J. Comput. Phys. 155 (1999) 410-438. | MR | Zbl

[38] P. Saint-Pierre, Approximation of viability kernel. Appl. Math. Optim. 29 (1994) 187-209. | MR | Zbl

[39] H.M. Soner, Optimal control with state-space constraint I. SIAM J. Control Optim. 24 (1986) 552-561. | MR | Zbl

[40] H.M. Soner, Optimal control with state-space constraint II. SIAM J. Control Optim. 24 (1986) 1110-1122. | MR | Zbl

[41] P. Soravia, Pursuit-evasion problems and viscosity solutions of Isaacs equations. SIAM J. Control Optim. 31 (1993) 604-623. | MR | Zbl

[42] P.P. Varaiya, On the existence of solutions to a differential game. SIAM J. Control 5 (1967) 153-162. | MR | Zbl