Discrete dynamic programming and viscosity solutions of the Bellman equation

Capuzzo Dolcetta, I.; Falcone, M.

Annales de l'I.H.P. Analyse non linéaire, Tome S6 (1989), pp. 161-183.

[1] M. Bardi, A boundary value problem for the minimum time function, to appear on SIAM J. Control and Optimization. | MR | Zbl

[2] M. Bardi, M. Falcone, An approximation scheme for the minimum time function, preprint, july 1988 | MR

[3] G. Barles, Inéquations quasi-variationnelles du premier ordre et equations d'Hamilton-Jacobi,, C.R.Acad.Sc.Paris,t.296, 1983. | MR

[4] G. Barles, Deterministic impulse control problems, SIAM J. Control and Optimization, vol.23, 3, 1985 | MR | Zbl

[5] G. Barles, B. Perthame, Discontinuous solutions of deterministic optimal stopping time problems, to appear in Math. Methods and Num. Anal. | Numdam | MR

[6] G. Barles, B.Perthame, Exit time problems in optimal control and vanishing viscosity method, preprint | MR

[7] R. Bellman, Dynamic programming, Princeton University Press, 1957 | MR | Zbl

[8] R. Bellman, S. Dreyfus, Applied Dynamic Programming, Princeton University Press, 1962 | MR | Zbl

[9] A. Bensoussan, Contrôle stochastique discret, Cahiers du Ceremade

[10] A. Bensoussan, W. Runggaldier, An approximation method for stochastic control problems with partial observation of the state-A method for constructing ∈-optimal controls, Acta Applicandae Mathematicae, vol. 10, 2, 1987 | MR | Zbl

[11] D.P. Bertsekas, S.E. Shreve, Stochastic optimal control: the discrete time case, Academic Press, New York, 1978. | MR | Zbl

[12] I. Capuzzo Dolcetta, On a discrete approximation of the Hamilton - Jacobi equation of dynamic programming, Appl.Math.and Optim. 10,1983 | MR | Zbl

[13] I. Capuzzo Dolcetta,L.C. Evans, Optimal switching for ordinary differential equations, SIAM J. Control and Optimization, vol.22,1984 | MR | Zbl

[14] I. Capuzzo Dolcetta, H. Ishii, Approximate solutions of the Bellman equation of deterministic control theory, Appl. Math.and Optim.11,1984. | MR | Zbl

[15] I. Capuzzo Dolcetta, P.L. Lions, Hamilton-Jacobi equations and state-constrained problems, to appear in Transactions of the AMS

[16] I. Capuzzo Dolcetta, M. Matzeu, On the dynamic programming inequalities associated with the optimal stopping problem in discrete and continuous time, Numer. Funct. Anal. Optim., 3 (4), 1981. | MR | Zbl

[17] I. Capuzzo Dolcetta,M. Matzeu,J.L. Menaldi, On a system of first order quasi-variational inequalities connected with the optimal switching problem, System and Control Letters, Vol.3,No.2,1983. | MR | Zbl

[18] D.A. Carlson, A. Haurie, Infinite horizon optimal control, Lecture Notes in Economics and Mathematical Systems, 290, Springer Verlag,1987. | MR | Zbl

[19] M.G. Crandall, H. Ishii, P.L. Lions, Uniqueness of viscosity solutions of Hamilton-Jacobi equations revisited, J. Math. Soc. Japan, vol. 39, 4, 1987 | MR | Zbl

[20] M.G. Crandall, P.L. Lions, Two approximations of solutions of Hamilton-Jacobi equations, Mathematics of Computation, vol.43, n.167, 1984 | MR | Zbl

[21] M.G. Crandall, P.L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. A.M.S. 277, 1983. | MR | Zbl

[22] J. Cullum, An explicit procedure for discretizing continuous optimal control problems, J. Optimization Theory and Applications, 8(1),1971. | MR | Zbl

[23] M. Falcone, A numerical approach to the infinite horizon problem of deterministic control theory, Appl. Math. Optim. 15, 1987 | MR | Zbl

[24] M. Falcone, Numerical solution of deterministic continuous control problems, Proceedings International Symposium on Numerical Analysis, Madrid, 1985

[25] M. Falcone, Approximate feedback controls for deterministic control problems, in preparation

[26] W.H. Fleming, Controlled Markov processes and viscosity solutions of nonlinear evolution equations, Lecture Notes, Scuola Normale Superiore Pisa, to appear | MR | Zbl

[27] W.H. Fleming, R. Rishel, Deterministic and stochastic optimal control, Springer Verlag, Berlin, 1975. | MR | Zbl

[28] R. Gonzales, E. Rofman, On deterministic control problems: an approximation procedure for the optimal cost, part I and II, SIAM J. Control and Optimization, vol. 23, n. 2, 1985 | Zbl

[29] R.A. Howard, Dynamic programming and Markov processes, Wiley, New York, 1960 | MR | Zbl

[30] M.M. Hrustalev, Necessary and sufficient optimality conditions in the form of Bellman's equation, Soviet Math.Dokl. ,19 (5), 1978. | Zbl

[31] R. Kalaba, On nonlinear differential equations, the maximum operation and monotone convergence, J. of Math. and Mech., vol 8, n. 4, 1959 | MR | Zbl

[32] H.J. Kushner, Probability methods for approximation in stochastic control and for elliptic equations, Academic Press, New York, 1977 | Zbl

[33] H.J. Kushner, A.J. Kleinman, Accelerated procedures for the solution of discrete Markov control problems, IEEE Trans. Automatic control, AC-16, 1971 | MR

[34] H.J. Kushner, A.J. Kleinman, Mathematical programming and the control of Markov chains, Int. J. Control, 13, 1971 | MR | Zbl

[35] R.E. Larson, State increment dynamic programming , American Elsevier, New York, 1967 | MR | Zbl

[36] R.E. Larson, Korsak, A dynamic programming successive approximaton technique with convergence proofs, parts I end II, Automatica, 1969 | Zbl

[37] E.B. Lee,L. Markus, Foundations of optimal control, J.Wiley, New York, 1967. | MR | Zbl

[38] P.L. Lions, Generalized solutions of Hamilton-Jacobi equations, Pitman, London, 1982. | MR | Zbl

[39] P.L. Lions, Optimal control and viscosity solutions, in I. Capuzzo Dolcetta-W.H. Fleming-T. Zolezzi (Eds.), Recent Mathematical Methods in Dynamic Programming, Lecture Notes in Mathematics 1119, Springer Verlag Berlin, 1985. | MR

[40] P.L. Lions, Recent progress on first order Hamilton-Jacobi equations, in Directions in Partial Differential Equations, Academic Press, 1987 | MR | Zbl

[41] P.L. Lions, B. Mercier, Approximation numérique des equations de Hamilton-Jacobi-Bellman, R.A.I.R.O Analyse numérique, vol.14, n. 4, 1980 | Numdam | MR | Zbl

[42] P.L. Lions, P.E. Souganidis, Differential games, optimal control and directional derivatives of viscosity solutions of Bellman-Isaacs equations, to appear in SIAM J. Control and Optimization | MR | Zbl

[43] P. Loreti, Approssimazione di soluzioni viscosità dell'equazione di Bellman, Boll. U.M.I (6), 5-B ,1986. | Zbl

[44] K. Malanowski, On convergence of finite difference approximations to optimal control problems for systems with control appearing linearly, Archivum Automatyki i Telemechaniki, 24, Zeszyt 2, 1979. | MR | Zbl

[45] E. Mascolo,L. Migliaccio, Relaxation methods in optimal control, to appear in J. Optim. Th. App.

[46] J.L. Menaldi, Probabilistic view of estimates for finite difference methods, to appear in Mathematicae Notae | MR | Zbl

[47] J.L. Menaldi, E. Rofman, An algorithm to compute the viscosity solution of the Hamilton-Jacobi-Bellman equation, in C.I. Byrnes, A. Lindquist (eds.), Theory and Applications of Nonlinear Control Systems, North-Holland, 1986 | MR

[48] M.L. Puterman, S.L. Brumelle, On the convergence of policy iteration in stationary dynamic programming, Math. of Operation Research, vol.4, n. 1, 1979 | MR | Zbl

[49] J.P. Quadrat, Existence de solution et algorothme de resolutions numeriques de problemes stochastiques degenerées ou non, SIAM J. Control and Optimization, vol.18, 1980 | MR | Zbl

[50] M.H. Soner Optimal control with state space constraint, SIAM J. Control and Optimization, vol.24, 3, 1986 | MR | Zbl

[51] P.E. Souganidis, Approximation schemes for viscosity solutions of Hamilton-Jacobi equations, Journal of Differential Equations,57,1985. | MR | Zbl

[52] J. Warga, Optimal control of differential and functional equations, Academic Press,New York,1972. | MR | Zbl

[53] L.C. Young, Lectures on the Calculus of Variations and Optimal Control Theory, W.B.Saunders, Philadelfia, 1969. | MR | Zbl