A semi-Lagrangian algorithm in policy space for hybrid optimal control problems
ESAIM: Control, Optimisation and Calculus of Variations, Volume 24 (2018) no. 3, pp. 965-983.

The mathematical framework of hybrid system is a recent and general tool to treat control systems involving control action of heterogeneous nature. In this paper, we construct and test a semi-Lagrangian numerical scheme for solving the Dynamic Programming equation of an infinite horizon optimal control problem for hybrid systems. In order to speed up convergence, we also propose and analyze an acceleration technique based on policy iteration. Finally, we validate the approach via some numerical tests in low dimension.

Received:
Accepted:
DOI: 10.1051/cocv/2017022
Classification: 34A38, 49L20, 65B99, 65N06
Keywords: Hybrid control, dynamic programming, semi-lagrangian schemes, policy iteration
Ferretti, Roberto 1; Sassi, Achille 1

1
@article{COCV_2018__24_3_965_0,
     author = {Ferretti, Roberto and Sassi, Achille},
     title = {A {semi-Lagrangian} algorithm in policy space for hybrid optimal control problems},
     journal = {ESAIM: Control, Optimisation and Calculus of Variations},
     pages = {965--983},
     publisher = {EDP-Sciences},
     volume = {24},
     number = {3},
     year = {2018},
     doi = {10.1051/cocv/2017022},
     mrnumber = {3877189},
     zbl = {1405.49024},
     language = {en},
     url = {http://archive.numdam.org/articles/10.1051/cocv/2017022/}
}
TY  - JOUR
AU  - Ferretti, Roberto
AU  - Sassi, Achille
TI  - A semi-Lagrangian algorithm in policy space for hybrid optimal control problems
JO  - ESAIM: Control, Optimisation and Calculus of Variations
PY  - 2018
SP  - 965
EP  - 983
VL  - 24
IS  - 3
PB  - EDP-Sciences
UR  - http://archive.numdam.org/articles/10.1051/cocv/2017022/
DO  - 10.1051/cocv/2017022
LA  - en
ID  - COCV_2018__24_3_965_0
ER  - 
%0 Journal Article
%A Ferretti, Roberto
%A Sassi, Achille
%T A semi-Lagrangian algorithm in policy space for hybrid optimal control problems
%J ESAIM: Control, Optimisation and Calculus of Variations
%D 2018
%P 965-983
%V 24
%N 3
%I EDP-Sciences
%U http://archive.numdam.org/articles/10.1051/cocv/2017022/
%R 10.1051/cocv/2017022
%G en
%F COCV_2018__24_3_965_0
Ferretti, Roberto; Sassi, Achille. A semi-Lagrangian algorithm in policy space for hybrid optimal control problems. ESAIM: Control, Optimisation and Calculus of Variations, Volume 24 (2018) no. 3, pp. 965-983. doi : 10.1051/cocv/2017022. http://archive.numdam.org/articles/10.1051/cocv/2017022/

[1] G. Barles and P.E. Souganidis, Convergence of approximation schemes for fully nonlinear second-order equations. Asymptotic Anal. 4 (1991) 271–283 | MR | Zbl

[2] R. Bellman, Dynamic Programming. Princeton University Press, Princeton NJ (1957) | MR | Zbl

[3] O. Bokanowski, S. Maroso and H. Zidani, Some convergence results for Howard’s algorithm. SIAM J. Numer. Anal. 47 (2009) 3001–3026 | DOI | MR | Zbl

[4] M.S. Branicky, V. Borkar and S. Mitter, A unified framework for hybrid control problem. IEEE Trans. Autom. Control 43 (1998) 31–45 | DOI | MR | Zbl

[5] J. Chai and R.G. Sanfelice, Hybrid feedback control methods for robust and global power conversion. IFAC–PapersOnLine 48 (2015) 298–303

[6] S. Dharmatti and M. Ramaswamy, Hybrid control system and viscosity solutions. SIAM J. Control Optimiz. 34 (2005), 1259–1288 | DOI | MR | Zbl

[7] M. Falcone and R. Ferretti, Semi-Lagrangian approximation schemes for linear and Hamilton–Jacobi equations. SIAM, Philadelphia (2013) | DOI | MR

[8] R. Ferretti and H. Zidani, Monotone numerical schemes and feedback construction for hybrid control systems. J. Optimiz. Theory Appl. 165 (2014) 507–531 | DOI | MR | Zbl

[9] R.A. Howard, Dynamic Programming and Markov processes. MIT Press, Cambridge MA (1960) | MR | Zbl

[10] U. Ledzewicz and H. Schättler, Optimal bang–bang control for a two-compartment model in cancer chemotherapy. J. Optimiz. Theory Appl. 114 (2002) 609–637 | DOI | MR | Zbl

[11] M.L. Puterman and S.L. Brumelle, On the convergence of policy iteration in stationary dynamic programming. Math. Oper. Res. 4 (1979) 60–69 | DOI | MR | Zbl

[12] M.L. Puterman and M.C. Shin, Modified policy iteration algorithms for discounted Markov decision problems. Manag. Sci. 24 (1978) 1127–1137 | DOI | MR | Zbl

[13] M.S. Santos and J. Rust, Convergence properties of policy iteration. SIAM J. Control Optimiz. 42 (2004) 2094–2115 | DOI | MR | Zbl

[14] A. Sassi, Tecniche di Programmazione Dinamica nell’ottimizzazione di sistemi di controllo ibridi. MSc Thesis, Università Roma Tre (2013)

Cited by Sources: