We consider an infinite-horizon discounted optimal control problem for piecewise deterministic Markov processes, where a piecewise open-loop control acts continuously on the jump dynamics and on the deterministic flow. For this class of control problems, the value function can in general be characterized as the unique viscosity solution to the corresponding Hamilton−Jacobi−Bellman equation. We prove that the value function can be represented by means of a backward stochastic differential equation (BSDE) on infinite horizon, driven by a random measure and with a sign constraint on its martingale part, for which we give existence and uniqueness results. This probabilistic representation is known as nonlinear Feynman−Kac formula. Finally we show that the constrained BSDE is related to an auxiliary dominated control problem, whose value function coincides with the value function of the original non-dominated control problem.

Keywords: Backward stochastic differential equations, optimal control problems, piecewise deterministic Markov processes, randomization of controls, discounted cost

^{1}

@article{COCV_2018__24_1_311_0, author = {Bandini, Elena}, title = {Optimal control of piecewise deterministic {Markov} processes: a {BSDE} representation of the value function}, journal = {ESAIM: Control, Optimisation and Calculus of Variations}, pages = {311--354}, publisher = {EDP-Sciences}, volume = {24}, number = {1}, year = {2018}, doi = {10.1051/cocv/2017006}, mrnumber = {3843187}, zbl = {1396.93131}, language = {en}, url = {http://archive.numdam.org/articles/10.1051/cocv/2017006/} }

TY - JOUR AU - Bandini, Elena TI - Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function JO - ESAIM: Control, Optimisation and Calculus of Variations PY - 2018 SP - 311 EP - 354 VL - 24 IS - 1 PB - EDP-Sciences UR - http://archive.numdam.org/articles/10.1051/cocv/2017006/ DO - 10.1051/cocv/2017006 LA - en ID - COCV_2018__24_1_311_0 ER -

%0 Journal Article %A Bandini, Elena %T Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function %J ESAIM: Control, Optimisation and Calculus of Variations %D 2018 %P 311-354 %V 24 %N 1 %I EDP-Sciences %U http://archive.numdam.org/articles/10.1051/cocv/2017006/ %R 10.1051/cocv/2017006 %G en %F COCV_2018__24_1_311_0

Bandini, Elena. Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function. ESAIM: Control, Optimisation and Calculus of Variations, Volume 24 (2018) no. 1, pp. 311-354. doi : 10.1051/cocv/2017006. http://archive.numdam.org/articles/10.1051/cocv/2017006/

[1] A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes. SIAM J. Control Optimiz. 40 (2000) 525–539. | DOI | MR | Zbl

,[2] Existence and uniqueness for backward stochastic differential equations driven by a random measure. Electr. Commun. Probab. 20 (2015) 1–13. | MR | Zbl

,[3] Optimal control of semi-Markov processes with a backward stochastic differential equations approach. Math. Control Signals Syst. 29 (2017) 1–35. | DOI | MR | Zbl

and .[4] Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes. Stochastic Proc. Appl. 127 (2017) 1441–1474. | DOI | MR | Zbl

and ,[5] Weak Dirichlet processes with jumps. Preprint arXiv:1512.06236 (2016). | MR

and ,[6] Special weak Dirichlet processes and BSDEs driven by a random measure. Preprint arXiv:1512.06234 (2016). To appear on Bernouilli (2017). | MR

and ,[7] Backward stochastic differential equations and integral-partial differential equations. Stochastics Rep. 60 (1997) 57–83. | DOI | MR | Zbl

, and .[8] Optimal control of piecewise deterministic Markov processes with finite time horizon. In Modern Trends in Controlled Stochastic Processes: Theory and Applications. Luniver Press, United Kingdom (2010) 123–143.

and ,[9] Bounded solutions to backward SDE’s with jumps for utility optimization and indifference hedging. Ann. Appl. Probab. 16 (2006) 2027–2054. | DOI | MR | Zbl

,[10] Point processes and queues, Martingale dynamics. Springer, Series in Statistics (1981). | DOI | MR | Zbl

,[11] Functional Analysis. Sobolev Spaces and Partial Differential Equations. Springer (2010). | MR | Zbl

,[12] Backward SDE Representation for Stochastic Control Problems with Non Dominated Controlled Intensity. Ann. Appl. Probab. 26 (2016) 1208–1259. | DOI | MR | Zbl

and ,[13] Reflected BSDEs with nonpositive jumps, and controller-and-stopper games. Stochastic Proc. Appl. 125 (2015) 597–633. | DOI | MR | Zbl

, and ,[14] Existence, uniqueness and comparisons for BSDEs in general spaces. Ann. Probab. 40 (2012) 2264–2297. | DOI | MR | Zbl

and ,[15] Backward stochastic differential equations and optimal control of marked point processes. SIAM J. Control Optim. 51 (2013) 3592–3623. | DOI | MR | Zbl

and ,[16] Backward stochastic differential equations associated to jump Markov processes and their applications. Stochastic Proc. Appl. 124 (2014) 289–316. | DOI | MR | Zbl

and ,[17] Backward stochastic differential equations driven by a marked point process: an elementary approach, with an application to optimal control. Ann. Appl. Probab. 26 (2016) 1743–1773. | DOI | MR | Zbl

, and ,[18] Long time asymptotics for fully nonlinear Bellman equations: a Backward SDE approach. Stochastic Proc. Appl. 126 (2016) 1932–1973. | DOI | MR | Zbl

, and ,[19] BSDEs with diffusion constraint and viscous Hamilton−Jacobi equations with unbounded data. Ann. Inst. Henri Poincaré Probab. Stat. 53 (2017) 1528–1547. | DOI | MR | Zbl

, and ,[20] Continuous Average Control of Piecewise Deterministic Markov Processes. Springer Briefs in Mathematics. Springer (2013). | MR | Zbl

and ,[21] Markov models and optimization. In Vol. 49 of Monographs on Statistics and Applied Probability. Chapman and Hall, London (1993). | MR | Zbl

,[22] Piecewise deterministic processes and viscosity solutions. A Volume in Honour of W. H. Fleming on Occasion of His 70th Birthday, edited by , Stochastic Analysis, Control Optimization and Applications. Birkhäuser (1999) 249–268. | DOI | MR | Zbl

and ,[23] Probabilities et Potentiel, Chapters I to IV. Hermann, Paris (1978).

and ,[24] Optimal control of piecewise deterministic Markov processes. Applied stochastic analysis, Vol. 5 of Stochastics Monogr. Gordon and Breach, New York (1991) 303–325. | MR | Zbl

,[25] Necessary and sufficient conditions for control of piecewise deterministic processes. Stochastics and Stochastics Reports 40 (1992) 125–145. | DOI | MR | Zbl

and ,[26] Backward stochastic differential equations in finance. Math. Finance 7 (1997) 1–71. | DOI | MR | Zbl

, and ,[27] Stochastic Calculus and its Applications. Springer (1982). | MR

,[28] Probabilistic representation and approximation for coupled systems of variational inequalities. Statist. Probab. Lett. 80 (2010) 1388–1396. | DOI | MR | Zbl

and ,[29] BSDE representations for optimal switching problems with controlled volatility. Stoch. Dyn. 14 (2014) 1450003. | DOI | MR | Zbl

and ,[30] Randomized and Backward SDE representation for optimal control of non-markovian SDEs. Ann. Appl. Probab. 25 (2015) 2134–2167. | DOI | MR | Zbl

and ,[31] Semimartingale theory and stochastic calculus, Science Press, Bejiing, New York (1992). | MR | Zbl

, and ,[32] Multivariate point processes: predictable projection, Radon−Nikodym derivatives, representation of martingales. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 31 (1974/75) 235–253. | DOI | MR | Zbl

,[33] Calcul stocastique et problèmes de martingales. Vol. 714 of Lect. Notes Math. Springer, Berlin (1979). | MR | Zbl

,[34] Probability essentials. Springer Science and Business Media (2003). | MR | Zbl

and ,[35] Limit Theorems for Stochastic Processes, 2nd edition, Springer, Berlin (2003). | DOI | MR

and ,[36] A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization. Monte Carlo Methods Appl. 20 (2014) 145–165. | DOI | MR | Zbl

, and ,[37] Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps. Ann. Appl. Probab. 25 (2015) 2301–2338. | DOI | MR | Zbl

, and ,[38] Feynman−Kac representation for Hamilton−Jacobi−Bellman IPDE. Ann. Probab. 43 (2015) 1823–1865. | MR | Zbl

and ,[39] Backward SDEs with contrained jumps and quasi-variational inequalities. Ann. Probab. 38 (2010) 794–840. | DOI | MR | Zbl

, , and ,[40] Monotonic limit theorem for BSDEs and non-linear Doob-Meyer decomposition. Probab. Theory Rel. 16 (2000) 225–234.

,[41] Continuous Martingales and Brownian Motion. Grundlehren der mathematischen Wissenschaften. Springer, 3rd edition (1999). | DOI | MR

and ,[42] Optimal control with state-space constraint II. SIAM J. Control Optim. 24 (1986) 1110–1122. | DOI | MR | Zbl

,[43] Multidimensional diffusion processes, Springer (2007). | MR | Zbl

and ,[44] Optimal control of piecewise deterministic Markov process. Stochastics 14 (1985) 165–207. | DOI | MR | Zbl

,[45] Backward stochastic differential equations with random measures. Acta Math. Appl. Sinica 16 (2000) 225–234. | DOI | MR | Zbl

,*Cited by Sources: *