Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures
DUFOUR, François
Institut Polytechnique de Bordeaux [Bordeaux INP]
Quality control and dynamic reliability [CQFD]
Institut de Mathématiques de Bordeaux [IMB]
Institut Polytechnique de Bordeaux [Bordeaux INP]
Quality control and dynamic reliability [CQFD]
Institut de Mathématiques de Bordeaux [IMB]
DUFOUR, François
Institut Polytechnique de Bordeaux [Bordeaux INP]
Quality control and dynamic reliability [CQFD]
Institut de Mathématiques de Bordeaux [IMB]
< Reduce
Institut Polytechnique de Bordeaux [Bordeaux INP]
Quality control and dynamic reliability [CQFD]
Institut de Mathématiques de Bordeaux [IMB]
Language
en
Article de revue
This item was published in
Journal of Mathematical Analysis and Applications. 2016, vol. 443, n° 2, p. 1323 - 1361
Elsevier
English Abstract
In this paper, we propose an approach for approximating the value function and an ϵ-optimal policy of continuous-time Markov decision processes with Borel state and action spaces, with possibly unbounded cost and transition ...Read more >
In this paper, we propose an approach for approximating the value function and an ϵ-optimal policy of continuous-time Markov decision processes with Borel state and action spaces, with possibly unbounded cost and transition rates, under the total expected discounted cost optimality criterion. Under adequate assumptions, which in particular include that the transition rate has a density function with respect to a reference measure, together with piecewise Lipschitz continuity of the elements of the control model, we approximate the original controlled process by a model with finite state and action spaces. The approximation error is related to the 1-Wasserstein distance between suitably defined probability measures and approximating measures with finite support. We also study the case when the reference measure is approximated with empirical distributions and we show that convergence of the approximations takes place at an exponential rate in probability.Read less <
English Keywords
Linear programming approach to control problems
Approximation of Markov decision processes
Constrained Markov decision processes
Origin
Hal imported