This technical note studies Markov decision processes under parameter uncertainty. We adapt the distributionally robust optimization framework, assume that the uncertain parameters are random variables following an unknown distribution, and seek the strategy which maximizes the expected performance under the most adversarial distribution. In particular, we generalize a previous study [1] which concentrates on distribution sets with very special structure to a considerably more generic class of distribution sets, and show that the optimal strategy can be obtained efficiently under mild technical conditions. This significantly extends the applicability of distributionally robust MDPs by incorporating probabilistic information of uncertainty in a more flexible way. Index Terms-Distributional robustness, Markov decision processes, parameter uncertainty.
These REF in which the optimal strategy maximizes the expected reward under the most adversarial distribution over the uncertainty set.
Distributionally Robust Counterpart in Markov Decision Processes
IEEE Transactions on Automatic Control
IEEE Transactions on Automatic Control
Mathematics Computer Science