Before revision:  In this study, we propose methods for OPE and OPL in two-player zero-sum Markov games.
After revision:  In this study, we propose OPE estimators constructed by the doubly robust and double reinforcement learning estimators in two-player zero-sum Markov games.