1) A new definition of Blackwell optimality via the Blackwell discount factor  $\gamma_{\mathrm{bw}} \in [0,1)$ . Our first main contribution is to highlight that the standard definition of Blackwell optimality cannot be used to compute Blackwell-optimal policies with simple algorithms. Standard definitions have focused on necessary condition for Blackwell optimal policies to be discount optimal. However, we show that this condition needs to be revised when one seeks to compute a Blackwell-optimal policy. We do so by highlighting the potential pathological behaviors of the value functions: a Blackwell-optimal policy may be optimal on an arbitrary number of arbitrary disjoint intervals, and other non-Blackwell optimal policies may also be discount-optimal for some discount factors very close to 1. Demonstrating this issue is important because previous literature has repeatedly overlooked it. To address this issue, we introduce and show the existence of a discount factor  $\gamma_{\mathrm{bw}}$  such that discount optimality for  $\gamma > \gamma_{\mathrm{bw}}$  is sufficient for Blackwell optimality. Knowing the discount factor  $\gamma_{\mathrm{bw}}$  is vital because it enables one to compute Blackwell- and average-optimal policies simply by solving a discounted MDP with  $\gamma \in (\gamma_{\mathrm{bw}}, 1)$ , for which there exist well-studied, simple, and efficient algorithms.
2) Upper-bound the Blackwell discount factor. As our second main contribution, we provide a strict upper bound on  $\gamma_{\mathrm{bw}}$  given an MDP instance. We show that an upper bound must depend on  $r$  and  $P$ , and we compute a bound that is parametrized by the number of states and the number of bits required to represent the MDP instance. Solving a discounted MDP with a discount factor larger or equal than our strict upper bound returns a Blackwell-optimal policy. Crucially, our strict upper bound requires no assumptions on the underlying structure of the MDP, which is a significant improvement on existing literature. Interestingly, the construction of our upper bound relies on novel techniques for analyzing MDPs. We interpret  $\gamma_{\mathrm{bw}} \in [0,1)$  as the root of a polynomial equation  $p(\gamma) = 0$  in  $\gamma$ , show  $p(1) = 0$ , and use a lower bound  $\mathsf{sep}(p)$  on the distance between any two roots of a polynomial  $p$ , known as the separation of algebraic numbers. This shows that  $\gamma_{\mathrm{bw}} < 1 - \mathsf{sep}(p)$ , where  $\mathsf{sep}(p)$  depends on the MDP instance. Since Blackwell optimality implies average optimality, we also obtain the first reduction from average optimality to discounted optimality, without any assumption on the MDP structure. Our upper bound on  $\gamma_{\mathrm{bw}}$  is itself of polynomial size in the bit-size of the MDP data. Combining this bound with interior-point methods for solving discounted MDPs, we obtain new weakly-polynomial time algorithms for computing Blackwell-optimal and average-optimal policies.
3) Blackwell discount factor for robust MDPs. We consider the case of robust reinforcement learning where the transition probabilities are unknown and, instead, belong to an uncertainty set. As our