Before revision:  To better utilize these samples, prioritized sampling is a promising technique to improve the performance of RL agents. Previous prioritization methods based on temporal-difference (TD ) errorare highly heuristic and divergent from the objective of RL . In this work, we analyze the optimal prioritization strategy that can minimize the regret of RL policytheoretically.
After revision:  To better utilize these samples, prioritized sampling is a promising technique to better utilize these samples. Previous criteria of prioritization include TD error, recentness and corrective feedback, which are mostly heuristically designed . In this work, we analyze the optimal prioritization strategy that can minimize the regret of RL policytheoretically.