Before revision:  Paramountly, the\tilde{O}\emph{  model-based method combining with our new analysis technique (singleton absorbing MDP) can be adapted to the new settings: offline task-agnostic and the offline reward-free with optimal complexity \tilde{O}(H^2\log(K)/d_m\epsilon^2) (K is the number of tasks) and \tilde{O}(H^2S/d_m\epsilon^2) respectively , which provides a unified framework for simultaneously solving different offline RL problems.
After revision:  Paramountly, the\tilde{O}\emph{ , which is a new sharp analysis tool that works with the model-based method combining with our new analysis technique (singleton absorbing MDP) can be adapted to the new settings: offline task-agnostic and the offline reward-free with optimal complexity \tilde{O}(H^2\log(K)/d_m\epsilon^2) (K is the number of tasks) and \tilde{O}(H^2S/d_m\epsilon^2) respectively , which provides a unified framework for simultaneously solving different offline RL problems.