Before revision:  For instance, the proposed analysis explains the overestimation phenomenon in Q-learning due to the maximization bias . Based on the control system theoretic argument and some nice structures of Q-learning , a new finite-time analysis of the Q-learning is given with a novel error bound .
After revision:  For instance, the proposed analysis explains the overestimation phenomenon in Q-learning due to the maximization bias . Based on the control system theoretic argument and some nice structures of Q-learning , a new finite-time analysis of the Q-learning is given with a finite-time error bound. It offers novel intuitive insights on analysis of Q-learning mainly based on control theoretic frameworks. By filling the gap between both domains in a synergistic way, this approach can potentially facilitate further progress in each field .