Before revision: This paper develops a novel  framework to analyze the  convergence of Q-learning algorithm by using its connections to dynamical systems.
After revision: This paper develops a novel control-theoretic framework to analyze the convergence of Q-learning algorithm by using its connections to dynamical systems.