Before revision: This paper develops a novel framework to analyze the convergence of Q-learning algorithm from a discrete-time switching system perspective .
After revision: This paper develops a novel framework to analyze the convergence of Q-learning algorithm by using its connections to dynamical systems .