Before revision:  We prove that asynchronous Q-learning with a constant step-size can be naturally formulated as  discrete-time stochastic switched linear systems .
After revision:  We prove that asynchronous Q-learning with a constant step-size can be naturally formulated as discrete-time stochastic affine switching system .