as#1 q-learning code refactoring