added some changes before q learning