2022-11-28 15:49:35 - r - INFO: - Hyperparameters: 2022-11-28 15:49:35 - r - INFO: - ================================================================================ 2022-11-28 15:49:35 - r - INFO: - Name Value Type 2022-11-28 15:49:35 - r - INFO: - env_name Racetrack-v0 2022-11-28 15:49:35 - r - INFO: - new_step_api 1 2022-11-28 15:49:35 - r - INFO: - wrapper None 2022-11-28 15:49:35 - r - INFO: - render 0 2022-11-28 15:49:35 - r - INFO: - algo_name QLearning 2022-11-28 15:49:35 - r - INFO: - mode train 2022-11-28 15:49:35 - r - INFO: - seed 10 2022-11-28 15:49:35 - r - INFO: - device cpu 2022-11-28 15:49:35 - r - INFO: - train_eps 400 2022-11-28 15:49:35 - r - INFO: - test_eps 20 2022-11-28 15:49:35 - r - INFO: - eval_eps 10 2022-11-28 15:49:35 - r - INFO: - eval_per_episode 5 2022-11-28 15:49:35 - r - INFO: - max_steps 200 2022-11-28 15:49:35 - r - INFO: - load_checkpoint 0 2022-11-28 15:49:35 - r - INFO: - load_path Train_CartPole-v1_DQN_20221026-054757 2022-11-28 15:49:35 - r - INFO: - show_fig 0 2022-11-28 15:49:35 - r - INFO: - save_fig 1 2022-11-28 15:49:35 - r - INFO: - epsilon_start 0.95 2022-11-28 15:49:35 - r - INFO: - epsilon_end 0.01 2022-11-28 15:49:35 - r - INFO: - epsilon_decay 300 2022-11-28 15:49:35 - r - INFO: - gamma 0.9 2022-11-28 15:49:35 - r - INFO: - lr 0.1 2022-11-28 15:49:35 - r - INFO: - ================================================================================ 2022-11-28 15:49:36 - r - INFO: - n_states: 4, n_actions: 9 2022-11-28 15:49:36 - r - INFO: - Start training! 2022-11-28 15:49:36 - r - INFO: - Env: Racetrack-v0, Algorithm: QLearning, Device: cpu 2022-11-28 15:49:36 - r - INFO: - Episode: 1/400, Reward: -850.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 2/400, Reward: -780.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 3/400, Reward: -730.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 4/400, Reward: -650.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 5/400, Reward: -540.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Current episode 5 has the best eval reward: -676.000 2022-11-28 15:49:36 - r - INFO: - Episode: 6/400, Reward: -560.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 7/400, Reward: -550.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 8/400, Reward: -580.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 9/400, Reward: -550.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 10/400, Reward: -560.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Current episode 10 has the best eval reward: -614.000 2022-11-28 15:49:36 - r - INFO: - Episode: 11/400, Reward: -560.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 12/400, Reward: -530.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 13/400, Reward: -550.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 14/400, Reward: -570.000, Step: 200 2022-11-28 15:49:36 - r - INFO: - Episode: 15/400, Reward: -620.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Current episode 15 has the best eval reward: -606.000 2022-11-28 15:49:37 - r - INFO: - Episode: 16/400, Reward: -510.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 17/400, Reward: -560.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 18/400, Reward: -550.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 19/400, Reward: -540.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 20/400, Reward: -520.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Current episode 20 has the best eval reward: -555.000 2022-11-28 15:49:37 - r - INFO: - Episode: 21/400, Reward: -550.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 22/400, Reward: -520.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 23/400, Reward: -520.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 24/400, Reward: -520.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 25/400, Reward: -510.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Current episode 25 has the best eval reward: -527.000 2022-11-28 15:49:37 - r - INFO: - Episode: 26/400, Reward: -540.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 27/400, Reward: -510.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 28/400, Reward: -460.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 29/400, Reward: -500.000, Step: 200 2022-11-28 15:49:37 - r - INFO: - Episode: 30/400, Reward: -520.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Current episode 30 has the best eval reward: -470.000 2022-11-28 15:49:38 - r - INFO: - Episode: 31/400, Reward: -470.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 32/400, Reward: -520.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 33/400, Reward: -530.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 34/400, Reward: -470.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 35/400, Reward: -480.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 36/400, Reward: -530.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 37/400, Reward: -510.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 38/400, Reward: -470.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 39/400, Reward: -366.000, Step: 156 2022-11-28 15:49:38 - r - INFO: - Episode: 40/400, Reward: -510.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 41/400, Reward: -412.000, Step: 172 2022-11-28 15:49:38 - r - INFO: - Episode: 42/400, Reward: -490.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 43/400, Reward: -480.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 44/400, Reward: -470.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Episode: 45/400, Reward: -470.000, Step: 200 2022-11-28 15:49:38 - r - INFO: - Current episode 45 has the best eval reward: -463.000 2022-11-28 15:49:38 - r - INFO: - Episode: 46/400, Reward: -460.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 47/400, Reward: -510.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 48/400, Reward: -480.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 49/400, Reward: -470.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 50/400, Reward: -460.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Current episode 50 has the best eval reward: -218.000 2022-11-28 15:49:39 - r - INFO: - Episode: 51/400, Reward: -510.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 52/400, Reward: -510.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 53/400, Reward: -500.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 54/400, Reward: -470.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 55/400, Reward: -369.000, Step: 169 2022-11-28 15:49:39 - r - INFO: - Episode: 56/400, Reward: -450.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 57/400, Reward: -540.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 58/400, Reward: -53.000, Step: 33 2022-11-28 15:49:39 - r - INFO: - Episode: 59/400, Reward: -500.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 60/400, Reward: -450.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 61/400, Reward: -530.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 62/400, Reward: -480.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 63/400, Reward: -500.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 64/400, Reward: -490.000, Step: 200 2022-11-28 15:49:39 - r - INFO: - Episode: 65/400, Reward: -520.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 66/400, Reward: -460.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 67/400, Reward: -490.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 68/400, Reward: -510.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 69/400, Reward: -480.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 70/400, Reward: -480.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Current episode 70 has the best eval reward: -159.900 2022-11-28 15:49:40 - r - INFO: - Episode: 71/400, Reward: -500.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 72/400, Reward: -343.000, Step: 153 2022-11-28 15:49:40 - r - INFO: - Episode: 73/400, Reward: -470.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 74/400, Reward: -460.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 75/400, Reward: -480.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 76/400, Reward: -490.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 77/400, Reward: -470.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 78/400, Reward: -480.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 79/400, Reward: -199.000, Step: 89 2022-11-28 15:49:40 - r - INFO: - Episode: 80/400, Reward: -480.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 81/400, Reward: -44.000, Step: 24 2022-11-28 15:49:40 - r - INFO: - Episode: 82/400, Reward: -520.000, Step: 200 2022-11-28 15:49:40 - r - INFO: - Episode: 83/400, Reward: -460.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 84/400, Reward: -480.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 85/400, Reward: -9.000, Step: 9 2022-11-28 15:49:41 - r - INFO: - Episode: 86/400, Reward: -450.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 87/400, Reward: -460.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 88/400, Reward: -134.000, Step: 64 2022-11-28 15:49:41 - r - INFO: - Episode: 89/400, Reward: -60.000, Step: 30 2022-11-28 15:49:41 - r - INFO: - Episode: 90/400, Reward: -510.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 91/400, Reward: -111.000, Step: 51 2022-11-28 15:49:41 - r - INFO: - Episode: 92/400, Reward: -99.000, Step: 49 2022-11-28 15:49:41 - r - INFO: - Episode: 93/400, Reward: -470.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 94/400, Reward: -323.000, Step: 133 2022-11-28 15:49:41 - r - INFO: - Episode: 95/400, Reward: -218.000, Step: 98 2022-11-28 15:49:41 - r - INFO: - Episode: 96/400, Reward: -29.000, Step: 19 2022-11-28 15:49:41 - r - INFO: - Episode: 97/400, Reward: -424.000, Step: 194 2022-11-28 15:49:41 - r - INFO: - Episode: 98/400, Reward: -470.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 99/400, Reward: -470.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 100/400, Reward: -240.000, Step: 120 2022-11-28 15:49:41 - r - INFO: - Episode: 101/400, Reward: -17.000, Step: 17 2022-11-28 15:49:41 - r - INFO: - Episode: 102/400, Reward: -11.000, Step: 11 2022-11-28 15:49:41 - r - INFO: - Episode: 103/400, Reward: -83.000, Step: 43 2022-11-28 15:49:41 - r - INFO: - Episode: 104/400, Reward: -460.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 105/400, Reward: -11.000, Step: 11 2022-11-28 15:49:41 - r - INFO: - Episode: 106/400, Reward: -470.000, Step: 200 2022-11-28 15:49:41 - r - INFO: - Episode: 107/400, Reward: 4.000, Step: 6 2022-11-28 15:49:41 - r - INFO: - Episode: 108/400, Reward: -85.000, Step: 45 2022-11-28 15:49:41 - r - INFO: - Episode: 109/400, Reward: -204.000, Step: 104 2022-11-28 15:49:41 - r - INFO: - Episode: 110/400, Reward: -139.000, Step: 59 2022-11-28 15:49:42 - r - INFO: - Current episode 110 has the best eval reward: -114.800 2022-11-28 15:49:42 - r - INFO: - Episode: 111/400, Reward: -39.000, Step: 29 2022-11-28 15:49:42 - r - INFO: - Episode: 112/400, Reward: -58.000, Step: 38 2022-11-28 15:49:42 - r - INFO: - Episode: 113/400, Reward: -200.000, Step: 90 2022-11-28 15:49:42 - r - INFO: - Episode: 114/400, Reward: -105.000, Step: 45 2022-11-28 15:49:42 - r - INFO: - Episode: 115/400, Reward: -107.000, Step: 47 2022-11-28 15:49:42 - r - INFO: - Episode: 116/400, Reward: -179.000, Step: 79 2022-11-28 15:49:42 - r - INFO: - Episode: 117/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 118/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 119/400, Reward: -37.000, Step: 27 2022-11-28 15:49:42 - r - INFO: - Episode: 120/400, Reward: 2.000, Step: 8 2022-11-28 15:49:42 - r - INFO: - Current episode 120 has the best eval reward: -17.200 2022-11-28 15:49:42 - r - INFO: - Episode: 121/400, Reward: -61.000, Step: 31 2022-11-28 15:49:42 - r - INFO: - Episode: 122/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 123/400, Reward: -21.000, Step: 21 2022-11-28 15:49:42 - r - INFO: - Episode: 124/400, Reward: -40.000, Step: 30 2022-11-28 15:49:42 - r - INFO: - Episode: 125/400, Reward: -49.000, Step: 29 2022-11-28 15:49:42 - r - INFO: - Episode: 126/400, Reward: -12.000, Step: 12 2022-11-28 15:49:42 - r - INFO: - Episode: 127/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 128/400, Reward: -140.000, Step: 70 2022-11-28 15:49:42 - r - INFO: - Episode: 129/400, Reward: -13.000, Step: 13 2022-11-28 15:49:42 - r - INFO: - Episode: 130/400, Reward: -115.000, Step: 55 2022-11-28 15:49:42 - r - INFO: - Episode: 131/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 132/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 133/400, Reward: -20.000, Step: 20 2022-11-28 15:49:42 - r - INFO: - Episode: 134/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 135/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 136/400, Reward: -17.000, Step: 17 2022-11-28 15:49:42 - r - INFO: - Episode: 137/400, Reward: -61.000, Step: 31 2022-11-28 15:49:42 - r - INFO: - Episode: 138/400, Reward: -19.000, Step: 19 2022-11-28 15:49:42 - r - INFO: - Episode: 139/400, Reward: -15.000, Step: 15 2022-11-28 15:49:42 - r - INFO: - Episode: 140/400, Reward: -16.000, Step: 16 2022-11-28 15:49:42 - r - INFO: - Current episode 140 has the best eval reward: -3.800 2022-11-28 15:49:42 - r - INFO: - Episode: 141/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 142/400, Reward: -13.000, Step: 13 2022-11-28 15:49:42 - r - INFO: - Episode: 143/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 144/400, Reward: -15.000, Step: 15 2022-11-28 15:49:42 - r - INFO: - Episode: 145/400, Reward: -108.000, Step: 58 2022-11-28 15:49:42 - r - INFO: - Episode: 146/400, Reward: -136.000, Step: 76 2022-11-28 15:49:42 - r - INFO: - Episode: 147/400, Reward: -99.000, Step: 49 2022-11-28 15:49:42 - r - INFO: - Episode: 148/400, Reward: -69.000, Step: 39 2022-11-28 15:49:42 - r - INFO: - Episode: 149/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 150/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 151/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 152/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 153/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 154/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 155/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 156/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 157/400, Reward: -140.000, Step: 70 2022-11-28 15:49:42 - r - INFO: - Episode: 158/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 159/400, Reward: -17.000, Step: 17 2022-11-28 15:49:42 - r - INFO: - Episode: 160/400, Reward: -15.000, Step: 15 2022-11-28 15:49:42 - r - INFO: - Episode: 161/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 162/400, Reward: -31.000, Step: 21 2022-11-28 15:49:42 - r - INFO: - Episode: 163/400, Reward: -18.000, Step: 18 2022-11-28 15:49:42 - r - INFO: - Episode: 164/400, Reward: -88.000, Step: 48 2022-11-28 15:49:42 - r - INFO: - Episode: 165/400, Reward: -1.000, Step: 11 2022-11-28 15:49:42 - r - INFO: - Episode: 166/400, Reward: -32.000, Step: 22 2022-11-28 15:49:42 - r - INFO: - Episode: 167/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 168/400, Reward: -34.000, Step: 24 2022-11-28 15:49:42 - r - INFO: - Episode: 169/400, Reward: -102.000, Step: 52 2022-11-28 15:49:42 - r - INFO: - Episode: 170/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Current episode 170 has the best eval reward: -3.800 2022-11-28 15:49:42 - r - INFO: - Episode: 171/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 172/400, Reward: -13.000, Step: 13 2022-11-28 15:49:42 - r - INFO: - Episode: 173/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 174/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 175/400, Reward: 2.000, Step: 8 2022-11-28 15:49:42 - r - INFO: - Episode: 176/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 177/400, Reward: -37.000, Step: 27 2022-11-28 15:49:42 - r - INFO: - Episode: 178/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 179/400, Reward: -45.000, Step: 25 2022-11-28 15:49:42 - r - INFO: - Episode: 180/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 181/400, Reward: -27.000, Step: 17 2022-11-28 15:49:42 - r - INFO: - Episode: 182/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 183/400, Reward: -42.000, Step: 22 2022-11-28 15:49:42 - r - INFO: - Episode: 184/400, Reward: -12.000, Step: 12 2022-11-28 15:49:42 - r - INFO: - Episode: 185/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 186/400, Reward: 3.000, Step: 7 2022-11-28 15:49:42 - r - INFO: - Episode: 187/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 188/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 189/400, Reward: -29.000, Step: 19 2022-11-28 15:49:42 - r - INFO: - Episode: 190/400, Reward: -19.000, Step: 19 2022-11-28 15:49:42 - r - INFO: - Episode: 191/400, Reward: -19.000, Step: 19 2022-11-28 15:49:42 - r - INFO: - Episode: 192/400, Reward: -19.000, Step: 19 2022-11-28 15:49:42 - r - INFO: - Episode: 193/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 194/400, Reward: -15.000, Step: 15 2022-11-28 15:49:42 - r - INFO: - Episode: 195/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 196/400, Reward: -147.000, Step: 77 2022-11-28 15:49:42 - r - INFO: - Episode: 197/400, Reward: -48.000, Step: 28 2022-11-28 15:49:42 - r - INFO: - Episode: 198/400, Reward: -1.000, Step: 11 2022-11-28 15:49:42 - r - INFO: - Episode: 199/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 200/400, Reward: -76.000, Step: 36 2022-11-28 15:49:42 - r - INFO: - Episode: 201/400, Reward: 0.000, Step: 10 2022-11-28 15:49:42 - r - INFO: - Episode: 202/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 203/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 204/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 205/400, Reward: -15.000, Step: 15 2022-11-28 15:49:42 - r - INFO: - Current episode 205 has the best eval reward: 1.600 2022-11-28 15:49:42 - r - INFO: - Episode: 206/400, Reward: 2.000, Step: 8 2022-11-28 15:49:42 - r - INFO: - Episode: 207/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 208/400, Reward: 2.000, Step: 8 2022-11-28 15:49:42 - r - INFO: - Episode: 209/400, Reward: 4.000, Step: 6 2022-11-28 15:49:42 - r - INFO: - Episode: 210/400, Reward: -49.000, Step: 29 2022-11-28 15:49:43 - r - INFO: - Episode: 211/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 212/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 213/400, Reward: -17.000, Step: 17 2022-11-28 15:49:43 - r - INFO: - Episode: 214/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 215/400, Reward: -13.000, Step: 13 2022-11-28 15:49:43 - r - INFO: - Episode: 216/400, Reward: -13.000, Step: 13 2022-11-28 15:49:43 - r - INFO: - Episode: 217/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 218/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 219/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 220/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 221/400, Reward: -17.000, Step: 17 2022-11-28 15:49:43 - r - INFO: - Episode: 222/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 223/400, Reward: -14.000, Step: 14 2022-11-28 15:49:43 - r - INFO: - Episode: 224/400, Reward: -15.000, Step: 15 2022-11-28 15:49:43 - r - INFO: - Episode: 225/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 226/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 227/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 228/400, Reward: -31.000, Step: 21 2022-11-28 15:49:43 - r - INFO: - Episode: 229/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 230/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 231/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 232/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 233/400, Reward: -15.000, Step: 15 2022-11-28 15:49:43 - r - INFO: - Episode: 234/400, Reward: -14.000, Step: 14 2022-11-28 15:49:43 - r - INFO: - Episode: 235/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 236/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 237/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 238/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 239/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 240/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 241/400, Reward: -21.000, Step: 21 2022-11-28 15:49:43 - r - INFO: - Episode: 242/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 243/400, Reward: -27.000, Step: 17 2022-11-28 15:49:43 - r - INFO: - Episode: 244/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 245/400, Reward: -16.000, Step: 16 2022-11-28 15:49:43 - r - INFO: - Episode: 246/400, Reward: -32.000, Step: 22 2022-11-28 15:49:43 - r - INFO: - Episode: 247/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 248/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 249/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 250/400, Reward: -12.000, Step: 12 2022-11-28 15:49:43 - r - INFO: - Episode: 251/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 252/400, Reward: -49.000, Step: 29 2022-11-28 15:49:43 - r - INFO: - Episode: 253/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 254/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 255/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 256/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 257/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 258/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 259/400, Reward: -54.000, Step: 34 2022-11-28 15:49:43 - r - INFO: - Episode: 260/400, Reward: -15.000, Step: 15 2022-11-28 15:49:43 - r - INFO: - Episode: 261/400, Reward: -34.000, Step: 24 2022-11-28 15:49:43 - r - INFO: - Episode: 262/400, Reward: -18.000, Step: 18 2022-11-28 15:49:43 - r - INFO: - Episode: 263/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 264/400, Reward: -45.000, Step: 25 2022-11-28 15:49:43 - r - INFO: - Episode: 265/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 266/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 267/400, Reward: -30.000, Step: 20 2022-11-28 15:49:43 - r - INFO: - Episode: 268/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 269/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 270/400, Reward: -16.000, Step: 16 2022-11-28 15:49:43 - r - INFO: - Episode: 271/400, Reward: -44.000, Step: 24 2022-11-28 15:49:43 - r - INFO: - Episode: 272/400, Reward: -4.000, Step: 14 2022-11-28 15:49:43 - r - INFO: - Episode: 273/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 274/400, Reward: -14.000, Step: 14 2022-11-28 15:49:43 - r - INFO: - Episode: 275/400, Reward: -29.000, Step: 19 2022-11-28 15:49:43 - r - INFO: - Episode: 276/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 277/400, Reward: -21.000, Step: 21 2022-11-28 15:49:43 - r - INFO: - Episode: 278/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 279/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 280/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 281/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 282/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 283/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 284/400, Reward: -14.000, Step: 14 2022-11-28 15:49:43 - r - INFO: - Episode: 285/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 286/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 287/400, Reward: -34.000, Step: 24 2022-11-28 15:49:43 - r - INFO: - Episode: 288/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 289/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 290/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Current episode 290 has the best eval reward: 3.300 2022-11-28 15:49:43 - r - INFO: - Episode: 291/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 292/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 293/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 294/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 295/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 296/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 297/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 298/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 299/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 300/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 301/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 302/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 303/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 304/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 305/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 306/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 307/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 308/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 309/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 310/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 311/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 312/400, Reward: -19.000, Step: 19 2022-11-28 15:49:43 - r - INFO: - Episode: 313/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 314/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 315/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 316/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 317/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 318/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 319/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 320/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 321/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 322/400, Reward: -29.000, Step: 19 2022-11-28 15:49:43 - r - INFO: - Episode: 323/400, Reward: -14.000, Step: 14 2022-11-28 15:49:43 - r - INFO: - Episode: 324/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 325/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 326/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 327/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 328/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 329/400, Reward: 1.000, Step: 9 2022-11-28 15:49:43 - r - INFO: - Episode: 330/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 331/400, Reward: -11.000, Step: 11 2022-11-28 15:49:43 - r - INFO: - Episode: 332/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 333/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 334/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 335/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 336/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 337/400, Reward: -15.000, Step: 15 2022-11-28 15:49:43 - r - INFO: - Episode: 338/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 339/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 340/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 341/400, Reward: -16.000, Step: 16 2022-11-28 15:49:43 - r - INFO: - Episode: 342/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 343/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 344/400, Reward: -39.000, Step: 29 2022-11-28 15:49:43 - r - INFO: - Episode: 345/400, Reward: -28.000, Step: 18 2022-11-28 15:49:43 - r - INFO: - Episode: 346/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 347/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 348/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 349/400, Reward: -17.000, Step: 17 2022-11-28 15:49:43 - r - INFO: - Episode: 350/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 351/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 352/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 353/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 354/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 355/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 356/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 357/400, Reward: 1.000, Step: 9 2022-11-28 15:49:43 - r - INFO: - Episode: 358/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 359/400, Reward: -13.000, Step: 13 2022-11-28 15:49:43 - r - INFO: - Episode: 360/400, Reward: -32.000, Step: 22 2022-11-28 15:49:43 - r - INFO: - Episode: 361/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 362/400, Reward: -14.000, Step: 14 2022-11-28 15:49:43 - r - INFO: - Episode: 363/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 364/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 365/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 366/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 367/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 368/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 369/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 370/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 371/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 372/400, Reward: -15.000, Step: 15 2022-11-28 15:49:43 - r - INFO: - Episode: 373/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 374/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 375/400, Reward: -19.000, Step: 19 2022-11-28 15:49:43 - r - INFO: - Episode: 376/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 377/400, Reward: -14.000, Step: 14 2022-11-28 15:49:43 - r - INFO: - Episode: 378/400, Reward: -35.000, Step: 25 2022-11-28 15:49:43 - r - INFO: - Episode: 379/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 380/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 381/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 382/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 383/400, Reward: -32.000, Step: 22 2022-11-28 15:49:43 - r - INFO: - Episode: 384/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 385/400, Reward: -13.000, Step: 13 2022-11-28 15:49:43 - r - INFO: - Episode: 386/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 387/400, Reward: -15.000, Step: 15 2022-11-28 15:49:43 - r - INFO: - Episode: 388/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 389/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 390/400, Reward: 0.000, Step: 10 2022-11-28 15:49:43 - r - INFO: - Episode: 391/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 392/400, Reward: 4.000, Step: 6 2022-11-28 15:49:43 - r - INFO: - Episode: 393/400, Reward: -36.000, Step: 26 2022-11-28 15:49:43 - r - INFO: - Episode: 394/400, Reward: -35.000, Step: 25 2022-11-28 15:49:43 - r - INFO: - Episode: 395/400, Reward: -15.000, Step: 15 2022-11-28 15:49:43 - r - INFO: - Episode: 396/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 397/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 398/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Episode: 399/400, Reward: 2.000, Step: 8 2022-11-28 15:49:43 - r - INFO: - Episode: 400/400, Reward: 3.000, Step: 7 2022-11-28 15:49:43 - r - INFO: - Finish training!