johnjim0816
update DDPG Pendulum-v1
56029d5
raw
history blame
36.5 kB
2023-03-05 11:42:17 - r - INFO: - Hyperparameters:
2023-03-05 11:42:17 - r - INFO: - ================================================================================
2023-03-05 11:42:17 - r - INFO: - Name Value Type
2023-03-05 11:42:17 - r - INFO: - env_name Pendulum-v1 <class 'str'>
2023-03-05 11:42:17 - r - INFO: - new_step_api 1 <class 'bool'>
2023-03-05 11:42:17 - r - INFO: - wrapper None <class 'str'>
2023-03-05 11:42:17 - r - INFO: - render 0 <class 'bool'>
2023-03-05 11:42:17 - r - INFO: - algo_name SAC <class 'str'>
2023-03-05 11:42:17 - r - INFO: - mode train <class 'str'>
2023-03-05 11:42:17 - r - INFO: - seed 10 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - device cuda <class 'str'>
2023-03-05 11:42:17 - r - INFO: - train_eps 400 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - test_eps 10 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - eval_eps 10 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - eval_per_episode 5 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - max_steps 200 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - load_checkpoint 0 <class 'bool'>
2023-03-05 11:42:17 - r - INFO: - load_path Train_CartPole-v1_DQN_20221026-054757 <class 'str'>
2023-03-05 11:42:17 - r - INFO: - show_fig 0 <class 'bool'>
2023-03-05 11:42:17 - r - INFO: - save_fig 1 <class 'bool'>
2023-03-05 11:42:17 - r - INFO: - policy_type Gaussian <class 'str'>
2023-03-05 11:42:17 - r - INFO: - lr 0.001 <class 'float'>
2023-03-05 11:42:17 - r - INFO: - gamma 0.99 <class 'float'>
2023-03-05 11:42:17 - r - INFO: - tau 0.005 <class 'float'>
2023-03-05 11:42:17 - r - INFO: - alpha 0.2 <class 'float'>
2023-03-05 11:42:17 - r - INFO: - automatic_entropy_tuning 0 <class 'bool'>
2023-03-05 11:42:17 - r - INFO: - batch_size 64 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - hidden_dim 64 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - n_epochs 1 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - start_steps 10000 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - target_update_fre 1 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - buffer_size 1000000 <class 'int'>
2023-03-05 11:42:17 - r - INFO: - continous 0 <class 'bool'>
2023-03-05 11:42:17 - r - INFO: - task_dir /home/dingli/joyrl_offline/tasks/Train_Pendulum-v1_SAC_20230305-114217 <class 'str'>
2023-03-05 11:42:17 - r - INFO: - model_dir /home/dingli/joyrl_offline/tasks/Train_Pendulum-v1_SAC_20230305-114217/models <class 'str'>
2023-03-05 11:42:17 - r - INFO: - res_dir /home/dingli/joyrl_offline/tasks/Train_Pendulum-v1_SAC_20230305-114217/results <class 'str'>
2023-03-05 11:42:17 - r - INFO: - log_dir /home/dingli/joyrl_offline/tasks/Train_Pendulum-v1_SAC_20230305-114217/logs <class 'str'>
2023-03-05 11:42:17 - r - INFO: - traj_dir /home/dingli/joyrl_offline/tasks/Train_Pendulum-v1_SAC_20230305-114217/traj <class 'str'>
2023-03-05 11:42:17 - r - INFO: - ================================================================================
2023-03-05 11:42:17 - r - INFO: - action_bound: 2.0
2023-03-05 11:42:17 - r - INFO: - n_states: 3, n_actions: 1
2023-03-05 11:42:19 - r - INFO: - Start training!
2023-03-05 11:42:19 - r - INFO: - Env: Pendulum-v1, Algorithm: SAC, Device: cuda
2023-03-05 11:42:21 - r - INFO: - Episode: 1/400, Reward: -1740.831, Step: 200
2023-03-05 11:42:23 - r - INFO: - Episode: 2/400, Reward: -961.069, Step: 200
2023-03-05 11:42:26 - r - INFO: - Episode: 3/400, Reward: -967.685, Step: 200
2023-03-05 11:42:28 - r - INFO: - Episode: 4/400, Reward: -1261.628, Step: 200
2023-03-05 11:42:31 - r - INFO: - Episode: 5/400, Reward: -966.460, Step: 200
2023-03-05 11:42:32 - r - INFO: - Current episode 5 has the best eval reward: -1766.117
2023-03-05 11:42:34 - r - INFO: - Episode: 6/400, Reward: -886.180, Step: 200
2023-03-05 11:42:37 - r - INFO: - Episode: 7/400, Reward: -1621.125, Step: 200
2023-03-05 11:42:40 - r - INFO: - Episode: 8/400, Reward: -886.914, Step: 200
2023-03-05 11:42:42 - r - INFO: - Episode: 9/400, Reward: -872.599, Step: 200
2023-03-05 11:42:45 - r - INFO: - Episode: 10/400, Reward: -1263.833, Step: 200
2023-03-05 11:42:46 - r - INFO: - Current episode 10 has the best eval reward: -875.561
2023-03-05 11:42:49 - r - INFO: - Episode: 11/400, Reward: -994.983, Step: 200
2023-03-05 11:42:51 - r - INFO: - Episode: 12/400, Reward: -768.399, Step: 200
2023-03-05 11:42:54 - r - INFO: - Episode: 13/400, Reward: -1660.733, Step: 200
2023-03-05 11:42:56 - r - INFO: - Episode: 14/400, Reward: -1576.356, Step: 200
2023-03-05 11:42:59 - r - INFO: - Episode: 15/400, Reward: -968.902, Step: 200
2023-03-05 11:43:00 - r - INFO: - Current episode 15 has the best eval reward: -106.637
2023-03-05 11:43:03 - r - INFO: - Episode: 16/400, Reward: -1071.378, Step: 200
2023-03-05 11:43:05 - r - INFO: - Episode: 17/400, Reward: -1156.690, Step: 200
2023-03-05 11:43:08 - r - INFO: - Episode: 18/400, Reward: -1514.475, Step: 200
2023-03-05 11:43:10 - r - INFO: - Episode: 19/400, Reward: -1093.737, Step: 200
2023-03-05 11:43:13 - r - INFO: - Episode: 20/400, Reward: -968.878, Step: 200
2023-03-05 11:43:17 - r - INFO: - Episode: 21/400, Reward: -1203.862, Step: 200
2023-03-05 11:43:19 - r - INFO: - Episode: 22/400, Reward: -1405.155, Step: 200
2023-03-05 11:43:22 - r - INFO: - Episode: 23/400, Reward: -917.430, Step: 200
2023-03-05 11:43:24 - r - INFO: - Episode: 24/400, Reward: -1087.190, Step: 200
2023-03-05 11:43:27 - r - INFO: - Episode: 25/400, Reward: -1116.189, Step: 200
2023-03-05 11:43:31 - r - INFO: - Episode: 26/400, Reward: -1823.221, Step: 200
2023-03-05 11:43:33 - r - INFO: - Episode: 27/400, Reward: -1379.747, Step: 200
2023-03-05 11:43:36 - r - INFO: - Episode: 28/400, Reward: -905.889, Step: 200
2023-03-05 11:43:38 - r - INFO: - Episode: 29/400, Reward: -1101.764, Step: 200
2023-03-05 11:43:41 - r - INFO: - Episode: 30/400, Reward: -1175.637, Step: 200
2023-03-05 11:43:45 - r - INFO: - Episode: 31/400, Reward: -1825.436, Step: 200
2023-03-05 11:43:47 - r - INFO: - Episode: 32/400, Reward: -1655.561, Step: 200
2023-03-05 11:43:50 - r - INFO: - Episode: 33/400, Reward: -1180.226, Step: 200
2023-03-05 11:43:52 - r - INFO: - Episode: 34/400, Reward: -1063.964, Step: 200
2023-03-05 11:43:55 - r - INFO: - Episode: 35/400, Reward: -1187.600, Step: 200
2023-03-05 11:43:59 - r - INFO: - Episode: 36/400, Reward: -960.925, Step: 200
2023-03-05 11:44:01 - r - INFO: - Episode: 37/400, Reward: -1522.463, Step: 200
2023-03-05 11:44:04 - r - INFO: - Episode: 38/400, Reward: -1310.517, Step: 200
2023-03-05 11:44:06 - r - INFO: - Episode: 39/400, Reward: -1008.963, Step: 200
2023-03-05 11:44:09 - r - INFO: - Episode: 40/400, Reward: -1062.704, Step: 200
2023-03-05 11:44:13 - r - INFO: - Episode: 41/400, Reward: -1519.136, Step: 200
2023-03-05 11:44:15 - r - INFO: - Episode: 42/400, Reward: -1652.808, Step: 200
2023-03-05 11:44:18 - r - INFO: - Episode: 43/400, Reward: -1333.117, Step: 200
2023-03-05 11:44:20 - r - INFO: - Episode: 44/400, Reward: -964.514, Step: 200
2023-03-05 11:44:23 - r - INFO: - Episode: 45/400, Reward: -906.041, Step: 200
2023-03-05 11:44:27 - r - INFO: - Episode: 46/400, Reward: -866.323, Step: 200
2023-03-05 11:44:29 - r - INFO: - Episode: 47/400, Reward: -1611.942, Step: 200
2023-03-05 11:44:32 - r - INFO: - Episode: 48/400, Reward: -865.089, Step: 200
2023-03-05 11:44:34 - r - INFO: - Episode: 49/400, Reward: -1789.956, Step: 200
2023-03-05 11:44:37 - r - INFO: - Episode: 50/400, Reward: -1261.456, Step: 200
2023-03-05 11:44:41 - r - INFO: - Episode: 51/400, Reward: -121.079, Step: 200
2023-03-05 11:44:43 - r - INFO: - Episode: 52/400, Reward: -125.196, Step: 200
2023-03-05 11:44:46 - r - INFO: - Episode: 53/400, Reward: -236.065, Step: 200
2023-03-05 11:44:48 - r - INFO: - Episode: 54/400, Reward: -230.632, Step: 200
2023-03-05 11:44:50 - r - INFO: - Episode: 55/400, Reward: -117.968, Step: 200
2023-03-05 11:44:55 - r - INFO: - Episode: 56/400, Reward: -359.631, Step: 200
2023-03-05 11:44:57 - r - INFO: - Episode: 57/400, Reward: -115.585, Step: 200
2023-03-05 11:44:59 - r - INFO: - Episode: 58/400, Reward: -126.048, Step: 200
2023-03-05 11:45:02 - r - INFO: - Episode: 59/400, Reward: -124.265, Step: 200
2023-03-05 11:45:05 - r - INFO: - Episode: 60/400, Reward: -122.481, Step: 200
2023-03-05 11:45:09 - r - INFO: - Episode: 61/400, Reward: -117.759, Step: 200
2023-03-05 11:45:11 - r - INFO: - Episode: 62/400, Reward: -0.930, Step: 200
2023-03-05 11:45:14 - r - INFO: - Episode: 63/400, Reward: -121.123, Step: 200
2023-03-05 11:45:16 - r - INFO: - Episode: 64/400, Reward: -230.469, Step: 200
2023-03-05 11:45:19 - r - INFO: - Episode: 65/400, Reward: -121.184, Step: 200
2023-03-05 11:45:23 - r - INFO: - Episode: 66/400, Reward: -291.294, Step: 200
2023-03-05 11:45:25 - r - INFO: - Episode: 67/400, Reward: -129.429, Step: 200
2023-03-05 11:45:28 - r - INFO: - Episode: 68/400, Reward: -123.449, Step: 200
2023-03-05 11:45:31 - r - INFO: - Episode: 69/400, Reward: -0.570, Step: 200
2023-03-05 11:45:33 - r - INFO: - Episode: 70/400, Reward: -117.034, Step: 200
2023-03-05 11:45:37 - r - INFO: - Episode: 71/400, Reward: -118.839, Step: 200
2023-03-05 11:45:39 - r - INFO: - Episode: 72/400, Reward: -117.737, Step: 200
2023-03-05 11:45:42 - r - INFO: - Episode: 73/400, Reward: -256.074, Step: 200
2023-03-05 11:45:45 - r - INFO: - Episode: 74/400, Reward: -230.478, Step: 200
2023-03-05 11:45:47 - r - INFO: - Episode: 75/400, Reward: -123.289, Step: 200
2023-03-05 11:45:49 - r - INFO: - Current episode 75 has the best eval reward: -85.034
2023-03-05 11:45:51 - r - INFO: - Episode: 76/400, Reward: -122.168, Step: 200
2023-03-05 11:45:54 - r - INFO: - Episode: 77/400, Reward: -230.310, Step: 200
2023-03-05 11:45:56 - r - INFO: - Episode: 78/400, Reward: -118.475, Step: 200
2023-03-05 11:45:59 - r - INFO: - Episode: 79/400, Reward: -119.281, Step: 200
2023-03-05 11:46:01 - r - INFO: - Episode: 80/400, Reward: -124.897, Step: 200
2023-03-05 11:46:05 - r - INFO: - Episode: 81/400, Reward: -243.982, Step: 200
2023-03-05 11:46:07 - r - INFO: - Episode: 82/400, Reward: -124.768, Step: 200
2023-03-05 11:46:10 - r - INFO: - Episode: 83/400, Reward: -227.730, Step: 200
2023-03-05 11:46:12 - r - INFO: - Episode: 84/400, Reward: -1.010, Step: 200
2023-03-05 11:46:15 - r - INFO: - Episode: 85/400, Reward: -124.081, Step: 200
2023-03-05 11:46:19 - r - INFO: - Episode: 86/400, Reward: -1.588, Step: 200
2023-03-05 11:46:21 - r - INFO: - Episode: 87/400, Reward: -121.960, Step: 200
2023-03-05 11:46:24 - r - INFO: - Episode: 88/400, Reward: -120.874, Step: 200
2023-03-05 11:46:26 - r - INFO: - Episode: 89/400, Reward: -0.759, Step: 200
2023-03-05 11:46:29 - r - INFO: - Episode: 90/400, Reward: -120.736, Step: 200
2023-03-05 11:46:33 - r - INFO: - Episode: 91/400, Reward: -125.261, Step: 200
2023-03-05 11:46:35 - r - INFO: - Episode: 92/400, Reward: -2.439, Step: 200
2023-03-05 11:46:38 - r - INFO: - Episode: 93/400, Reward: -119.618, Step: 200
2023-03-05 11:46:40 - r - INFO: - Episode: 94/400, Reward: -248.041, Step: 200
2023-03-05 11:46:43 - r - INFO: - Episode: 95/400, Reward: -117.193, Step: 200
2023-03-05 11:46:47 - r - INFO: - Episode: 96/400, Reward: -237.092, Step: 200
2023-03-05 11:46:49 - r - INFO: - Episode: 97/400, Reward: -123.495, Step: 200
2023-03-05 11:46:52 - r - INFO: - Episode: 98/400, Reward: -1.174, Step: 200
2023-03-05 11:46:54 - r - INFO: - Episode: 99/400, Reward: -128.283, Step: 200
2023-03-05 11:46:57 - r - INFO: - Episode: 100/400, Reward: -241.865, Step: 200
2023-03-05 11:47:01 - r - INFO: - Episode: 101/400, Reward: -237.591, Step: 200
2023-03-05 11:47:03 - r - INFO: - Episode: 102/400, Reward: -2.152, Step: 200
2023-03-05 11:47:06 - r - INFO: - Episode: 103/400, Reward: -116.715, Step: 200
2023-03-05 11:47:09 - r - INFO: - Episode: 104/400, Reward: -234.190, Step: 200
2023-03-05 11:47:11 - r - INFO: - Episode: 105/400, Reward: -117.637, Step: 200
2023-03-05 11:47:15 - r - INFO: - Episode: 106/400, Reward: -232.310, Step: 200
2023-03-05 11:47:18 - r - INFO: - Episode: 107/400, Reward: -121.094, Step: 200
2023-03-05 11:47:20 - r - INFO: - Episode: 108/400, Reward: -223.660, Step: 200
2023-03-05 11:47:23 - r - INFO: - Episode: 109/400, Reward: -121.571, Step: 200
2023-03-05 11:47:25 - r - INFO: - Episode: 110/400, Reward: -1.335, Step: 200
2023-03-05 11:47:29 - r - INFO: - Episode: 111/400, Reward: -123.449, Step: 200
2023-03-05 11:47:32 - r - INFO: - Episode: 112/400, Reward: -1.152, Step: 200
2023-03-05 11:47:35 - r - INFO: - Episode: 113/400, Reward: -236.364, Step: 200
2023-03-05 11:47:37 - r - INFO: - Episode: 114/400, Reward: -2.099, Step: 200
2023-03-05 11:47:40 - r - INFO: - Episode: 115/400, Reward: -117.181, Step: 200
2023-03-05 11:47:44 - r - INFO: - Episode: 116/400, Reward: -2.308, Step: 200
2023-03-05 11:47:46 - r - INFO: - Episode: 117/400, Reward: -122.470, Step: 200
2023-03-05 11:47:49 - r - INFO: - Episode: 118/400, Reward: -269.947, Step: 200
2023-03-05 11:47:52 - r - INFO: - Episode: 119/400, Reward: -1.709, Step: 200
2023-03-05 11:47:54 - r - INFO: - Episode: 120/400, Reward: -123.374, Step: 200
2023-03-05 11:47:58 - r - INFO: - Episode: 121/400, Reward: -120.821, Step: 200
2023-03-05 11:48:01 - r - INFO: - Episode: 122/400, Reward: -1.854, Step: 200
2023-03-05 11:48:03 - r - INFO: - Episode: 123/400, Reward: -122.845, Step: 200
2023-03-05 11:48:06 - r - INFO: - Episode: 124/400, Reward: -1.950, Step: 200
2023-03-05 11:48:08 - r - INFO: - Episode: 125/400, Reward: -125.031, Step: 200
2023-03-05 11:48:12 - r - INFO: - Episode: 126/400, Reward: -119.509, Step: 200
2023-03-05 11:48:15 - r - INFO: - Episode: 127/400, Reward: -119.325, Step: 200
2023-03-05 11:48:17 - r - INFO: - Episode: 128/400, Reward: -119.516, Step: 200
2023-03-05 11:48:20 - r - INFO: - Episode: 129/400, Reward: -120.932, Step: 200
2023-03-05 11:48:22 - r - INFO: - Episode: 130/400, Reward: -125.258, Step: 200
2023-03-05 11:48:26 - r - INFO: - Episode: 131/400, Reward: -230.162, Step: 200
2023-03-05 11:48:29 - r - INFO: - Episode: 132/400, Reward: -343.086, Step: 200
2023-03-05 11:48:31 - r - INFO: - Episode: 133/400, Reward: -124.625, Step: 200
2023-03-05 11:48:34 - r - INFO: - Episode: 134/400, Reward: -117.051, Step: 200
2023-03-05 11:48:36 - r - INFO: - Episode: 135/400, Reward: -123.435, Step: 200
2023-03-05 11:48:40 - r - INFO: - Episode: 136/400, Reward: -228.665, Step: 200
2023-03-05 11:48:43 - r - INFO: - Episode: 137/400, Reward: -120.986, Step: 200
2023-03-05 11:48:45 - r - INFO: - Episode: 138/400, Reward: -123.391, Step: 200
2023-03-05 11:48:48 - r - INFO: - Episode: 139/400, Reward: -125.514, Step: 200
2023-03-05 11:48:50 - r - INFO: - Episode: 140/400, Reward: -123.420, Step: 200
2023-03-05 11:48:54 - r - INFO: - Episode: 141/400, Reward: -120.776, Step: 200
2023-03-05 11:48:57 - r - INFO: - Episode: 142/400, Reward: -127.112, Step: 200
2023-03-05 11:48:59 - r - INFO: - Episode: 143/400, Reward: -123.433, Step: 200
2023-03-05 11:49:02 - r - INFO: - Episode: 144/400, Reward: -226.254, Step: 200
2023-03-05 11:49:04 - r - INFO: - Episode: 145/400, Reward: -230.954, Step: 200
2023-03-05 11:49:08 - r - INFO: - Episode: 146/400, Reward: -128.639, Step: 200
2023-03-05 11:49:11 - r - INFO: - Episode: 147/400, Reward: -129.375, Step: 200
2023-03-05 11:49:13 - r - INFO: - Episode: 148/400, Reward: -129.193, Step: 200
2023-03-05 11:49:16 - r - INFO: - Episode: 149/400, Reward: -238.002, Step: 200
2023-03-05 11:49:18 - r - INFO: - Episode: 150/400, Reward: -234.694, Step: 200
2023-03-05 11:49:22 - r - INFO: - Episode: 151/400, Reward: -128.960, Step: 200
2023-03-05 11:49:25 - r - INFO: - Episode: 152/400, Reward: -322.122, Step: 200
2023-03-05 11:49:27 - r - INFO: - Episode: 153/400, Reward: -242.238, Step: 200
2023-03-05 11:49:30 - r - INFO: - Episode: 154/400, Reward: -10.029, Step: 200
2023-03-05 11:49:32 - r - INFO: - Episode: 155/400, Reward: -366.905, Step: 200
2023-03-05 11:49:37 - r - INFO: - Episode: 156/400, Reward: -230.781, Step: 200
2023-03-05 11:49:39 - r - INFO: - Episode: 157/400, Reward: -122.230, Step: 200
2023-03-05 11:49:41 - r - INFO: - Episode: 158/400, Reward: -121.343, Step: 200
2023-03-05 11:49:44 - r - INFO: - Episode: 159/400, Reward: -9.152, Step: 200
2023-03-05 11:49:47 - r - INFO: - Episode: 160/400, Reward: -130.778, Step: 200
2023-03-05 11:49:51 - r - INFO: - Episode: 161/400, Reward: -128.777, Step: 200
2023-03-05 11:49:53 - r - INFO: - Episode: 162/400, Reward: -327.974, Step: 200
2023-03-05 11:49:56 - r - INFO: - Episode: 163/400, Reward: -231.541, Step: 200
2023-03-05 11:49:58 - r - INFO: - Episode: 164/400, Reward: -118.463, Step: 200
2023-03-05 11:50:01 - r - INFO: - Episode: 165/400, Reward: -119.919, Step: 200
2023-03-05 11:50:05 - r - INFO: - Episode: 166/400, Reward: -227.345, Step: 200
2023-03-05 11:50:07 - r - INFO: - Episode: 167/400, Reward: -347.262, Step: 200
2023-03-05 11:50:10 - r - INFO: - Episode: 168/400, Reward: -117.324, Step: 200
2023-03-05 11:50:12 - r - INFO: - Episode: 169/400, Reward: -126.777, Step: 200
2023-03-05 11:50:15 - r - INFO: - Episode: 170/400, Reward: -222.402, Step: 200
2023-03-05 11:50:19 - r - INFO: - Episode: 171/400, Reward: -336.967, Step: 200
2023-03-05 11:50:21 - r - INFO: - Episode: 172/400, Reward: -4.295, Step: 200
2023-03-05 11:50:24 - r - INFO: - Episode: 173/400, Reward: -127.348, Step: 200
2023-03-05 11:50:26 - r - INFO: - Episode: 174/400, Reward: -230.956, Step: 200
2023-03-05 11:50:29 - r - INFO: - Episode: 175/400, Reward: -350.648, Step: 200
2023-03-05 11:50:33 - r - INFO: - Episode: 176/400, Reward: -129.078, Step: 200
2023-03-05 11:50:35 - r - INFO: - Episode: 177/400, Reward: -126.609, Step: 200
2023-03-05 11:50:38 - r - INFO: - Episode: 178/400, Reward: -120.766, Step: 200
2023-03-05 11:50:40 - r - INFO: - Episode: 179/400, Reward: -121.202, Step: 200
2023-03-05 11:50:43 - r - INFO: - Episode: 180/400, Reward: -125.929, Step: 200
2023-03-05 11:50:47 - r - INFO: - Episode: 181/400, Reward: -124.739, Step: 200
2023-03-05 11:50:49 - r - INFO: - Episode: 182/400, Reward: -245.240, Step: 200
2023-03-05 11:50:52 - r - INFO: - Episode: 183/400, Reward: -234.015, Step: 200
2023-03-05 11:50:55 - r - INFO: - Episode: 184/400, Reward: -351.502, Step: 200
2023-03-05 11:50:57 - r - INFO: - Episode: 185/400, Reward: -126.190, Step: 200
2023-03-05 11:51:01 - r - INFO: - Episode: 186/400, Reward: -119.752, Step: 200
2023-03-05 11:51:04 - r - INFO: - Episode: 187/400, Reward: -124.533, Step: 200
2023-03-05 11:51:06 - r - INFO: - Episode: 188/400, Reward: -121.198, Step: 200
2023-03-05 11:51:09 - r - INFO: - Episode: 189/400, Reward: -116.399, Step: 200
2023-03-05 11:51:11 - r - INFO: - Episode: 190/400, Reward: -237.938, Step: 200
2023-03-05 11:51:15 - r - INFO: - Episode: 191/400, Reward: -342.369, Step: 200
2023-03-05 11:51:18 - r - INFO: - Episode: 192/400, Reward: -225.576, Step: 200
2023-03-05 11:51:20 - r - INFO: - Episode: 193/400, Reward: -120.088, Step: 200
2023-03-05 11:51:23 - r - INFO: - Episode: 194/400, Reward: -126.699, Step: 200
2023-03-05 11:51:25 - r - INFO: - Episode: 195/400, Reward: -129.906, Step: 200
2023-03-05 11:51:29 - r - INFO: - Episode: 196/400, Reward: -121.470, Step: 200
2023-03-05 11:51:32 - r - INFO: - Episode: 197/400, Reward: -2.127, Step: 200
2023-03-05 11:51:35 - r - INFO: - Episode: 198/400, Reward: -122.947, Step: 200
2023-03-05 11:51:37 - r - INFO: - Episode: 199/400, Reward: -223.389, Step: 200
2023-03-05 11:51:40 - r - INFO: - Episode: 200/400, Reward: -131.037, Step: 200
2023-03-05 11:51:44 - r - INFO: - Episode: 201/400, Reward: -121.685, Step: 200
2023-03-05 11:51:46 - r - INFO: - Episode: 202/400, Reward: -123.564, Step: 200
2023-03-05 11:51:49 - r - INFO: - Episode: 203/400, Reward: -234.580, Step: 200
2023-03-05 11:51:51 - r - INFO: - Episode: 204/400, Reward: -123.657, Step: 200
2023-03-05 11:51:54 - r - INFO: - Episode: 205/400, Reward: -234.739, Step: 200
2023-03-05 11:51:58 - r - INFO: - Episode: 206/400, Reward: -119.959, Step: 200
2023-03-05 11:52:00 - r - INFO: - Episode: 207/400, Reward: -230.505, Step: 200
2023-03-05 11:52:03 - r - INFO: - Episode: 208/400, Reward: -232.542, Step: 200
2023-03-05 11:52:05 - r - INFO: - Episode: 209/400, Reward: -124.964, Step: 200
2023-03-05 11:52:08 - r - INFO: - Episode: 210/400, Reward: -231.458, Step: 200
2023-03-05 11:52:12 - r - INFO: - Episode: 211/400, Reward: -122.316, Step: 200
2023-03-05 11:52:14 - r - INFO: - Episode: 212/400, Reward: -2.820, Step: 200
2023-03-05 11:52:16 - r - INFO: - Episode: 213/400, Reward: -126.555, Step: 200
2023-03-05 11:52:19 - r - INFO: - Episode: 214/400, Reward: -121.396, Step: 200
2023-03-05 11:52:22 - r - INFO: - Episode: 215/400, Reward: -126.893, Step: 200
2023-03-05 11:52:26 - r - INFO: - Episode: 216/400, Reward: -123.645, Step: 200
2023-03-05 11:52:28 - r - INFO: - Episode: 217/400, Reward: -365.787, Step: 200
2023-03-05 11:52:31 - r - INFO: - Episode: 218/400, Reward: -125.840, Step: 200
2023-03-05 11:52:33 - r - INFO: - Episode: 219/400, Reward: -123.968, Step: 200
2023-03-05 11:52:36 - r - INFO: - Episode: 220/400, Reward: -126.313, Step: 200
2023-03-05 11:52:40 - r - INFO: - Episode: 221/400, Reward: -121.046, Step: 200
2023-03-05 11:52:42 - r - INFO: - Episode: 222/400, Reward: -232.492, Step: 200
2023-03-05 11:52:45 - r - INFO: - Episode: 223/400, Reward: -121.855, Step: 200
2023-03-05 11:52:47 - r - INFO: - Episode: 224/400, Reward: -1.313, Step: 200
2023-03-05 11:52:50 - r - INFO: - Episode: 225/400, Reward: -120.833, Step: 200
2023-03-05 11:52:54 - r - INFO: - Episode: 226/400, Reward: -121.842, Step: 200
2023-03-05 11:52:56 - r - INFO: - Episode: 227/400, Reward: -2.767, Step: 200
2023-03-05 11:52:59 - r - INFO: - Episode: 228/400, Reward: -122.450, Step: 200
2023-03-05 11:53:01 - r - INFO: - Episode: 229/400, Reward: -117.011, Step: 200
2023-03-05 11:53:04 - r - INFO: - Episode: 230/400, Reward: -121.282, Step: 200
2023-03-05 11:53:08 - r - INFO: - Episode: 231/400, Reward: -119.668, Step: 200
2023-03-05 11:53:10 - r - INFO: - Episode: 232/400, Reward: -1.676, Step: 200
2023-03-05 11:53:13 - r - INFO: - Episode: 233/400, Reward: -325.939, Step: 200
2023-03-05 11:53:15 - r - INFO: - Episode: 234/400, Reward: -118.521, Step: 200
2023-03-05 11:53:18 - r - INFO: - Episode: 235/400, Reward: -230.993, Step: 200
2023-03-05 11:53:22 - r - INFO: - Episode: 236/400, Reward: -126.674, Step: 200
2023-03-05 11:53:24 - r - INFO: - Episode: 237/400, Reward: -223.550, Step: 200
2023-03-05 11:53:27 - r - INFO: - Episode: 238/400, Reward: -254.288, Step: 200
2023-03-05 11:53:30 - r - INFO: - Episode: 239/400, Reward: -238.798, Step: 200
2023-03-05 11:53:32 - r - INFO: - Episode: 240/400, Reward: -119.634, Step: 200
2023-03-05 11:53:36 - r - INFO: - Episode: 241/400, Reward: -240.899, Step: 200
2023-03-05 11:53:39 - r - INFO: - Episode: 242/400, Reward: -231.824, Step: 200
2023-03-05 11:53:41 - r - INFO: - Episode: 243/400, Reward: -122.801, Step: 200
2023-03-05 11:53:44 - r - INFO: - Episode: 244/400, Reward: -1.265, Step: 200
2023-03-05 11:53:46 - r - INFO: - Episode: 245/400, Reward: -242.958, Step: 200
2023-03-05 11:53:50 - r - INFO: - Episode: 246/400, Reward: -232.867, Step: 200
2023-03-05 11:53:53 - r - INFO: - Episode: 247/400, Reward: -118.480, Step: 200
2023-03-05 11:53:55 - r - INFO: - Episode: 248/400, Reward: -114.560, Step: 200
2023-03-05 11:53:58 - r - INFO: - Episode: 249/400, Reward: -1.909, Step: 200
2023-03-05 11:54:00 - r - INFO: - Episode: 250/400, Reward: -117.991, Step: 200
2023-03-05 11:54:04 - r - INFO: - Episode: 251/400, Reward: -227.719, Step: 200
2023-03-05 11:54:07 - r - INFO: - Episode: 252/400, Reward: -121.283, Step: 200
2023-03-05 11:54:09 - r - INFO: - Episode: 253/400, Reward: -225.416, Step: 200
2023-03-05 11:54:12 - r - INFO: - Episode: 254/400, Reward: -124.251, Step: 200
2023-03-05 11:54:14 - r - INFO: - Episode: 255/400, Reward: -125.539, Step: 200
2023-03-05 11:54:18 - r - INFO: - Episode: 256/400, Reward: -123.485, Step: 200
2023-03-05 11:54:21 - r - INFO: - Episode: 257/400, Reward: -122.450, Step: 200
2023-03-05 11:54:23 - r - INFO: - Episode: 258/400, Reward: -120.159, Step: 200
2023-03-05 11:54:26 - r - INFO: - Episode: 259/400, Reward: -125.485, Step: 200
2023-03-05 11:54:28 - r - INFO: - Episode: 260/400, Reward: -122.159, Step: 200
2023-03-05 11:54:32 - r - INFO: - Episode: 261/400, Reward: -233.975, Step: 200
2023-03-05 11:54:34 - r - INFO: - Episode: 262/400, Reward: -119.895, Step: 200
2023-03-05 11:54:37 - r - INFO: - Episode: 263/400, Reward: -124.272, Step: 200
2023-03-05 11:54:39 - r - INFO: - Episode: 264/400, Reward: -125.546, Step: 200
2023-03-05 11:54:42 - r - INFO: - Episode: 265/400, Reward: -118.443, Step: 200
2023-03-05 11:54:46 - r - INFO: - Episode: 266/400, Reward: -119.949, Step: 200
2023-03-05 11:54:48 - r - INFO: - Episode: 267/400, Reward: -122.551, Step: 200
2023-03-05 11:54:51 - r - INFO: - Episode: 268/400, Reward: -2.975, Step: 200
2023-03-05 11:54:53 - r - INFO: - Episode: 269/400, Reward: -123.863, Step: 200
2023-03-05 11:54:56 - r - INFO: - Episode: 270/400, Reward: -121.591, Step: 200
2023-03-05 11:55:00 - r - INFO: - Episode: 271/400, Reward: -2.258, Step: 200
2023-03-05 11:55:02 - r - INFO: - Episode: 272/400, Reward: -363.247, Step: 200
2023-03-05 11:55:05 - r - INFO: - Episode: 273/400, Reward: -3.673, Step: 200
2023-03-05 11:55:07 - r - INFO: - Episode: 274/400, Reward: -124.923, Step: 200
2023-03-05 11:55:10 - r - INFO: - Episode: 275/400, Reward: -127.559, Step: 200
2023-03-05 11:55:14 - r - INFO: - Episode: 276/400, Reward: -117.881, Step: 200
2023-03-05 11:55:17 - r - INFO: - Episode: 277/400, Reward: -121.423, Step: 200
2023-03-05 11:55:19 - r - INFO: - Episode: 278/400, Reward: -126.849, Step: 200
2023-03-05 11:55:22 - r - INFO: - Episode: 279/400, Reward: -126.276, Step: 200
2023-03-05 11:55:24 - r - INFO: - Episode: 280/400, Reward: -2.250, Step: 200
2023-03-05 11:55:28 - r - INFO: - Episode: 281/400, Reward: -121.136, Step: 200
2023-03-05 11:55:31 - r - INFO: - Episode: 282/400, Reward: -224.689, Step: 200
2023-03-05 11:55:33 - r - INFO: - Episode: 283/400, Reward: -124.130, Step: 200
2023-03-05 11:55:35 - r - INFO: - Episode: 284/400, Reward: -120.099, Step: 200
2023-03-05 11:55:38 - r - INFO: - Episode: 285/400, Reward: -120.851, Step: 200
2023-03-05 11:55:42 - r - INFO: - Episode: 286/400, Reward: -241.842, Step: 200
2023-03-05 11:55:44 - r - INFO: - Episode: 287/400, Reward: -1.888, Step: 200
2023-03-05 11:55:47 - r - INFO: - Episode: 288/400, Reward: -232.046, Step: 200
2023-03-05 11:55:49 - r - INFO: - Episode: 289/400, Reward: -0.992, Step: 200
2023-03-05 11:55:52 - r - INFO: - Episode: 290/400, Reward: -242.567, Step: 200
2023-03-05 11:55:56 - r - INFO: - Episode: 291/400, Reward: -246.054, Step: 200
2023-03-05 11:55:58 - r - INFO: - Episode: 292/400, Reward: -122.682, Step: 200
2023-03-05 11:56:01 - r - INFO: - Episode: 293/400, Reward: -241.528, Step: 200
2023-03-05 11:56:04 - r - INFO: - Episode: 294/400, Reward: -227.902, Step: 200
2023-03-05 11:56:06 - r - INFO: - Episode: 295/400, Reward: -118.493, Step: 200
2023-03-05 11:56:10 - r - INFO: - Episode: 296/400, Reward: -122.008, Step: 200
2023-03-05 11:56:13 - r - INFO: - Episode: 297/400, Reward: -1.455, Step: 200
2023-03-05 11:56:15 - r - INFO: - Episode: 298/400, Reward: -0.942, Step: 200
2023-03-05 11:56:18 - r - INFO: - Episode: 299/400, Reward: -232.474, Step: 200
2023-03-05 11:56:20 - r - INFO: - Episode: 300/400, Reward: -118.554, Step: 200
2023-03-05 11:56:24 - r - INFO: - Episode: 301/400, Reward: -116.931, Step: 200
2023-03-05 11:56:27 - r - INFO: - Episode: 302/400, Reward: -233.801, Step: 200
2023-03-05 11:56:29 - r - INFO: - Episode: 303/400, Reward: -122.254, Step: 200
2023-03-05 11:56:32 - r - INFO: - Episode: 304/400, Reward: -235.866, Step: 200
2023-03-05 11:56:34 - r - INFO: - Episode: 305/400, Reward: -1.511, Step: 200
2023-03-05 11:56:38 - r - INFO: - Episode: 306/400, Reward: -119.010, Step: 200
2023-03-05 11:56:41 - r - INFO: - Episode: 307/400, Reward: -121.740, Step: 200
2023-03-05 11:56:43 - r - INFO: - Episode: 308/400, Reward: -122.898, Step: 200
2023-03-05 11:56:46 - r - INFO: - Episode: 309/400, Reward: -118.441, Step: 200
2023-03-05 11:56:48 - r - INFO: - Episode: 310/400, Reward: -120.226, Step: 200
2023-03-05 11:56:52 - r - INFO: - Episode: 311/400, Reward: -118.058, Step: 200
2023-03-05 11:56:54 - r - INFO: - Episode: 312/400, Reward: -120.073, Step: 200
2023-03-05 11:56:57 - r - INFO: - Episode: 313/400, Reward: -123.118, Step: 200
2023-03-05 11:57:00 - r - INFO: - Episode: 314/400, Reward: -307.235, Step: 200
2023-03-05 11:57:03 - r - INFO: - Episode: 315/400, Reward: -2.070, Step: 200
2023-03-05 11:57:06 - r - INFO: - Episode: 316/400, Reward: -0.628, Step: 200
2023-03-05 11:57:09 - r - INFO: - Episode: 317/400, Reward: -230.086, Step: 200
2023-03-05 11:57:12 - r - INFO: - Episode: 318/400, Reward: -1.442, Step: 200
2023-03-05 11:57:14 - r - INFO: - Episode: 319/400, Reward: -248.928, Step: 200
2023-03-05 11:57:17 - r - INFO: - Episode: 320/400, Reward: -246.539, Step: 200
2023-03-05 11:57:21 - r - INFO: - Episode: 321/400, Reward: -228.111, Step: 200
2023-03-05 11:57:23 - r - INFO: - Episode: 322/400, Reward: -254.951, Step: 200
2023-03-05 11:57:26 - r - INFO: - Episode: 323/400, Reward: -120.730, Step: 200
2023-03-05 11:57:28 - r - INFO: - Episode: 324/400, Reward: -123.069, Step: 200
2023-03-05 11:57:31 - r - INFO: - Episode: 325/400, Reward: -346.035, Step: 200
2023-03-05 11:57:35 - r - INFO: - Episode: 326/400, Reward: -118.853, Step: 200
2023-03-05 11:57:37 - r - INFO: - Episode: 327/400, Reward: -231.077, Step: 200
2023-03-05 11:57:40 - r - INFO: - Episode: 328/400, Reward: -126.975, Step: 200
2023-03-05 11:57:42 - r - INFO: - Episode: 329/400, Reward: -121.876, Step: 200
2023-03-05 11:57:45 - r - INFO: - Episode: 330/400, Reward: -238.283, Step: 200
2023-03-05 11:57:49 - r - INFO: - Episode: 331/400, Reward: -233.785, Step: 200
2023-03-05 11:57:52 - r - INFO: - Episode: 332/400, Reward: -239.539, Step: 200
2023-03-05 11:57:54 - r - INFO: - Episode: 333/400, Reward: -123.722, Step: 200
2023-03-05 11:57:57 - r - INFO: - Episode: 334/400, Reward: -244.016, Step: 200
2023-03-05 11:57:59 - r - INFO: - Episode: 335/400, Reward: -122.493, Step: 200
2023-03-05 11:58:03 - r - INFO: - Episode: 336/400, Reward: -132.925, Step: 200
2023-03-05 11:58:06 - r - INFO: - Episode: 337/400, Reward: -123.566, Step: 200
2023-03-05 11:58:09 - r - INFO: - Episode: 338/400, Reward: -228.249, Step: 200
2023-03-05 11:58:11 - r - INFO: - Episode: 339/400, Reward: -232.270, Step: 200
2023-03-05 11:58:14 - r - INFO: - Episode: 340/400, Reward: -1.657, Step: 200
2023-03-05 11:58:18 - r - INFO: - Episode: 341/400, Reward: -3.349, Step: 200
2023-03-05 11:58:21 - r - INFO: - Episode: 342/400, Reward: -227.909, Step: 200
2023-03-05 11:58:23 - r - INFO: - Episode: 343/400, Reward: -125.091, Step: 200
2023-03-05 11:58:26 - r - INFO: - Episode: 344/400, Reward: -119.591, Step: 200
2023-03-05 11:58:28 - r - INFO: - Episode: 345/400, Reward: -230.799, Step: 200
2023-03-05 11:58:32 - r - INFO: - Episode: 346/400, Reward: -4.066, Step: 200
2023-03-05 11:58:35 - r - INFO: - Episode: 347/400, Reward: -118.917, Step: 200
2023-03-05 11:58:37 - r - INFO: - Episode: 348/400, Reward: -231.116, Step: 200
2023-03-05 11:58:40 - r - INFO: - Episode: 349/400, Reward: -115.684, Step: 200
2023-03-05 11:58:42 - r - INFO: - Episode: 350/400, Reward: -123.263, Step: 200
2023-03-05 11:58:46 - r - INFO: - Episode: 351/400, Reward: -126.788, Step: 200
2023-03-05 11:58:49 - r - INFO: - Episode: 352/400, Reward: -122.307, Step: 200
2023-03-05 11:58:51 - r - INFO: - Episode: 353/400, Reward: -123.405, Step: 200
2023-03-05 11:58:54 - r - INFO: - Episode: 354/400, Reward: -126.424, Step: 200
2023-03-05 11:58:56 - r - INFO: - Episode: 355/400, Reward: -118.180, Step: 200
2023-03-05 11:59:00 - r - INFO: - Episode: 356/400, Reward: -125.828, Step: 200
2023-03-05 11:59:03 - r - INFO: - Episode: 357/400, Reward: -127.383, Step: 200
2023-03-05 11:59:05 - r - INFO: - Episode: 358/400, Reward: -124.455, Step: 200
2023-03-05 11:59:08 - r - INFO: - Episode: 359/400, Reward: -233.062, Step: 200
2023-03-05 11:59:10 - r - INFO: - Episode: 360/400, Reward: -118.345, Step: 200
2023-03-05 11:59:14 - r - INFO: - Episode: 361/400, Reward: -1.072, Step: 200
2023-03-05 11:59:17 - r - INFO: - Episode: 362/400, Reward: -127.160, Step: 200
2023-03-05 11:59:20 - r - INFO: - Episode: 363/400, Reward: -121.396, Step: 200
2023-03-05 11:59:22 - r - INFO: - Episode: 364/400, Reward: -115.136, Step: 200
2023-03-05 11:59:25 - r - INFO: - Episode: 365/400, Reward: -233.369, Step: 200
2023-03-05 11:59:29 - r - INFO: - Episode: 366/400, Reward: -124.808, Step: 200
2023-03-05 11:59:31 - r - INFO: - Episode: 367/400, Reward: -122.788, Step: 200
2023-03-05 11:59:34 - r - INFO: - Episode: 368/400, Reward: -123.264, Step: 200
2023-03-05 11:59:36 - r - INFO: - Episode: 369/400, Reward: -259.776, Step: 200
2023-03-05 11:59:39 - r - INFO: - Episode: 370/400, Reward: -227.237, Step: 200
2023-03-05 11:59:43 - r - INFO: - Episode: 371/400, Reward: -1.756, Step: 200
2023-03-05 11:59:45 - r - INFO: - Episode: 372/400, Reward: -119.952, Step: 200
2023-03-05 11:59:48 - r - INFO: - Episode: 373/400, Reward: -1.477, Step: 200
2023-03-05 11:59:50 - r - INFO: - Episode: 374/400, Reward: -128.200, Step: 200
2023-03-05 11:59:53 - r - INFO: - Episode: 375/400, Reward: -2.361, Step: 200
2023-03-05 11:59:57 - r - INFO: - Episode: 376/400, Reward: -2.999, Step: 200
2023-03-05 11:59:59 - r - INFO: - Episode: 377/400, Reward: -119.141, Step: 200
2023-03-05 12:00:02 - r - INFO: - Episode: 378/400, Reward: -228.394, Step: 200
2023-03-05 12:00:04 - r - INFO: - Episode: 379/400, Reward: -118.631, Step: 200
2023-03-05 12:00:07 - r - INFO: - Episode: 380/400, Reward: -118.296, Step: 200
2023-03-05 12:00:11 - r - INFO: - Episode: 381/400, Reward: -120.981, Step: 200
2023-03-05 12:00:13 - r - INFO: - Episode: 382/400, Reward: -242.309, Step: 200
2023-03-05 12:00:16 - r - INFO: - Episode: 383/400, Reward: -128.361, Step: 200
2023-03-05 12:00:18 - r - INFO: - Episode: 384/400, Reward: -243.632, Step: 200
2023-03-05 12:00:21 - r - INFO: - Episode: 385/400, Reward: -118.637, Step: 200
2023-03-05 12:00:25 - r - INFO: - Episode: 386/400, Reward: -330.532, Step: 200
2023-03-05 12:00:27 - r - INFO: - Episode: 387/400, Reward: -119.244, Step: 200
2023-03-05 12:00:30 - r - INFO: - Episode: 388/400, Reward: -118.364, Step: 200
2023-03-05 12:00:32 - r - INFO: - Episode: 389/400, Reward: -118.736, Step: 200
2023-03-05 12:00:35 - r - INFO: - Episode: 390/400, Reward: -347.443, Step: 200
2023-03-05 12:00:39 - r - INFO: - Episode: 391/400, Reward: -121.889, Step: 200
2023-03-05 12:00:42 - r - INFO: - Episode: 392/400, Reward: -123.754, Step: 200
2023-03-05 12:00:44 - r - INFO: - Episode: 393/400, Reward: -0.573, Step: 200
2023-03-05 12:00:47 - r - INFO: - Episode: 394/400, Reward: -229.850, Step: 200
2023-03-05 12:00:49 - r - INFO: - Episode: 395/400, Reward: -343.326, Step: 200
2023-03-05 12:00:53 - r - INFO: - Episode: 396/400, Reward: -227.782, Step: 200
2023-03-05 12:00:55 - r - INFO: - Episode: 397/400, Reward: -122.918, Step: 200
2023-03-05 12:00:58 - r - INFO: - Episode: 398/400, Reward: -119.106, Step: 200
2023-03-05 12:01:01 - r - INFO: - Episode: 399/400, Reward: -242.566, Step: 200
2023-03-05 12:01:03 - r - INFO: - Episode: 400/400, Reward: -364.784, Step: 200
2023-03-05 12:01:05 - r - INFO: - Finish training!