| 2023-05-27 00:28:59 - SimpleLog - INFO: - General Configs: | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - env_name gym <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - algo_name DDPG <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - mode train <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - device cuda <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - seed 10 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - max_episode 200 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - max_step 200 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - collect_traj 0 <class 'bool'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - mp_backend single <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - n_workers 4 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - n_learners 1 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - share_buffer 1 <class 'bool'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - online_eval 1 <class 'bool'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - online_eval_episode 20 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - model_save_fre 2000 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - load_path Train_single_CartPole-v1_DQN_20230515-211721 <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - load_model_step best <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - Algo Configs: | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - action_type dpg <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - buffer_size 8000 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - batch_size 128 <class 'int'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - gamma 0.99 <class 'float'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - policy_loss_weight 0.002 <class 'float'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - critic_lr 0.001 <class 'float'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - actor_lr 0.0001 <class 'float'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - tau 0.001 <class 'float'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - value_min -inf <class 'float'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - value_max inf <class 'float'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - actor_layers [{'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}] <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - critic_layers [{'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}] <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - Env Configs: | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - id Pendulum-v1 <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - render_mode None <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - wrapper None <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'> | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - obs_space: Box([-1. -1. -8.], [1. 1. 8.], (3,), float32), n_actions: Box(-2.0, 2.0, (1,), float32) | |
| 2023-05-27 00:28:59 - SimpleLog - INFO: - Start training! | |
| 2023-05-27 00:29:02 - SimpleLog - INFO: - episode: 0, ep_reward: -1661.201, ep_step: 200 | |
| 2023-05-27 00:29:04 - SimpleLog - INFO: - episode: 1, ep_reward: -1466.432, ep_step: 200 | |
| 2023-05-27 00:29:05 - SimpleLog - INFO: - episode: 2, ep_reward: -1506.946, ep_step: 200 | |
| 2023-05-27 00:29:07 - SimpleLog - INFO: - episode: 3, ep_reward: -1431.371, ep_step: 200 | |
| 2023-05-27 00:29:08 - SimpleLog - INFO: - episode: 4, ep_reward: -1450.385, ep_step: 200 | |
| 2023-05-27 00:29:10 - SimpleLog - INFO: - episode: 5, ep_reward: -1512.022, ep_step: 200 | |
| 2023-05-27 00:29:11 - SimpleLog - INFO: - episode: 6, ep_reward: -1225.477, ep_step: 200 | |
| 2023-05-27 00:29:13 - SimpleLog - INFO: - episode: 7, ep_reward: -1240.132, ep_step: 200 | |
| 2023-05-27 00:29:14 - SimpleLog - INFO: - episode: 8, ep_reward: -1578.078, ep_step: 200 | |
| 2023-05-27 00:29:16 - SimpleLog - INFO: - episode: 9, ep_reward: -1573.127, ep_step: 200 | |
| 2023-05-27 00:29:19 - SimpleLog - INFO: - update_step: 2000, online_eval_reward: -1512.958 | |
| 2023-05-27 00:29:19 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1512.958, save the best model! | |
| 2023-05-27 00:29:19 - SimpleLog - INFO: - episode: 10, ep_reward: -1694.954, ep_step: 200 | |
| 2023-05-27 00:29:21 - SimpleLog - INFO: - episode: 11, ep_reward: -1551.799, ep_step: 200 | |
| 2023-05-27 00:29:23 - SimpleLog - INFO: - episode: 12, ep_reward: -1573.322, ep_step: 200 | |
| 2023-05-27 00:29:24 - SimpleLog - INFO: - episode: 13, ep_reward: -1530.919, ep_step: 200 | |
| 2023-05-27 00:29:26 - SimpleLog - INFO: - episode: 14, ep_reward: -1577.197, ep_step: 200 | |
| 2023-05-27 00:29:28 - SimpleLog - INFO: - episode: 15, ep_reward: -1576.294, ep_step: 200 | |
| 2023-05-27 00:29:30 - SimpleLog - INFO: - episode: 16, ep_reward: -1519.319, ep_step: 200 | |
| 2023-05-27 00:29:31 - SimpleLog - INFO: - episode: 17, ep_reward: -1491.932, ep_step: 200 | |
| 2023-05-27 00:29:33 - SimpleLog - INFO: - episode: 18, ep_reward: -1440.715, ep_step: 200 | |
| 2023-05-27 00:29:34 - SimpleLog - INFO: - episode: 19, ep_reward: -1511.762, ep_step: 200 | |
| 2023-05-27 00:29:37 - SimpleLog - INFO: - update_step: 4000, online_eval_reward: -1492.909 | |
| 2023-05-27 00:29:37 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1492.909, save the best model! | |
| 2023-05-27 00:29:38 - SimpleLog - INFO: - episode: 20, ep_reward: -1555.738, ep_step: 200 | |
| 2023-05-27 00:29:39 - SimpleLog - INFO: - episode: 21, ep_reward: -1527.226, ep_step: 200 | |
| 2023-05-27 00:29:41 - SimpleLog - INFO: - episode: 22, ep_reward: -1264.275, ep_step: 200 | |
| 2023-05-27 00:29:43 - SimpleLog - INFO: - episode: 23, ep_reward: -1258.880, ep_step: 200 | |
| 2023-05-27 00:29:44 - SimpleLog - INFO: - episode: 24, ep_reward: -1512.860, ep_step: 200 | |
| 2023-05-27 00:29:46 - SimpleLog - INFO: - episode: 25, ep_reward: -1531.582, ep_step: 200 | |
| 2023-05-27 00:29:47 - SimpleLog - INFO: - episode: 26, ep_reward: -1530.531, ep_step: 200 | |
| 2023-05-27 00:29:49 - SimpleLog - INFO: - episode: 27, ep_reward: -1343.932, ep_step: 200 | |
| 2023-05-27 00:29:50 - SimpleLog - INFO: - episode: 28, ep_reward: -1537.344, ep_step: 200 | |
| 2023-05-27 00:29:52 - SimpleLog - INFO: - episode: 29, ep_reward: -1409.571, ep_step: 200 | |
| 2023-05-27 00:29:55 - SimpleLog - INFO: - update_step: 6000, online_eval_reward: -1492.954 | |
| 2023-05-27 00:29:56 - SimpleLog - INFO: - episode: 30, ep_reward: -1519.580, ep_step: 200 | |
| 2023-05-27 00:29:57 - SimpleLog - INFO: - episode: 31, ep_reward: -1388.753, ep_step: 200 | |
| 2023-05-27 00:29:59 - SimpleLog - INFO: - episode: 32, ep_reward: -1063.628, ep_step: 200 | |
| 2023-05-27 00:30:01 - SimpleLog - INFO: - episode: 33, ep_reward: -1168.097, ep_step: 200 | |
| 2023-05-27 00:30:02 - SimpleLog - INFO: - episode: 34, ep_reward: -1331.366, ep_step: 200 | |
| 2023-05-27 00:30:04 - SimpleLog - INFO: - episode: 35, ep_reward: -1293.768, ep_step: 200 | |
| 2023-05-27 00:30:05 - SimpleLog - INFO: - episode: 36, ep_reward: -1034.570, ep_step: 200 | |
| 2023-05-27 00:30:07 - SimpleLog - INFO: - episode: 37, ep_reward: -1553.501, ep_step: 200 | |
| 2023-05-27 00:30:09 - SimpleLog - INFO: - episode: 38, ep_reward: -1328.453, ep_step: 200 | |
| 2023-05-27 00:30:10 - SimpleLog - INFO: - episode: 39, ep_reward: -1194.988, ep_step: 200 | |
| 2023-05-27 00:30:13 - SimpleLog - INFO: - update_step: 8000, online_eval_reward: -1493.162 | |
| 2023-05-27 00:30:14 - SimpleLog - INFO: - episode: 40, ep_reward: -1460.393, ep_step: 200 | |
| 2023-05-27 00:30:15 - SimpleLog - INFO: - episode: 41, ep_reward: -914.100, ep_step: 200 | |
| 2023-05-27 00:30:17 - SimpleLog - INFO: - episode: 42, ep_reward: -808.302, ep_step: 200 | |
| 2023-05-27 00:30:18 - SimpleLog - INFO: - episode: 43, ep_reward: -955.122, ep_step: 200 | |
| 2023-05-27 00:30:20 - SimpleLog - INFO: - episode: 44, ep_reward: -565.088, ep_step: 200 | |
| 2023-05-27 00:30:21 - SimpleLog - INFO: - episode: 45, ep_reward: -656.654, ep_step: 200 | |
| 2023-05-27 00:30:23 - SimpleLog - INFO: - episode: 46, ep_reward: -948.915, ep_step: 200 | |
| 2023-05-27 00:30:25 - SimpleLog - INFO: - episode: 47, ep_reward: -1058.994, ep_step: 200 | |
| 2023-05-27 00:30:26 - SimpleLog - INFO: - episode: 48, ep_reward: -828.764, ep_step: 200 | |
| 2023-05-27 00:30:28 - SimpleLog - INFO: - episode: 49, ep_reward: -785.656, ep_step: 200 | |
| 2023-05-27 00:30:30 - SimpleLog - INFO: - update_step: 10000, online_eval_reward: -513.639 | |
| 2023-05-27 00:30:30 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -513.639, save the best model! | |
| 2023-05-27 00:30:31 - SimpleLog - INFO: - episode: 50, ep_reward: -509.437, ep_step: 200 | |
| 2023-05-27 00:30:33 - SimpleLog - INFO: - episode: 51, ep_reward: -617.535, ep_step: 200 | |
| 2023-05-27 00:30:34 - SimpleLog - INFO: - episode: 52, ep_reward: -511.001, ep_step: 200 | |
| 2023-05-27 00:30:36 - SimpleLog - INFO: - episode: 53, ep_reward: -726.522, ep_step: 200 | |
| 2023-05-27 00:30:38 - SimpleLog - INFO: - episode: 54, ep_reward: -638.109, ep_step: 200 | |
| 2023-05-27 00:30:39 - SimpleLog - INFO: - episode: 55, ep_reward: -765.023, ep_step: 200 | |
| 2023-05-27 00:30:41 - SimpleLog - INFO: - episode: 56, ep_reward: -515.006, ep_step: 200 | |
| 2023-05-27 00:30:42 - SimpleLog - INFO: - episode: 57, ep_reward: -518.109, ep_step: 200 | |
| 2023-05-27 00:30:44 - SimpleLog - INFO: - episode: 58, ep_reward: -498.920, ep_step: 200 | |
| 2023-05-27 00:30:46 - SimpleLog - INFO: - episode: 59, ep_reward: -742.844, ep_step: 200 | |
| 2023-05-27 00:30:49 - SimpleLog - INFO: - update_step: 12000, online_eval_reward: -361.109 | |
| 2023-05-27 00:30:49 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -361.109, save the best model! | |
| 2023-05-27 00:30:49 - SimpleLog - INFO: - episode: 60, ep_reward: -793.589, ep_step: 200 | |
| 2023-05-27 00:30:51 - SimpleLog - INFO: - episode: 61, ep_reward: -581.931, ep_step: 200 | |
| 2023-05-27 00:30:53 - SimpleLog - INFO: - episode: 62, ep_reward: -739.699, ep_step: 200 | |
| 2023-05-27 00:30:54 - SimpleLog - INFO: - episode: 63, ep_reward: -587.436, ep_step: 200 | |
| 2023-05-27 00:30:56 - SimpleLog - INFO: - episode: 64, ep_reward: -454.861, ep_step: 200 | |
| 2023-05-27 00:30:57 - SimpleLog - INFO: - episode: 65, ep_reward: -429.372, ep_step: 200 | |
| 2023-05-27 00:30:59 - SimpleLog - INFO: - episode: 66, ep_reward: -335.385, ep_step: 200 | |
| 2023-05-27 00:31:01 - SimpleLog - INFO: - episode: 67, ep_reward: -343.434, ep_step: 200 | |
| 2023-05-27 00:31:02 - SimpleLog - INFO: - episode: 68, ep_reward: -449.724, ep_step: 200 | |
| 2023-05-27 00:31:04 - SimpleLog - INFO: - episode: 69, ep_reward: -336.069, ep_step: 200 | |
| 2023-05-27 00:31:07 - SimpleLog - INFO: - update_step: 14000, online_eval_reward: -335.937 | |
| 2023-05-27 00:31:07 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -335.937, save the best model! | |
| 2023-05-27 00:31:08 - SimpleLog - INFO: - episode: 70, ep_reward: -334.573, ep_step: 200 | |
| 2023-05-27 00:31:10 - SimpleLog - INFO: - episode: 71, ep_reward: -455.101, ep_step: 200 | |
| 2023-05-27 00:31:11 - SimpleLog - INFO: - episode: 72, ep_reward: -450.986, ep_step: 200 | |
| 2023-05-27 00:31:13 - SimpleLog - INFO: - episode: 73, ep_reward: -325.188, ep_step: 200 | |
| 2023-05-27 00:31:14 - SimpleLog - INFO: - episode: 74, ep_reward: -335.417, ep_step: 200 | |
| 2023-05-27 00:31:16 - SimpleLog - INFO: - episode: 75, ep_reward: -448.173, ep_step: 200 | |
| 2023-05-27 00:31:18 - SimpleLog - INFO: - episode: 76, ep_reward: -317.437, ep_step: 200 | |
| 2023-05-27 00:31:20 - SimpleLog - INFO: - episode: 77, ep_reward: -303.536, ep_step: 200 | |
| 2023-05-27 00:31:21 - SimpleLog - INFO: - episode: 78, ep_reward: -325.795, ep_step: 200 | |
| 2023-05-27 00:31:23 - SimpleLog - INFO: - episode: 79, ep_reward: -315.077, ep_step: 200 | |
| 2023-05-27 00:31:26 - SimpleLog - INFO: - update_step: 16000, online_eval_reward: -323.018 | |
| 2023-05-27 00:31:26 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -323.018, save the best model! | |
| 2023-05-27 00:31:27 - SimpleLog - INFO: - episode: 80, ep_reward: -426.788, ep_step: 200 | |
| 2023-05-27 00:31:29 - SimpleLog - INFO: - episode: 81, ep_reward: -431.983, ep_step: 200 | |
| 2023-05-27 00:31:31 - SimpleLog - INFO: - episode: 82, ep_reward: -315.165, ep_step: 200 | |
| 2023-05-27 00:31:33 - SimpleLog - INFO: - episode: 83, ep_reward: -324.079, ep_step: 200 | |
| 2023-05-27 00:31:34 - SimpleLog - INFO: - episode: 84, ep_reward: -321.491, ep_step: 200 | |
| 2023-05-27 00:31:36 - SimpleLog - INFO: - episode: 85, ep_reward: -396.571, ep_step: 200 | |
| 2023-05-27 00:31:37 - SimpleLog - INFO: - episode: 86, ep_reward: -397.450, ep_step: 200 | |
| 2023-05-27 00:31:39 - SimpleLog - INFO: - episode: 87, ep_reward: -401.147, ep_step: 200 | |
| 2023-05-27 00:31:41 - SimpleLog - INFO: - episode: 88, ep_reward: -402.048, ep_step: 200 | |
| 2023-05-27 00:31:42 - SimpleLog - INFO: - episode: 89, ep_reward: -400.057, ep_step: 200 | |
| 2023-05-27 00:31:46 - SimpleLog - INFO: - update_step: 18000, online_eval_reward: -401.499 | |
| 2023-05-27 00:31:46 - SimpleLog - INFO: - episode: 90, ep_reward: -403.737, ep_step: 200 | |
| 2023-05-27 00:31:48 - SimpleLog - INFO: - episode: 91, ep_reward: -395.337, ep_step: 200 | |
| 2023-05-27 00:31:49 - SimpleLog - INFO: - episode: 92, ep_reward: -405.396, ep_step: 200 | |
| 2023-05-27 00:31:51 - SimpleLog - INFO: - episode: 93, ep_reward: -394.181, ep_step: 200 | |
| 2023-05-27 00:31:52 - SimpleLog - INFO: - episode: 94, ep_reward: -396.762, ep_step: 200 | |
| 2023-05-27 00:31:54 - SimpleLog - INFO: - episode: 95, ep_reward: -395.173, ep_step: 200 | |
| 2023-05-27 00:31:56 - SimpleLog - INFO: - episode: 96, ep_reward: -1220.214, ep_step: 200 | |
| 2023-05-27 00:31:57 - SimpleLog - INFO: - episode: 97, ep_reward: -391.593, ep_step: 200 | |
| 2023-05-27 00:31:59 - SimpleLog - INFO: - episode: 98, ep_reward: -408.688, ep_step: 200 | |
| 2023-05-27 00:32:01 - SimpleLog - INFO: - episode: 99, ep_reward: -392.954, ep_step: 200 | |
| 2023-05-27 00:32:04 - SimpleLog - INFO: - update_step: 20000, online_eval_reward: -265.208 | |
| 2023-05-27 00:32:04 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -265.208, save the best model! | |
| 2023-05-27 00:32:04 - SimpleLog - INFO: - episode: 100, ep_reward: -400.185, ep_step: 200 | |
| 2023-05-27 00:32:06 - SimpleLog - INFO: - episode: 101, ep_reward: -261.834, ep_step: 200 | |
| 2023-05-27 00:32:08 - SimpleLog - INFO: - episode: 102, ep_reward: -386.378, ep_step: 200 | |
| 2023-05-27 00:32:09 - SimpleLog - INFO: - episode: 103, ep_reward: -398.440, ep_step: 200 | |
| 2023-05-27 00:32:11 - SimpleLog - INFO: - episode: 104, ep_reward: -290.212, ep_step: 200 | |
| 2023-05-27 00:32:13 - SimpleLog - INFO: - episode: 105, ep_reward: -276.961, ep_step: 200 | |
| 2023-05-27 00:32:14 - SimpleLog - INFO: - episode: 106, ep_reward: -326.268, ep_step: 200 | |
| 2023-05-27 00:32:16 - SimpleLog - INFO: - episode: 107, ep_reward: -318.556, ep_step: 200 | |
| 2023-05-27 00:32:18 - SimpleLog - INFO: - episode: 108, ep_reward: -310.143, ep_step: 200 | |
| 2023-05-27 00:32:19 - SimpleLog - INFO: - episode: 109, ep_reward: -296.360, ep_step: 200 | |
| 2023-05-27 00:32:22 - SimpleLog - INFO: - update_step: 22000, online_eval_reward: -319.053 | |
| 2023-05-27 00:32:23 - SimpleLog - INFO: - episode: 110, ep_reward: -312.254, ep_step: 200 | |
| 2023-05-27 00:32:24 - SimpleLog - INFO: - episode: 111, ep_reward: -304.765, ep_step: 200 | |
| 2023-05-27 00:32:26 - SimpleLog - INFO: - episode: 112, ep_reward: -299.560, ep_step: 200 | |
| 2023-05-27 00:32:28 - SimpleLog - INFO: - episode: 113, ep_reward: -295.238, ep_step: 200 | |
| 2023-05-27 00:32:29 - SimpleLog - INFO: - episode: 114, ep_reward: -322.348, ep_step: 200 | |
| 2023-05-27 00:32:31 - SimpleLog - INFO: - episode: 115, ep_reward: -315.010, ep_step: 200 | |
| 2023-05-27 00:32:33 - SimpleLog - INFO: - episode: 116, ep_reward: -317.994, ep_step: 200 | |
| 2023-05-27 00:32:34 - SimpleLog - INFO: - episode: 117, ep_reward: -301.996, ep_step: 200 | |
| 2023-05-27 00:32:36 - SimpleLog - INFO: - episode: 118, ep_reward: -323.265, ep_step: 200 | |
| 2023-05-27 00:32:37 - SimpleLog - INFO: - episode: 119, ep_reward: -332.644, ep_step: 200 | |
| 2023-05-27 00:32:40 - SimpleLog - INFO: - update_step: 24000, online_eval_reward: -330.781 | |
| 2023-05-27 00:32:41 - SimpleLog - INFO: - episode: 120, ep_reward: -340.317, ep_step: 200 | |
| 2023-05-27 00:32:43 - SimpleLog - INFO: - episode: 121, ep_reward: -330.843, ep_step: 200 | |
| 2023-05-27 00:32:44 - SimpleLog - INFO: - episode: 122, ep_reward: -324.560, ep_step: 200 | |
| 2023-05-27 00:32:46 - SimpleLog - INFO: - episode: 123, ep_reward: -303.582, ep_step: 200 | |
| 2023-05-27 00:32:47 - SimpleLog - INFO: - episode: 124, ep_reward: -318.335, ep_step: 200 | |
| 2023-05-27 00:32:49 - SimpleLog - INFO: - episode: 125, ep_reward: -329.885, ep_step: 200 | |
| 2023-05-27 00:32:51 - SimpleLog - INFO: - episode: 126, ep_reward: -473.694, ep_step: 200 | |
| 2023-05-27 00:32:52 - SimpleLog - INFO: - episode: 127, ep_reward: -335.948, ep_step: 200 | |
| 2023-05-27 00:32:54 - SimpleLog - INFO: - episode: 128, ep_reward: -462.891, ep_step: 200 | |
| 2023-05-27 00:32:56 - SimpleLog - INFO: - episode: 129, ep_reward: -332.484, ep_step: 200 | |
| 2023-05-27 00:32:59 - SimpleLog - INFO: - update_step: 26000, online_eval_reward: -335.792 | |
| 2023-05-27 00:32:59 - SimpleLog - INFO: - episode: 130, ep_reward: -331.908, ep_step: 200 | |
| 2023-05-27 00:33:01 - SimpleLog - INFO: - episode: 131, ep_reward: -333.130, ep_step: 200 | |
| 2023-05-27 00:33:03 - SimpleLog - INFO: - episode: 132, ep_reward: -313.720, ep_step: 200 | |
| 2023-05-27 00:33:04 - SimpleLog - INFO: - episode: 133, ep_reward: -337.996, ep_step: 200 | |
| 2023-05-27 00:33:06 - SimpleLog - INFO: - episode: 134, ep_reward: -345.811, ep_step: 200 | |
| 2023-05-27 00:33:08 - SimpleLog - INFO: - episode: 135, ep_reward: -340.689, ep_step: 200 | |
| 2023-05-27 00:33:09 - SimpleLog - INFO: - episode: 136, ep_reward: -344.886, ep_step: 200 | |
| 2023-05-27 00:33:11 - SimpleLog - INFO: - episode: 137, ep_reward: -311.142, ep_step: 200 | |
| 2023-05-27 00:33:13 - SimpleLog - INFO: - episode: 138, ep_reward: -348.848, ep_step: 200 | |
| 2023-05-27 00:33:14 - SimpleLog - INFO: - episode: 139, ep_reward: -339.404, ep_step: 200 | |
| 2023-05-27 00:33:17 - SimpleLog - INFO: - update_step: 28000, online_eval_reward: -338.074 | |
| 2023-05-27 00:33:18 - SimpleLog - INFO: - episode: 140, ep_reward: -326.129, ep_step: 200 | |
| 2023-05-27 00:33:19 - SimpleLog - INFO: - episode: 141, ep_reward: -347.216, ep_step: 200 | |
| 2023-05-27 00:33:21 - SimpleLog - INFO: - episode: 142, ep_reward: -330.485, ep_step: 200 | |
| 2023-05-27 00:33:23 - SimpleLog - INFO: - episode: 143, ep_reward: -333.542, ep_step: 200 | |
| 2023-05-27 00:33:24 - SimpleLog - INFO: - episode: 144, ep_reward: -331.096, ep_step: 200 | |
| 2023-05-27 00:33:26 - SimpleLog - INFO: - episode: 145, ep_reward: -338.880, ep_step: 200 | |
| 2023-05-27 00:33:27 - SimpleLog - INFO: - episode: 146, ep_reward: -337.393, ep_step: 200 | |
| 2023-05-27 00:33:29 - SimpleLog - INFO: - episode: 147, ep_reward: -318.588, ep_step: 200 | |
| 2023-05-27 00:33:31 - SimpleLog - INFO: - episode: 148, ep_reward: -336.684, ep_step: 200 | |
| 2023-05-27 00:33:32 - SimpleLog - INFO: - episode: 149, ep_reward: -336.559, ep_step: 200 | |
| 2023-05-27 00:33:36 - SimpleLog - INFO: - update_step: 30000, online_eval_reward: -336.016 | |
| 2023-05-27 00:33:36 - SimpleLog - INFO: - episode: 150, ep_reward: -313.757, ep_step: 200 | |
| 2023-05-27 00:33:38 - SimpleLog - INFO: - episode: 151, ep_reward: -344.355, ep_step: 200 | |
| 2023-05-27 00:33:39 - SimpleLog - INFO: - episode: 152, ep_reward: -323.297, ep_step: 200 | |
| 2023-05-27 00:33:41 - SimpleLog - INFO: - episode: 153, ep_reward: -327.646, ep_step: 200 | |
| 2023-05-27 00:33:42 - SimpleLog - INFO: - episode: 154, ep_reward: -335.975, ep_step: 200 | |
| 2023-05-27 00:33:44 - SimpleLog - INFO: - episode: 155, ep_reward: -318.358, ep_step: 200 | |
| 2023-05-27 00:33:45 - SimpleLog - INFO: - episode: 156, ep_reward: -324.404, ep_step: 200 | |
| 2023-05-27 00:33:47 - SimpleLog - INFO: - episode: 157, ep_reward: -325.595, ep_step: 200 | |
| 2023-05-27 00:33:48 - SimpleLog - INFO: - episode: 158, ep_reward: -336.706, ep_step: 200 | |
| 2023-05-27 00:33:50 - SimpleLog - INFO: - episode: 159, ep_reward: -324.430, ep_step: 200 | |
| 2023-05-27 00:33:53 - SimpleLog - INFO: - update_step: 32000, online_eval_reward: -330.636 | |
| 2023-05-27 00:33:54 - SimpleLog - INFO: - episode: 160, ep_reward: -322.437, ep_step: 200 | |
| 2023-05-27 00:33:55 - SimpleLog - INFO: - episode: 161, ep_reward: -336.050, ep_step: 200 | |
| 2023-05-27 00:33:57 - SimpleLog - INFO: - episode: 162, ep_reward: -331.872, ep_step: 200 | |
| 2023-05-27 00:33:59 - SimpleLog - INFO: - episode: 163, ep_reward: -321.575, ep_step: 200 | |
| 2023-05-27 00:34:00 - SimpleLog - INFO: - episode: 164, ep_reward: -296.710, ep_step: 200 | |
| 2023-05-27 00:34:02 - SimpleLog - INFO: - episode: 165, ep_reward: -326.305, ep_step: 200 | |
| 2023-05-27 00:34:03 - SimpleLog - INFO: - episode: 166, ep_reward: -315.655, ep_step: 200 | |
| 2023-05-27 00:34:05 - SimpleLog - INFO: - episode: 167, ep_reward: -327.117, ep_step: 200 | |
| 2023-05-27 00:34:07 - SimpleLog - INFO: - episode: 168, ep_reward: -315.649, ep_step: 200 | |
| 2023-05-27 00:34:08 - SimpleLog - INFO: - episode: 169, ep_reward: -325.160, ep_step: 200 | |
| 2023-05-27 00:34:11 - SimpleLog - INFO: - update_step: 34000, online_eval_reward: -325.015 | |
| 2023-05-27 00:34:12 - SimpleLog - INFO: - episode: 170, ep_reward: -328.977, ep_step: 200 | |
| 2023-05-27 00:34:13 - SimpleLog - INFO: - episode: 171, ep_reward: -322.324, ep_step: 200 | |
| 2023-05-27 00:34:15 - SimpleLog - INFO: - episode: 172, ep_reward: -294.012, ep_step: 200 | |
| 2023-05-27 00:34:16 - SimpleLog - INFO: - episode: 173, ep_reward: -319.424, ep_step: 200 | |
| 2023-05-27 00:34:18 - SimpleLog - INFO: - episode: 174, ep_reward: -304.872, ep_step: 200 | |
| 2023-05-27 00:34:19 - SimpleLog - INFO: - episode: 175, ep_reward: -330.653, ep_step: 200 | |
| 2023-05-27 00:34:21 - SimpleLog - INFO: - episode: 176, ep_reward: -330.855, ep_step: 200 | |
| 2023-05-27 00:34:23 - SimpleLog - INFO: - episode: 177, ep_reward: -320.936, ep_step: 200 | |
| 2023-05-27 00:34:24 - SimpleLog - INFO: - episode: 178, ep_reward: -293.672, ep_step: 200 | |
| 2023-05-27 00:34:26 - SimpleLog - INFO: - episode: 179, ep_reward: -305.842, ep_step: 200 | |
| 2023-05-27 00:34:29 - SimpleLog - INFO: - update_step: 36000, online_eval_reward: -316.303 | |
| 2023-05-27 00:34:29 - SimpleLog - INFO: - episode: 180, ep_reward: -304.273, ep_step: 200 | |
| 2023-05-27 00:34:31 - SimpleLog - INFO: - episode: 181, ep_reward: -319.363, ep_step: 200 | |
| 2023-05-27 00:34:33 - SimpleLog - INFO: - episode: 182, ep_reward: -309.749, ep_step: 200 | |
| 2023-05-27 00:34:34 - SimpleLog - INFO: - episode: 183, ep_reward: -295.360, ep_step: 200 | |
| 2023-05-27 00:34:36 - SimpleLog - INFO: - episode: 184, ep_reward: -260.432, ep_step: 200 | |
| 2023-05-27 00:34:38 - SimpleLog - INFO: - episode: 185, ep_reward: -299.548, ep_step: 200 | |
| 2023-05-27 00:34:39 - SimpleLog - INFO: - episode: 186, ep_reward: -306.128, ep_step: 200 | |
| 2023-05-27 00:34:41 - SimpleLog - INFO: - episode: 187, ep_reward: -1411.605, ep_step: 200 | |
| 2023-05-27 00:34:42 - SimpleLog - INFO: - episode: 188, ep_reward: -1556.318, ep_step: 200 | |
| 2023-05-27 00:34:44 - SimpleLog - INFO: - episode: 189, ep_reward: -1544.945, ep_step: 200 | |
| 2023-05-27 00:34:47 - SimpleLog - INFO: - update_step: 38000, online_eval_reward: -304.749 | |
| 2023-05-27 00:34:47 - SimpleLog - INFO: - episode: 190, ep_reward: -383.180, ep_step: 200 | |
| 2023-05-27 00:34:49 - SimpleLog - INFO: - episode: 191, ep_reward: -1478.870, ep_step: 200 | |
| 2023-05-27 00:34:51 - SimpleLog - INFO: - episode: 192, ep_reward: -278.173, ep_step: 200 | |
| 2023-05-27 00:34:52 - SimpleLog - INFO: - episode: 193, ep_reward: -274.785, ep_step: 200 | |
| 2023-05-27 00:34:54 - SimpleLog - INFO: - episode: 194, ep_reward: -258.812, ep_step: 200 | |
| 2023-05-27 00:34:56 - SimpleLog - INFO: - episode: 195, ep_reward: -302.596, ep_step: 200 | |
| 2023-05-27 00:34:57 - SimpleLog - INFO: - episode: 196, ep_reward: -300.993, ep_step: 200 | |
| 2023-05-27 00:34:59 - SimpleLog - INFO: - episode: 197, ep_reward: -298.122, ep_step: 200 | |
| 2023-05-27 00:35:00 - SimpleLog - INFO: - episode: 198, ep_reward: -292.155, ep_step: 200 | |
| 2023-05-27 00:35:02 - SimpleLog - INFO: - episode: 199, ep_reward: -278.364, ep_step: 200 | |
| 2023-05-27 00:35:02 - SimpleLog - INFO: - Finish training! total time consumed: 362.67s | |