| 2023-05-27 00:17:15 - SimpleLog - INFO: - General Configs: | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - env_name gym <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - algo_name DDPG <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - mode train <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - device cuda <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - seed 10 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - max_episode 200 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - max_step 200 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - collect_traj 0 <class 'bool'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - mp_backend ray <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - n_workers 4 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - n_learners 1 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - share_buffer 1 <class 'bool'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - online_eval 1 <class 'bool'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - online_eval_episode 20 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - model_save_fre 2000 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - load_path Train_single_CartPole-v1_DQN_20230515-211721 <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - load_model_step best <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - Algo Configs: | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - action_type dpg <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - buffer_size 8000 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - batch_size 128 <class 'int'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - gamma 0.99 <class 'float'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - policy_loss_weight 0.002 <class 'float'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - critic_lr 0.001 <class 'float'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - actor_lr 0.0001 <class 'float'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - tau 0.001 <class 'float'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - value_min -inf <class 'float'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - value_max inf <class 'float'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - actor_layers [{'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}] <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - critic_layers [{'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}] <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - Env Configs: | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - id Pendulum-v1 <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - render_mode None <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - wrapper None <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'> | |
| 2023-05-27 00:17:15 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-27 00:17:20 - SimpleLog - INFO: - obs_space: Box([-1. -1. -8.], [1. 1. 8.], (3,), float32), n_actions: Box(-2.0, 2.0, (1,), float32) | |
| 2023-05-27 00:17:36 - RayLog - INFO: - Worker 2 finished episode 0 with reward -1451.948 in 200 steps | |
| 2023-05-27 00:17:36 - RayLog - INFO: - Worker 1 finished episode 0 with reward -1426.784 in 200 steps | |
| 2023-05-27 00:17:36 - RayLog - INFO: - Worker 0 finished episode 0 with reward -1551.199 in 200 steps | |
| 2023-05-27 00:17:36 - RayLog - INFO: - Worker 3 finished episode 0 with reward -1445.478 in 200 steps | |
| 2023-05-27 00:17:47 - RayLog - INFO: - Worker 2 finished episode 1 with reward -1166.916 in 200 steps | |
| 2023-05-27 00:17:47 - RayLog - INFO: - Worker 1 finished episode 3 with reward -1593.319 in 200 steps | |
| 2023-05-27 00:17:47 - RayLog - INFO: - Worker 0 finished episode 3 with reward -1603.294 in 200 steps | |
| 2023-05-27 00:17:47 - RayLog - INFO: - Worker 3 finished episode 4 with reward -1605.817 in 200 steps | |
| 2023-05-27 00:17:58 - RayLog - INFO: - learner id: 0, update_step: 2000, online_eval_reward: -1548.226 | |
| 2023-05-27 00:17:58 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -1548.226, save the best model! | |
| 2023-05-27 00:18:02 - RayLog - INFO: - Worker 2 finished episode 5 with reward -1538.540 in 200 steps | |
| 2023-05-27 00:18:02 - RayLog - INFO: - Worker 1 finished episode 6 with reward -1569.025 in 200 steps | |
| 2023-05-27 00:18:02 - RayLog - INFO: - Worker 0 finished episode 7 with reward -1578.207 in 200 steps | |
| 2023-05-27 00:18:02 - RayLog - INFO: - Worker 3 finished episode 8 with reward -1565.720 in 200 steps | |
| 2023-05-27 00:18:13 - RayLog - INFO: - Worker 2 finished episode 9 with reward -1543.753 in 200 steps | |
| 2023-05-27 00:18:13 - RayLog - INFO: - Worker 1 finished episode 10 with reward -1593.543 in 200 steps | |
| 2023-05-27 00:18:13 - RayLog - INFO: - Worker 0 finished episode 11 with reward -1599.652 in 200 steps | |
| 2023-05-27 00:18:13 - RayLog - INFO: - Worker 3 finished episode 12 with reward -1576.990 in 200 steps | |
| 2023-05-27 00:18:24 - RayLog - INFO: - Worker 2 finished episode 13 with reward -1494.538 in 200 steps | |
| 2023-05-27 00:18:24 - RayLog - INFO: - Worker 1 finished episode 14 with reward -1561.907 in 200 steps | |
| 2023-05-27 00:18:24 - RayLog - INFO: - Worker 0 finished episode 15 with reward -1576.500 in 200 steps | |
| 2023-05-27 00:18:24 - RayLog - INFO: - Worker 3 finished episode 16 with reward -1562.714 in 200 steps | |
| 2023-05-27 00:18:28 - RayLog - INFO: - learner id: 0, update_step: 4000, online_eval_reward: -1492.934 | |
| 2023-05-27 00:18:28 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -1492.934, save the best model! | |
| 2023-05-27 00:18:38 - RayLog - INFO: - Worker 2 finished episode 17 with reward -1441.521 in 200 steps | |
| 2023-05-27 00:18:38 - RayLog - INFO: - Worker 1 finished episode 18 with reward -1524.784 in 200 steps | |
| 2023-05-27 00:18:38 - RayLog - INFO: - Worker 0 finished episode 19 with reward -1460.997 in 200 steps | |
| 2023-05-27 00:18:38 - RayLog - INFO: - Worker 3 finished episode 20 with reward -1504.649 in 200 steps | |
| 2023-05-27 00:18:49 - RayLog - INFO: - Worker 2 finished episode 21 with reward -1159.768 in 200 steps | |
| 2023-05-27 00:18:49 - RayLog - INFO: - Worker 0 finished episode 23 with reward -1547.452 in 200 steps | |
| 2023-05-27 00:18:49 - RayLog - INFO: - Worker 1 finished episode 22 with reward -1245.359 in 200 steps | |
| 2023-05-27 00:18:49 - RayLog - INFO: - Worker 3 finished episode 24 with reward -1316.269 in 200 steps | |
| 2023-05-27 00:18:58 - RayLog - INFO: - learner id: 0, update_step: 6000, online_eval_reward: -1493.150 | |
| 2023-05-27 00:19:02 - RayLog - INFO: - Worker 2 finished episode 25 with reward -1123.863 in 200 steps | |
| 2023-05-27 00:19:02 - RayLog - INFO: - Worker 0 finished episode 26 with reward -1249.138 in 200 steps | |
| 2023-05-27 00:19:02 - RayLog - INFO: - Worker 1 finished episode 27 with reward -1208.721 in 200 steps | |
| 2023-05-27 00:19:02 - RayLog - INFO: - Worker 3 finished episode 28 with reward -1142.612 in 200 steps | |
| 2023-05-27 00:19:15 - RayLog - INFO: - Worker 2 finished episode 29 with reward -997.769 in 200 steps | |
| 2023-05-27 00:19:15 - RayLog - INFO: - Worker 0 finished episode 31 with reward -1118.224 in 200 steps | |
| 2023-05-27 00:19:15 - RayLog - INFO: - Worker 1 finished episode 31 with reward -1066.523 in 200 steps | |
| 2023-05-27 00:19:15 - RayLog - INFO: - Worker 3 finished episode 32 with reward -1068.746 in 200 steps | |
| 2023-05-27 00:19:28 - RayLog - INFO: - Worker 2 finished episode 33 with reward -937.183 in 200 steps | |
| 2023-05-27 00:19:28 - RayLog - INFO: - Worker 0 finished episode 34 with reward -1376.815 in 200 steps | |
| 2023-05-27 00:19:28 - RayLog - INFO: - Worker 1 finished episode 35 with reward -966.611 in 200 steps | |
| 2023-05-27 00:19:28 - RayLog - INFO: - Worker 3 finished episode 36 with reward -1073.693 in 200 steps | |
| 2023-05-27 00:19:32 - RayLog - INFO: - learner id: 0, update_step: 8000, online_eval_reward: -1493.374 | |
| 2023-05-27 00:19:42 - RayLog - INFO: - Worker 2 finished episode 37 with reward -878.047 in 200 steps | |
| 2023-05-27 00:19:42 - RayLog - INFO: - Worker 0 finished episode 38 with reward -976.531 in 200 steps | |
| 2023-05-27 00:19:42 - RayLog - INFO: - Worker 1 finished episode 39 with reward -956.583 in 200 steps | |
| 2023-05-27 00:19:42 - RayLog - INFO: - Worker 3 finished episode 40 with reward -805.417 in 200 steps | |
| 2023-05-27 00:19:54 - RayLog - INFO: - Worker 2 finished episode 42 with reward -532.475 in 200 steps | |
| 2023-05-27 00:19:54 - RayLog - INFO: - Worker 0 finished episode 42 with reward -1061.521 in 200 steps | |
| 2023-05-27 00:19:54 - RayLog - INFO: - Worker 1 finished episode 43 with reward -553.406 in 200 steps | |
| 2023-05-27 00:19:54 - RayLog - INFO: - Worker 3 finished episode 44 with reward -668.325 in 200 steps | |
| 2023-05-27 00:20:04 - RayLog - INFO: - learner id: 0, update_step: 10000, online_eval_reward: -536.765 | |
| 2023-05-27 00:20:04 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -536.765, save the best model! | |
| 2023-05-27 00:20:08 - RayLog - INFO: - Worker 2 finished episode 45 with reward -410.737 in 200 steps | |
| 2023-05-27 00:20:08 - RayLog - INFO: - Worker 0 finished episode 46 with reward -1107.134 in 200 steps | |
| 2023-05-27 00:20:08 - RayLog - INFO: - Worker 1 finished episode 47 with reward -571.280 in 200 steps | |
| 2023-05-27 00:20:08 - RayLog - INFO: - Worker 3 finished episode 48 with reward -464.369 in 200 steps | |
| 2023-05-27 00:20:20 - RayLog - INFO: - Worker 2 finished episode 49 with reward -534.199 in 200 steps | |
| 2023-05-27 00:20:20 - RayLog - INFO: - Worker 0 finished episode 50 with reward -539.787 in 200 steps | |
| 2023-05-27 00:20:21 - RayLog - INFO: - Worker 1 finished episode 51 with reward -536.010 in 200 steps | |
| 2023-05-27 00:20:21 - RayLog - INFO: - Worker 3 finished episode 52 with reward -390.752 in 200 steps | |
| 2023-05-27 00:20:33 - RayLog - INFO: - Worker 2 finished episode 53 with reward -273.564 in 200 steps | |
| 2023-05-27 00:20:33 - RayLog - INFO: - Worker 0 finished episode 54 with reward -552.384 in 200 steps | |
| 2023-05-27 00:20:33 - RayLog - INFO: - Worker 1 finished episode 55 with reward -275.812 in 200 steps | |
| 2023-05-27 00:20:33 - RayLog - INFO: - Worker 3 finished episode 56 with reward -391.565 in 200 steps | |
| 2023-05-27 00:20:37 - RayLog - INFO: - learner id: 0, update_step: 12000, online_eval_reward: -374.204 | |
| 2023-05-27 00:20:37 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -374.204, save the best model! | |
| 2023-05-27 00:20:47 - RayLog - INFO: - Worker 2 finished episode 57 with reward -141.707 in 200 steps | |
| 2023-05-27 00:20:47 - RayLog - INFO: - Worker 0 finished episode 58 with reward -372.130 in 200 steps | |
| 2023-05-27 00:20:47 - RayLog - INFO: - Worker 1 finished episode 59 with reward -268.024 in 200 steps | |
| 2023-05-27 00:20:47 - RayLog - INFO: - Worker 3 finished episode 60 with reward -252.713 in 200 steps | |
| 2023-05-27 00:20:59 - RayLog - INFO: - Worker 2 finished episode 61 with reward -137.422 in 200 steps | |
| 2023-05-27 00:20:59 - RayLog - INFO: - Worker 0 finished episode 62 with reward -362.890 in 200 steps | |
| 2023-05-27 00:20:59 - RayLog - INFO: - Worker 1 finished episode 63 with reward -260.108 in 200 steps | |
| 2023-05-27 00:20:59 - RayLog - INFO: - Worker 3 finished episode 64 with reward -249.668 in 200 steps | |
| 2023-05-27 00:21:09 - RayLog - INFO: - learner id: 0, update_step: 14000, online_eval_reward: -351.751 | |
| 2023-05-27 00:21:09 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -351.751, save the best model! | |
| 2023-05-27 00:21:13 - RayLog - INFO: - Worker 2 finished episode 65 with reward -136.902 in 200 steps | |
| 2023-05-27 00:21:13 - RayLog - INFO: - Worker 0 finished episode 66 with reward -363.455 in 200 steps | |
| 2023-05-27 00:21:13 - RayLog - INFO: - Worker 1 finished episode 67 with reward -255.716 in 200 steps | |
| 2023-05-27 00:21:13 - RayLog - INFO: - Worker 3 finished episode 68 with reward -245.107 in 200 steps | |
| 2023-05-27 00:21:26 - RayLog - INFO: - Worker 2 finished episode 69 with reward -133.383 in 200 steps | |
| 2023-05-27 00:21:26 - RayLog - INFO: - Worker 0 finished episode 70 with reward -346.740 in 200 steps | |
| 2023-05-27 00:21:26 - RayLog - INFO: - Worker 1 finished episode 71 with reward -257.755 in 200 steps | |
| 2023-05-27 00:21:26 - RayLog - INFO: - Worker 3 finished episode 72 with reward -242.626 in 200 steps | |
| 2023-05-27 00:21:38 - RayLog - INFO: - Worker 2 finished episode 73 with reward -132.665 in 200 steps | |
| 2023-05-27 00:21:38 - RayLog - INFO: - Worker 0 finished episode 74 with reward -460.159 in 200 steps | |
| 2023-05-27 00:21:38 - RayLog - INFO: - Worker 1 finished episode 75 with reward -260.655 in 200 steps | |
| 2023-05-27 00:21:38 - RayLog - INFO: - Worker 3 finished episode 76 with reward -242.048 in 200 steps | |
| 2023-05-27 00:21:42 - RayLog - INFO: - learner id: 0, update_step: 16000, online_eval_reward: -261.486 | |
| 2023-05-27 00:21:42 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -261.486, save the best model! | |
| 2023-05-27 00:21:52 - RayLog - INFO: - Worker 2 finished episode 77 with reward -130.662 in 200 steps | |
| 2023-05-27 00:21:52 - RayLog - INFO: - Worker 0 finished episode 78 with reward -392.058 in 200 steps | |
| 2023-05-27 00:21:52 - RayLog - INFO: - Worker 1 finished episode 79 with reward -258.029 in 200 steps | |
| 2023-05-27 00:21:52 - RayLog - INFO: - Worker 3 finished episode 80 with reward -240.002 in 200 steps | |
| 2023-05-27 00:22:05 - RayLog - INFO: - Worker 2 finished episode 81 with reward -128.995 in 200 steps | |
| 2023-05-27 00:22:05 - RayLog - INFO: - Worker 0 finished episode 82 with reward -390.709 in 200 steps | |
| 2023-05-27 00:22:05 - RayLog - INFO: - Worker 1 finished episode 83 with reward -256.941 in 200 steps | |
| 2023-05-27 00:22:05 - RayLog - INFO: - Worker 3 finished episode 84 with reward -241.232 in 200 steps | |
| 2023-05-27 00:22:16 - RayLog - INFO: - learner id: 0, update_step: 18000, online_eval_reward: -261.173 | |
| 2023-05-27 00:22:16 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -261.173, save the best model! | |
| 2023-05-27 00:22:20 - RayLog - INFO: - Worker 2 finished episode 85 with reward -129.842 in 200 steps | |
| 2023-05-27 00:22:20 - RayLog - INFO: - Worker 0 finished episode 86 with reward -385.724 in 200 steps | |
| 2023-05-27 00:22:20 - RayLog - INFO: - Worker 1 finished episode 87 with reward -259.445 in 200 steps | |
| 2023-05-27 00:22:20 - RayLog - INFO: - Worker 3 finished episode 88 with reward -239.046 in 200 steps | |
| 2023-05-27 00:22:33 - RayLog - INFO: - Worker 2 finished episode 89 with reward -129.676 in 200 steps | |
| 2023-05-27 00:22:33 - RayLog - INFO: - Worker 0 finished episode 90 with reward -384.466 in 200 steps | |
| 2023-05-27 00:22:33 - RayLog - INFO: - Worker 1 finished episode 91 with reward -254.073 in 200 steps | |
| 2023-05-27 00:22:33 - RayLog - INFO: - Worker 3 finished episode 92 with reward -236.958 in 200 steps | |
| 2023-05-27 00:22:45 - RayLog - INFO: - Worker 2 finished episode 93 with reward -128.925 in 200 steps | |
| 2023-05-27 00:22:45 - RayLog - INFO: - Worker 0 finished episode 94 with reward -389.155 in 200 steps | |
| 2023-05-27 00:22:45 - RayLog - INFO: - Worker 1 finished episode 95 with reward -248.248 in 200 steps | |
| 2023-05-27 00:22:45 - RayLog - INFO: - Worker 3 finished episode 96 with reward -240.203 in 200 steps | |
| 2023-05-27 00:22:50 - RayLog - INFO: - learner id: 0, update_step: 20000, online_eval_reward: -260.301 | |
| 2023-05-27 00:22:50 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -260.301, save the best model! | |
| 2023-05-27 00:23:00 - RayLog - INFO: - Worker 2 finished episode 97 with reward -130.815 in 200 steps | |
| 2023-05-27 00:23:00 - RayLog - INFO: - Worker 0 finished episode 98 with reward -264.699 in 200 steps | |
| 2023-05-27 00:23:01 - RayLog - INFO: - Worker 1 finished episode 99 with reward -252.631 in 200 steps | |
| 2023-05-27 00:23:01 - RayLog - INFO: - Worker 3 finished episode 100 with reward -241.614 in 200 steps | |
| 2023-05-27 00:23:13 - RayLog - INFO: - Worker 2 finished episode 101 with reward -127.897 in 200 steps | |
| 2023-05-27 00:23:13 - RayLog - INFO: - Worker 0 finished episode 102 with reward -382.318 in 200 steps | |
| 2023-05-27 00:23:13 - RayLog - INFO: - Worker 1 finished episode 103 with reward -249.393 in 200 steps | |
| 2023-05-27 00:23:13 - RayLog - INFO: - Worker 3 finished episode 104 with reward -236.064 in 200 steps | |
| 2023-05-27 00:23:24 - RayLog - INFO: - learner id: 0, update_step: 22000, online_eval_reward: -257.503 | |
| 2023-05-27 00:23:24 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -257.503, save the best model! | |
| 2023-05-27 00:23:28 - RayLog - INFO: - Worker 2 finished episode 105 with reward -126.619 in 200 steps | |
| 2023-05-27 00:23:28 - RayLog - INFO: - Worker 0 finished episode 106 with reward -261.980 in 200 steps | |
| 2023-05-27 00:23:28 - RayLog - INFO: - Worker 1 finished episode 107 with reward -247.431 in 200 steps | |
| 2023-05-27 00:23:28 - RayLog - INFO: - Worker 3 finished episode 108 with reward -238.771 in 200 steps | |
| 2023-05-27 00:23:41 - RayLog - INFO: - Worker 2 finished episode 109 with reward -125.812 in 200 steps | |
| 2023-05-27 00:23:41 - RayLog - INFO: - Worker 0 finished episode 110 with reward -261.920 in 200 steps | |
| 2023-05-27 00:23:41 - RayLog - INFO: - Worker 1 finished episode 111 with reward -245.626 in 200 steps | |
| 2023-05-27 00:23:41 - RayLog - INFO: - Worker 3 finished episode 112 with reward -238.486 in 200 steps | |
| 2023-05-27 00:23:55 - RayLog - INFO: - Worker 2 finished episode 113 with reward -128.340 in 200 steps | |
| 2023-05-27 00:23:55 - RayLog - INFO: - Worker 0 finished episode 114 with reward -258.998 in 200 steps | |
| 2023-05-27 00:23:55 - RayLog - INFO: - Worker 1 finished episode 115 with reward -247.832 in 200 steps | |
| 2023-05-27 00:23:55 - RayLog - INFO: - Worker 3 finished episode 116 with reward -239.194 in 200 steps | |
| 2023-05-27 00:23:59 - RayLog - INFO: - learner id: 0, update_step: 24000, online_eval_reward: -257.609 | |
| 2023-05-27 00:24:10 - RayLog - INFO: - Worker 2 finished episode 117 with reward -125.382 in 200 steps | |
| 2023-05-27 00:24:10 - RayLog - INFO: - Worker 0 finished episode 118 with reward -382.680 in 200 steps | |
| 2023-05-27 00:24:10 - RayLog - INFO: - Worker 1 finished episode 120 with reward -249.722 in 200 steps | |
| 2023-05-27 00:24:10 - RayLog - INFO: - Worker 3 finished episode 120 with reward -232.887 in 200 steps | |
| 2023-05-27 00:24:22 - RayLog - INFO: - Worker 2 finished episode 121 with reward -126.514 in 200 steps | |
| 2023-05-27 00:24:22 - RayLog - INFO: - Worker 0 finished episode 122 with reward -264.696 in 200 steps | |
| 2023-05-27 00:24:22 - RayLog - INFO: - Worker 1 finished episode 123 with reward -247.312 in 200 steps | |
| 2023-05-27 00:24:22 - RayLog - INFO: - Worker 3 finished episode 124 with reward -237.931 in 200 steps | |
| 2023-05-27 00:24:33 - RayLog - INFO: - learner id: 0, update_step: 26000, online_eval_reward: -257.615 | |
| 2023-05-27 00:24:37 - RayLog - INFO: - Worker 2 finished episode 125 with reward -125.401 in 200 steps | |
| 2023-05-27 00:24:37 - RayLog - INFO: - Worker 0 finished episode 126 with reward -391.984 in 200 steps | |
| 2023-05-27 00:24:37 - RayLog - INFO: - Worker 1 finished episode 128 with reward -245.055 in 200 steps | |
| 2023-05-27 00:24:37 - RayLog - INFO: - Worker 3 finished episode 128 with reward -241.654 in 200 steps | |
| 2023-05-27 00:24:49 - RayLog - INFO: - Worker 2 finished episode 129 with reward -129.201 in 200 steps | |
| 2023-05-27 00:24:49 - RayLog - INFO: - Worker 0 finished episode 130 with reward -383.073 in 200 steps | |
| 2023-05-27 00:24:49 - RayLog - INFO: - Worker 1 finished episode 131 with reward -249.794 in 200 steps | |
| 2023-05-27 00:24:49 - RayLog - INFO: - Worker 3 finished episode 132 with reward -638.571 in 200 steps | |
| 2023-05-27 00:25:01 - RayLog - INFO: - Worker 2 finished episode 133 with reward -128.480 in 200 steps | |
| 2023-05-27 00:25:01 - RayLog - INFO: - Worker 0 finished episode 134 with reward -257.693 in 200 steps | |
| 2023-05-27 00:25:01 - RayLog - INFO: - Worker 1 finished episode 135 with reward -251.526 in 200 steps | |
| 2023-05-27 00:25:01 - RayLog - INFO: - Worker 3 finished episode 136 with reward -237.464 in 200 steps | |
| 2023-05-27 00:25:05 - RayLog - INFO: - learner id: 0, update_step: 28000, online_eval_reward: -258.711 | |
| 2023-05-27 00:25:16 - RayLog - INFO: - Worker 2 finished episode 137 with reward -125.328 in 200 steps | |
| 2023-05-27 00:25:16 - RayLog - INFO: - Worker 0 finished episode 138 with reward -396.933 in 200 steps | |
| 2023-05-27 00:25:16 - RayLog - INFO: - Worker 1 finished episode 139 with reward -243.056 in 200 steps | |
| 2023-05-27 00:25:16 - RayLog - INFO: - Worker 3 finished episode 140 with reward -236.837 in 200 steps | |
| 2023-05-27 00:25:29 - RayLog - INFO: - Worker 2 finished episode 141 with reward -125.565 in 200 steps | |
| 2023-05-27 00:25:29 - RayLog - INFO: - Worker 0 finished episode 142 with reward -385.128 in 200 steps | |
| 2023-05-27 00:25:29 - RayLog - INFO: - Worker 1 finished episode 143 with reward -247.311 in 200 steps | |
| 2023-05-27 00:25:29 - RayLog - INFO: - Worker 3 finished episode 144 with reward -237.032 in 200 steps | |
| 2023-05-27 00:25:40 - RayLog - INFO: - learner id: 0, update_step: 30000, online_eval_reward: -321.698 | |
| 2023-05-27 00:25:44 - RayLog - INFO: - Worker 2 finished episode 145 with reward -125.422 in 200 steps | |
| 2023-05-27 00:25:44 - RayLog - INFO: - Worker 0 finished episode 146 with reward -318.491 in 200 steps | |
| 2023-05-27 00:25:44 - RayLog - INFO: - Worker 1 finished episode 147 with reward -243.852 in 200 steps | |
| 2023-05-27 00:25:44 - RayLog - INFO: - Worker 3 finished episode 148 with reward -237.555 in 200 steps | |
| 2023-05-27 00:25:57 - RayLog - INFO: - Worker 2 finished episode 149 with reward -125.752 in 200 steps | |
| 2023-05-27 00:25:57 - RayLog - INFO: - Worker 0 finished episode 150 with reward -291.795 in 200 steps | |
| 2023-05-27 00:25:57 - RayLog - INFO: - Worker 1 finished episode 151 with reward -250.404 in 200 steps | |
| 2023-05-27 00:25:57 - RayLog - INFO: - Worker 3 finished episode 152 with reward -235.658 in 200 steps | |
| 2023-05-27 00:26:10 - RayLog - INFO: - Worker 2 finished episode 153 with reward -126.320 in 200 steps | |
| 2023-05-27 00:26:10 - RayLog - INFO: - Worker 0 finished episode 154 with reward -316.699 in 200 steps | |
| 2023-05-27 00:26:10 - RayLog - INFO: - Worker 1 finished episode 155 with reward -247.839 in 200 steps | |
| 2023-05-27 00:26:10 - RayLog - INFO: - Worker 3 finished episode 156 with reward -238.029 in 200 steps | |
| 2023-05-27 00:26:14 - RayLog - INFO: - learner id: 0, update_step: 32000, online_eval_reward: -336.660 | |
| 2023-05-27 00:26:24 - RayLog - INFO: - Worker 2 finished episode 157 with reward -123.561 in 200 steps | |
| 2023-05-27 00:26:24 - RayLog - INFO: - Worker 0 finished episode 158 with reward -336.021 in 200 steps | |
| 2023-05-27 00:26:24 - RayLog - INFO: - Worker 1 finished episode 159 with reward -243.522 in 200 steps | |
| 2023-05-27 00:26:24 - RayLog - INFO: - Worker 3 finished episode 160 with reward -235.183 in 200 steps | |
| 2023-05-27 00:26:37 - RayLog - INFO: - Worker 2 finished episode 161 with reward -127.010 in 200 steps | |
| 2023-05-27 00:26:37 - RayLog - INFO: - Worker 0 finished episode 162 with reward -314.601 in 200 steps | |
| 2023-05-27 00:26:37 - RayLog - INFO: - Worker 1 finished episode 164 with reward -249.200 in 200 steps | |
| 2023-05-27 00:26:38 - RayLog - INFO: - Worker 3 finished episode 164 with reward -236.858 in 200 steps | |
| 2023-05-27 00:26:48 - RayLog - INFO: - learner id: 0, update_step: 34000, online_eval_reward: -336.947 | |
| 2023-05-27 00:26:52 - RayLog - INFO: - Worker 2 finished episode 165 with reward -123.759 in 200 steps | |
| 2023-05-27 00:26:52 - RayLog - INFO: - Worker 0 finished episode 167 with reward -337.302 in 200 steps | |
| 2023-05-27 00:26:52 - RayLog - INFO: - Worker 1 finished episode 167 with reward -244.929 in 200 steps | |
| 2023-05-27 00:26:52 - RayLog - INFO: - Worker 3 finished episode 168 with reward -235.501 in 200 steps | |
| 2023-05-27 00:27:04 - RayLog - INFO: - Worker 2 finished episode 169 with reward -126.060 in 200 steps | |
| 2023-05-27 00:27:04 - RayLog - INFO: - Worker 0 finished episode 170 with reward -342.141 in 200 steps | |
| 2023-05-27 00:27:04 - RayLog - INFO: - Worker 1 finished episode 171 with reward -247.857 in 200 steps | |
| 2023-05-27 00:27:04 - RayLog - INFO: - Worker 3 finished episode 172 with reward -235.949 in 200 steps | |
| 2023-05-27 00:27:16 - RayLog - INFO: - Worker 2 finished episode 173 with reward -125.042 in 200 steps | |
| 2023-05-27 00:27:16 - RayLog - INFO: - Worker 0 finished episode 174 with reward -337.675 in 200 steps | |
| 2023-05-27 00:27:16 - RayLog - INFO: - Worker 1 finished episode 175 with reward -244.513 in 200 steps | |
| 2023-05-27 00:27:16 - RayLog - INFO: - Worker 3 finished episode 176 with reward -236.820 in 200 steps | |
| 2023-05-27 00:27:21 - RayLog - INFO: - learner id: 0, update_step: 36000, online_eval_reward: -334.958 | |
| 2023-05-27 00:27:30 - RayLog - INFO: - Worker 2 finished episode 177 with reward -125.483 in 200 steps | |
| 2023-05-27 00:27:31 - RayLog - INFO: - Worker 0 finished episode 178 with reward -331.356 in 200 steps | |
| 2023-05-27 00:27:31 - RayLog - INFO: - Worker 1 finished episode 179 with reward -245.782 in 200 steps | |
| 2023-05-27 00:27:31 - RayLog - INFO: - Worker 3 finished episode 180 with reward -236.418 in 200 steps | |
| 2023-05-27 00:27:43 - RayLog - INFO: - Worker 2 finished episode 181 with reward -123.913 in 200 steps | |
| 2023-05-27 00:27:43 - RayLog - INFO: - Worker 0 finished episode 183 with reward -330.790 in 200 steps | |
| 2023-05-27 00:27:43 - RayLog - INFO: - Worker 1 finished episode 183 with reward -244.373 in 200 steps | |
| 2023-05-27 00:27:43 - RayLog - INFO: - Worker 3 finished episode 184 with reward -237.850 in 200 steps | |
| 2023-05-27 00:27:56 - RayLog - INFO: - learner id: 0, update_step: 38000, online_eval_reward: -328.723 | |
| 2023-05-27 00:28:01 - RayLog - INFO: - Worker 2 finished episode 185 with reward -125.132 in 200 steps | |
| 2023-05-27 00:28:01 - RayLog - INFO: - Worker 0 finished episode 186 with reward -333.844 in 200 steps | |
| 2023-05-27 00:28:01 - RayLog - INFO: - Worker 1 finished episode 187 with reward -245.075 in 200 steps | |
| 2023-05-27 00:28:01 - RayLog - INFO: - Worker 3 finished episode 188 with reward -236.059 in 200 steps | |
| 2023-05-27 00:28:13 - RayLog - INFO: - Worker 2 finished episode 189 with reward -125.009 in 200 steps | |
| 2023-05-27 00:28:13 - RayLog - INFO: - Worker 0 finished episode 190 with reward -319.946 in 200 steps | |
| 2023-05-27 00:28:13 - RayLog - INFO: - Worker 1 finished episode 191 with reward -245.653 in 200 steps | |
| 2023-05-27 00:28:13 - RayLog - INFO: - Worker 3 finished episode 192 with reward -238.445 in 200 steps | |
| 2023-05-27 00:28:25 - RayLog - INFO: - Worker 2 finished episode 193 with reward -124.193 in 200 steps | |
| 2023-05-27 00:28:25 - RayLog - INFO: - Worker 0 finished episode 194 with reward -291.396 in 200 steps | |
| 2023-05-27 00:28:25 - RayLog - INFO: - Worker 1 finished episode 195 with reward -243.751 in 200 steps | |
| 2023-05-27 00:28:25 - RayLog - INFO: - Worker 3 finished episode 196 with reward -236.249 in 200 steps | |
| 2023-05-27 00:28:29 - RayLog - INFO: - learner id: 0, update_step: 40000, online_eval_reward: -253.906 | |
| 2023-05-27 00:28:29 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: -253.906, save the best model! | |
| 2023-05-27 00:28:38 - RayLog - INFO: - Worker 2 finished episode 197 with reward -123.794 in 200 steps | |
| 2023-05-27 00:28:38 - RayLog - INFO: - Worker 0 finished episode 198 with reward -256.651 in 200 steps | |
| 2023-05-27 00:28:38 - RayLog - INFO: - Worker 1 finished episode 199 with reward -244.366 in 200 steps | |
| 2023-05-27 00:28:38 - RayLog - INFO: - Worker 3 finished episode 200 with reward -236.650 in 200 steps | |
| 2023-05-27 00:28:41 - SimpleLog - INFO: - Finish training! total time consumed: 685.93s | |