| 2023-05-17 13:44:40 - SimpleLog - INFO: - General Configs: | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - env_name gym <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - algo_name PPO <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - mode train <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - device cpu <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - seed 1 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - max_episode 200 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - max_step 200 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - collect_traj 0 <class 'bool'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - mp_backend single <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - n_workers 2 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - online_eval 1 <class 'bool'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - model_save_fre 10 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - load_path Train_single_CartPole-v1_DQN_20230515-211721 <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - load_model_step best <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - Algo Configs: | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - independ_actor 1 <class 'bool'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - share_optimizer 0 <class 'bool'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - ppo_type clip <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - eps_clip 0.2 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - kl_target 0.1 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - kl_lambda 0.5 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - kl_beta 1.5 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - kl_alpha 2 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - continuous 0 <class 'bool'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - gamma 0.99 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - k_epochs 4 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - lr 0.0001 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - actor_lr 0.0003 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - critic_lr 0.001 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - critic_loss_coef 0.5 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - entropy_coef 0.01 <class 'float'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - batch_size 256 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - sgd_batch_size 128 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - actor_hidden_dim 256 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - critic_hidden_dim 256 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - min_policy 0 <class 'int'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - actor_layers [{'layer_type': 'linear', 'layer_dim': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_dim': [256], 'activation': 'relu'}] <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - critic_layers [{'layer_type': 'linear', 'layer_dim': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_dim': [256], 'activation': 'relu'}] <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - buffer_type ONPOLICY_QUE <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - Env Configs: | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - id CartPole-v1 <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - render_mode None <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - wrapper None <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'> | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2) | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - Start training! | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - episode: 0, ep_reward: 13.0, ep_step: 13 | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - episode: 1, ep_reward: 30.0, ep_step: 30 | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - episode: 2, ep_reward: 15.0, ep_step: 15 | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - episode: 3, ep_reward: 10.0, ep_step: 10 | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - episode: 4, ep_reward: 38.0, ep_step: 38 | |
| 2023-05-17 13:44:40 - SimpleLog - INFO: - episode: 5, ep_reward: 18.0, ep_step: 18 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 6, ep_reward: 20.0, ep_step: 20 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 7, ep_reward: 26.0, ep_step: 26 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 8, ep_reward: 9.0, ep_step: 9 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 9, ep_reward: 12.0, ep_step: 12 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 10, ep_reward: 47.0, ep_step: 47 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 11, ep_reward: 21.0, ep_step: 21 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 12, ep_reward: 27.0, ep_step: 27 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 13, ep_reward: 12.0, ep_step: 12 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 14, ep_reward: 37.0, ep_step: 37 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 15, ep_reward: 20.0, ep_step: 20 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 16, ep_reward: 36.0, ep_step: 36 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 17, ep_reward: 21.0, ep_step: 21 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 18, ep_reward: 40.0, ep_step: 40 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 19, ep_reward: 15.0, ep_step: 15 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 20, ep_reward: 14.0, ep_step: 14 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 21, ep_reward: 89.0, ep_step: 89 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 22, ep_reward: 27.0, ep_step: 27 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 23, ep_reward: 38.0, ep_step: 38 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 24, ep_reward: 29.0, ep_step: 29 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 25, ep_reward: 20.0, ep_step: 20 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 26, ep_reward: 50.0, ep_step: 50 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 27, ep_reward: 44.0, ep_step: 44 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 28, ep_reward: 30.0, ep_step: 30 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 29, ep_reward: 24.0, ep_step: 24 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 30, ep_reward: 19.0, ep_step: 19 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 31, ep_reward: 19.0, ep_step: 19 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 32, ep_reward: 28.0, ep_step: 28 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 33, ep_reward: 59.0, ep_step: 59 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 34, ep_reward: 22.0, ep_step: 22 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 35, ep_reward: 33.0, ep_step: 33 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 36, ep_reward: 18.0, ep_step: 18 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 37, ep_reward: 13.0, ep_step: 13 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 38, ep_reward: 68.0, ep_step: 68 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 39, ep_reward: 20.0, ep_step: 20 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 40, ep_reward: 17.0, ep_step: 17 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 41, ep_reward: 32.0, ep_step: 32 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 42, ep_reward: 17.0, ep_step: 17 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 43, ep_reward: 35.0, ep_step: 35 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 44, ep_reward: 18.0, ep_step: 18 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 45, ep_reward: 23.0, ep_step: 23 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 46, ep_reward: 15.0, ep_step: 15 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 47, ep_reward: 19.0, ep_step: 19 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 48, ep_reward: 31.0, ep_step: 31 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 49, ep_reward: 39.0, ep_step: 39 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 50, ep_reward: 26.0, ep_step: 26 | |
| 2023-05-17 13:44:41 - SimpleLog - INFO: - episode: 51, ep_reward: 82.0, ep_step: 82 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 52, ep_reward: 80.0, ep_step: 80 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 53, ep_reward: 20.0, ep_step: 20 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 54, ep_reward: 39.0, ep_step: 39 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 55, ep_reward: 74.0, ep_step: 74 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 56, ep_reward: 72.0, ep_step: 72 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 57, ep_reward: 28.0, ep_step: 28 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 58, ep_reward: 65.0, ep_step: 65 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 59, ep_reward: 54.0, ep_step: 54 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 60, ep_reward: 79.0, ep_step: 79 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 61, ep_reward: 55.0, ep_step: 55 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 62, ep_reward: 43.0, ep_step: 43 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 63, ep_reward: 84.0, ep_step: 84 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 64, ep_reward: 39.0, ep_step: 39 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 65, ep_reward: 115.0, ep_step: 115 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 66, ep_reward: 87.0, ep_step: 87 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 67, ep_reward: 71.0, ep_step: 71 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 68, ep_reward: 52.0, ep_step: 52 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - update_step: 10, online_eval_reward: 147.000 | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 147.000, save the best model! | |
| 2023-05-17 13:44:42 - SimpleLog - INFO: - episode: 69, ep_reward: 71.0, ep_step: 71 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 70, ep_reward: 48.0, ep_step: 48 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 71, ep_reward: 104.0, ep_step: 104 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 72, ep_reward: 30.0, ep_step: 30 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 73, ep_reward: 152.0, ep_step: 152 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 74, ep_reward: 51.0, ep_step: 51 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 75, ep_reward: 19.0, ep_step: 19 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 76, ep_reward: 44.0, ep_step: 44 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 77, ep_reward: 138.0, ep_step: 138 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 78, ep_reward: 26.0, ep_step: 26 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 79, ep_reward: 53.0, ep_step: 53 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 80, ep_reward: 108.0, ep_step: 108 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 81, ep_reward: 97.0, ep_step: 97 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 82, ep_reward: 87.0, ep_step: 87 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 83, ep_reward: 51.0, ep_step: 51 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 84, ep_reward: 101.0, ep_step: 101 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 85, ep_reward: 32.0, ep_step: 32 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 86, ep_reward: 76.0, ep_step: 76 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 87, ep_reward: 75.0, ep_step: 75 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 88, ep_reward: 44.0, ep_step: 44 | |
| 2023-05-17 13:44:43 - SimpleLog - INFO: - episode: 89, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 90, ep_reward: 126.0, ep_step: 126 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 91, ep_reward: 115.0, ep_step: 115 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 92, ep_reward: 29.0, ep_step: 29 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 93, ep_reward: 152.0, ep_step: 152 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 94, ep_reward: 78.0, ep_step: 78 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 95, ep_reward: 120.0, ep_step: 120 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 96, ep_reward: 140.0, ep_step: 140 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 97, ep_reward: 128.0, ep_step: 128 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - update_step: 20, online_eval_reward: 129.000 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 98, ep_reward: 102.0, ep_step: 102 | |
| 2023-05-17 13:44:44 - SimpleLog - INFO: - episode: 99, ep_reward: 87.0, ep_step: 87 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 100, ep_reward: 107.0, ep_step: 107 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 101, ep_reward: 66.0, ep_step: 66 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 102, ep_reward: 192.0, ep_step: 192 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 103, ep_reward: 125.0, ep_step: 125 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 104, ep_reward: 113.0, ep_step: 113 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 105, ep_reward: 138.0, ep_step: 138 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 106, ep_reward: 112.0, ep_step: 112 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 107, ep_reward: 164.0, ep_step: 164 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 108, ep_reward: 108.0, ep_step: 108 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 109, ep_reward: 28.0, ep_step: 28 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 110, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:45 - SimpleLog - INFO: - episode: 111, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 112, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 113, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 114, ep_reward: 151.0, ep_step: 151 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 115, ep_reward: 107.0, ep_step: 107 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 116, ep_reward: 66.0, ep_step: 66 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 117, ep_reward: 94.0, ep_step: 94 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 118, ep_reward: 25.0, ep_step: 25 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 119, ep_reward: 47.0, ep_step: 47 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - update_step: 30, online_eval_reward: 155.000 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 155.000, save the best model! | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 120, ep_reward: 136.0, ep_step: 136 | |
| 2023-05-17 13:44:46 - SimpleLog - INFO: - episode: 121, ep_reward: 119.0, ep_step: 119 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 122, ep_reward: 136.0, ep_step: 136 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 123, ep_reward: 182.0, ep_step: 182 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 124, ep_reward: 99.0, ep_step: 99 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 125, ep_reward: 33.0, ep_step: 33 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 126, ep_reward: 161.0, ep_step: 161 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 127, ep_reward: 171.0, ep_step: 171 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 128, ep_reward: 172.0, ep_step: 172 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 129, ep_reward: 178.0, ep_step: 178 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 130, ep_reward: 167.0, ep_step: 167 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 131, ep_reward: 149.0, ep_step: 149 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 132, ep_reward: 127.0, ep_step: 127 | |
| 2023-05-17 13:44:47 - SimpleLog - INFO: - episode: 133, ep_reward: 112.0, ep_step: 112 | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - episode: 134, ep_reward: 52.0, ep_step: 52 | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - episode: 135, ep_reward: 48.0, ep_step: 48 | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - episode: 136, ep_reward: 110.0, ep_step: 110 | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - episode: 137, ep_reward: 170.0, ep_step: 170 | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - episode: 138, ep_reward: 147.0, ep_step: 147 | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - update_step: 40, online_eval_reward: 200.000 | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 200.000, save the best model! | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - episode: 139, ep_reward: 127.0, ep_step: 127 | |
| 2023-05-17 13:44:48 - SimpleLog - INFO: - episode: 140, ep_reward: 164.0, ep_step: 164 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 141, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 142, ep_reward: 103.0, ep_step: 103 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 143, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 144, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 145, ep_reward: 147.0, ep_step: 147 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 146, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 147, ep_reward: 157.0, ep_step: 157 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 148, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:49 - SimpleLog - INFO: - episode: 149, ep_reward: 163.0, ep_step: 163 | |
| 2023-05-17 13:44:50 - SimpleLog - INFO: - episode: 150, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:50 - SimpleLog - INFO: - episode: 151, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:50 - SimpleLog - INFO: - episode: 152, ep_reward: 146.0, ep_step: 146 | |
| 2023-05-17 13:44:50 - SimpleLog - INFO: - episode: 153, ep_reward: 118.0, ep_step: 118 | |
| 2023-05-17 13:44:50 - SimpleLog - INFO: - update_step: 50, online_eval_reward: 200.000 | |
| 2023-05-17 13:44:50 - SimpleLog - INFO: - episode: 154, ep_reward: 191.0, ep_step: 191 | |
| 2023-05-17 13:44:50 - SimpleLog - INFO: - episode: 155, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 156, ep_reward: 153.0, ep_step: 153 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 157, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 158, ep_reward: 160.0, ep_step: 160 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 159, ep_reward: 137.0, ep_step: 137 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 160, ep_reward: 132.0, ep_step: 132 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 161, ep_reward: 147.0, ep_step: 147 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 162, ep_reward: 142.0, ep_step: 142 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 163, ep_reward: 133.0, ep_step: 133 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 164, ep_reward: 183.0, ep_step: 183 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 165, ep_reward: 161.0, ep_step: 161 | |
| 2023-05-17 13:44:51 - SimpleLog - INFO: - episode: 166, ep_reward: 186.0, ep_step: 186 | |
| 2023-05-17 13:44:52 - SimpleLog - INFO: - episode: 167, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:52 - SimpleLog - INFO: - episode: 168, ep_reward: 195.0, ep_step: 195 | |
| 2023-05-17 13:44:52 - SimpleLog - INFO: - update_step: 60, online_eval_reward: 200.000 | |
| 2023-05-17 13:44:52 - SimpleLog - INFO: - episode: 169, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:52 - SimpleLog - INFO: - episode: 170, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:52 - SimpleLog - INFO: - episode: 171, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 172, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 173, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 174, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 175, ep_reward: 153.0, ep_step: 153 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 176, ep_reward: 158.0, ep_step: 158 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 177, ep_reward: 53.0, ep_step: 53 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 178, ep_reward: 157.0, ep_step: 157 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 179, ep_reward: 133.0, ep_step: 133 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 180, ep_reward: 126.0, ep_step: 126 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 181, ep_reward: 147.0, ep_step: 147 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 182, ep_reward: 145.0, ep_step: 145 | |
| 2023-05-17 13:44:53 - SimpleLog - INFO: - episode: 183, ep_reward: 32.0, ep_step: 32 | |
| 2023-05-17 13:44:54 - SimpleLog - INFO: - episode: 184, ep_reward: 136.0, ep_step: 136 | |
| 2023-05-17 13:44:54 - SimpleLog - INFO: - episode: 185, ep_reward: 153.0, ep_step: 153 | |
| 2023-05-17 13:44:54 - SimpleLog - INFO: - update_step: 70, online_eval_reward: 200.000 | |
| 2023-05-17 13:44:54 - SimpleLog - INFO: - episode: 186, ep_reward: 199.0, ep_step: 199 | |
| 2023-05-17 13:44:54 - SimpleLog - INFO: - episode: 187, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:54 - SimpleLog - INFO: - episode: 188, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:54 - SimpleLog - INFO: - episode: 189, ep_reward: 193.0, ep_step: 193 | |
| 2023-05-17 13:44:55 - SimpleLog - INFO: - episode: 190, ep_reward: 167.0, ep_step: 167 | |
| 2023-05-17 13:44:55 - SimpleLog - INFO: - episode: 191, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:55 - SimpleLog - INFO: - episode: 192, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:55 - SimpleLog - INFO: - episode: 193, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:55 - SimpleLog - INFO: - episode: 194, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:55 - SimpleLog - INFO: - episode: 195, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:55 - SimpleLog - INFO: - episode: 196, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:55 - SimpleLog - INFO: - episode: 197, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:56 - SimpleLog - INFO: - episode: 198, ep_reward: 200.0, ep_step: 200 | |
| 2023-05-17 13:44:56 - SimpleLog - INFO: - update_step: 80, online_eval_reward: 200.000 | |
| 2023-05-17 13:44:56 - SimpleLog - INFO: - episode: 199, ep_reward: 187.0, ep_step: 187 | |
| 2023-05-17 13:44:56 - SimpleLog - INFO: - Finish training! total time consumed: 15.62s | |