| 2023-05-18 13:39:47 - SimpleLog - INFO: - General Configs: | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - env_name gym <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - algo_name NoisyDQN <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - mode train <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - device cpu <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - seed 1 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - max_episode 100 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - max_step 200 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - collect_traj 0 <class 'bool'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - mp_backend ray <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - n_workers 2 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - n_learners 1 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - share_buffer 1 <class 'bool'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - online_eval 1 <class 'bool'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - model_save_fre 500 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - load_path Train_single_CartPole-v1_NoisyDQN_20230518-133737 <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - load_model_step best <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - Algo Configs: | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_decay 500 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - gamma 0.99 <class 'float'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - lr 0.0001 <class 'float'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - buffer_size 100000 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - batch_size 64 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - target_update 4 <class 'int'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - value_layers [{'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}, {'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}] <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - Env Configs: | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - id CartPole-v1 <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - render_mode None <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - wrapper None <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'> | |
| 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================ | |
| 2023-05-18 13:39:54 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2) | |
| 2023-05-18 13:39:59 - RayLog - INFO: - Worker 0 finished episode 0 with reward 14.0 in 14 steps | |
| 2023-05-18 13:39:59 - RayLog - INFO: - Worker 1 finished episode 0 with reward 22.0 in 22 steps | |
| 2023-05-18 13:40:01 - RayLog - INFO: - Worker 1 finished episode 2 with reward 33.0 in 33 steps | |
| 2023-05-18 13:40:01 - RayLog - INFO: - Worker 0 finished episode 2 with reward 38.0 in 38 steps | |
| 2023-05-18 13:40:01 - RayLog - INFO: - Worker 0 finished episode 4 with reward 15.0 in 15 steps | |
| 2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 5 with reward 26.0 in 26 steps | |
| 2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 3 with reward 53.0 in 53 steps | |
| 2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 6 with reward 11.0 in 11 steps | |
| 2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 7 with reward 12.0 in 12 steps | |
| 2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 8 with reward 14.0 in 14 steps | |
| 2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 9 with reward 13.0 in 13 steps | |
| 2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 10 with reward 12.0 in 12 steps | |
| 2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 11 with reward 10.0 in 10 steps | |
| 2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 12 with reward 9.0 in 9 steps | |
| 2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 13 with reward 12.0 in 12 steps | |
| 2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 14 with reward 11.0 in 11 steps | |
| 2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 15 with reward 11.0 in 11 steps | |
| 2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 16 with reward 9.0 in 9 steps | |
| 2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 17 with reward 20.0 in 20 steps | |
| 2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 18 with reward 20.0 in 20 steps | |
| 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 20 with reward 12.0 in 12 steps | |
| 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 21 with reward 13.0 in 13 steps | |
| 2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 19 with reward 31.0 in 31 steps | |
| 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 22 with reward 10.0 in 10 steps | |
| 2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 23 with reward 9.0 in 9 steps | |
| 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 24 with reward 12.0 in 12 steps | |
| 2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 25 with reward 11.0 in 11 steps | |
| 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 26 with reward 12.0 in 12 steps | |
| 2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 27 with reward 12.0 in 12 steps | |
| 2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 28 with reward 13.0 in 13 steps | |
| 2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 29 with reward 13.0 in 13 steps | |
| 2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 30 with reward 12.0 in 12 steps | |
| 2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 31 with reward 14.0 in 14 steps | |
| 2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 32 with reward 14.0 in 14 steps | |
| 2023-05-18 13:40:07 - RayLog - INFO: - learner id: 0, update_step: 500, online_eval_reward: 10.000 | |
| 2023-05-18 13:40:07 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 10.000, save the best model! | |
| 2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 33 with reward 14.0 in 14 steps | |
| 2023-05-18 13:40:07 - RayLog - INFO: - Worker 0 finished episode 34 with reward 11.0 in 11 steps | |
| 2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 35 with reward 13.0 in 13 steps | |
| 2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 37 with reward 10.0 in 10 steps | |
| 2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 38 with reward 11.0 in 11 steps | |
| 2023-05-18 13:40:07 - RayLog - INFO: - Worker 0 finished episode 36 with reward 29.0 in 29 steps | |
| 2023-05-18 13:40:08 - RayLog - INFO: - Worker 1 finished episode 39 with reward 9.0 in 9 steps | |
| 2023-05-18 13:40:08 - RayLog - INFO: - Worker 0 finished episode 40 with reward 11.0 in 11 steps | |
| 2023-05-18 13:40:08 - RayLog - INFO: - Worker 0 finished episode 42 with reward 35.0 in 35 steps | |
| 2023-05-18 13:40:08 - RayLog - INFO: - Worker 1 finished episode 41 with reward 49.0 in 49 steps | |
| 2023-05-18 13:40:09 - RayLog - INFO: - Worker 0 finished episode 43 with reward 36.0 in 36 steps | |
| 2023-05-18 13:40:09 - RayLog - INFO: - Worker 1 finished episode 44 with reward 35.0 in 35 steps | |
| 2023-05-18 13:40:10 - RayLog - INFO: - Worker 0 finished episode 45 with reward 42.0 in 42 steps | |
| 2023-05-18 13:40:10 - RayLog - INFO: - Worker 1 finished episode 46 with reward 44.0 in 44 steps | |
| 2023-05-18 13:40:10 - RayLog - INFO: - Worker 0 finished episode 47 with reward 36.0 in 36 steps | |
| 2023-05-18 13:40:11 - RayLog - INFO: - Worker 1 finished episode 48 with reward 44.0 in 44 steps | |
| 2023-05-18 13:40:11 - RayLog - INFO: - Worker 0 finished episode 49 with reward 37.0 in 37 steps | |
| 2023-05-18 13:40:12 - RayLog - INFO: - learner id: 0, update_step: 1000, online_eval_reward: 51.000 | |
| 2023-05-18 13:40:12 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 51.000, save the best model! | |
| 2023-05-18 13:40:13 - RayLog - INFO: - Worker 0 finished episode 51 with reward 76.0 in 76 steps | |
| 2023-05-18 13:40:13 - RayLog - INFO: - Worker 1 finished episode 50 with reward 109.0 in 109 steps | |
| 2023-05-18 13:40:15 - RayLog - INFO: - Worker 0 finished episode 52 with reward 119.0 in 119 steps | |
| 2023-05-18 13:40:15 - RayLog - INFO: - Worker 1 finished episode 53 with reward 120.0 in 120 steps | |
| 2023-05-18 13:40:16 - RayLog - INFO: - Worker 0 finished episode 54 with reward 57.0 in 57 steps | |
| 2023-05-18 13:40:16 - RayLog - INFO: - Worker 1 finished episode 55 with reward 44.0 in 44 steps | |
| 2023-05-18 13:40:16 - RayLog - INFO: - learner id: 0, update_step: 1500, online_eval_reward: 53.000 | |
| 2023-05-18 13:40:16 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 53.000, save the best model! | |
| 2023-05-18 13:40:17 - RayLog - INFO: - Worker 1 finished episode 57 with reward 43.0 in 43 steps | |
| 2023-05-18 13:40:18 - RayLog - INFO: - Worker 0 finished episode 56 with reward 72.0 in 72 steps | |
| 2023-05-18 13:40:18 - RayLog - INFO: - Worker 1 finished episode 58 with reward 35.0 in 35 steps | |
| 2023-05-18 13:40:19 - RayLog - INFO: - Worker 1 finished episode 60 with reward 48.0 in 48 steps | |
| 2023-05-18 13:40:19 - RayLog - INFO: - Worker 0 finished episode 59 with reward 64.0 in 64 steps | |
| 2023-05-18 13:40:19 - RayLog - INFO: - Worker 1 finished episode 61 with reward 36.0 in 36 steps | |
| 2023-05-18 13:40:20 - RayLog - INFO: - Worker 0 finished episode 62 with reward 57.0 in 57 steps | |
| 2023-05-18 13:40:20 - RayLog - INFO: - Worker 1 finished episode 63 with reward 42.0 in 42 steps | |
| 2023-05-18 13:40:21 - RayLog - INFO: - learner id: 0, update_step: 2000, online_eval_reward: 143.000 | |
| 2023-05-18 13:40:21 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 143.000, save the best model! | |
| 2023-05-18 13:40:23 - RayLog - INFO: - Worker 1 finished episode 65 with reward 100.0 in 100 steps | |
| 2023-05-18 13:40:23 - RayLog - INFO: - Worker 0 finished episode 64 with reward 153.0 in 153 steps | |
| 2023-05-18 13:40:24 - RayLog - INFO: - Worker 1 finished episode 66 with reward 61.0 in 61 steps | |
| 2023-05-18 13:40:25 - RayLog - INFO: - Worker 0 finished episode 67 with reward 111.0 in 111 steps | |
| 2023-05-18 13:40:26 - RayLog - INFO: - Worker 1 finished episode 68 with reward 103.0 in 103 steps | |
| 2023-05-18 13:40:27 - RayLog - INFO: - learner id: 0, update_step: 2500, online_eval_reward: 85.000 | |
| 2023-05-18 13:40:28 - RayLog - INFO: - Worker 1 finished episode 70 with reward 127.0 in 127 steps | |
| 2023-05-18 13:40:29 - RayLog - INFO: - Worker 0 finished episode 69 with reward 164.0 in 164 steps | |
| 2023-05-18 13:40:32 - RayLog - INFO: - learner id: 0, update_step: 3000, online_eval_reward: 74.000 | |
| 2023-05-18 13:40:32 - RayLog - INFO: - Worker 1 finished episode 71 with reward 200.0 in 200 steps | |
| 2023-05-18 13:40:32 - RayLog - INFO: - Worker 0 finished episode 72 with reward 186.0 in 186 steps | |
| 2023-05-18 13:40:35 - RayLog - INFO: - Worker 1 finished episode 73 with reward 114.0 in 114 steps | |
| 2023-05-18 13:40:35 - RayLog - INFO: - Worker 0 finished episode 74 with reward 109.0 in 109 steps | |
| 2023-05-18 13:40:36 - RayLog - INFO: - Worker 1 finished episode 75 with reward 72.0 in 72 steps | |
| 2023-05-18 13:40:37 - RayLog - INFO: - learner id: 0, update_step: 3500, online_eval_reward: 83.000 | |
| 2023-05-18 13:40:37 - RayLog - INFO: - Worker 0 finished episode 76 with reward 96.0 in 96 steps | |
| 2023-05-18 13:40:37 - RayLog - INFO: - Worker 1 finished episode 77 with reward 62.0 in 62 steps | |
| 2023-05-18 13:40:38 - RayLog - INFO: - Worker 0 finished episode 78 with reward 88.0 in 88 steps | |
| 2023-05-18 13:40:39 - RayLog - INFO: - Worker 1 finished episode 79 with reward 70.0 in 70 steps | |
| 2023-05-18 13:40:40 - RayLog - INFO: - Worker 1 finished episode 81 with reward 77.0 in 77 steps | |
| 2023-05-18 13:40:42 - RayLog - INFO: - learner id: 0, update_step: 4000, online_eval_reward: 123.000 | |
| 2023-05-18 13:40:42 - RayLog - INFO: - Worker 0 finished episode 80 with reward 181.0 in 181 steps | |
| 2023-05-18 13:40:42 - RayLog - INFO: - Worker 1 finished episode 82 with reward 105.0 in 105 steps | |
| 2023-05-18 13:40:46 - RayLog - INFO: - Worker 0 finished episode 83 with reward 200.0 in 200 steps | |
| 2023-05-18 13:40:47 - RayLog - INFO: - Worker 1 finished episode 84 with reward 200.0 in 200 steps | |
| 2023-05-18 13:40:47 - RayLog - INFO: - learner id: 0, update_step: 4500, online_eval_reward: 200.000 | |
| 2023-05-18 13:40:47 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 200.000, save the best model! | |
| 2023-05-18 13:40:51 - RayLog - INFO: - Worker 0 finished episode 85 with reward 200.0 in 200 steps | |
| 2023-05-18 13:40:51 - RayLog - INFO: - Worker 1 finished episode 86 with reward 200.0 in 200 steps | |
| 2023-05-18 13:40:53 - RayLog - INFO: - learner id: 0, update_step: 5000, online_eval_reward: 200.000 | |
| 2023-05-18 13:40:56 - RayLog - INFO: - Worker 0 finished episode 87 with reward 200.0 in 200 steps | |
| 2023-05-18 13:40:56 - RayLog - INFO: - Worker 1 finished episode 88 with reward 200.0 in 200 steps | |
| 2023-05-18 13:40:59 - RayLog - INFO: - learner id: 0, update_step: 5500, online_eval_reward: 200.000 | |
| 2023-05-18 13:41:00 - RayLog - INFO: - Worker 0 finished episode 89 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:01 - RayLog - INFO: - Worker 1 finished episode 90 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:05 - RayLog - INFO: - learner id: 0, update_step: 6000, online_eval_reward: 200.000 | |
| 2023-05-18 13:41:05 - RayLog - INFO: - Worker 0 finished episode 91 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:05 - RayLog - INFO: - Worker 1 finished episode 92 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:10 - RayLog - INFO: - Worker 0 finished episode 93 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:10 - RayLog - INFO: - Worker 1 finished episode 94 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:11 - RayLog - INFO: - learner id: 0, update_step: 6500, online_eval_reward: 200.000 | |
| 2023-05-18 13:41:15 - RayLog - INFO: - Worker 0 finished episode 95 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:15 - RayLog - INFO: - Worker 1 finished episode 96 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:17 - RayLog - INFO: - learner id: 0, update_step: 7000, online_eval_reward: 200.000 | |
| 2023-05-18 13:41:19 - RayLog - INFO: - Worker 0 finished episode 97 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:20 - RayLog - INFO: - Worker 1 finished episode 98 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:23 - RayLog - INFO: - learner id: 0, update_step: 7500, online_eval_reward: 200.000 | |
| 2023-05-18 13:41:24 - RayLog - INFO: - Worker 0 finished episode 99 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:25 - RayLog - INFO: - Worker 1 finished episode 100 with reward 200.0 in 200 steps | |
| 2023-05-18 13:41:28 - SimpleLog - INFO: - Finish training! total time consumed: 100.97s | |