File size: 15,798 Bytes
ccb908b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 |
2023-05-16 11:51:26 - SimpleLog - INFO: - General Configs:
2023-05-16 11:51:26 - SimpleLog - INFO: - ================================================================================
2023-05-16 11:51:26 - SimpleLog - INFO: - Name Value Type
2023-05-16 11:51:26 - SimpleLog - INFO: - env_name gym <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - algo_name DoubleDQN <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - mode train <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - device cpu <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - seed 1 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - max_episode 100 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - max_step 200 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
2023-05-16 11:51:26 - SimpleLog - INFO: - mp_backend ray <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - n_workers 2 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - online_eval 1 <class 'bool'>
2023-05-16 11:51:26 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'>
2023-05-16 11:51:26 - SimpleLog - INFO: - load_path Train_CartPole-v1_DQN_20221026-054757 <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - load_model_step best <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - ================================================================================
2023-05-16 11:51:26 - SimpleLog - INFO: - Algo Configs:
2023-05-16 11:51:26 - SimpleLog - INFO: - ================================================================================
2023-05-16 11:51:26 - SimpleLog - INFO: - Name Value Type
2023-05-16 11:51:26 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
2023-05-16 11:51:26 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
2023-05-16 11:51:26 - SimpleLog - INFO: - epsilon_decay 500 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
2023-05-16 11:51:26 - SimpleLog - INFO: - lr 0.0001 <class 'float'>
2023-05-16 11:51:26 - SimpleLog - INFO: - buffer_size 100000 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - batch_size 64 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - target_update 4 <class 'int'>
2023-05-16 11:51:26 - SimpleLog - INFO: - value_layers [{'layer_type': 'linear', 'layer_dim': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_dim': [256], 'activation': 'relu'}] <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - ================================================================================
2023-05-16 11:51:26 - SimpleLog - INFO: - Env Configs:
2023-05-16 11:51:26 - SimpleLog - INFO: - ================================================================================
2023-05-16 11:51:26 - SimpleLog - INFO: - Name Value Type
2023-05-16 11:51:26 - SimpleLog - INFO: - id CartPole-v1 <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - render_mode None <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - wrapper None <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
2023-05-16 11:51:26 - SimpleLog - INFO: - ================================================================================
2023-05-16 11:51:32 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2)
2023-05-16 11:51:37 - RayLog - INFO: - Worker 0 finished episode 0 with reward 16.0 in 16 steps
2023-05-16 11:51:37 - RayLog - INFO: - Worker 1 finished episode 0 with reward 20.0 in 20 steps
2023-05-16 11:51:39 - RayLog - INFO: - Worker 0 finished episode 2 with reward 11.0 in 11 steps
2023-05-16 11:51:39 - RayLog - INFO: - Worker 1 finished episode 2 with reward 15.0 in 15 steps
2023-05-16 11:51:39 - RayLog - INFO: - Worker 1 finished episode 4 with reward 13.0 in 13 steps
2023-05-16 11:51:39 - RayLog - INFO: - Worker 0 finished episode 3 with reward 22.0 in 22 steps
2023-05-16 11:51:39 - RayLog - INFO: - Worker 1 finished episode 5 with reward 9.0 in 9 steps
2023-05-16 11:51:40 - RayLog - INFO: - Worker 0 finished episode 6 with reward 14.0 in 14 steps
2023-05-16 11:51:40 - RayLog - INFO: - Worker 1 finished episode 7 with reward 12.0 in 12 steps
2023-05-16 11:51:40 - RayLog - INFO: - Worker 1 finished episode 9 with reward 13.0 in 13 steps
2023-05-16 11:51:40 - RayLog - INFO: - Worker 1 finished episode 10 with reward 14.0 in 14 steps
2023-05-16 11:51:40 - RayLog - INFO: - Worker 0 finished episode 8 with reward 35.0 in 35 steps
2023-05-16 11:51:40 - RayLog - INFO: - Worker 1 finished episode 11 with reward 13.0 in 13 steps
2023-05-16 11:51:40 - RayLog - INFO: - Worker 0 finished episode 12 with reward 15.0 in 15 steps
2023-05-16 11:51:40 - RayLog - INFO: - Worker 1 finished episode 13 with reward 12.0 in 12 steps
2023-05-16 11:51:41 - RayLog - INFO: - Worker 0 finished episode 14 with reward 10.0 in 10 steps
2023-05-16 11:51:41 - RayLog - INFO: - Worker 1 finished episode 15 with reward 11.0 in 11 steps
2023-05-16 11:51:41 - RayLog - INFO: - Worker 0 finished episode 16 with reward 11.0 in 11 steps
2023-05-16 11:51:41 - RayLog - INFO: - Worker 0 finished episode 18 with reward 12.0 in 12 steps
2023-05-16 11:51:41 - RayLog - INFO: - Worker 1 finished episode 17 with reward 21.0 in 21 steps
2023-05-16 11:51:41 - RayLog - INFO: - Worker 0 finished episode 19 with reward 15.0 in 15 steps
2023-05-16 11:51:41 - RayLog - INFO: - Worker 1 finished episode 20 with reward 18.0 in 18 steps
2023-05-16 11:51:41 - RayLog - INFO: - Worker 0 finished episode 21 with reward 13.0 in 13 steps
2023-05-16 11:51:42 - RayLog - INFO: - Worker 1 finished episode 22 with reward 10.0 in 10 steps
2023-05-16 11:51:42 - RayLog - INFO: - Worker 0 finished episode 23 with reward 16.0 in 16 steps
2023-05-16 11:51:42 - RayLog - INFO: - Worker 1 finished episode 24 with reward 19.0 in 19 steps
2023-05-16 11:51:42 - RayLog - INFO: - Worker 0 finished episode 25 with reward 10.0 in 10 steps
2023-05-16 11:51:42 - RayLog - INFO: - Worker 0 finished episode 27 with reward 10.0 in 10 steps
2023-05-16 11:51:42 - RayLog - INFO: - Worker 0 finished episode 28 with reward 10.0 in 10 steps
2023-05-16 11:51:42 - RayLog - INFO: - Worker 1 finished episode 26 with reward 26.0 in 26 steps
2023-05-16 11:51:42 - RayLog - INFO: - Worker 0 finished episode 29 with reward 12.0 in 12 steps
2023-05-16 11:51:43 - RayLog - INFO: - Worker 1 finished episode 30 with reward 15.0 in 15 steps
2023-05-16 11:51:43 - RayLog - INFO: - Worker 1 finished episode 32 with reward 10.0 in 10 steps
2023-05-16 11:51:43 - RayLog - INFO: - Worker 0 finished episode 31 with reward 18.0 in 18 steps
2023-05-16 11:51:43 - RayLog - INFO: - Worker 1 finished episode 33 with reward 13.0 in 13 steps
2023-05-16 11:51:43 - RayLog - INFO: - Worker 0 finished episode 34 with reward 10.0 in 10 steps
2023-05-16 11:51:43 - RayLog - INFO: - Worker 0 finished episode 36 with reward 9.0 in 9 steps
2023-05-16 11:51:43 - RayLog - INFO: - Worker 1 finished episode 35 with reward 12.0 in 12 steps
2023-05-16 11:51:45 - RayLog - INFO: - update_step: 500, online_eval_reward: 9.000
2023-05-16 11:51:45 - RayLog - INFO: - current update step obtain a better online_eval_reward: 9.000, save the best model!
2023-05-16 11:51:45 - RayLog - INFO: - Worker 1 finished episode 38 with reward 10.0 in 10 steps
2023-05-16 11:51:45 - RayLog - INFO: - Worker 0 finished episode 37 with reward 13.0 in 13 steps
2023-05-16 11:51:45 - RayLog - INFO: - Worker 0 finished episode 40 with reward 9.0 in 9 steps
2023-05-16 11:51:45 - RayLog - INFO: - Worker 1 finished episode 39 with reward 14.0 in 14 steps
2023-05-16 11:51:45 - RayLog - INFO: - Worker 0 finished episode 41 with reward 14.0 in 14 steps
2023-05-16 11:51:45 - RayLog - INFO: - Worker 1 finished episode 42 with reward 20.0 in 20 steps
2023-05-16 11:51:45 - RayLog - INFO: - Worker 0 finished episode 43 with reward 15.0 in 15 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 1 finished episode 44 with reward 19.0 in 19 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 0 finished episode 45 with reward 17.0 in 17 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 0 finished episode 47 with reward 12.0 in 12 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 1 finished episode 46 with reward 15.0 in 15 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 0 finished episode 48 with reward 14.0 in 14 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 1 finished episode 49 with reward 16.0 in 16 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 0 finished episode 50 with reward 9.0 in 9 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 1 finished episode 51 with reward 13.0 in 13 steps
2023-05-16 11:51:46 - RayLog - INFO: - Worker 0 finished episode 52 with reward 10.0 in 10 steps
2023-05-16 11:51:47 - RayLog - INFO: - Worker 1 finished episode 53 with reward 13.0 in 13 steps
2023-05-16 11:51:47 - RayLog - INFO: - Worker 0 finished episode 54 with reward 13.0 in 13 steps
2023-05-16 11:51:47 - RayLog - INFO: - Worker 0 finished episode 56 with reward 14.0 in 14 steps
2023-05-16 11:51:47 - RayLog - INFO: - Worker 1 finished episode 55 with reward 20.0 in 20 steps
2023-05-16 11:51:47 - RayLog - INFO: - Worker 0 finished episode 57 with reward 14.0 in 14 steps
2023-05-16 11:51:47 - RayLog - INFO: - Worker 1 finished episode 58 with reward 16.0 in 16 steps
2023-05-16 11:51:47 - RayLog - INFO: - Worker 0 finished episode 59 with reward 11.0 in 11 steps
2023-05-16 11:51:48 - RayLog - INFO: - Worker 1 finished episode 60 with reward 14.0 in 14 steps
2023-05-16 11:51:48 - RayLog - INFO: - Worker 0 finished episode 61 with reward 9.0 in 9 steps
2023-05-16 11:51:48 - RayLog - INFO: - Worker 0 finished episode 63 with reward 19.0 in 19 steps
2023-05-16 11:51:48 - RayLog - INFO: - Worker 1 finished episode 62 with reward 23.0 in 23 steps
2023-05-16 11:51:48 - RayLog - INFO: - Worker 0 finished episode 64 with reward 18.0 in 18 steps
2023-05-16 11:51:49 - RayLog - INFO: - Worker 1 finished episode 65 with reward 26.0 in 26 steps
2023-05-16 11:51:49 - RayLog - INFO: - update_step: 1000, online_eval_reward: 69.000
2023-05-16 11:51:49 - RayLog - INFO: - current update step obtain a better online_eval_reward: 69.000, save the best model!
2023-05-16 11:51:50 - RayLog - INFO: - Worker 1 finished episode 67 with reward 93.0 in 93 steps
2023-05-16 11:51:51 - RayLog - INFO: - Worker 0 finished episode 66 with reward 127.0 in 127 steps
2023-05-16 11:51:51 - RayLog - INFO: - Worker 1 finished episode 68 with reward 40.0 in 40 steps
2023-05-16 11:51:52 - RayLog - INFO: - Worker 0 finished episode 69 with reward 54.0 in 54 steps
2023-05-16 11:51:52 - RayLog - INFO: - Worker 1 finished episode 70 with reward 48.0 in 48 steps
2023-05-16 11:51:53 - RayLog - INFO: - Worker 0 finished episode 71 with reward 62.0 in 62 steps
2023-05-16 11:51:53 - RayLog - INFO: - Worker 1 finished episode 72 with reward 60.0 in 60 steps
2023-05-16 11:51:54 - RayLog - INFO: - Worker 1 finished episode 74 with reward 35.0 in 35 steps
2023-05-16 11:51:54 - RayLog - INFO: - Worker 0 finished episode 73 with reward 47.0 in 47 steps
2023-05-16 11:51:54 - RayLog - INFO: - update_step: 1500, online_eval_reward: 63.000
2023-05-16 11:51:54 - RayLog - INFO: - Worker 1 finished episode 75 with reward 38.0 in 38 steps
2023-05-16 11:51:54 - RayLog - INFO: - Worker 0 finished episode 76 with reward 46.0 in 46 steps
2023-05-16 11:51:55 - RayLog - INFO: - Worker 1 finished episode 77 with reward 40.0 in 40 steps
2023-05-16 11:51:55 - RayLog - INFO: - Worker 0 finished episode 78 with reward 57.0 in 57 steps
2023-05-16 11:51:56 - RayLog - INFO: - Worker 1 finished episode 79 with reward 38.0 in 38 steps
2023-05-16 11:51:56 - RayLog - INFO: - Worker 1 finished episode 81 with reward 33.0 in 33 steps
2023-05-16 11:51:56 - RayLog - INFO: - Worker 0 finished episode 80 with reward 51.0 in 51 steps
2023-05-16 11:51:57 - RayLog - INFO: - Worker 1 finished episode 82 with reward 44.0 in 44 steps
2023-05-16 11:51:58 - RayLog - INFO: - Worker 0 finished episode 83 with reward 70.0 in 70 steps
2023-05-16 11:51:58 - RayLog - INFO: - Worker 1 finished episode 84 with reward 55.0 in 55 steps
2023-05-16 11:51:58 - RayLog - INFO: - update_step: 2000, online_eval_reward: 82.000
2023-05-16 11:51:58 - RayLog - INFO: - current update step obtain a better online_eval_reward: 82.000, save the best model!
2023-05-16 11:51:59 - RayLog - INFO: - Worker 0 finished episode 85 with reward 66.0 in 66 steps
2023-05-16 11:51:59 - RayLog - INFO: - Worker 1 finished episode 86 with reward 56.0 in 56 steps
2023-05-16 11:52:00 - RayLog - INFO: - Worker 1 finished episode 88 with reward 45.0 in 45 steps
2023-05-16 11:52:00 - RayLog - INFO: - Worker 0 finished episode 87 with reward 68.0 in 68 steps
2023-05-16 11:52:01 - RayLog - INFO: - Worker 1 finished episode 89 with reward 50.0 in 50 steps
2023-05-16 11:52:02 - RayLog - INFO: - Worker 0 finished episode 90 with reward 79.0 in 79 steps
2023-05-16 11:52:02 - RayLog - INFO: - Worker 1 finished episode 91 with reward 57.0 in 57 steps
2023-05-16 11:52:04 - RayLog - INFO: - update_step: 2500, online_eval_reward: 77.000
2023-05-16 11:52:04 - RayLog - INFO: - Worker 1 finished episode 93 with reward 66.0 in 66 steps
2023-05-16 11:52:04 - RayLog - INFO: - Worker 0 finished episode 92 with reward 84.0 in 84 steps
2023-05-16 11:52:05 - RayLog - INFO: - Worker 1 finished episode 94 with reward 56.0 in 56 steps
2023-05-16 11:52:07 - RayLog - INFO: - Worker 0 finished episode 95 with reward 134.0 in 134 steps
2023-05-16 11:52:08 - RayLog - INFO: - Worker 1 finished episode 96 with reward 115.0 in 115 steps
2023-05-16 11:52:10 - RayLog - INFO: - update_step: 3000, online_eval_reward: 200.000
2023-05-16 11:52:10 - RayLog - INFO: - current update step obtain a better online_eval_reward: 200.000, save the best model!
2023-05-16 11:52:12 - RayLog - INFO: - Worker 0 finished episode 97 with reward 200.0 in 200 steps
2023-05-16 11:52:13 - RayLog - INFO: - Worker 1 finished episode 98 with reward 200.0 in 200 steps
2023-05-16 11:52:15 - RayLog - INFO: - update_step: 3500, online_eval_reward: 200.000
2023-05-16 11:52:16 - RayLog - INFO: - Worker 0 finished episode 99 with reward 200.0 in 200 steps
2023-05-16 11:52:17 - RayLog - INFO: - Worker 1 finished episode 100 with reward 200.0 in 200 steps
2023-05-16 11:52:19 - SimpleLog - INFO: - Finish training! total time consumed: 53.70s
|