Hugging Face's logo

joyrl
/
joyrl-benchmarks

TensorBoard
Model card Files Metrics Community
15
johnjim0816 commited on
Commit
24d6ada
•
1 Parent(s): 6a67ba4

update NoisyDQN Cartpolev1

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. ClassControl/CartPole-v1/{Train_single_CartPole-v1_NoisyDQN_20230518-133737 → Test_single_CartPole-v1_NoisyDQN_20230518-173426}/config.yaml +4 -4
  2. ClassControl/CartPole-v1/{Test_single_CartPole-v1_NoisyDQN_20230518-133824 → Test_single_CartPole-v1_NoisyDQN_20230518-173426}/logs/log.txt +57 -57
  3. ClassControl/CartPole-v1/{Test_single_CartPole-v1_NoisyDQN_20230518-133824/tb_logs/interact/events.out.tfevents.1684388304.JMac.local.86491.0 → Test_single_CartPole-v1_NoisyDQN_20230518-173426/tb_logs/interact/events.out.tfevents.1684402466.JMac.local.93018.0} +1 -1
  4. ClassControl/CartPole-v1/{Train_ray_CartPole-v1_NoisyDQN_20230518-133947/tb_logs/interact/events.out.tfevents.1684388387.JMac.local.86757.0 → Test_single_CartPole-v1_NoisyDQN_20230518-173426/tb_logs/model/events.out.tfevents.1684402466.JMac.local.93018.1} +1 -1
  5. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/.DS_Store +0 -0
  6. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/logs/log.txt +0 -168
  7. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/1000 +0 -0
  8. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/1500 +0 -0
  9. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/2000 +0 -0
  10. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/2500 +0 -0
  11. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/3000 +0 -0
  12. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/3500 +0 -0
  13. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/4000 +0 -0
  14. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/4500 +0 -0
  15. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/500 +0 -0
  16. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/5000 +0 -0
  17. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/5500 +0 -0
  18. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/6000 +0 -0
  19. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/6500 +0 -0
  20. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/7000 +0 -0
  21. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/7500 +0 -0
  22. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/best +0 -0
  23. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/tb_logs/model/events.out.tfevents.1684388398.JMac.local.86815.1 +0 -3
  24. ClassControl/CartPole-v1/{Train_ray_CartPole-v1_NoisyDQN_20230518-133947 → Train_ray_CartPole-v1_NoisyDQN_20230518-172642}/config.yaml +0 -0
  25. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/logs/log.txt +168 -0
  26. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/1000 +3 -0
  27. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/1500 +3 -0
  28. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/2000 +3 -0
  29. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/2500 +3 -0
  30. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/3000 +3 -0
  31. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/3500 +3 -0
  32. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/4000 +3 -0
  33. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/4500 +3 -0
  34. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/500 +3 -0
  35. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/5000 +3 -0
  36. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/5500 +3 -0
  37. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/6000 +3 -0
  38. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/6500 +3 -0
  39. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/7000 +3 -0
  40. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/7500 +3 -0
  41. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/8000 +3 -0
  42. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/best +3 -0
  43. ClassControl/CartPole-v1/{Train_ray_CartPole-v1_NoisyDQN_20230518-133947/tb_logs/model/events.out.tfevents.1684388387.JMac.local.86757.1 → Train_ray_CartPole-v1_NoisyDQN_20230518-172642/tb_logs/interact/events.out.tfevents.1684402002.JMac.local.91984.0} +1 -1
  44. ClassControl/CartPole-v1/{Train_ray_CartPole-v1_NoisyDQN_20230518-133947/tb_logs/interact/events.out.tfevents.1684388398.JMac.local.86815.0 → Train_ray_CartPole-v1_NoisyDQN_20230518-172642/tb_logs/interact/events.out.tfevents.1684402014.JMac.local.92035.0} +1 -1
  45. ClassControl/CartPole-v1/{Test_single_CartPole-v1_NoisyDQN_20230518-133824/tb_logs/model/events.out.tfevents.1684388304.JMac.local.86491.1 → Train_ray_CartPole-v1_NoisyDQN_20230518-172642/tb_logs/model/events.out.tfevents.1684402002.JMac.local.91984.1} +1 -1
  46. ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/tb_logs/model/events.out.tfevents.1684402014.JMac.local.92035.1 +3 -0
  47. ClassControl/CartPole-v1/Train_single_CartPole-v1_NoisyDQN_20230518-133737/logs/log.txt +0 -164
  48. ClassControl/CartPole-v1/Train_single_CartPole-v1_NoisyDQN_20230518-133737/models/1000 +0 -0
  49. ClassControl/CartPole-v1/Train_single_CartPole-v1_NoisyDQN_20230518-133737/models/1500 +0 -0
  50. ClassControl/CartPole-v1/Train_single_CartPole-v1_NoisyDQN_20230518-133737/models/2000 +0 -0
ClassControl/CartPole-v1/{Train_single_CartPole-v1_NoisyDQN_20230518-133737 → Test_single_CartPole-v1_NoisyDQN_20230518-173426}/config.yaml RENAMED
@@ -3,12 +3,12 @@ general_cfg:
3
  collect_traj: false
4
  device: cpu
5
  env_name: gym
6
- load_checkpoint: false
7
  load_model_step: best
8
- load_path: Train_CartPole-v1_DQN_20221026-054757
9
- max_episode: 100
10
  max_step: 200
11
- mode: train
12
  model_save_fre: 500
13
  mp_backend: single
14
  n_learners: 1
 
3
  collect_traj: false
4
  device: cpu
5
  env_name: gym
6
+ load_checkpoint: true
7
  load_model_step: best
8
+ load_path: Train_ray_CartPole-v1_NoisyDQN_20230518-172642
9
+ max_episode: 10
10
  max_step: 200
11
+ mode: test
12
  model_save_fre: 500
13
  mp_backend: single
14
  n_learners: 1
ClassControl/CartPole-v1/{Test_single_CartPole-v1_NoisyDQN_20230518-133824 → Test_single_CartPole-v1_NoisyDQN_20230518-173426}/logs/log.txt RENAMED
@@ -1,57 +1,57 @@
1
- 2023-05-18 13:38:24 - SimpleLog - INFO: - General Configs:
2
- 2023-05-18 13:38:24 - SimpleLog - INFO: - ================================================================================
3
- 2023-05-18 13:38:24 - SimpleLog - INFO: - Name Value Type
4
- 2023-05-18 13:38:24 - SimpleLog - INFO: - env_name gym <class 'str'>
5
- 2023-05-18 13:38:24 - SimpleLog - INFO: - algo_name NoisyDQN <class 'str'>
6
- 2023-05-18 13:38:24 - SimpleLog - INFO: - mode test <class 'str'>
7
- 2023-05-18 13:38:24 - SimpleLog - INFO: - device cpu <class 'str'>
8
- 2023-05-18 13:38:24 - SimpleLog - INFO: - seed 1 <class 'int'>
9
- 2023-05-18 13:38:24 - SimpleLog - INFO: - max_episode 10 <class 'int'>
10
- 2023-05-18 13:38:24 - SimpleLog - INFO: - max_step 200 <class 'int'>
11
- 2023-05-18 13:38:24 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
12
- 2023-05-18 13:38:24 - SimpleLog - INFO: - mp_backend single <class 'str'>
13
- 2023-05-18 13:38:24 - SimpleLog - INFO: - n_workers 2 <class 'int'>
14
- 2023-05-18 13:38:24 - SimpleLog - INFO: - n_learners 1 <class 'int'>
15
- 2023-05-18 13:38:24 - SimpleLog - INFO: - share_buffer 1 <class 'bool'>
16
- 2023-05-18 13:38:24 - SimpleLog - INFO: - online_eval 1 <class 'bool'>
17
- 2023-05-18 13:38:24 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
18
- 2023-05-18 13:38:24 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
19
- 2023-05-18 13:38:24 - SimpleLog - INFO: - load_checkpoint 1 <class 'bool'>
20
- 2023-05-18 13:38:24 - SimpleLog - INFO: - load_path Train_single_CartPole-v1_NoisyDQN_20230518-133737 <class 'str'>
21
- 2023-05-18 13:38:24 - SimpleLog - INFO: - load_model_step best <class 'str'>
22
- 2023-05-18 13:38:24 - SimpleLog - INFO: - ================================================================================
23
- 2023-05-18 13:38:24 - SimpleLog - INFO: - Algo Configs:
24
- 2023-05-18 13:38:24 - SimpleLog - INFO: - ================================================================================
25
- 2023-05-18 13:38:24 - SimpleLog - INFO: - Name Value Type
26
- 2023-05-18 13:38:24 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
27
- 2023-05-18 13:38:24 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
28
- 2023-05-18 13:38:24 - SimpleLog - INFO: - epsilon_decay 500 <class 'int'>
29
- 2023-05-18 13:38:24 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
30
- 2023-05-18 13:38:24 - SimpleLog - INFO: - lr 0.0001 <class 'float'>
31
- 2023-05-18 13:38:24 - SimpleLog - INFO: - buffer_size 100000 <class 'int'>
32
- 2023-05-18 13:38:24 - SimpleLog - INFO: - batch_size 64 <class 'int'>
33
- 2023-05-18 13:38:24 - SimpleLog - INFO: - target_update 4 <class 'int'>
34
- 2023-05-18 13:38:24 - SimpleLog - INFO: - value_layers [{'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}, {'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}] <class 'str'>
35
- 2023-05-18 13:38:24 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'>
36
- 2023-05-18 13:38:24 - SimpleLog - INFO: - ================================================================================
37
- 2023-05-18 13:38:24 - SimpleLog - INFO: - Env Configs:
38
- 2023-05-18 13:38:24 - SimpleLog - INFO: - ================================================================================
39
- 2023-05-18 13:38:24 - SimpleLog - INFO: - Name Value Type
40
- 2023-05-18 13:38:24 - SimpleLog - INFO: - id CartPole-v1 <class 'str'>
41
- 2023-05-18 13:38:24 - SimpleLog - INFO: - render_mode None <class 'str'>
42
- 2023-05-18 13:38:24 - SimpleLog - INFO: - wrapper None <class 'str'>
43
- 2023-05-18 13:38:24 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
44
- 2023-05-18 13:38:24 - SimpleLog - INFO: - ================================================================================
45
- 2023-05-18 13:38:24 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2)
46
- 2023-05-18 13:38:24 - SimpleLog - INFO: - Start testing!
47
- 2023-05-18 13:38:24 - SimpleLog - INFO: - episode: 0, ep_reward: 200.0, ep_step: 200
48
- 2023-05-18 13:38:24 - SimpleLog - INFO: - episode: 1, ep_reward: 200.0, ep_step: 200
49
- 2023-05-18 13:38:24 - SimpleLog - INFO: - episode: 2, ep_reward: 200.0, ep_step: 200
50
- 2023-05-18 13:38:24 - SimpleLog - INFO: - episode: 3, ep_reward: 200.0, ep_step: 200
51
- 2023-05-18 13:38:24 - SimpleLog - INFO: - episode: 4, ep_reward: 200.0, ep_step: 200
52
- 2023-05-18 13:38:25 - SimpleLog - INFO: - episode: 5, ep_reward: 200.0, ep_step: 200
53
- 2023-05-18 13:38:25 - SimpleLog - INFO: - episode: 6, ep_reward: 200.0, ep_step: 200
54
- 2023-05-18 13:38:25 - SimpleLog - INFO: - episode: 7, ep_reward: 200.0, ep_step: 200
55
- 2023-05-18 13:38:25 - SimpleLog - INFO: - episode: 8, ep_reward: 200.0, ep_step: 200
56
- 2023-05-18 13:38:25 - SimpleLog - INFO: - episode: 9, ep_reward: 200.0, ep_step: 200
57
- 2023-05-18 13:38:25 - SimpleLog - INFO: - Finish testing! total time consumed: 0.26s
 
1
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - General Configs:
2
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - ================================================================================
3
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - Name Value Type
4
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - env_name gym <class 'str'>
5
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - algo_name NoisyDQN <class 'str'>
6
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - mode test <class 'str'>
7
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - device cpu <class 'str'>
8
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - seed 1 <class 'int'>
9
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - max_episode 10 <class 'int'>
10
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - max_step 200 <class 'int'>
11
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
12
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - mp_backend single <class 'str'>
13
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - n_workers 2 <class 'int'>
14
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - n_learners 1 <class 'int'>
15
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - share_buffer 1 <class 'bool'>
16
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - online_eval 1 <class 'bool'>
17
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
18
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
19
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - load_checkpoint 1 <class 'bool'>
20
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - load_path Train_ray_CartPole-v1_NoisyDQN_20230518-172642 <class 'str'>
21
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - load_model_step best <class 'str'>
22
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - ================================================================================
23
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - Algo Configs:
24
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - ================================================================================
25
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - Name Value Type
26
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
27
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
28
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - epsilon_decay 500 <class 'int'>
29
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
30
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - lr 0.0001 <class 'float'>
31
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - buffer_size 100000 <class 'int'>
32
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - batch_size 64 <class 'int'>
33
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - target_update 4 <class 'int'>
34
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - value_layers [{'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}, {'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}] <class 'str'>
35
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'>
36
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - ================================================================================
37
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - Env Configs:
38
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - ================================================================================
39
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - Name Value Type
40
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - id CartPole-v1 <class 'str'>
41
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - render_mode None <class 'str'>
42
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - wrapper None <class 'str'>
43
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
44
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - ================================================================================
45
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2)
46
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - Start testing!
47
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 0, ep_reward: 200.0, ep_step: 200
48
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 1, ep_reward: 200.0, ep_step: 200
49
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 2, ep_reward: 200.0, ep_step: 200
50
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 3, ep_reward: 200.0, ep_step: 200
51
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 4, ep_reward: 200.0, ep_step: 200
52
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 5, ep_reward: 200.0, ep_step: 200
53
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 6, ep_reward: 200.0, ep_step: 200
54
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 7, ep_reward: 200.0, ep_step: 200
55
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 8, ep_reward: 200.0, ep_step: 200
56
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - episode: 9, ep_reward: 200.0, ep_step: 200
57
+ 2023-05-18 17:34:26 - SimpleLog - INFO: - Finish testing! total time consumed: 0.40s
ClassControl/CartPole-v1/{Test_single_CartPole-v1_NoisyDQN_20230518-133824/tb_logs/interact/events.out.tfevents.1684388304.JMac.local.86491.0 → Test_single_CartPole-v1_NoisyDQN_20230518-173426/tb_logs/interact/events.out.tfevents.1684402466.JMac.local.93018.0} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a888c1468be41447e5aec55b8e1cac6bb28792c9a6d11eb37f43f8db1a1ec315
3
  size 1056
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:258880c4de60e1559379aafb2b9a88c8acae2bc5837b2aac244ad616e98826bb
3
  size 1056
ClassControl/CartPole-v1/{Train_ray_CartPole-v1_NoisyDQN_20230518-133947/tb_logs/interact/events.out.tfevents.1684388387.JMac.local.86757.0 → Test_single_CartPole-v1_NoisyDQN_20230518-173426/tb_logs/model/events.out.tfevents.1684402466.JMac.local.93018.1} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e571710bb077a186d5841f41570cb79c0d8364b23ad6c763b1b9296ceaf0284f
3
  size 40
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf8f09f32b8bf3b696ef8955e087d48e6aa80c7f6a142450bf7bcc7d25282a26
3
  size 40
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/.DS_Store DELETED
Binary file (6.15 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/logs/log.txt DELETED
@@ -1,168 +0,0 @@
1
- 2023-05-18 13:39:47 - SimpleLog - INFO: - General Configs:
2
- 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
3
- 2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type
4
- 2023-05-18 13:39:47 - SimpleLog - INFO: - env_name gym <class 'str'>
5
- 2023-05-18 13:39:47 - SimpleLog - INFO: - algo_name NoisyDQN <class 'str'>
6
- 2023-05-18 13:39:47 - SimpleLog - INFO: - mode train <class 'str'>
7
- 2023-05-18 13:39:47 - SimpleLog - INFO: - device cpu <class 'str'>
8
- 2023-05-18 13:39:47 - SimpleLog - INFO: - seed 1 <class 'int'>
9
- 2023-05-18 13:39:47 - SimpleLog - INFO: - max_episode 100 <class 'int'>
10
- 2023-05-18 13:39:47 - SimpleLog - INFO: - max_step 200 <class 'int'>
11
- 2023-05-18 13:39:47 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
12
- 2023-05-18 13:39:47 - SimpleLog - INFO: - mp_backend ray <class 'str'>
13
- 2023-05-18 13:39:47 - SimpleLog - INFO: - n_workers 2 <class 'int'>
14
- 2023-05-18 13:39:47 - SimpleLog - INFO: - n_learners 1 <class 'int'>
15
- 2023-05-18 13:39:47 - SimpleLog - INFO: - share_buffer 1 <class 'bool'>
16
- 2023-05-18 13:39:47 - SimpleLog - INFO: - online_eval 1 <class 'bool'>
17
- 2023-05-18 13:39:47 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
18
- 2023-05-18 13:39:47 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
19
- 2023-05-18 13:39:47 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'>
20
- 2023-05-18 13:39:47 - SimpleLog - INFO: - load_path Train_single_CartPole-v1_NoisyDQN_20230518-133737 <class 'str'>
21
- 2023-05-18 13:39:47 - SimpleLog - INFO: - load_model_step best <class 'str'>
22
- 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
23
- 2023-05-18 13:39:47 - SimpleLog - INFO: - Algo Configs:
24
- 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
25
- 2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type
26
- 2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
27
- 2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
28
- 2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_decay 500 <class 'int'>
29
- 2023-05-18 13:39:47 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
30
- 2023-05-18 13:39:47 - SimpleLog - INFO: - lr 0.0001 <class 'float'>
31
- 2023-05-18 13:39:47 - SimpleLog - INFO: - buffer_size 100000 <class 'int'>
32
- 2023-05-18 13:39:47 - SimpleLog - INFO: - batch_size 64 <class 'int'>
33
- 2023-05-18 13:39:47 - SimpleLog - INFO: - target_update 4 <class 'int'>
34
- 2023-05-18 13:39:47 - SimpleLog - INFO: - value_layers [{'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}, {'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}] <class 'str'>
35
- 2023-05-18 13:39:47 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'>
36
- 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
37
- 2023-05-18 13:39:47 - SimpleLog - INFO: - Env Configs:
38
- 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
39
- 2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type
40
- 2023-05-18 13:39:47 - SimpleLog - INFO: - id CartPole-v1 <class 'str'>
41
- 2023-05-18 13:39:47 - SimpleLog - INFO: - render_mode None <class 'str'>
42
- 2023-05-18 13:39:47 - SimpleLog - INFO: - wrapper None <class 'str'>
43
- 2023-05-18 13:39:47 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
44
- 2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
45
- 2023-05-18 13:39:54 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2)
46
- 2023-05-18 13:39:59 - RayLog - INFO: - Worker 0 finished episode 0 with reward 14.0 in 14 steps
47
- 2023-05-18 13:39:59 - RayLog - INFO: - Worker 1 finished episode 0 with reward 22.0 in 22 steps
48
- 2023-05-18 13:40:01 - RayLog - INFO: - Worker 1 finished episode 2 with reward 33.0 in 33 steps
49
- 2023-05-18 13:40:01 - RayLog - INFO: - Worker 0 finished episode 2 with reward 38.0 in 38 steps
50
- 2023-05-18 13:40:01 - RayLog - INFO: - Worker 0 finished episode 4 with reward 15.0 in 15 steps
51
- 2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 5 with reward 26.0 in 26 steps
52
- 2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 3 with reward 53.0 in 53 steps
53
- 2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 6 with reward 11.0 in 11 steps
54
- 2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 7 with reward 12.0 in 12 steps
55
- 2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 8 with reward 14.0 in 14 steps
56
- 2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 9 with reward 13.0 in 13 steps
57
- 2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 10 with reward 12.0 in 12 steps
58
- 2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 11 with reward 10.0 in 10 steps
59
- 2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 12 with reward 9.0 in 9 steps
60
- 2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 13 with reward 12.0 in 12 steps
61
- 2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 14 with reward 11.0 in 11 steps
62
- 2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 15 with reward 11.0 in 11 steps
63
- 2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 16 with reward 9.0 in 9 steps
64
- 2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 17 with reward 20.0 in 20 steps
65
- 2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 18 with reward 20.0 in 20 steps
66
- 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 20 with reward 12.0 in 12 steps
67
- 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 21 with reward 13.0 in 13 steps
68
- 2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 19 with reward 31.0 in 31 steps
69
- 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 22 with reward 10.0 in 10 steps
70
- 2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 23 with reward 9.0 in 9 steps
71
- 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 24 with reward 12.0 in 12 steps
72
- 2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 25 with reward 11.0 in 11 steps
73
- 2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 26 with reward 12.0 in 12 steps
74
- 2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 27 with reward 12.0 in 12 steps
75
- 2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 28 with reward 13.0 in 13 steps
76
- 2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 29 with reward 13.0 in 13 steps
77
- 2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 30 with reward 12.0 in 12 steps
78
- 2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 31 with reward 14.0 in 14 steps
79
- 2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 32 with reward 14.0 in 14 steps
80
- 2023-05-18 13:40:07 - RayLog - INFO: - learner id: 0, update_step: 500, online_eval_reward: 10.000
81
- 2023-05-18 13:40:07 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 10.000, save the best model!
82
- 2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 33 with reward 14.0 in 14 steps
83
- 2023-05-18 13:40:07 - RayLog - INFO: - Worker 0 finished episode 34 with reward 11.0 in 11 steps
84
- 2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 35 with reward 13.0 in 13 steps
85
- 2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 37 with reward 10.0 in 10 steps
86
- 2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 38 with reward 11.0 in 11 steps
87
- 2023-05-18 13:40:07 - RayLog - INFO: - Worker 0 finished episode 36 with reward 29.0 in 29 steps
88
- 2023-05-18 13:40:08 - RayLog - INFO: - Worker 1 finished episode 39 with reward 9.0 in 9 steps
89
- 2023-05-18 13:40:08 - RayLog - INFO: - Worker 0 finished episode 40 with reward 11.0 in 11 steps
90
- 2023-05-18 13:40:08 - RayLog - INFO: - Worker 0 finished episode 42 with reward 35.0 in 35 steps
91
- 2023-05-18 13:40:08 - RayLog - INFO: - Worker 1 finished episode 41 with reward 49.0 in 49 steps
92
- 2023-05-18 13:40:09 - RayLog - INFO: - Worker 0 finished episode 43 with reward 36.0 in 36 steps
93
- 2023-05-18 13:40:09 - RayLog - INFO: - Worker 1 finished episode 44 with reward 35.0 in 35 steps
94
- 2023-05-18 13:40:10 - RayLog - INFO: - Worker 0 finished episode 45 with reward 42.0 in 42 steps
95
- 2023-05-18 13:40:10 - RayLog - INFO: - Worker 1 finished episode 46 with reward 44.0 in 44 steps
96
- 2023-05-18 13:40:10 - RayLog - INFO: - Worker 0 finished episode 47 with reward 36.0 in 36 steps
97
- 2023-05-18 13:40:11 - RayLog - INFO: - Worker 1 finished episode 48 with reward 44.0 in 44 steps
98
- 2023-05-18 13:40:11 - RayLog - INFO: - Worker 0 finished episode 49 with reward 37.0 in 37 steps
99
- 2023-05-18 13:40:12 - RayLog - INFO: - learner id: 0, update_step: 1000, online_eval_reward: 51.000
100
- 2023-05-18 13:40:12 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 51.000, save the best model!
101
- 2023-05-18 13:40:13 - RayLog - INFO: - Worker 0 finished episode 51 with reward 76.0 in 76 steps
102
- 2023-05-18 13:40:13 - RayLog - INFO: - Worker 1 finished episode 50 with reward 109.0 in 109 steps
103
- 2023-05-18 13:40:15 - RayLog - INFO: - Worker 0 finished episode 52 with reward 119.0 in 119 steps
104
- 2023-05-18 13:40:15 - RayLog - INFO: - Worker 1 finished episode 53 with reward 120.0 in 120 steps
105
- 2023-05-18 13:40:16 - RayLog - INFO: - Worker 0 finished episode 54 with reward 57.0 in 57 steps
106
- 2023-05-18 13:40:16 - RayLog - INFO: - Worker 1 finished episode 55 with reward 44.0 in 44 steps
107
- 2023-05-18 13:40:16 - RayLog - INFO: - learner id: 0, update_step: 1500, online_eval_reward: 53.000
108
- 2023-05-18 13:40:16 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 53.000, save the best model!
109
- 2023-05-18 13:40:17 - RayLog - INFO: - Worker 1 finished episode 57 with reward 43.0 in 43 steps
110
- 2023-05-18 13:40:18 - RayLog - INFO: - Worker 0 finished episode 56 with reward 72.0 in 72 steps
111
- 2023-05-18 13:40:18 - RayLog - INFO: - Worker 1 finished episode 58 with reward 35.0 in 35 steps
112
- 2023-05-18 13:40:19 - RayLog - INFO: - Worker 1 finished episode 60 with reward 48.0 in 48 steps
113
- 2023-05-18 13:40:19 - RayLog - INFO: - Worker 0 finished episode 59 with reward 64.0 in 64 steps
114
- 2023-05-18 13:40:19 - RayLog - INFO: - Worker 1 finished episode 61 with reward 36.0 in 36 steps
115
- 2023-05-18 13:40:20 - RayLog - INFO: - Worker 0 finished episode 62 with reward 57.0 in 57 steps
116
- 2023-05-18 13:40:20 - RayLog - INFO: - Worker 1 finished episode 63 with reward 42.0 in 42 steps
117
- 2023-05-18 13:40:21 - RayLog - INFO: - learner id: 0, update_step: 2000, online_eval_reward: 143.000
118
- 2023-05-18 13:40:21 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 143.000, save the best model!
119
- 2023-05-18 13:40:23 - RayLog - INFO: - Worker 1 finished episode 65 with reward 100.0 in 100 steps
120
- 2023-05-18 13:40:23 - RayLog - INFO: - Worker 0 finished episode 64 with reward 153.0 in 153 steps
121
- 2023-05-18 13:40:24 - RayLog - INFO: - Worker 1 finished episode 66 with reward 61.0 in 61 steps
122
- 2023-05-18 13:40:25 - RayLog - INFO: - Worker 0 finished episode 67 with reward 111.0 in 111 steps
123
- 2023-05-18 13:40:26 - RayLog - INFO: - Worker 1 finished episode 68 with reward 103.0 in 103 steps
124
- 2023-05-18 13:40:27 - RayLog - INFO: - learner id: 0, update_step: 2500, online_eval_reward: 85.000
125
- 2023-05-18 13:40:28 - RayLog - INFO: - Worker 1 finished episode 70 with reward 127.0 in 127 steps
126
- 2023-05-18 13:40:29 - RayLog - INFO: - Worker 0 finished episode 69 with reward 164.0 in 164 steps
127
- 2023-05-18 13:40:32 - RayLog - INFO: - learner id: 0, update_step: 3000, online_eval_reward: 74.000
128
- 2023-05-18 13:40:32 - RayLog - INFO: - Worker 1 finished episode 71 with reward 200.0 in 200 steps
129
- 2023-05-18 13:40:32 - RayLog - INFO: - Worker 0 finished episode 72 with reward 186.0 in 186 steps
130
- 2023-05-18 13:40:35 - RayLog - INFO: - Worker 1 finished episode 73 with reward 114.0 in 114 steps
131
- 2023-05-18 13:40:35 - RayLog - INFO: - Worker 0 finished episode 74 with reward 109.0 in 109 steps
132
- 2023-05-18 13:40:36 - RayLog - INFO: - Worker 1 finished episode 75 with reward 72.0 in 72 steps
133
- 2023-05-18 13:40:37 - RayLog - INFO: - learner id: 0, update_step: 3500, online_eval_reward: 83.000
134
- 2023-05-18 13:40:37 - RayLog - INFO: - Worker 0 finished episode 76 with reward 96.0 in 96 steps
135
- 2023-05-18 13:40:37 - RayLog - INFO: - Worker 1 finished episode 77 with reward 62.0 in 62 steps
136
- 2023-05-18 13:40:38 - RayLog - INFO: - Worker 0 finished episode 78 with reward 88.0 in 88 steps
137
- 2023-05-18 13:40:39 - RayLog - INFO: - Worker 1 finished episode 79 with reward 70.0 in 70 steps
138
- 2023-05-18 13:40:40 - RayLog - INFO: - Worker 1 finished episode 81 with reward 77.0 in 77 steps
139
- 2023-05-18 13:40:42 - RayLog - INFO: - learner id: 0, update_step: 4000, online_eval_reward: 123.000
140
- 2023-05-18 13:40:42 - RayLog - INFO: - Worker 0 finished episode 80 with reward 181.0 in 181 steps
141
- 2023-05-18 13:40:42 - RayLog - INFO: - Worker 1 finished episode 82 with reward 105.0 in 105 steps
142
- 2023-05-18 13:40:46 - RayLog - INFO: - Worker 0 finished episode 83 with reward 200.0 in 200 steps
143
- 2023-05-18 13:40:47 - RayLog - INFO: - Worker 1 finished episode 84 with reward 200.0 in 200 steps
144
- 2023-05-18 13:40:47 - RayLog - INFO: - learner id: 0, update_step: 4500, online_eval_reward: 200.000
145
- 2023-05-18 13:40:47 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 200.000, save the best model!
146
- 2023-05-18 13:40:51 - RayLog - INFO: - Worker 0 finished episode 85 with reward 200.0 in 200 steps
147
- 2023-05-18 13:40:51 - RayLog - INFO: - Worker 1 finished episode 86 with reward 200.0 in 200 steps
148
- 2023-05-18 13:40:53 - RayLog - INFO: - learner id: 0, update_step: 5000, online_eval_reward: 200.000
149
- 2023-05-18 13:40:56 - RayLog - INFO: - Worker 0 finished episode 87 with reward 200.0 in 200 steps
150
- 2023-05-18 13:40:56 - RayLog - INFO: - Worker 1 finished episode 88 with reward 200.0 in 200 steps
151
- 2023-05-18 13:40:59 - RayLog - INFO: - learner id: 0, update_step: 5500, online_eval_reward: 200.000
152
- 2023-05-18 13:41:00 - RayLog - INFO: - Worker 0 finished episode 89 with reward 200.0 in 200 steps
153
- 2023-05-18 13:41:01 - RayLog - INFO: - Worker 1 finished episode 90 with reward 200.0 in 200 steps
154
- 2023-05-18 13:41:05 - RayLog - INFO: - learner id: 0, update_step: 6000, online_eval_reward: 200.000
155
- 2023-05-18 13:41:05 - RayLog - INFO: - Worker 0 finished episode 91 with reward 200.0 in 200 steps
156
- 2023-05-18 13:41:05 - RayLog - INFO: - Worker 1 finished episode 92 with reward 200.0 in 200 steps
157
- 2023-05-18 13:41:10 - RayLog - INFO: - Worker 0 finished episode 93 with reward 200.0 in 200 steps
158
- 2023-05-18 13:41:10 - RayLog - INFO: - Worker 1 finished episode 94 with reward 200.0 in 200 steps
159
- 2023-05-18 13:41:11 - RayLog - INFO: - learner id: 0, update_step: 6500, online_eval_reward: 200.000
160
- 2023-05-18 13:41:15 - RayLog - INFO: - Worker 0 finished episode 95 with reward 200.0 in 200 steps
161
- 2023-05-18 13:41:15 - RayLog - INFO: - Worker 1 finished episode 96 with reward 200.0 in 200 steps
162
- 2023-05-18 13:41:17 - RayLog - INFO: - learner id: 0, update_step: 7000, online_eval_reward: 200.000
163
- 2023-05-18 13:41:19 - RayLog - INFO: - Worker 0 finished episode 97 with reward 200.0 in 200 steps
164
- 2023-05-18 13:41:20 - RayLog - INFO: - Worker 1 finished episode 98 with reward 200.0 in 200 steps
165
- 2023-05-18 13:41:23 - RayLog - INFO: - learner id: 0, update_step: 7500, online_eval_reward: 200.000
166
- 2023-05-18 13:41:24 - RayLog - INFO: - Worker 0 finished episode 99 with reward 200.0 in 200 steps
167
- 2023-05-18 13:41:25 - RayLog - INFO: - Worker 1 finished episode 100 with reward 200.0 in 200 steps
168
- 2023-05-18 13:41:28 - SimpleLog - INFO: - Finish training! total time consumed: 100.97s
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/1000 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/1500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/2000 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/2500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/3000 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/3500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/4000 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/4500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/5000 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/5500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/6000 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/6500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/7000 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/7500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/models/best DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-133947/tb_logs/model/events.out.tfevents.1684388398.JMac.local.86815.1 DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:8fd21541f93e709c2d2db525eb5b8fb4569896f9ec747de14ab5193752825d5f
3
- size 375106
 
 
 
 
ClassControl/CartPole-v1/{Train_ray_CartPole-v1_NoisyDQN_20230518-133947 → Train_ray_CartPole-v1_NoisyDQN_20230518-172642}/config.yaml RENAMED
File without changes
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/logs/log.txt ADDED
@@ -0,0 +1,168 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - General Configs:
2
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - ================================================================================
3
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - Name Value Type
4
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - env_name gym <class 'str'>
5
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - algo_name NoisyDQN <class 'str'>
6
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - mode train <class 'str'>
7
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - device cpu <class 'str'>
8
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - seed 1 <class 'int'>
9
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - max_episode 100 <class 'int'>
10
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - max_step 200 <class 'int'>
11
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
12
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - mp_backend ray <class 'str'>
13
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - n_workers 2 <class 'int'>
14
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - n_learners 1 <class 'int'>
15
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - share_buffer 1 <class 'bool'>
16
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - online_eval 1 <class 'bool'>
17
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
18
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
19
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'>
20
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - load_path Train_single_CartPole-v1_NoisyDQN_20230518-133737 <class 'str'>
21
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - load_model_step best <class 'str'>
22
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - ================================================================================
23
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - Algo Configs:
24
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - ================================================================================
25
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - Name Value Type
26
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
27
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
28
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - epsilon_decay 500 <class 'int'>
29
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
30
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - lr 0.0001 <class 'float'>
31
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - buffer_size 100000 <class 'int'>
32
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - batch_size 64 <class 'int'>
33
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - target_update 4 <class 'int'>
34
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - value_layers [{'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}, {'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}] <class 'str'>
35
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'>
36
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - ================================================================================
37
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - Env Configs:
38
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - ================================================================================
39
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - Name Value Type
40
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - id CartPole-v1 <class 'str'>
41
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - render_mode None <class 'str'>
42
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - wrapper None <class 'str'>
43
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
44
+ 2023-05-18 17:26:42 - SimpleLog - INFO: - ================================================================================
45
+ 2023-05-18 17:26:49 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2)
46
+ 2023-05-18 17:26:56 - RayLog - INFO: - Worker 1 finished episode 0 with reward 41.0 in 41 steps
47
+ 2023-05-18 17:26:57 - RayLog - INFO: - Worker 0 finished episode 0 with reward 55.0 in 55 steps
48
+ 2023-05-18 17:26:57 - RayLog - INFO: - Worker 1 finished episode 1 with reward 15.0 in 15 steps
49
+ 2023-05-18 17:26:57 - RayLog - INFO: - Worker 0 finished episode 2 with reward 17.0 in 17 steps
50
+ 2023-05-18 17:26:57 - RayLog - INFO: - Worker 1 finished episode 3 with reward 24.0 in 24 steps
51
+ 2023-05-18 17:26:57 - RayLog - INFO: - Worker 0 finished episode 4 with reward 12.0 in 12 steps
52
+ 2023-05-18 17:26:57 - RayLog - INFO: - Worker 0 finished episode 6 with reward 9.0 in 9 steps
53
+ 2023-05-18 17:26:57 - RayLog - INFO: - Worker 1 finished episode 5 with reward 17.0 in 17 steps
54
+ 2023-05-18 17:26:58 - RayLog - INFO: - Worker 0 finished episode 7 with reward 11.0 in 11 steps
55
+ 2023-05-18 17:26:58 - RayLog - INFO: - Worker 1 finished episode 8 with reward 11.0 in 11 steps
56
+ 2023-05-18 17:26:58 - RayLog - INFO: - Worker 0 finished episode 9 with reward 11.0 in 11 steps
57
+ 2023-05-18 17:26:58 - RayLog - INFO: - Worker 1 finished episode 10 with reward 13.0 in 13 steps
58
+ 2023-05-18 17:26:58 - RayLog - INFO: - Worker 0 finished episode 11 with reward 11.0 in 11 steps
59
+ 2023-05-18 17:26:58 - RayLog - INFO: - Worker 1 finished episode 12 with reward 16.0 in 16 steps
60
+ 2023-05-18 17:26:58 - RayLog - INFO: - Worker 0 finished episode 13 with reward 12.0 in 12 steps
61
+ 2023-05-18 17:26:59 - RayLog - INFO: - Worker 0 finished episode 15 with reward 11.0 in 11 steps
62
+ 2023-05-18 17:26:59 - RayLog - INFO: - Worker 1 finished episode 14 with reward 13.0 in 13 steps
63
+ 2023-05-18 17:26:59 - RayLog - INFO: - Worker 0 finished episode 16 with reward 14.0 in 14 steps
64
+ 2023-05-18 17:26:59 - RayLog - INFO: - Worker 0 finished episode 18 with reward 10.0 in 10 steps
65
+ 2023-05-18 17:26:59 - RayLog - INFO: - Worker 1 finished episode 17 with reward 24.0 in 24 steps
66
+ 2023-05-18 17:26:59 - RayLog - INFO: - Worker 0 finished episode 19 with reward 12.0 in 12 steps
67
+ 2023-05-18 17:27:00 - RayLog - INFO: - Worker 1 finished episode 20 with reward 17.0 in 17 steps
68
+ 2023-05-18 17:27:00 - RayLog - INFO: - Worker 0 finished episode 21 with reward 14.0 in 14 steps
69
+ 2023-05-18 17:27:00 - RayLog - INFO: - Worker 1 finished episode 22 with reward 9.0 in 9 steps
70
+ 2023-05-18 17:27:00 - RayLog - INFO: - Worker 1 finished episode 24 with reward 9.0 in 9 steps
71
+ 2023-05-18 17:27:00 - RayLog - INFO: - Worker 0 finished episode 23 with reward 17.0 in 17 steps
72
+ 2023-05-18 17:27:00 - RayLog - INFO: - Worker 1 finished episode 25 with reward 13.0 in 13 steps
73
+ 2023-05-18 17:27:01 - RayLog - INFO: - Worker 0 finished episode 26 with reward 15.0 in 15 steps
74
+ 2023-05-18 17:27:01 - RayLog - INFO: - Worker 1 finished episode 27 with reward 13.0 in 13 steps
75
+ 2023-05-18 17:27:01 - RayLog - INFO: - Worker 0 finished episode 28 with reward 11.0 in 11 steps
76
+ 2023-05-18 17:27:01 - RayLog - INFO: - Worker 1 finished episode 29 with reward 9.0 in 9 steps
77
+ 2023-05-18 17:27:01 - RayLog - INFO: - Worker 0 finished episode 30 with reward 15.0 in 15 steps
78
+ 2023-05-18 17:27:01 - RayLog - INFO: - Worker 1 finished episode 31 with reward 16.0 in 16 steps
79
+ 2023-05-18 17:27:01 - RayLog - INFO: - Worker 0 finished episode 32 with reward 10.0 in 10 steps
80
+ 2023-05-18 17:27:01 - RayLog - INFO: - Worker 1 finished episode 33 with reward 9.0 in 9 steps
81
+ 2023-05-18 17:27:02 - RayLog - INFO: - Worker 0 finished episode 34 with reward 10.0 in 10 steps
82
+ 2023-05-18 17:27:02 - RayLog - INFO: - Worker 1 finished episode 35 with reward 9.0 in 9 steps
83
+ 2023-05-18 17:27:03 - RayLog - INFO: - learner id: 0, update_step: 500, online_eval_reward: 10.000
84
+ 2023-05-18 17:27:03 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 10.000, save the best model!
85
+ 2023-05-18 17:27:03 - RayLog - INFO: - Worker 0 finished episode 36 with reward 12.0 in 12 steps
86
+ 2023-05-18 17:27:03 - RayLog - INFO: - Worker 1 finished episode 37 with reward 17.0 in 17 steps
87
+ 2023-05-18 17:27:05 - RayLog - INFO: - Worker 0 finished episode 38 with reward 92.0 in 92 steps
88
+ 2023-05-18 17:27:06 - RayLog - INFO: - Worker 1 finished episode 39 with reward 99.0 in 99 steps
89
+ 2023-05-18 17:27:06 - RayLog - INFO: - Worker 0 finished episode 40 with reward 26.0 in 26 steps
90
+ 2023-05-18 17:27:07 - RayLog - INFO: - Worker 0 finished episode 42 with reward 23.0 in 23 steps
91
+ 2023-05-18 17:27:07 - RayLog - INFO: - Worker 1 finished episode 41 with reward 40.0 in 40 steps
92
+ 2023-05-18 17:27:07 - RayLog - INFO: - Worker 0 finished episode 43 with reward 21.0 in 21 steps
93
+ 2023-05-18 17:27:07 - RayLog - INFO: - Worker 1 finished episode 44 with reward 32.0 in 32 steps
94
+ 2023-05-18 17:27:08 - RayLog - INFO: - Worker 0 finished episode 45 with reward 20.0 in 20 steps
95
+ 2023-05-18 17:27:08 - RayLog - INFO: - Worker 1 finished episode 46 with reward 28.0 in 28 steps
96
+ 2023-05-18 17:27:08 - RayLog - INFO: - Worker 0 finished episode 47 with reward 27.0 in 27 steps
97
+ 2023-05-18 17:27:09 - RayLog - INFO: - Worker 0 finished episode 49 with reward 21.0 in 21 steps
98
+ 2023-05-18 17:27:09 - RayLog - INFO: - Worker 1 finished episode 48 with reward 30.0 in 30 steps
99
+ 2023-05-18 17:27:09 - RayLog - INFO: - learner id: 0, update_step: 1000, online_eval_reward: 25.000
100
+ 2023-05-18 17:27:09 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 25.000, save the best model!
101
+ 2023-05-18 17:27:09 - RayLog - INFO: - Worker 1 finished episode 51 with reward 18.0 in 18 steps
102
+ 2023-05-18 17:27:09 - RayLog - INFO: - Worker 0 finished episode 50 with reward 24.0 in 24 steps
103
+ 2023-05-18 17:27:10 - RayLog - INFO: - Worker 0 finished episode 53 with reward 24.0 in 24 steps
104
+ 2023-05-18 17:27:10 - RayLog - INFO: - Worker 1 finished episode 52 with reward 29.0 in 29 steps
105
+ 2023-05-18 17:27:11 - RayLog - INFO: - Worker 0 finished episode 54 with reward 24.0 in 24 steps
106
+ 2023-05-18 17:27:11 - RayLog - INFO: - Worker 1 finished episode 55 with reward 29.0 in 29 steps
107
+ 2023-05-18 17:27:11 - RayLog - INFO: - Worker 0 finished episode 56 with reward 23.0 in 23 steps
108
+ 2023-05-18 17:27:11 - RayLog - INFO: - Worker 1 finished episode 57 with reward 33.0 in 33 steps
109
+ 2023-05-18 17:27:12 - RayLog - INFO: - Worker 0 finished episode 58 with reward 31.0 in 31 steps
110
+ 2023-05-18 17:27:12 - RayLog - INFO: - Worker 0 finished episode 60 with reward 25.0 in 25 steps
111
+ 2023-05-18 17:27:12 - RayLog - INFO: - Worker 1 finished episode 59 with reward 39.0 in 39 steps
112
+ 2023-05-18 17:27:13 - RayLog - INFO: - Worker 0 finished episode 61 with reward 33.0 in 33 steps
113
+ 2023-05-18 17:27:14 - RayLog - INFO: - Worker 1 finished episode 62 with reward 60.0 in 60 steps
114
+ 2023-05-18 17:27:14 - RayLog - INFO: - Worker 0 finished episode 63 with reward 39.0 in 39 steps
115
+ 2023-05-18 17:27:15 - RayLog - INFO: - Worker 0 finished episode 65 with reward 37.0 in 37 steps
116
+ 2023-05-18 17:27:16 - RayLog - INFO: - learner id: 0, update_step: 1500, online_eval_reward: 35.000
117
+ 2023-05-18 17:27:16 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 35.000, save the best model!
118
+ 2023-05-18 17:27:16 - RayLog - INFO: - Worker 1 finished episode 64 with reward 58.0 in 58 steps
119
+ 2023-05-18 17:27:17 - RayLog - INFO: - Worker 0 finished episode 66 with reward 53.0 in 53 steps
120
+ 2023-05-18 17:27:19 - RayLog - INFO: - Worker 1 finished episode 67 with reward 99.0 in 99 steps
121
+ 2023-05-18 17:27:19 - RayLog - INFO: - Worker 0 finished episode 68 with reward 91.0 in 91 steps
122
+ 2023-05-18 17:27:21 - RayLog - INFO: - Worker 1 finished episode 69 with reward 106.0 in 106 steps
123
+ 2023-05-18 17:27:23 - RayLog - INFO: - learner id: 0, update_step: 2000, online_eval_reward: 200.000
124
+ 2023-05-18 17:27:23 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 200.000, save the best model!
125
+ 2023-05-18 17:27:25 - RayLog - INFO: - Worker 0 finished episode 70 with reward 200.0 in 200 steps
126
+ 2023-05-18 17:27:27 - RayLog - INFO: - Worker 1 finished episode 71 with reward 200.0 in 200 steps
127
+ 2023-05-18 17:27:29 - RayLog - INFO: - learner id: 0, update_step: 2500, online_eval_reward: 200.000
128
+ 2023-05-18 17:27:30 - RayLog - INFO: - Worker 0 finished episode 72 with reward 200.0 in 200 steps
129
+ 2023-05-18 17:27:32 - RayLog - INFO: - Worker 1 finished episode 73 with reward 200.0 in 200 steps
130
+ 2023-05-18 17:27:36 - RayLog - INFO: - Worker 0 finished episode 74 with reward 200.0 in 200 steps
131
+ 2023-05-18 17:27:37 - RayLog - INFO: - learner id: 0, update_step: 3000, online_eval_reward: 200.000
132
+ 2023-05-18 17:27:39 - RayLog - INFO: - Worker 1 finished episode 75 with reward 200.0 in 200 steps
133
+ 2023-05-18 17:27:42 - RayLog - INFO: - Worker 0 finished episode 76 with reward 200.0 in 200 steps
134
+ 2023-05-18 17:27:44 - RayLog - INFO: - learner id: 0, update_step: 3500, online_eval_reward: 200.000
135
+ 2023-05-18 17:27:44 - RayLog - INFO: - Worker 1 finished episode 77 with reward 200.0 in 200 steps
136
+ 2023-05-18 17:27:48 - RayLog - INFO: - Worker 0 finished episode 78 with reward 200.0 in 200 steps
137
+ 2023-05-18 17:27:50 - RayLog - INFO: - Worker 1 finished episode 79 with reward 200.0 in 200 steps
138
+ 2023-05-18 17:27:52 - RayLog - INFO: - learner id: 0, update_step: 4000, online_eval_reward: 200.000
139
+ 2023-05-18 17:27:54 - RayLog - INFO: - Worker 0 finished episode 80 with reward 200.0 in 200 steps
140
+ 2023-05-18 17:27:56 - RayLog - INFO: - Worker 1 finished episode 81 with reward 200.0 in 200 steps
141
+ 2023-05-18 17:27:59 - RayLog - INFO: - learner id: 0, update_step: 4500, online_eval_reward: 200.000
142
+ 2023-05-18 17:28:00 - RayLog - INFO: - Worker 0 finished episode 82 with reward 200.0 in 200 steps
143
+ 2023-05-18 17:28:02 - RayLog - INFO: - Worker 1 finished episode 83 with reward 200.0 in 200 steps
144
+ 2023-05-18 17:28:06 - RayLog - INFO: - Worker 0 finished episode 84 with reward 200.0 in 200 steps
145
+ 2023-05-18 17:28:06 - RayLog - INFO: - learner id: 0, update_step: 5000, online_eval_reward: 200.000
146
+ 2023-05-18 17:28:08 - RayLog - INFO: - Worker 1 finished episode 85 with reward 200.0 in 200 steps
147
+ 2023-05-18 17:28:12 - RayLog - INFO: - Worker 0 finished episode 86 with reward 200.0 in 200 steps
148
+ 2023-05-18 17:28:14 - RayLog - INFO: - learner id: 0, update_step: 5500, online_eval_reward: 200.000
149
+ 2023-05-18 17:28:14 - RayLog - INFO: - Worker 1 finished episode 87 with reward 200.0 in 200 steps
150
+ 2023-05-18 17:28:18 - RayLog - INFO: - Worker 0 finished episode 88 with reward 200.0 in 200 steps
151
+ 2023-05-18 17:28:20 - RayLog - INFO: - Worker 1 finished episode 89 with reward 200.0 in 200 steps
152
+ 2023-05-18 17:28:21 - RayLog - INFO: - learner id: 0, update_step: 6000, online_eval_reward: 200.000
153
+ 2023-05-18 17:28:24 - RayLog - INFO: - Worker 0 finished episode 90 with reward 200.0 in 200 steps
154
+ 2023-05-18 17:28:26 - RayLog - INFO: - Worker 1 finished episode 91 with reward 200.0 in 200 steps
155
+ 2023-05-18 17:28:29 - RayLog - INFO: - learner id: 0, update_step: 6500, online_eval_reward: 200.000
156
+ 2023-05-18 17:28:30 - RayLog - INFO: - Worker 0 finished episode 92 with reward 200.0 in 200 steps
157
+ 2023-05-18 17:28:32 - RayLog - INFO: - Worker 1 finished episode 93 with reward 200.0 in 200 steps
158
+ 2023-05-18 17:28:37 - RayLog - INFO: - Worker 0 finished episode 94 with reward 200.0 in 200 steps
159
+ 2023-05-18 17:28:38 - RayLog - INFO: - learner id: 0, update_step: 7000, online_eval_reward: 200.000
160
+ 2023-05-18 17:28:40 - RayLog - INFO: - Worker 1 finished episode 95 with reward 200.0 in 200 steps
161
+ 2023-05-18 17:28:44 - RayLog - INFO: - Worker 0 finished episode 96 with reward 200.0 in 200 steps
162
+ 2023-05-18 17:28:47 - RayLog - INFO: - learner id: 0, update_step: 7500, online_eval_reward: 200.000
163
+ 2023-05-18 17:28:47 - RayLog - INFO: - Worker 1 finished episode 97 with reward 200.0 in 200 steps
164
+ 2023-05-18 17:28:52 - RayLog - INFO: - Worker 0 finished episode 98 with reward 200.0 in 200 steps
165
+ 2023-05-18 17:28:54 - RayLog - INFO: - Worker 1 finished episode 99 with reward 200.0 in 200 steps
166
+ 2023-05-18 17:28:56 - RayLog - INFO: - learner id: 0, update_step: 8000, online_eval_reward: 200.000
167
+ 2023-05-18 17:28:58 - RayLog - INFO: - Worker 0 finished episode 100 with reward 200.0 in 200 steps
168
+ 2023-05-18 17:29:01 - SimpleLog - INFO: - Finish training! total time consumed: 138.97s
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/1000 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d57deeb0464b6e9f89819ad546824a3f1e2e8002aeade1276ae5ac422b081c5
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/1500 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b19df30e46730d16ec3413c9280b0e4f07632abe78cd54eda08ea40ae04ad21
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/2000 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a27127e0838f8ad9672fae3534b7bd294ad20c815367138448f010969632952
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/2500 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03690ae5376aff9ec01509b969d554b453b7b2ecf5770777009f6cd80a02034d
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/3000 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99d1fab6ad841dfd4d452cd883b9cc61d6a6d3fec15a540c8de29e92926ede97
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/3500 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:619aa81383729fe98f60d0fdb71018605f747253585949decb77c121a8979586
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/4000 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e37b97bc88d3e21e31fdd4d53e7a16c90a73c850156cc5bba2076113a599b80
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/4500 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aca2009138f869be797b7f9e4efa341d831e9ec677607648c46790983e480d93
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/500 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4089a151d252a9e0cf8721e74b8b636eded43d173e8f1848c12d4240d4e47ec1
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/5000 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18c22f9833c9e8c13d971b2c012fdb3cc4e85d76bb8efaccc5d3767e4d7befd5
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/5500 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4273ff1ab2f0024f3562fac0d9b57e356410996ac13f6a0c512e918b1dce59f6
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/6000 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dcc1f49e420050d54de512a0ec1d1a9713b10017e4159527cef1812df30a5fd9
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/6500 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:937fdc23e792e243262fa7bacf8e1f28b092ed885df060a6aab7baa953cfd941
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/7000 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f766ef6a6a2270f18a559238f85b14f3ca5beb4fa98bb45d85aba35629541732
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/7500 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:26d765cdf390b5197972a3d250edd2993bb91c764bfacac17fe66207b336276d
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/8000 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04527e16df0a53f53764515da214ee67a5e4c5cf821e612c35df5ec94b43d6d8
3
+ size 1622673
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/models/best ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a27127e0838f8ad9672fae3534b7bd294ad20c815367138448f010969632952
3
+ size 1622673
ClassControl/CartPole-v1/{Train_ray_CartPole-v1_NoisyDQN_20230518-133947/tb_logs/model/events.out.tfevents.1684388387.JMac.local.86757.1 → Train_ray_CartPole-v1_NoisyDQN_20230518-172642/tb_logs/interact/events.out.tfevents.1684402002.JMac.local.91984.0} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:104eaae7368044a1396e1f0288356e64792ffebc2905456d3a98edd39438d03c
3
  size 40
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9dbe30ce02b42e95b4b867ddfc2e8e68cc94f87181bc659a19e4f5c174c5b93b
3
  size 40
ClassControl/CartPole-v1/{Train_ray_CartPole-v1_NoisyDQN_20230518-133947/tb_logs/interact/events.out.tfevents.1684388398.JMac.local.86815.0 → Train_ray_CartPole-v1_NoisyDQN_20230518-172642/tb_logs/interact/events.out.tfevents.1684402014.JMac.local.92035.0} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:836310578ed6206604d3b1432fc5f35019c383105fbb64f1505a5bd72baf68cb
3
  size 10028
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a0232091096dd5fa4d72d8b15deebf885a69d7fe03e4dff2c31ddbc5901549b
3
  size 10028
ClassControl/CartPole-v1/{Test_single_CartPole-v1_NoisyDQN_20230518-133824/tb_logs/model/events.out.tfevents.1684388304.JMac.local.86491.1 → Train_ray_CartPole-v1_NoisyDQN_20230518-172642/tb_logs/model/events.out.tfevents.1684402002.JMac.local.91984.1} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e5ef74ab61b4fd87898bdb34cad1df3bbf05d7edf30ae81504f7d09bb4f75ffd
3
  size 40
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c1036e7f4f8114338b83bb2e5160a9a5dd32b4633240d896320ca7ab39b3071
3
  size 40
ClassControl/CartPole-v1/Train_ray_CartPole-v1_NoisyDQN_20230518-172642/tb_logs/model/events.out.tfevents.1684402014.JMac.local.92035.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad53b18f0687e7143053cd41109992ca0386d5cda19cedc69f9af251ef4d746e
3
+ size 394608
ClassControl/CartPole-v1/Train_single_CartPole-v1_NoisyDQN_20230518-133737/logs/log.txt DELETED
@@ -1,164 +0,0 @@
1
- 2023-05-18 13:37:37 - SimpleLog - INFO: - General Configs:
2
- 2023-05-18 13:37:37 - SimpleLog - INFO: - ================================================================================
3
- 2023-05-18 13:37:37 - SimpleLog - INFO: - Name Value Type
4
- 2023-05-18 13:37:37 - SimpleLog - INFO: - env_name gym <class 'str'>
5
- 2023-05-18 13:37:37 - SimpleLog - INFO: - algo_name NoisyDQN <class 'str'>
6
- 2023-05-18 13:37:37 - SimpleLog - INFO: - mode train <class 'str'>
7
- 2023-05-18 13:37:37 - SimpleLog - INFO: - device cpu <class 'str'>
8
- 2023-05-18 13:37:37 - SimpleLog - INFO: - seed 1 <class 'int'>
9
- 2023-05-18 13:37:37 - SimpleLog - INFO: - max_episode 100 <class 'int'>
10
- 2023-05-18 13:37:37 - SimpleLog - INFO: - max_step 200 <class 'int'>
11
- 2023-05-18 13:37:37 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
12
- 2023-05-18 13:37:37 - SimpleLog - INFO: - mp_backend single <class 'str'>
13
- 2023-05-18 13:37:37 - SimpleLog - INFO: - n_workers 2 <class 'int'>
14
- 2023-05-18 13:37:37 - SimpleLog - INFO: - n_learners 1 <class 'int'>
15
- 2023-05-18 13:37:37 - SimpleLog - INFO: - share_buffer 1 <class 'bool'>
16
- 2023-05-18 13:37:37 - SimpleLog - INFO: - online_eval 1 <class 'bool'>
17
- 2023-05-18 13:37:37 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
18
- 2023-05-18 13:37:37 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
19
- 2023-05-18 13:37:37 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'>
20
- 2023-05-18 13:37:37 - SimpleLog - INFO: - load_path Train_CartPole-v1_DQN_20221026-054757 <class 'str'>
21
- 2023-05-18 13:37:37 - SimpleLog - INFO: - load_model_step best <class 'str'>
22
- 2023-05-18 13:37:37 - SimpleLog - INFO: - ================================================================================
23
- 2023-05-18 13:37:37 - SimpleLog - INFO: - Algo Configs:
24
- 2023-05-18 13:37:37 - SimpleLog - INFO: - ================================================================================
25
- 2023-05-18 13:37:37 - SimpleLog - INFO: - Name Value Type
26
- 2023-05-18 13:37:37 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
27
- 2023-05-18 13:37:37 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
28
- 2023-05-18 13:37:37 - SimpleLog - INFO: - epsilon_decay 500 <class 'int'>
29
- 2023-05-18 13:37:37 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
30
- 2023-05-18 13:37:37 - SimpleLog - INFO: - lr 0.0001 <class 'float'>
31
- 2023-05-18 13:37:37 - SimpleLog - INFO: - buffer_size 100000 <class 'int'>
32
- 2023-05-18 13:37:37 - SimpleLog - INFO: - batch_size 64 <class 'int'>
33
- 2023-05-18 13:37:37 - SimpleLog - INFO: - target_update 4 <class 'int'>
34
- 2023-05-18 13:37:37 - SimpleLog - INFO: - value_layers [{'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}, {'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}] <class 'str'>
35
- 2023-05-18 13:37:37 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'>
36
- 2023-05-18 13:37:37 - SimpleLog - INFO: - ================================================================================
37
- 2023-05-18 13:37:37 - SimpleLog - INFO: - Env Configs:
38
- 2023-05-18 13:37:37 - SimpleLog - INFO: - ================================================================================
39
- 2023-05-18 13:37:37 - SimpleLog - INFO: - Name Value Type
40
- 2023-05-18 13:37:37 - SimpleLog - INFO: - id CartPole-v1 <class 'str'>
41
- 2023-05-18 13:37:37 - SimpleLog - INFO: - render_mode None <class 'str'>
42
- 2023-05-18 13:37:37 - SimpleLog - INFO: - wrapper None <class 'str'>
43
- 2023-05-18 13:37:37 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
44
- 2023-05-18 13:37:37 - SimpleLog - INFO: - ================================================================================
45
- 2023-05-18 13:37:37 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2)
46
- 2023-05-18 13:37:37 - SimpleLog - INFO: - Start training!
47
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 0, ep_reward: 12.0, ep_step: 12
48
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 1, ep_reward: 23.0, ep_step: 23
49
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 2, ep_reward: 21.0, ep_step: 21
50
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 3, ep_reward: 14.0, ep_step: 14
51
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 4, ep_reward: 24.0, ep_step: 24
52
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 5, ep_reward: 18.0, ep_step: 18
53
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 6, ep_reward: 45.0, ep_step: 45
54
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 7, ep_reward: 30.0, ep_step: 30
55
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 8, ep_reward: 25.0, ep_step: 25
56
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 9, ep_reward: 29.0, ep_step: 29
57
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 10, ep_reward: 13.0, ep_step: 13
58
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 11, ep_reward: 10.0, ep_step: 10
59
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 12, ep_reward: 11.0, ep_step: 11
60
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 13, ep_reward: 14.0, ep_step: 14
61
- 2023-05-18 13:37:37 - SimpleLog - INFO: - episode: 14, ep_reward: 12.0, ep_step: 12
62
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 15, ep_reward: 11.0, ep_step: 11
63
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 16, ep_reward: 11.0, ep_step: 11
64
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 17, ep_reward: 28.0, ep_step: 28
65
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 18, ep_reward: 10.0, ep_step: 10
66
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 19, ep_reward: 13.0, ep_step: 13
67
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 20, ep_reward: 15.0, ep_step: 15
68
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 21, ep_reward: 13.0, ep_step: 13
69
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 22, ep_reward: 10.0, ep_step: 10
70
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 23, ep_reward: 17.0, ep_step: 17
71
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 24, ep_reward: 21.0, ep_step: 21
72
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 25, ep_reward: 12.0, ep_step: 12
73
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 26, ep_reward: 16.0, ep_step: 16
74
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 27, ep_reward: 13.0, ep_step: 13
75
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 28, ep_reward: 10.0, ep_step: 10
76
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 29, ep_reward: 12.0, ep_step: 12
77
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 30, ep_reward: 22.0, ep_step: 22
78
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 31, ep_reward: 10.0, ep_step: 10
79
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 32, ep_reward: 13.0, ep_step: 13
80
- 2023-05-18 13:37:38 - SimpleLog - INFO: - update_step: 500, online_eval_reward: 10.000
81
- 2023-05-18 13:37:38 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 10.000, save the best model!
82
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 33, ep_reward: 14.0, ep_step: 14
83
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 34, ep_reward: 9.0, ep_step: 9
84
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 35, ep_reward: 10.0, ep_step: 10
85
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 36, ep_reward: 11.0, ep_step: 11
86
- 2023-05-18 13:37:38 - SimpleLog - INFO: - episode: 37, ep_reward: 9.0, ep_step: 9
87
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 38, ep_reward: 10.0, ep_step: 10
88
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 39, ep_reward: 14.0, ep_step: 14
89
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 40, ep_reward: 9.0, ep_step: 9
90
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 41, ep_reward: 11.0, ep_step: 11
91
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 42, ep_reward: 9.0, ep_step: 9
92
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 43, ep_reward: 9.0, ep_step: 9
93
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 44, ep_reward: 15.0, ep_step: 15
94
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 45, ep_reward: 15.0, ep_step: 15
95
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 46, ep_reward: 9.0, ep_step: 9
96
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 47, ep_reward: 11.0, ep_step: 11
97
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 48, ep_reward: 10.0, ep_step: 10
98
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 49, ep_reward: 9.0, ep_step: 9
99
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 50, ep_reward: 12.0, ep_step: 12
100
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 51, ep_reward: 10.0, ep_step: 10
101
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 52, ep_reward: 10.0, ep_step: 10
102
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 53, ep_reward: 10.0, ep_step: 10
103
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 54, ep_reward: 10.0, ep_step: 10
104
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 55, ep_reward: 10.0, ep_step: 10
105
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 56, ep_reward: 39.0, ep_step: 39
106
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 57, ep_reward: 17.0, ep_step: 17
107
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 58, ep_reward: 20.0, ep_step: 20
108
- 2023-05-18 13:37:39 - SimpleLog - INFO: - episode: 59, ep_reward: 21.0, ep_step: 21
109
- 2023-05-18 13:37:40 - SimpleLog - INFO: - episode: 60, ep_reward: 55.0, ep_step: 55
110
- 2023-05-18 13:37:40 - SimpleLog - INFO: - episode: 61, ep_reward: 26.0, ep_step: 26
111
- 2023-05-18 13:37:40 - SimpleLog - INFO: - episode: 62, ep_reward: 29.0, ep_step: 29
112
- 2023-05-18 13:37:40 - SimpleLog - INFO: - episode: 63, ep_reward: 35.0, ep_step: 35
113
- 2023-05-18 13:37:40 - SimpleLog - INFO: - update_step: 1000, online_eval_reward: 27.000
114
- 2023-05-18 13:37:40 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 27.000, save the best model!
115
- 2023-05-18 13:37:40 - SimpleLog - INFO: - episode: 64, ep_reward: 27.0, ep_step: 27
116
- 2023-05-18 13:37:40 - SimpleLog - INFO: - episode: 65, ep_reward: 41.0, ep_step: 41
117
- 2023-05-18 13:37:40 - SimpleLog - INFO: - episode: 66, ep_reward: 91.0, ep_step: 91
118
- 2023-05-18 13:37:41 - SimpleLog - INFO: - episode: 67, ep_reward: 71.0, ep_step: 71
119
- 2023-05-18 13:37:41 - SimpleLog - INFO: - episode: 68, ep_reward: 72.0, ep_step: 72
120
- 2023-05-18 13:37:41 - SimpleLog - INFO: - episode: 69, ep_reward: 24.0, ep_step: 24
121
- 2023-05-18 13:37:41 - SimpleLog - INFO: - episode: 70, ep_reward: 50.0, ep_step: 50
122
- 2023-05-18 13:37:41 - SimpleLog - INFO: - episode: 71, ep_reward: 85.0, ep_step: 85
123
- 2023-05-18 13:37:42 - SimpleLog - INFO: - update_step: 1500, online_eval_reward: 66.000
124
- 2023-05-18 13:37:42 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 66.000, save the best model!
125
- 2023-05-18 13:37:42 - SimpleLog - INFO: - episode: 72, ep_reward: 183.0, ep_step: 183
126
- 2023-05-18 13:37:42 - SimpleLog - INFO: - episode: 73, ep_reward: 124.0, ep_step: 124
127
- 2023-05-18 13:37:43 - SimpleLog - INFO: - episode: 74, ep_reward: 123.0, ep_step: 123
128
- 2023-05-18 13:37:43 - SimpleLog - INFO: - update_step: 2000, online_eval_reward: 102.000
129
- 2023-05-18 13:37:43 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 102.000, save the best model!
130
- 2023-05-18 13:37:43 - SimpleLog - INFO: - episode: 75, ep_reward: 192.0, ep_step: 192
131
- 2023-05-18 13:37:44 - SimpleLog - INFO: - episode: 76, ep_reward: 115.0, ep_step: 115
132
- 2023-05-18 13:37:44 - SimpleLog - INFO: - episode: 77, ep_reward: 200.0, ep_step: 200
133
- 2023-05-18 13:37:45 - SimpleLog - INFO: - update_step: 2500, online_eval_reward: 99.000
134
- 2023-05-18 13:37:45 - SimpleLog - INFO: - episode: 78, ep_reward: 171.0, ep_step: 171
135
- 2023-05-18 13:37:45 - SimpleLog - INFO: - episode: 79, ep_reward: 200.0, ep_step: 200
136
- 2023-05-18 13:37:46 - SimpleLog - INFO: - episode: 80, ep_reward: 125.0, ep_step: 125
137
- 2023-05-18 13:37:46 - SimpleLog - INFO: - update_step: 3000, online_eval_reward: 90.000
138
- 2023-05-18 13:37:46 - SimpleLog - INFO: - episode: 81, ep_reward: 200.0, ep_step: 200
139
- 2023-05-18 13:37:47 - SimpleLog - INFO: - episode: 82, ep_reward: 140.0, ep_step: 140
140
- 2023-05-18 13:37:47 - SimpleLog - INFO: - episode: 83, ep_reward: 172.0, ep_step: 172
141
- 2023-05-18 13:37:48 - SimpleLog - INFO: - episode: 84, ep_reward: 78.0, ep_step: 78
142
- 2023-05-18 13:37:48 - SimpleLog - INFO: - update_step: 3500, online_eval_reward: 82.000
143
- 2023-05-18 13:37:48 - SimpleLog - INFO: - episode: 85, ep_reward: 92.0, ep_step: 92
144
- 2023-05-18 13:37:48 - SimpleLog - INFO: - episode: 86, ep_reward: 104.0, ep_step: 104
145
- 2023-05-18 13:37:49 - SimpleLog - INFO: - episode: 87, ep_reward: 99.0, ep_step: 99
146
- 2023-05-18 13:37:49 - SimpleLog - INFO: - episode: 88, ep_reward: 81.0, ep_step: 81
147
- 2023-05-18 13:37:49 - SimpleLog - INFO: - episode: 89, ep_reward: 97.0, ep_step: 97
148
- 2023-05-18 13:37:49 - SimpleLog - INFO: - update_step: 4000, online_eval_reward: 106.000
149
- 2023-05-18 13:37:49 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 106.000, save the best model!
150
- 2023-05-18 13:37:49 - SimpleLog - INFO: - episode: 90, ep_reward: 99.0, ep_step: 99
151
- 2023-05-18 13:37:50 - SimpleLog - INFO: - episode: 91, ep_reward: 200.0, ep_step: 200
152
- 2023-05-18 13:37:50 - SimpleLog - INFO: - episode: 92, ep_reward: 93.0, ep_step: 93
153
- 2023-05-18 13:37:51 - SimpleLog - INFO: - episode: 93, ep_reward: 127.0, ep_step: 127
154
- 2023-05-18 13:37:51 - SimpleLog - INFO: - update_step: 4500, online_eval_reward: 200.000
155
- 2023-05-18 13:37:51 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 200.000, save the best model!
156
- 2023-05-18 13:37:51 - SimpleLog - INFO: - episode: 94, ep_reward: 200.0, ep_step: 200
157
- 2023-05-18 13:37:52 - SimpleLog - INFO: - episode: 95, ep_reward: 145.0, ep_step: 145
158
- 2023-05-18 13:37:53 - SimpleLog - INFO: - update_step: 5000, online_eval_reward: 200.000
159
- 2023-05-18 13:37:53 - SimpleLog - INFO: - episode: 96, ep_reward: 200.0, ep_step: 200
160
- 2023-05-18 13:37:53 - SimpleLog - INFO: - episode: 97, ep_reward: 200.0, ep_step: 200
161
- 2023-05-18 13:37:54 - SimpleLog - INFO: - episode: 98, ep_reward: 168.0, ep_step: 168
162
- 2023-05-18 13:37:54 - SimpleLog - INFO: - update_step: 5500, online_eval_reward: 200.000
163
- 2023-05-18 13:37:55 - SimpleLog - INFO: - episode: 99, ep_reward: 200.0, ep_step: 200
164
- 2023-05-18 13:37:55 - SimpleLog - INFO: - Finish training! total time consumed: 17.76s
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ClassControl/CartPole-v1/Train_single_CartPole-v1_NoisyDQN_20230518-133737/models/1000 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_single_CartPole-v1_NoisyDQN_20230518-133737/models/1500 DELETED
Binary file (545 kB)
 
ClassControl/CartPole-v1/Train_single_CartPole-v1_NoisyDQN_20230518-133737/models/2000 DELETED
Binary file (545 kB)