Commit
•
3e87e2a
1
Parent(s):
42df274
Upload folder using huggingface_hub
Browse files- .summary/0/events.out.tfevents.1696850003.rhmmedcatt-proliant-ml350-gen10 +3 -0
- .summary/1/events.out.tfevents.1696850003.rhmmedcatt-proliant-ml350-gen10 +3 -0
- README.md +117 -4
- checkpoint_p0/best_000003488_3571712_reward_10.000.pth +3 -0
- checkpoint_p0/checkpoint_000096864_99188736.pth +3 -0
- checkpoint_p0/checkpoint_000097664_100007936.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000008320_8519680.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000016800_17203200.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000025280_25886720.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000033792_34603008.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000042304_43319296.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000050816_52035584.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000059360_60784640.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000067872_69500928.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000076416_78249984.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000084928_86966272.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000093472_95715328.pth +3 -0
- checkpoint_p1/best_000002112_2162688_reward_10.000.pth +3 -0
- checkpoint_p1/checkpoint_000097248_99581952.pth +3 -0
- checkpoint_p1/checkpoint_000098048_100401152.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000008352_8552448.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000016864_17268736.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000025408_26017792.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000033952_34766848.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000042496_43515904.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000051040_52264960.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000059584_61014016.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000068160_69795840.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000076736_78577664.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000085280_87326720.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000093824_96075776.pth +3 -0
- config.json +40 -22
- git.diff +2 -2
- replay.mp4 +2 -2
- sf_log.txt +0 -0
.summary/0/events.out.tfevents.1696850003.rhmmedcatt-proliant-ml350-gen10
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:15bbc6440beb58c912a18f326ad8f34f626a7509effb485fb0df9cfb362080a8
|
3 |
+
size 7524729
|
.summary/1/events.out.tfevents.1696850003.rhmmedcatt-proliant-ml350-gen10
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:4e64db3e80a13def93ba9d15a03193e594d76f443aede20d9685821caf12ed37
|
3 |
+
size 4348897
|
README.md
CHANGED
@@ -15,7 +15,7 @@ model-index:
|
|
15 |
type: atari_bowling
|
16 |
metrics:
|
17 |
- type: mean_reward
|
18 |
-
value:
|
19 |
name: mean_reward
|
20 |
verified: false
|
21 |
---
|
@@ -30,15 +30,128 @@ Documentation for how to use Sample-Factory can be found at https://www.samplefa
|
|
30 |
|
31 |
After installing Sample-Factory, download the model with:
|
32 |
```
|
33 |
-
python -m sample_factory.huggingface.load_from_hub -r MattStammers/
|
34 |
```
|
35 |
|
36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
## Using the model
|
38 |
|
39 |
To run the model after download, use the `enjoy` script corresponding to this environment:
|
40 |
```
|
41 |
-
python -m sf_examples.atari.enjoy_atari --algo=APPO --env=atari_bowling --train_dir=./train_dir --experiment=
|
42 |
```
|
43 |
|
44 |
|
@@ -49,7 +162,7 @@ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
|
|
49 |
|
50 |
To continue training with this model, use the `train` script corresponding to this environment:
|
51 |
```
|
52 |
-
python -m sf_examples.atari.train_atari --algo=APPO --env=atari_bowling --train_dir=./train_dir --experiment=
|
53 |
```
|
54 |
|
55 |
Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
|
|
|
15 |
type: atari_bowling
|
16 |
metrics:
|
17 |
- type: mean_reward
|
18 |
+
value: 46.40 +/- 5.28
|
19 |
name: mean_reward
|
20 |
verified: false
|
21 |
---
|
|
|
30 |
|
31 |
After installing Sample-Factory, download the model with:
|
32 |
```
|
33 |
+
python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_bowling
|
34 |
```
|
35 |
|
36 |
|
37 |
+
## About the Model
|
38 |
+
|
39 |
+
This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it.
|
40 |
+
|
41 |
+
The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels.
|
42 |
+
|
43 |
+
The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
|
44 |
+
I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
|
45 |
+
```
|
46 |
+
hyperparameters = {
|
47 |
+
"device": "gpu",
|
48 |
+
"seed": 1234,
|
49 |
+
"num_policies": 2,
|
50 |
+
"async_rl": true,
|
51 |
+
"serial_mode": false,
|
52 |
+
"batched_sampling": true,
|
53 |
+
"num_batches_to_accumulate": 2,
|
54 |
+
"worker_num_splits": 1,
|
55 |
+
"policy_workers_per_policy": 1,
|
56 |
+
"max_policy_lag": 1000,
|
57 |
+
"num_workers": 16,
|
58 |
+
"num_envs_per_worker": 2,
|
59 |
+
"batch_size": 1024,
|
60 |
+
"num_batches_per_epoch": 8,
|
61 |
+
"num_epochs": 4,
|
62 |
+
"rollout": 128,
|
63 |
+
"recurrence": 1,
|
64 |
+
"shuffle_minibatches": false,
|
65 |
+
"gamma": 0.99,
|
66 |
+
"reward_scale": 1.0,
|
67 |
+
"reward_clip": 1000.0,
|
68 |
+
"value_bootstrap": false,
|
69 |
+
"normalize_returns": true,
|
70 |
+
"exploration_loss_coeff": 0.0004677351413,
|
71 |
+
"value_loss_coeff": 0.5,
|
72 |
+
"kl_loss_coeff": 0.0,
|
73 |
+
"exploration_loss": "entropy",
|
74 |
+
"gae_lambda": 0.95,
|
75 |
+
"ppo_clip_ratio": 0.1,
|
76 |
+
"ppo_clip_value": 1.0,
|
77 |
+
"with_vtrace": false,
|
78 |
+
"vtrace_rho": 1.0,
|
79 |
+
"vtrace_c": 1.0,
|
80 |
+
"optimizer": "adam",
|
81 |
+
"adam_eps": 1e-05,
|
82 |
+
"adam_beta1": 0.9,
|
83 |
+
"adam_beta2": 0.999,
|
84 |
+
"max_grad_norm": 0.0,
|
85 |
+
"learning_rate": 0.0003033891184,
|
86 |
+
"lr_schedule": "linear_decay",
|
87 |
+
"lr_schedule_kl_threshold": 0.008,
|
88 |
+
"lr_adaptive_min": 1e-06,
|
89 |
+
"lr_adaptive_max": 0.01,
|
90 |
+
"obs_subtract_mean": 0.0,
|
91 |
+
"obs_scale": 255.0,
|
92 |
+
"normalize_input": true,
|
93 |
+
"normalize_input_keys": [
|
94 |
+
"obs"
|
95 |
+
],
|
96 |
+
"decorrelate_experience_max_seconds": 0,
|
97 |
+
"decorrelate_envs_on_one_worker": true,
|
98 |
+
"actor_worker_gpus": [],
|
99 |
+
"set_workers_cpu_affinity": true,
|
100 |
+
"force_envs_single_thread": false,
|
101 |
+
"default_niceness": 0,
|
102 |
+
"log_to_file": true,
|
103 |
+
"experiment_summaries_interval": 3,
|
104 |
+
"flush_summaries_interval": 30,
|
105 |
+
"stats_avg": 100,
|
106 |
+
"summaries_use_frameskip": true,
|
107 |
+
"heartbeat_interval": 10,
|
108 |
+
"heartbeat_reporting_interval": 60,
|
109 |
+
"train_for_env_steps": 100000000,
|
110 |
+
"train_for_seconds": 10000000000,
|
111 |
+
"save_every_sec": 120,
|
112 |
+
"keep_checkpoints": 2,
|
113 |
+
"load_checkpoint_kind": "latest",
|
114 |
+
"save_milestones_sec": 1200,
|
115 |
+
"save_best_every_sec": 5,
|
116 |
+
"save_best_metric": "reward",
|
117 |
+
"save_best_after": 100000,
|
118 |
+
"benchmark": false,
|
119 |
+
"encoder_mlp_layers": [
|
120 |
+
512,
|
121 |
+
512
|
122 |
+
],
|
123 |
+
"encoder_conv_architecture": "convnet_atari",
|
124 |
+
"encoder_conv_mlp_layers": [
|
125 |
+
512
|
126 |
+
],
|
127 |
+
"use_rnn": false,
|
128 |
+
"rnn_size": 512,
|
129 |
+
"rnn_type": "gru",
|
130 |
+
"rnn_num_layers": 1,
|
131 |
+
"decoder_mlp_layers": [],
|
132 |
+
"nonlinearity": "relu",
|
133 |
+
"policy_initialization": "orthogonal",
|
134 |
+
"policy_init_gain": 1.0,
|
135 |
+
"actor_critic_share_weights": true,
|
136 |
+
"adaptive_stddev": false,
|
137 |
+
"continuous_tanh_scale": 0.0,
|
138 |
+
"initial_stddev": 1.0,
|
139 |
+
"use_env_info_cache": false,
|
140 |
+
"env_gpu_actions": false,
|
141 |
+
"env_gpu_observations": true,
|
142 |
+
"env_frameskip": 4,
|
143 |
+
"env_framestack": 4,
|
144 |
+
}
|
145 |
+
|
146 |
+
```
|
147 |
+
|
148 |
+
|
149 |
+
|
150 |
## Using the model
|
151 |
|
152 |
To run the model after download, use the `enjoy` script corresponding to this environment:
|
153 |
```
|
154 |
+
python -m sf_examples.atari.enjoy_atari --algo=APPO --env=atari_bowling --train_dir=./train_dir --experiment=APPO-atari_bowling
|
155 |
```
|
156 |
|
157 |
|
|
|
162 |
|
163 |
To continue training with this model, use the `train` script corresponding to this environment:
|
164 |
```
|
165 |
+
python -m sf_examples.atari.train_atari --algo=APPO --env=atari_bowling --train_dir=./train_dir --experiment=APPO-atari_bowling --restart_behavior=resume --train_for_env_steps=10000000000
|
166 |
```
|
167 |
|
168 |
Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
|
checkpoint_p0/best_000003488_3571712_reward_10.000.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:52c6b5c52858e5e90774d5a443c8dd8db7965ecf2cab3f9c9093b4525708f704
|
3 |
+
size 20721843
|
checkpoint_p0/checkpoint_000096864_99188736.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3d946d635799c2b57ae22a26d4a67cfc8ec89a573f2927d9ce8227cedb764a02
|
3 |
+
size 20722179
|
checkpoint_p0/checkpoint_000097664_100007936.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ee0154c14cf8a653a7d93e1c5dfe588d698b8a65d5549f1db9472d5aaf08ad82
|
3 |
+
size 20722179
|
checkpoint_p0/milestones/checkpoint_000008320_8519680.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:723e0b18dc1bc44c11fe49c0626f272d326aae92618a2b14b8dc3d015cdbe341
|
3 |
+
size 20723035
|
checkpoint_p0/milestones/checkpoint_000016800_17203200.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3382249f65d2578c8b874fbfa36a94bf3e826cac32a6add875fecbc21ae9dde1
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000025280_25886720.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:882ff54c0633c37bd9ad5ae5fb3e213ce5cff5478ce83c9fd0a706f6c07b5e66
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000033792_34603008.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ad1d3df45cf2893e060c1a788b43c40a7738c78d8982ba00672666171cbda1aa
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000042304_43319296.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:914877f0663a2d21c2ad110f9b6246cb332e2bd62b371ae49d6d4d0f9d602a73
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000050816_52035584.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:e7ad477595705efd140070ba13496c8afe3a6ecffa5444908d5fede266c21fb7
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000059360_60784640.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:0c41f7b1567c2ecb0a0a7e3370ab9e4c5663ca3a6af82cea729d6c1f887b3bc6
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000067872_69500928.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:14bfd9838036f1b2da989705daa9dcd1a70c860ced25ba8aee1a19eecf6b3b4d
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000076416_78249984.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:c2016c47609cd2b788bb4c125afd6a134791576fcb938730aa9b3900e622cbe0
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000084928_86966272.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:4fdb4d35b8cf0d3456c20e8caadb10917f368d3fe280183aa8b9b9a2658b4c69
|
3 |
+
size 20723091
|
checkpoint_p0/milestones/checkpoint_000093472_95715328.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:fbcdaeabf520a76a3b6061f241d488564fafea4b8daf4082e47732fe1bf41540
|
3 |
+
size 20723091
|
checkpoint_p1/best_000002112_2162688_reward_10.000.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:903912659bbb60f4df9b724798b7f668e4461d8355ab196cd9c84023785ecd78
|
3 |
+
size 20721843
|
checkpoint_p1/checkpoint_000097248_99581952.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:fdb5a2d79410190167183988376968aeb497e1324e05d28674927ba8aec66935
|
3 |
+
size 20722179
|
checkpoint_p1/checkpoint_000098048_100401152.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:7a1716beb779a9efea0e8f0d5952e9b9e60b53e8bd02447ff60fb560d6d7c3a2
|
3 |
+
size 20722179
|
checkpoint_p1/milestones/checkpoint_000008352_8552448.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:63750b67f39a10c0b44efb659157aaef13820847428c35f8f10ebb642601580e
|
3 |
+
size 20723035
|
checkpoint_p1/milestones/checkpoint_000016864_17268736.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:f176ebe3075ff9122a4255152ea73cca36a7f9aaa132854b8f1307f99f4ebe3a
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000025408_26017792.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:6f7febe1090240899df26ad3d1ca65282aaf11e1b398737b48a8f6f239079966
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000033952_34766848.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:471099ae7be2438a14be44e08ded1adcd2dd5a944d130e1cebe1e30185919ec0
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000042496_43515904.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:2dc0ff91adf258665bdd4d7abcf706f50101949be582bc248b8c9a061df5e150
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000051040_52264960.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:5af3c5ec50ef7e91f3f6057d73e8321af4eda5ff94d01e8b49d9836a6ebfe83f
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000059584_61014016.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:b1033f05059633d7871c468c3fba2d27b87ea1d530effa043dd44536363688e6
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000068160_69795840.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:a77e2fdf721ad57bc902b4d10b997088f9752613107ef398af050dc259df0aab
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000076736_78577664.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:d530cd97ca08c4755d01689cb642c008946c65c878e25cb3dc56eda2dd1bdbaf
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000085280_87326720.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:1441ba0022935e006fb5d7520210b99690a0713024f0420732d95d501b2b3788
|
3 |
+
size 20723091
|
checkpoint_p1/milestones/checkpoint_000093824_96075776.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:a7f72130f55113af491193d6532c0225af6fd27bb94c4fc5fc70d8023abcd4e5
|
3 |
+
size 20723091
|
config.json
CHANGED
@@ -2,23 +2,23 @@
|
|
2 |
"help": false,
|
3 |
"algo": "APPO",
|
4 |
"env": "atari_bowling",
|
5 |
-
"experiment": "
|
6 |
"train_dir": "./train_atari",
|
7 |
-
"restart_behavior": "
|
8 |
"device": "gpu",
|
9 |
-
"seed":
|
10 |
"num_policies": 2,
|
11 |
-
"async_rl":
|
12 |
"serial_mode": false,
|
13 |
-
"batched_sampling":
|
14 |
"num_batches_to_accumulate": 2,
|
15 |
"worker_num_splits": 1,
|
16 |
"policy_workers_per_policy": 1,
|
17 |
"max_policy_lag": 1000,
|
18 |
-
"num_workers":
|
19 |
-
"num_envs_per_worker":
|
20 |
-
"batch_size":
|
21 |
-
"num_batches_per_epoch":
|
22 |
"num_epochs": 4,
|
23 |
"rollout": 128,
|
24 |
"recurrence": 1,
|
@@ -28,7 +28,7 @@
|
|
28 |
"reward_clip": 1000.0,
|
29 |
"value_bootstrap": false,
|
30 |
"normalize_returns": true,
|
31 |
-
"exploration_loss_coeff": 0.
|
32 |
"value_loss_coeff": 0.5,
|
33 |
"kl_loss_coeff": 0.0,
|
34 |
"exploration_loss": "entropy",
|
@@ -42,8 +42,8 @@
|
|
42 |
"adam_eps": 1e-05,
|
43 |
"adam_beta1": 0.9,
|
44 |
"adam_beta2": 0.999,
|
45 |
-
"max_grad_norm": 0.
|
46 |
-
"learning_rate": 0.
|
47 |
"lr_schedule": "linear_decay",
|
48 |
"lr_schedule_kl_threshold": 0.008,
|
49 |
"lr_adaptive_min": 1e-06,
|
@@ -51,7 +51,9 @@
|
|
51 |
"obs_subtract_mean": 0.0,
|
52 |
"obs_scale": 255.0,
|
53 |
"normalize_input": true,
|
54 |
-
"normalize_input_keys":
|
|
|
|
|
55 |
"decorrelate_experience_max_seconds": 0,
|
56 |
"decorrelate_envs_on_one_worker": true,
|
57 |
"actor_worker_gpus": [],
|
@@ -63,14 +65,14 @@
|
|
63 |
"flush_summaries_interval": 30,
|
64 |
"stats_avg": 100,
|
65 |
"summaries_use_frameskip": true,
|
66 |
-
"heartbeat_interval":
|
67 |
-
"heartbeat_reporting_interval":
|
68 |
-
"train_for_env_steps":
|
69 |
"train_for_seconds": 10000000000,
|
70 |
"save_every_sec": 120,
|
71 |
"keep_checkpoints": 2,
|
72 |
"load_checkpoint_kind": "latest",
|
73 |
-
"save_milestones_sec":
|
74 |
"save_best_every_sec": 5,
|
75 |
"save_best_metric": "reward",
|
76 |
"save_best_after": 100000,
|
@@ -104,7 +106,7 @@
|
|
104 |
"use_record_episode_statistics": true,
|
105 |
"with_wandb": true,
|
106 |
"wandb_user": "matt-stammers",
|
107 |
-
"wandb_project": "
|
108 |
"wandb_group": "atari_bowling",
|
109 |
"wandb_job_type": "SF",
|
110 |
"wandb_tags": [
|
@@ -122,16 +124,32 @@
|
|
122 |
"pbt_target_objective": "true_objective",
|
123 |
"pbt_perturb_min": 1.1,
|
124 |
"pbt_perturb_max": 1.5,
|
125 |
-
"command_line": "--algo=APPO --env=atari_bowling --experiment=
|
126 |
"cli_args": {
|
127 |
"algo": "APPO",
|
128 |
"env": "atari_bowling",
|
129 |
-
"experiment": "
|
130 |
"train_dir": "./train_atari",
|
|
|
|
|
131 |
"num_policies": 2,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
132 |
"with_wandb": true,
|
133 |
"wandb_user": "matt-stammers",
|
134 |
-
"wandb_project": "
|
135 |
"wandb_group": "atari_bowling",
|
136 |
"wandb_job_type": "SF",
|
137 |
"wandb_tags": [
|
@@ -140,5 +158,5 @@
|
|
140 |
},
|
141 |
"git_hash": "5fff97c2f535da5987d358cdbe6927cccd43621e",
|
142 |
"git_repo_name": "not a git repository",
|
143 |
-
"wandb_unique_id": "
|
144 |
}
|
|
|
2 |
"help": false,
|
3 |
"algo": "APPO",
|
4 |
"env": "atari_bowling",
|
5 |
+
"experiment": "atari_bowling_APPO",
|
6 |
"train_dir": "./train_atari",
|
7 |
+
"restart_behavior": "restart",
|
8 |
"device": "gpu",
|
9 |
+
"seed": 1234,
|
10 |
"num_policies": 2,
|
11 |
+
"async_rl": true,
|
12 |
"serial_mode": false,
|
13 |
+
"batched_sampling": true,
|
14 |
"num_batches_to_accumulate": 2,
|
15 |
"worker_num_splits": 1,
|
16 |
"policy_workers_per_policy": 1,
|
17 |
"max_policy_lag": 1000,
|
18 |
+
"num_workers": 16,
|
19 |
+
"num_envs_per_worker": 2,
|
20 |
+
"batch_size": 1024,
|
21 |
+
"num_batches_per_epoch": 8,
|
22 |
"num_epochs": 4,
|
23 |
"rollout": 128,
|
24 |
"recurrence": 1,
|
|
|
28 |
"reward_clip": 1000.0,
|
29 |
"value_bootstrap": false,
|
30 |
"normalize_returns": true,
|
31 |
+
"exploration_loss_coeff": 0.0004677351413,
|
32 |
"value_loss_coeff": 0.5,
|
33 |
"kl_loss_coeff": 0.0,
|
34 |
"exploration_loss": "entropy",
|
|
|
42 |
"adam_eps": 1e-05,
|
43 |
"adam_beta1": 0.9,
|
44 |
"adam_beta2": 0.999,
|
45 |
+
"max_grad_norm": 0.0,
|
46 |
+
"learning_rate": 0.0003033891184,
|
47 |
"lr_schedule": "linear_decay",
|
48 |
"lr_schedule_kl_threshold": 0.008,
|
49 |
"lr_adaptive_min": 1e-06,
|
|
|
51 |
"obs_subtract_mean": 0.0,
|
52 |
"obs_scale": 255.0,
|
53 |
"normalize_input": true,
|
54 |
+
"normalize_input_keys": [
|
55 |
+
"obs"
|
56 |
+
],
|
57 |
"decorrelate_experience_max_seconds": 0,
|
58 |
"decorrelate_envs_on_one_worker": true,
|
59 |
"actor_worker_gpus": [],
|
|
|
65 |
"flush_summaries_interval": 30,
|
66 |
"stats_avg": 100,
|
67 |
"summaries_use_frameskip": true,
|
68 |
+
"heartbeat_interval": 10,
|
69 |
+
"heartbeat_reporting_interval": 60,
|
70 |
+
"train_for_env_steps": 100000000,
|
71 |
"train_for_seconds": 10000000000,
|
72 |
"save_every_sec": 120,
|
73 |
"keep_checkpoints": 2,
|
74 |
"load_checkpoint_kind": "latest",
|
75 |
+
"save_milestones_sec": 1200,
|
76 |
"save_best_every_sec": 5,
|
77 |
"save_best_metric": "reward",
|
78 |
"save_best_after": 100000,
|
|
|
106 |
"use_record_episode_statistics": true,
|
107 |
"with_wandb": true,
|
108 |
"wandb_user": "matt-stammers",
|
109 |
+
"wandb_project": "atari_APPO",
|
110 |
"wandb_group": "atari_bowling",
|
111 |
"wandb_job_type": "SF",
|
112 |
"wandb_tags": [
|
|
|
124 |
"pbt_target_objective": "true_objective",
|
125 |
"pbt_perturb_min": 1.1,
|
126 |
"pbt_perturb_max": 1.5,
|
127 |
+
"command_line": "--algo=APPO --env=atari_bowling --experiment=atari_bowling_APPO --num_policies=2 --restart_behavior=restart --train_dir=./train_atari --train_for_env_steps=100000000 --seed=1234 --num_workers=16 --num_envs_per_worker=2 --num_batches_per_epoch=8 --async_rl=true --batched_sampling=true --batch_size=1024 --max_grad_norm=0 --learning_rate=0.0003033891184 --heartbeat_interval=10 --heartbeat_reporting_interval=60 --save_milestones_sec=1200 --num_epochs=4 --exploration_loss_coeff=0.0004677351413 --with_wandb=true --wandb_user=matt-stammers --wandb_project=atari_APPO --wandb_group=atari_bowling --wandb_job_type=SF --wandb_tags=atari",
|
128 |
"cli_args": {
|
129 |
"algo": "APPO",
|
130 |
"env": "atari_bowling",
|
131 |
+
"experiment": "atari_bowling_APPO",
|
132 |
"train_dir": "./train_atari",
|
133 |
+
"restart_behavior": "restart",
|
134 |
+
"seed": 1234,
|
135 |
"num_policies": 2,
|
136 |
+
"async_rl": true,
|
137 |
+
"batched_sampling": true,
|
138 |
+
"num_workers": 16,
|
139 |
+
"num_envs_per_worker": 2,
|
140 |
+
"batch_size": 1024,
|
141 |
+
"num_batches_per_epoch": 8,
|
142 |
+
"num_epochs": 4,
|
143 |
+
"exploration_loss_coeff": 0.0004677351413,
|
144 |
+
"max_grad_norm": 0.0,
|
145 |
+
"learning_rate": 0.0003033891184,
|
146 |
+
"heartbeat_interval": 10,
|
147 |
+
"heartbeat_reporting_interval": 60,
|
148 |
+
"train_for_env_steps": 100000000,
|
149 |
+
"save_milestones_sec": 1200,
|
150 |
"with_wandb": true,
|
151 |
"wandb_user": "matt-stammers",
|
152 |
+
"wandb_project": "atari_APPO",
|
153 |
"wandb_group": "atari_bowling",
|
154 |
"wandb_job_type": "SF",
|
155 |
"wandb_tags": [
|
|
|
158 |
},
|
159 |
"git_hash": "5fff97c2f535da5987d358cdbe6927cccd43621e",
|
160 |
"git_repo_name": "not a git repository",
|
161 |
+
"wandb_unique_id": "atari_bowling_APPO_20231009_121320_509191"
|
162 |
}
|
git.diff
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3357904f421d3f4924836316b1741bf64d5dd0e807d5e80ac07059b4c52a7008
|
3 |
+
size 14426734
|
replay.mp4
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:36bcd5c67c731b3a640581cd4f87dc3fdc35da3aab4bfc3064197ee816bdd44c
|
3 |
+
size 1486697
|
sf_log.txt
CHANGED
The diff for this file is too large to render.
See raw diff
|
|