Commit
•
6159b0a
1
Parent(s):
a00e6b7
Upload folder using huggingface_hub
Browse files- .summary/0/events.out.tfevents.1696820851.rhmmedcatt-proliant-ml350-gen10 +3 -0
- .summary/1/events.out.tfevents.1696820851.rhmmedcatt-proliant-ml350-gen10 +3 -0
- README.md +117 -4
- checkpoint_p0/best_000093792_96043008_reward_41.570.pth +3 -0
- checkpoint_p0/checkpoint_000097344_99680256.pth +3 -0
- checkpoint_p0/checkpoint_000097664_100007936.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000007904_8093696.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000015872_16252928.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000023872_24444928.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000031840_32604160.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000039840_40796160.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000047808_48955392.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000055808_57147392.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000063808_65339392.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000071808_73531392.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000079776_81690624.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000087776_89882624.pth +3 -0
- checkpoint_p0/milestones/checkpoint_000095744_98041856.pth +3 -0
- checkpoint_p1/best_000090432_92602368_reward_39.550.pth +3 -0
- checkpoint_p1/checkpoint_000098464_100827136.pth +3 -0
- checkpoint_p1/checkpoint_000098784_101154816.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000007968_8159232.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000016032_16416768.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000024128_24707072.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000032192_32964608.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000040288_41254912.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000048384_49545216.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000056448_57802752.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000064544_66093056.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000072608_74350592.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000080672_82608128.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000088768_90898432.pth +3 -0
- checkpoint_p1/milestones/checkpoint_000096864_99188736.pth +3 -0
- config.json +40 -22
- git.diff +2 -2
- replay.mp4 +2 -2
- sf_log.txt +0 -0
.summary/0/events.out.tfevents.1696820851.rhmmedcatt-proliant-ml350-gen10
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:da7362d86ae6c23586ceca9dc4d98f8c60245cb78ecd549423d6238e2734e189
|
3 |
+
size 8019896
|
.summary/1/events.out.tfevents.1696820851.rhmmedcatt-proliant-ml350-gen10
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:65ef01d4349c47eab61b423aa1c6dcfb57859eb5d2a23ed810a675afabce0a57
|
3 |
+
size 4645004
|
README.md
CHANGED
@@ -15,7 +15,7 @@ model-index:
|
|
15 |
type: atari_beamrider
|
16 |
metrics:
|
17 |
- type: mean_reward
|
18 |
-
value:
|
19 |
name: mean_reward
|
20 |
verified: false
|
21 |
---
|
@@ -30,15 +30,128 @@ Documentation for how to use Sample-Factory can be found at https://www.samplefa
|
|
30 |
|
31 |
After installing Sample-Factory, download the model with:
|
32 |
```
|
33 |
-
python -m sample_factory.huggingface.load_from_hub -r MattStammers/
|
34 |
```
|
35 |
|
36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
## Using the model
|
38 |
|
39 |
To run the model after download, use the `enjoy` script corresponding to this environment:
|
40 |
```
|
41 |
-
python -m sf_examples.atari.enjoy_atari --algo=APPO --env=atari_beamrider --train_dir=./train_dir --experiment=
|
42 |
```
|
43 |
|
44 |
|
@@ -49,7 +162,7 @@ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
|
|
49 |
|
50 |
To continue training with this model, use the `train` script corresponding to this environment:
|
51 |
```
|
52 |
-
python -m sf_examples.atari.train_atari --algo=APPO --env=atari_beamrider --train_dir=./train_dir --experiment=
|
53 |
```
|
54 |
|
55 |
Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
|
|
|
15 |
type: atari_beamrider
|
16 |
metrics:
|
17 |
- type: mean_reward
|
18 |
+
value: 7105.40 +/- 2931.13
|
19 |
name: mean_reward
|
20 |
verified: false
|
21 |
---
|
|
|
30 |
|
31 |
After installing Sample-Factory, download the model with:
|
32 |
```
|
33 |
+
python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_beamrider
|
34 |
```
|
35 |
|
36 |
|
37 |
+
## About the Model
|
38 |
+
|
39 |
+
This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it.
|
40 |
+
|
41 |
+
The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels.
|
42 |
+
|
43 |
+
The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
|
44 |
+
I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
|
45 |
+
```
|
46 |
+
hyperparameters = {
|
47 |
+
"device": "gpu",
|
48 |
+
"seed": 1234,
|
49 |
+
"num_policies": 2,
|
50 |
+
"async_rl": true,
|
51 |
+
"serial_mode": false,
|
52 |
+
"batched_sampling": true,
|
53 |
+
"num_batches_to_accumulate": 2,
|
54 |
+
"worker_num_splits": 1,
|
55 |
+
"policy_workers_per_policy": 1,
|
56 |
+
"max_policy_lag": 1000,
|
57 |
+
"num_workers": 16,
|
58 |
+
"num_envs_per_worker": 2,
|
59 |
+
"batch_size": 1024,
|
60 |
+
"num_batches_per_epoch": 8,
|
61 |
+
"num_epochs": 4,
|
62 |
+
"rollout": 128,
|
63 |
+
"recurrence": 1,
|
64 |
+
"shuffle_minibatches": false,
|
65 |
+
"gamma": 0.99,
|
66 |
+
"reward_scale": 1.0,
|
67 |
+
"reward_clip": 1000.0,
|
68 |
+
"value_bootstrap": false,
|
69 |
+
"normalize_returns": true,
|
70 |
+
"exploration_loss_coeff": 0.0004677351413,
|
71 |
+
"value_loss_coeff": 0.5,
|
72 |
+
"kl_loss_coeff": 0.0,
|
73 |
+
"exploration_loss": "entropy",
|
74 |
+
"gae_lambda": 0.95,
|
75 |
+
"ppo_clip_ratio": 0.1,
|
76 |
+
"ppo_clip_value": 1.0,
|
77 |
+
"with_vtrace": false,
|
78 |
+
"vtrace_rho": 1.0,
|
79 |
+
"vtrace_c": 1.0,
|
80 |
+
"optimizer": "adam",
|
81 |
+
"adam_eps": 1e-05,
|
82 |
+
"adam_beta1": 0.9,
|
83 |
+
"adam_beta2": 0.999,
|
84 |
+
"max_grad_norm": 0.0,
|
85 |
+
"learning_rate": 0.0003033891184,
|
86 |
+
"lr_schedule": "linear_decay",
|
87 |
+
"lr_schedule_kl_threshold": 0.008,
|
88 |
+
"lr_adaptive_min": 1e-06,
|
89 |
+
"lr_adaptive_max": 0.01,
|
90 |
+
"obs_subtract_mean": 0.0,
|
91 |
+
"obs_scale": 255.0,
|
92 |
+
"normalize_input": true,
|
93 |
+
"normalize_input_keys": [
|
94 |
+
"obs"
|
95 |
+
],
|
96 |
+
"decorrelate_experience_max_seconds": 0,
|
97 |
+
"decorrelate_envs_on_one_worker": true,
|
98 |
+
"actor_worker_gpus": [],
|
99 |
+
"set_workers_cpu_affinity": true,
|
100 |
+
"force_envs_single_thread": false,
|
101 |
+
"default_niceness": 0,
|
102 |
+
"log_to_file": true,
|
103 |
+
"experiment_summaries_interval": 3,
|
104 |
+
"flush_summaries_interval": 30,
|
105 |
+
"stats_avg": 100,
|
106 |
+
"summaries_use_frameskip": true,
|
107 |
+
"heartbeat_interval": 10,
|
108 |
+
"heartbeat_reporting_interval": 60,
|
109 |
+
"train_for_env_steps": 100000000,
|
110 |
+
"train_for_seconds": 10000000000,
|
111 |
+
"save_every_sec": 120,
|
112 |
+
"keep_checkpoints": 2,
|
113 |
+
"load_checkpoint_kind": "latest",
|
114 |
+
"save_milestones_sec": 1200,
|
115 |
+
"save_best_every_sec": 5,
|
116 |
+
"save_best_metric": "reward",
|
117 |
+
"save_best_after": 100000,
|
118 |
+
"benchmark": false,
|
119 |
+
"encoder_mlp_layers": [
|
120 |
+
512,
|
121 |
+
512
|
122 |
+
],
|
123 |
+
"encoder_conv_architecture": "convnet_atari",
|
124 |
+
"encoder_conv_mlp_layers": [
|
125 |
+
512
|
126 |
+
],
|
127 |
+
"use_rnn": false,
|
128 |
+
"rnn_size": 512,
|
129 |
+
"rnn_type": "gru",
|
130 |
+
"rnn_num_layers": 1,
|
131 |
+
"decoder_mlp_layers": [],
|
132 |
+
"nonlinearity": "relu",
|
133 |
+
"policy_initialization": "orthogonal",
|
134 |
+
"policy_init_gain": 1.0,
|
135 |
+
"actor_critic_share_weights": true,
|
136 |
+
"adaptive_stddev": false,
|
137 |
+
"continuous_tanh_scale": 0.0,
|
138 |
+
"initial_stddev": 1.0,
|
139 |
+
"use_env_info_cache": false,
|
140 |
+
"env_gpu_actions": false,
|
141 |
+
"env_gpu_observations": true,
|
142 |
+
"env_frameskip": 4,
|
143 |
+
"env_framestack": 4,
|
144 |
+
}
|
145 |
+
|
146 |
+
```
|
147 |
+
|
148 |
+
|
149 |
+
|
150 |
## Using the model
|
151 |
|
152 |
To run the model after download, use the `enjoy` script corresponding to this environment:
|
153 |
```
|
154 |
+
python -m sf_examples.atari.enjoy_atari --algo=APPO --env=atari_beamrider --train_dir=./train_dir --experiment=APPO-atari_beamrider
|
155 |
```
|
156 |
|
157 |
|
|
|
162 |
|
163 |
To continue training with this model, use the `train` script corresponding to this environment:
|
164 |
```
|
165 |
+
python -m sf_examples.atari.train_atari --algo=APPO --env=atari_beamrider --train_dir=./train_dir --experiment=APPO-atari_beamrider --restart_behavior=resume --train_for_env_steps=10000000000
|
166 |
```
|
167 |
|
168 |
Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
|
checkpoint_p0/best_000093792_96043008_reward_41.570.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3dd6f929e8e3050ce836f7bc890ed2093ca69a5b6d2fb5bf0f7bc80128353555
|
3 |
+
size 20740275
|
checkpoint_p0/checkpoint_000097344_99680256.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:fdf15979bd57b31a05e65351b426e17e0f30c73178c45fd2cb260969999b3b69
|
3 |
+
size 20740611
|
checkpoint_p0/checkpoint_000097664_100007936.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:4591ca1cabda604ed3f08f3192887886b372ab41ee4cd37775f5df6446903a98
|
3 |
+
size 20740611
|
checkpoint_p0/milestones/checkpoint_000007904_8093696.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:b772a8693fda3c1cbbdeb9ef22f2c275a2d6b3d2579c93da47d2e9d83e9bc2b6
|
3 |
+
size 20741467
|
checkpoint_p0/milestones/checkpoint_000015872_16252928.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ada809150ce1d470fa5ffa1bd61c77a1542ea1ca6d3f060828d742d0eb2bca7b
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000023872_24444928.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:e37fdbab1e97aa40080d6a3ef67ae36c53f3a8f2732991cf14e6b6e08e198934
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000031840_32604160.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:58bcc154e235b42230a2a77c9bfb5b1a6c84f44643a55fc81ccdafa80defacae
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000039840_40796160.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3ef7bef446fb55426022cbe3d35929b10f27a9dd11d847977477d56c090f1ccf
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000047808_48955392.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:4001234168ff892a41694a32ed9ab14abbc7e1842c9975fea744e5911f672a54
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000055808_57147392.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:74815a68302f85ddde3d2ce064927a6423d10a8c0b93ff8e2258828b8a875763
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000063808_65339392.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:2296578fe732af280207783beefd0114b9f0704a2cc729293de24658a21af417
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000071808_73531392.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:7fafc48c2f162989dfc3dd6a197dd297b4d68869cd46b62fbd326542fb032d08
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000079776_81690624.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:786fab8a815cdb369cc9541f00d4dc400a6088f1dd946c034a38e59bef02a0fa
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000087776_89882624.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:c5616ddf79efeaa7bc888b28903b500759a45847e57a6de99577fb84b76f7dd0
|
3 |
+
size 20741523
|
checkpoint_p0/milestones/checkpoint_000095744_98041856.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:638b6be15d652e2efeb0c703295f494d9f71148325c65daf91d5e2943b9e432c
|
3 |
+
size 20741523
|
checkpoint_p1/best_000090432_92602368_reward_39.550.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:bdbe689f1c3b4e12232ee3ebb571796ede6bef4869eedb9e93678a64548a10c3
|
3 |
+
size 20740275
|
checkpoint_p1/checkpoint_000098464_100827136.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:2cbc567f1f3c05221e1e97942ac30154f7305559a0610bfb06d6dbf72f09eaea
|
3 |
+
size 20740611
|
checkpoint_p1/checkpoint_000098784_101154816.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:60dec3b235cb2e92a97bbd90fa5ea58d126978a54e12e949e5f08f896f1168ff
|
3 |
+
size 20740611
|
checkpoint_p1/milestones/checkpoint_000007968_8159232.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:974b99a2716ca84a77f08286011257140608a9ca9cc81ed17237bf09fecbb6dd
|
3 |
+
size 20741467
|
checkpoint_p1/milestones/checkpoint_000016032_16416768.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:c7db2171703e14d718935e3d2b80cd5a22caadd75578d6c28f304ddeb4c940d0
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000024128_24707072.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:d1f91f37d413b56b761803c50ce1fa63df64b7cadbab4d98235b320b69e89697
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000032192_32964608.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3451241775acbc04f104e8b1bdc274a873efe69e807c6f688dd132a377c0794c
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000040288_41254912.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:214359c90825ec650d1cae8070032a6268f1551ed779c828bf68fd3618321c87
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000048384_49545216.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:bf927dff7eaee7dcc8c757f19d952528d3623d7efd638735c0b5218f25354cbb
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000056448_57802752.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:b3806932d775d0bf009d2005f6e3164051689198135e57ef36c05388cc9d7eed
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000064544_66093056.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:8b72e641648c40e1413f896b329c38c5e6a92825790228561a6e6a2334d4c329
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000072608_74350592.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:6476db6165e05e66862dbfe38ea48176c45fe444d7a93f355c59718d22af1399
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000080672_82608128.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:78b287b0f081db8463eaf7cc8769041f61ddfc5c96ee770c3d8501bd25a901fc
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000088768_90898432.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:266cd456c6b0230f55898ddf66cb29cd1b7131ebb95475ba466ff2b7b2be7b18
|
3 |
+
size 20741523
|
checkpoint_p1/milestones/checkpoint_000096864_99188736.pth
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:022cea07e9bc8af6c95ae96b3f49339613560dd3b6c259ee46e29b013ad40209
|
3 |
+
size 20741523
|
config.json
CHANGED
@@ -2,23 +2,23 @@
|
|
2 |
"help": false,
|
3 |
"algo": "APPO",
|
4 |
"env": "atari_beamrider",
|
5 |
-
"experiment": "
|
6 |
"train_dir": "./train_atari",
|
7 |
-
"restart_behavior": "
|
8 |
"device": "gpu",
|
9 |
-
"seed":
|
10 |
"num_policies": 2,
|
11 |
-
"async_rl":
|
12 |
"serial_mode": false,
|
13 |
-
"batched_sampling":
|
14 |
"num_batches_to_accumulate": 2,
|
15 |
"worker_num_splits": 1,
|
16 |
"policy_workers_per_policy": 1,
|
17 |
"max_policy_lag": 1000,
|
18 |
-
"num_workers":
|
19 |
-
"num_envs_per_worker":
|
20 |
-
"batch_size":
|
21 |
-
"num_batches_per_epoch":
|
22 |
"num_epochs": 4,
|
23 |
"rollout": 128,
|
24 |
"recurrence": 1,
|
@@ -28,7 +28,7 @@
|
|
28 |
"reward_clip": 1000.0,
|
29 |
"value_bootstrap": false,
|
30 |
"normalize_returns": true,
|
31 |
-
"exploration_loss_coeff": 0.
|
32 |
"value_loss_coeff": 0.5,
|
33 |
"kl_loss_coeff": 0.0,
|
34 |
"exploration_loss": "entropy",
|
@@ -42,8 +42,8 @@
|
|
42 |
"adam_eps": 1e-05,
|
43 |
"adam_beta1": 0.9,
|
44 |
"adam_beta2": 0.999,
|
45 |
-
"max_grad_norm": 0.
|
46 |
-
"learning_rate": 0.
|
47 |
"lr_schedule": "linear_decay",
|
48 |
"lr_schedule_kl_threshold": 0.008,
|
49 |
"lr_adaptive_min": 1e-06,
|
@@ -51,7 +51,9 @@
|
|
51 |
"obs_subtract_mean": 0.0,
|
52 |
"obs_scale": 255.0,
|
53 |
"normalize_input": true,
|
54 |
-
"normalize_input_keys":
|
|
|
|
|
55 |
"decorrelate_experience_max_seconds": 0,
|
56 |
"decorrelate_envs_on_one_worker": true,
|
57 |
"actor_worker_gpus": [],
|
@@ -63,14 +65,14 @@
|
|
63 |
"flush_summaries_interval": 30,
|
64 |
"stats_avg": 100,
|
65 |
"summaries_use_frameskip": true,
|
66 |
-
"heartbeat_interval":
|
67 |
-
"heartbeat_reporting_interval":
|
68 |
-
"train_for_env_steps":
|
69 |
"train_for_seconds": 10000000000,
|
70 |
"save_every_sec": 120,
|
71 |
"keep_checkpoints": 2,
|
72 |
"load_checkpoint_kind": "latest",
|
73 |
-
"save_milestones_sec":
|
74 |
"save_best_every_sec": 5,
|
75 |
"save_best_metric": "reward",
|
76 |
"save_best_after": 100000,
|
@@ -104,7 +106,7 @@
|
|
104 |
"use_record_episode_statistics": true,
|
105 |
"with_wandb": true,
|
106 |
"wandb_user": "matt-stammers",
|
107 |
-
"wandb_project": "
|
108 |
"wandb_group": "atari_beamrider",
|
109 |
"wandb_job_type": "SF",
|
110 |
"wandb_tags": [
|
@@ -122,16 +124,32 @@
|
|
122 |
"pbt_target_objective": "true_objective",
|
123 |
"pbt_perturb_min": 1.1,
|
124 |
"pbt_perturb_max": 1.5,
|
125 |
-
"command_line": "--algo=APPO --env=atari_beamrider --experiment=
|
126 |
"cli_args": {
|
127 |
"algo": "APPO",
|
128 |
"env": "atari_beamrider",
|
129 |
-
"experiment": "
|
130 |
"train_dir": "./train_atari",
|
|
|
|
|
131 |
"num_policies": 2,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
132 |
"with_wandb": true,
|
133 |
"wandb_user": "matt-stammers",
|
134 |
-
"wandb_project": "
|
135 |
"wandb_group": "atari_beamrider",
|
136 |
"wandb_job_type": "SF",
|
137 |
"wandb_tags": [
|
@@ -140,5 +158,5 @@
|
|
140 |
},
|
141 |
"git_hash": "5fff97c2f535da5987d358cdbe6927cccd43621e",
|
142 |
"git_repo_name": "not a git repository",
|
143 |
-
"wandb_unique_id": "
|
144 |
}
|
|
|
2 |
"help": false,
|
3 |
"algo": "APPO",
|
4 |
"env": "atari_beamrider",
|
5 |
+
"experiment": "atari_beamrider_APPO",
|
6 |
"train_dir": "./train_atari",
|
7 |
+
"restart_behavior": "restart",
|
8 |
"device": "gpu",
|
9 |
+
"seed": 1234,
|
10 |
"num_policies": 2,
|
11 |
+
"async_rl": true,
|
12 |
"serial_mode": false,
|
13 |
+
"batched_sampling": true,
|
14 |
"num_batches_to_accumulate": 2,
|
15 |
"worker_num_splits": 1,
|
16 |
"policy_workers_per_policy": 1,
|
17 |
"max_policy_lag": 1000,
|
18 |
+
"num_workers": 16,
|
19 |
+
"num_envs_per_worker": 2,
|
20 |
+
"batch_size": 1024,
|
21 |
+
"num_batches_per_epoch": 8,
|
22 |
"num_epochs": 4,
|
23 |
"rollout": 128,
|
24 |
"recurrence": 1,
|
|
|
28 |
"reward_clip": 1000.0,
|
29 |
"value_bootstrap": false,
|
30 |
"normalize_returns": true,
|
31 |
+
"exploration_loss_coeff": 0.0004677351413,
|
32 |
"value_loss_coeff": 0.5,
|
33 |
"kl_loss_coeff": 0.0,
|
34 |
"exploration_loss": "entropy",
|
|
|
42 |
"adam_eps": 1e-05,
|
43 |
"adam_beta1": 0.9,
|
44 |
"adam_beta2": 0.999,
|
45 |
+
"max_grad_norm": 0.0,
|
46 |
+
"learning_rate": 0.0003033891184,
|
47 |
"lr_schedule": "linear_decay",
|
48 |
"lr_schedule_kl_threshold": 0.008,
|
49 |
"lr_adaptive_min": 1e-06,
|
|
|
51 |
"obs_subtract_mean": 0.0,
|
52 |
"obs_scale": 255.0,
|
53 |
"normalize_input": true,
|
54 |
+
"normalize_input_keys": [
|
55 |
+
"obs"
|
56 |
+
],
|
57 |
"decorrelate_experience_max_seconds": 0,
|
58 |
"decorrelate_envs_on_one_worker": true,
|
59 |
"actor_worker_gpus": [],
|
|
|
65 |
"flush_summaries_interval": 30,
|
66 |
"stats_avg": 100,
|
67 |
"summaries_use_frameskip": true,
|
68 |
+
"heartbeat_interval": 10,
|
69 |
+
"heartbeat_reporting_interval": 60,
|
70 |
+
"train_for_env_steps": 100000000,
|
71 |
"train_for_seconds": 10000000000,
|
72 |
"save_every_sec": 120,
|
73 |
"keep_checkpoints": 2,
|
74 |
"load_checkpoint_kind": "latest",
|
75 |
+
"save_milestones_sec": 1200,
|
76 |
"save_best_every_sec": 5,
|
77 |
"save_best_metric": "reward",
|
78 |
"save_best_after": 100000,
|
|
|
106 |
"use_record_episode_statistics": true,
|
107 |
"with_wandb": true,
|
108 |
"wandb_user": "matt-stammers",
|
109 |
+
"wandb_project": "atari_APPO",
|
110 |
"wandb_group": "atari_beamrider",
|
111 |
"wandb_job_type": "SF",
|
112 |
"wandb_tags": [
|
|
|
124 |
"pbt_target_objective": "true_objective",
|
125 |
"pbt_perturb_min": 1.1,
|
126 |
"pbt_perturb_max": 1.5,
|
127 |
+
"command_line": "--algo=APPO --env=atari_beamrider --experiment=atari_beamrider_APPO --num_policies=2 --restart_behavior=restart --train_dir=./train_atari --train_for_env_steps=100000000 --seed=1234 --num_workers=16 --num_envs_per_worker=2 --num_batches_per_epoch=8 --async_rl=true --batched_sampling=true --batch_size=1024 --max_grad_norm=0 --learning_rate=0.0003033891184 --heartbeat_interval=10 --heartbeat_reporting_interval=60 --save_milestones_sec=1200 --num_epochs=4 --exploration_loss_coeff=0.0004677351413 --with_wandb=true --wandb_user=matt-stammers --wandb_project=atari_APPO --wandb_group=atari_beamrider --wandb_job_type=SF --wandb_tags=atari",
|
128 |
"cli_args": {
|
129 |
"algo": "APPO",
|
130 |
"env": "atari_beamrider",
|
131 |
+
"experiment": "atari_beamrider_APPO",
|
132 |
"train_dir": "./train_atari",
|
133 |
+
"restart_behavior": "restart",
|
134 |
+
"seed": 1234,
|
135 |
"num_policies": 2,
|
136 |
+
"async_rl": true,
|
137 |
+
"batched_sampling": true,
|
138 |
+
"num_workers": 16,
|
139 |
+
"num_envs_per_worker": 2,
|
140 |
+
"batch_size": 1024,
|
141 |
+
"num_batches_per_epoch": 8,
|
142 |
+
"num_epochs": 4,
|
143 |
+
"exploration_loss_coeff": 0.0004677351413,
|
144 |
+
"max_grad_norm": 0.0,
|
145 |
+
"learning_rate": 0.0003033891184,
|
146 |
+
"heartbeat_interval": 10,
|
147 |
+
"heartbeat_reporting_interval": 60,
|
148 |
+
"train_for_env_steps": 100000000,
|
149 |
+
"save_milestones_sec": 1200,
|
150 |
"with_wandb": true,
|
151 |
"wandb_user": "matt-stammers",
|
152 |
+
"wandb_project": "atari_APPO",
|
153 |
"wandb_group": "atari_beamrider",
|
154 |
"wandb_job_type": "SF",
|
155 |
"wandb_tags": [
|
|
|
158 |
},
|
159 |
"git_hash": "5fff97c2f535da5987d358cdbe6927cccd43621e",
|
160 |
"git_repo_name": "not a git repository",
|
161 |
+
"wandb_unique_id": "atari_beamrider_APPO_20231009_040728_476822"
|
162 |
}
|
git.diff
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3357904f421d3f4924836316b1741bf64d5dd0e807d5e80ac07059b4c52a7008
|
3 |
+
size 14426734
|
replay.mp4
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:1cb49473c602f7d682d0503b509f4c91414ceb33f30e084ccbc5976243d2eb78
|
3 |
+
size 24413627
|
sf_log.txt
CHANGED
The diff for this file is too large to render.
See raw diff
|
|