Hugging Face's logo

MattStammers
/
appo-atari_qbert-superhuman2

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files Metrics Community
MattStammers commited on
Commit
27ea712
•
1 Parent(s): b037d17

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +2 -0
  2. .summary/0/events.out.tfevents.1700364845.rhmmedcatt-proliant-ml350-gen10 +3 -0
  3. .summary/1/events.out.tfevents.1700364845.rhmmedcatt-proliant-ml350-gen10 +3 -0
  4. README.md +189 -0
  5. checkpoint_p0/best_001911416_489324544_reward_216.830.pth +3 -0
  6. checkpoint_p0/checkpoint_001961560_504324096.pth +3 -0
  7. checkpoint_p0/checkpoint_001961752_504422400.pth +3 -0
  8. checkpoint_p0/milestones/checkpoint_000012576_3219456.pth +3 -0
  9. checkpoint_p0/milestones/checkpoint_000025440_6512640.pth +3 -0
  10. checkpoint_p0/milestones/checkpoint_000038464_9846784.pth +3 -0
  11. checkpoint_p0/milestones/checkpoint_000051456_13172736.pth +3 -0
  12. checkpoint_p0/milestones/checkpoint_000064416_16490496.pth +3 -0
  13. checkpoint_p0/milestones/checkpoint_000077408_19816448.pth +3 -0
  14. checkpoint_p0/milestones/checkpoint_000090368_23134208.pth +3 -0
  15. checkpoint_p0/milestones/checkpoint_000103296_26443776.pth +3 -0
  16. checkpoint_p0/milestones/checkpoint_000116256_29761536.pth +3 -0
  17. checkpoint_p0/milestones/checkpoint_000129248_33087488.pth +3 -0
  18. checkpoint_p0/milestones/checkpoint_000142208_36405248.pth +3 -0
  19. checkpoint_p0/milestones/checkpoint_000155200_39731200.pth +3 -0
  20. checkpoint_p0/milestones/checkpoint_000168160_43048960.pth +3 -0
  21. checkpoint_p0/milestones/checkpoint_000181120_46366720.pth +3 -0
  22. checkpoint_p0/milestones/checkpoint_000194080_49684480.pth +3 -0
  23. checkpoint_p0/milestones/checkpoint_000207040_53002240.pth +3 -0
  24. checkpoint_p0/milestones/checkpoint_000219968_56311808.pth +3 -0
  25. checkpoint_p0/milestones/checkpoint_000232960_59637760.pth +3 -0
  26. checkpoint_p0/milestones/checkpoint_000245952_62963712.pth +3 -0
  27. checkpoint_p0/milestones/checkpoint_000258912_66281472.pth +3 -0
  28. checkpoint_p0/milestones/checkpoint_000271904_69607424.pth +3 -0
  29. checkpoint_p0/milestones/checkpoint_000284864_72925184.pth +3 -0
  30. checkpoint_p0/milestones/checkpoint_000297824_76242944.pth +3 -0
  31. checkpoint_p0/milestones/checkpoint_000310848_79577088.pth +3 -0
  32. checkpoint_p0/milestones/checkpoint_000323776_82886656.pth +3 -0
  33. checkpoint_p0/milestones/checkpoint_000336672_86188032.pth +3 -0
  34. checkpoint_p0/milestones/checkpoint_000349696_89522176.pth +3 -0
  35. checkpoint_p0/milestones/checkpoint_000362752_92864512.pth +3 -0
  36. checkpoint_p0/milestones/checkpoint_000375712_96182272.pth +3 -0
  37. checkpoint_p0/milestones/checkpoint_000388736_99516416.pth +3 -0
  38. checkpoint_p0/milestones/checkpoint_000401728_102842368.pth +3 -0
  39. checkpoint_p0/milestones/checkpoint_000414752_106176512.pth +3 -0
  40. checkpoint_p0/milestones/checkpoint_000427808_109518848.pth +3 -0
  41. checkpoint_p0/milestones/checkpoint_000440768_112836608.pth +3 -0
  42. checkpoint_p0/milestones/checkpoint_000453728_116154368.pth +3 -0
  43. checkpoint_p0/milestones/checkpoint_000466688_119472128.pth +3 -0
  44. checkpoint_p0/milestones/checkpoint_000479680_122798080.pth +3 -0
  45. checkpoint_p0/milestones/checkpoint_000492640_126115840.pth +3 -0
  46. checkpoint_p0/milestones/checkpoint_000505568_129425408.pth +3 -0
  47. checkpoint_p0/milestones/checkpoint_000518560_132751360.pth +3 -0
  48. checkpoint_p0/milestones/checkpoint_000531552_136077312.pth +3 -0
  49. checkpoint_p0/milestones/checkpoint_000544544_139403264.pth +3 -0
  50. checkpoint_p0/milestones/checkpoint_000557440_142704640.pth +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ replay.mp4 filter=lfs diff=lfs merge=lfs -text
37
+ sf_log.txt filter=lfs diff=lfs merge=lfs -text
.summary/0/events.out.tfevents.1700364845.rhmmedcatt-proliant-ml350-gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2df6b5f974b4b478379efdd7c94b17e734e474dda873a49ce97a87c01e9ae24
3
+ size 89592527
.summary/1/events.out.tfevents.1700364845.rhmmedcatt-proliant-ml350-gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d111d8bbaf68873cb57f1cd57bda5ec46bca63ab8467c029c043bff5e09be130
3
+ size 47062591
README.md ADDED
@@ -0,0 +1,189 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sample-factory
3
+ tags:
4
+ - deep-reinforcement-learning
5
+ - reinforcement-learning
6
+ - sample-factory
7
+ model-index:
8
+ - name: APPO
9
+ results:
10
+ - task:
11
+ type: reinforcement-learning
12
+ name: reinforcement-learning
13
+ dataset:
14
+ name: atari_qbert
15
+ type: atari_qbert
16
+ metrics:
17
+ - type: mean_reward
18
+ value: 45602.50 +/- 2068.10
19
+ name: mean_reward
20
+ verified: false
21
+ ---
22
+
23
+ ## About the Project
24
+
25
+ This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist).
26
+
27
+ In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
28
+
29
+ ## Project Aims
30
+
31
+ This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance.
32
+
33
+ I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota.
34
+
35
+ The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels.
36
+
37
+ After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
38
+
39
+
40
+ ## About the Model
41
+
42
+ The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
43
+ ```
44
+ hyperparameters = {
45
+ "help": false,
46
+ "algo": "APPO",
47
+ "env": "atari_asteroid",
48
+ "experiment": "atari_asteroid_APPO",
49
+ "train_dir": "./train_atari",
50
+ "restart_behavior": "restart",
51
+ "device": "gpu",
52
+ "seed": 1234,
53
+ "num_policies": 2,
54
+ "async_rl": true,
55
+ "serial_mode": false,
56
+ "batched_sampling": true,
57
+ "num_batches_to_accumulate": 2,
58
+ "worker_num_splits": 1,
59
+ "policy_workers_per_policy": 1,
60
+ "max_policy_lag": 1000,
61
+ "num_workers": 16,
62
+ "num_envs_per_worker": 2,
63
+ "batch_size": 1024,
64
+ "num_batches_per_epoch": 8,
65
+ "num_epochs": 4,
66
+ "rollout": 128,
67
+ "recurrence": 1,
68
+ "shuffle_minibatches": false,
69
+ "gamma": 0.99,
70
+ "reward_scale": 1.0,
71
+ "reward_clip": 1000.0,
72
+ "value_bootstrap": false,
73
+ "normalize_returns": true,
74
+ "exploration_loss_coeff": 0.0004677351413,
75
+ "value_loss_coeff": 0.5,
76
+ "kl_loss_coeff": 0.0,
77
+ "exploration_loss": "entropy",
78
+ "gae_lambda": 0.95,
79
+ "ppo_clip_ratio": 0.1,
80
+ "ppo_clip_value": 1.0,
81
+ "with_vtrace": false,
82
+ "vtrace_rho": 1.0,
83
+ "vtrace_c": 1.0,
84
+ "optimizer": "adam",
85
+ "adam_eps": 1e-05,
86
+ "adam_beta1": 0.9,
87
+ "adam_beta2": 0.999,
88
+ "max_grad_norm": 0.0,
89
+ "learning_rate": 0.0003033891184,
90
+ "lr_schedule": "linear_decay",
91
+ "lr_schedule_kl_threshold": 0.008,
92
+ "lr_adaptive_min": 1e-06,
93
+ "lr_adaptive_max": 0.01,
94
+ "obs_subtract_mean": 0.0,
95
+ "obs_scale": 255.0,
96
+ "normalize_input": true,
97
+ "normalize_input_keys": [
98
+ "obs"
99
+ ],
100
+ "decorrelate_experience_max_seconds": 0,
101
+ "decorrelate_envs_on_one_worker": true,
102
+ "actor_worker_gpus": [],
103
+ "set_workers_cpu_affinity": true,
104
+ "force_envs_single_thread": false,
105
+ "default_niceness": 0,
106
+ "log_to_file": true,
107
+ "experiment_summaries_interval": 3,
108
+ "flush_summaries_interval": 30,
109
+ "stats_avg": 100,
110
+ "summaries_use_frameskip": true,
111
+ "heartbeat_interval": 10,
112
+ "heartbeat_reporting_interval": 60,
113
+ "train_for_env_steps": 100000000,
114
+ "train_for_seconds": 10000000000,
115
+ "save_every_sec": 120,
116
+ "keep_checkpoints": 2,
117
+ "load_checkpoint_kind": "latest",
118
+ "save_milestones_sec": 1200,
119
+ "save_best_every_sec": 5,
120
+ "save_best_metric": "reward",
121
+ "save_best_after": 100000,
122
+ "benchmark": false,
123
+ "encoder_mlp_layers": [
124
+ 512,
125
+ 512
126
+ ],
127
+ "encoder_conv_architecture": "convnet_atari",
128
+ "encoder_conv_mlp_layers": [
129
+ 512
130
+ ],
131
+ "use_rnn": false,
132
+ "rnn_size": 512,
133
+ "rnn_type": "gru",
134
+ "rnn_num_layers": 1,
135
+ "decoder_mlp_layers": [],
136
+ "nonlinearity": "relu",
137
+ "policy_initialization": "orthogonal",
138
+ "policy_init_gain": 1.0,
139
+ "actor_critic_share_weights": true,
140
+ "adaptive_stddev": false,
141
+ "continuous_tanh_scale": 0.0,
142
+ "initial_stddev": 1.0,
143
+ "use_env_info_cache": false,
144
+ "env_gpu_actions": false,
145
+ "env_gpu_observations": true,
146
+ "env_frameskip": 4,
147
+ "env_framestack": 4,
148
+ "pixel_format": "CHW"
149
+ }
150
+
151
+ ```
152
+
153
+
154
+
155
+ A(n) **APPO** model trained on the **atari_qbert** environment.
156
+
157
+ This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a
158
+ high throughput on-policy RL framework. I have been using
159
+ Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
160
+
161
+
162
+ ## Downloading the model
163
+
164
+ After installing Sample-Factory, download the model with:
165
+ ```
166
+ python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_qbert
167
+ ```
168
+
169
+
170
+ ## Using the model
171
+
172
+ To run the model after download, use the `enjoy` script corresponding to this environment:
173
+ ```
174
+ python -m sf_examples.atari.enjoy_atari --algo=APPO --env=atari_qbert --train_dir=./train_dir --experiment=APPO-atari_qbert
175
+ ```
176
+
177
+
178
+ You can also upload models to the Hugging Face Hub using the same script with the `--push_to_hub` flag.
179
+ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
180
+
181
+ ## Training with this model
182
+
183
+ To continue training with this model, use the `train` script corresponding to this environment:
184
+ ```
185
+ python -m sf_examples.atari.train_atari --algo=APPO --env=atari_qbert --train_dir=./train_dir --experiment=APPO-atari_qbert --restart_behavior=resume --train_for_env_steps=10000000000
186
+ ```
187
+
188
+ Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
189
+
checkpoint_p0/best_001911416_489324544_reward_216.830.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5aec131ae9aaf68fc810a4ed74da365fb81720e873ce44497795a3ea34c48d3
3
+ size 20721843
checkpoint_p0/checkpoint_001961560_504324096.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1df17ec575a0e265341485ef1bc0a835f49f822dea6fc1896188b0d8514ba240
3
+ size 20722179
checkpoint_p0/checkpoint_001961752_504422400.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ab53dd824b95c3d2c96baae921235bfeedb429e21ef9459b8e0aa4e1c2005e7
3
+ size 20722179
checkpoint_p0/milestones/checkpoint_000012576_3219456.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ef9a921a37c4e3134d3b0b962b065ffbffee909d6247facc8b5a152f8e62851
3
+ size 20723035
checkpoint_p0/milestones/checkpoint_000025440_6512640.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bc89f2fa2b37a4ca99ec69d6c871b1e48f50e1b5311c99e8465cf7647f1bc08
3
+ size 20723035
checkpoint_p0/milestones/checkpoint_000038464_9846784.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6100cacac77d348bc7992dc0419b377cac68f99a3b855359aee8680635065af6
3
+ size 20723035
checkpoint_p0/milestones/checkpoint_000051456_13172736.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59057c5e0c7e476155b3481e59a63eecd6604abad590cc97fffbeaaae8896ec7
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000064416_16490496.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5fcff4250a73eef24cd2e0ef4a771ce5202b265c2a1a1633ff5b04c3d111b911
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000077408_19816448.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75236bd736c3f67789f06f177f1e9d70078b0beb29efce8b5184f897fc887332
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000090368_23134208.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a6bcfc22e3e7bdf5fb1a1fe8b576f2e40c2fde5b9d21022b3a9fa8ea5a7c881
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000103296_26443776.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3264b4453671233dedab9fc48394f815629cf69743593731cb7b6b9a6ed17c6
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000116256_29761536.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b76614b369e1888255644c42422dfbe99bcca848ed8489237aed35f61b0f386
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000129248_33087488.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58f6b2e15d82d4aa0210b0fed6b54a5efe7b60d6f9846eacac9e40c2596fb3ba
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000142208_36405248.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6dc3f2405505a3b817372315f89beacc3e199c9872c089f05643f62e5875add
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000155200_39731200.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f9aaaeeca9b024f271270473ab5b48aac72a5f7e0e5b19a289f089ea4aff292
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000168160_43048960.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f83b96ba715d52f7842be695026b05e69a68ef241b16882d0a029c935f9ba8b2
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000181120_46366720.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a160a593b7e2c2c3bf1385ef5cbe4d80abb270156e51931c4e7400f78cd6e9b3
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000194080_49684480.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5024b8b88c9009346836bb93626daee8f0d60cc17be9c5545f34e582106c861
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000207040_53002240.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:140d4dde7032fa189d7df8e356cd37dcdf1817a12c64347e958f0a7c2ecf8f7b
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000219968_56311808.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:00b702e17c20965b010e4660a55dd662cd50d76f6e972dce96c7e9ed87aa2d02
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000232960_59637760.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:24bbc9807ae5f57e1fe9a16d6348fc4e9bf865ef4643a8926cb98fa489ea82ce
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000245952_62963712.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4e90ccbd3a82ef12c87d9700aeb9e229ee52d6feb2e33e9426fea823103b525
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000258912_66281472.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40cf6ca24acba6321a0e62162ed2d1e3b1ef534e3dd6bd8743ee1099f14a414f
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000271904_69607424.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d79d3c029dc197f5f5b37eb565e917cf8940e0de647ba4a07effac720fc524d
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000284864_72925184.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc5216a5bf8e9a7b0c1f032ad4f1fad49f41bab98781e8975ac3bd4d37e91e3c
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000297824_76242944.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:855e46d015eb2830655787fc7f4c889cb56fca06f11647ab027fe5806df0f429
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000310848_79577088.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d0bb3afb08ddd97ebad2bf041139a679c7e42b6e38ad858631a37fe44d2c826
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000323776_82886656.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:107ace74ee0165cbb5e66c686ac99ebc797f2119cafd9edd6bf995aeb2e3497c
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000336672_86188032.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87dfa5cf43505790e960101ca5ec0f50ad381a5d8ad79b55e8306ec35a29da1f
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000349696_89522176.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be13370090e4f582b200f8ec4c4532e6de55dbd359517dea8c9a23ea3bffa7de
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000362752_92864512.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7841197c8c0be19fe4a6b6bce98cc39f95b22599dd1a7a86f994fe404694d4b
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000375712_96182272.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5f7b6e7fbf6f9dc06d86fc798707bb3c287827979ff6bd0d5728d6297e5e2bf
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000388736_99516416.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a44131a26c9bbbda1240154162ebf2968b8a27e93e9870c553310f47cd7cf6b2
3
+ size 20723091
checkpoint_p0/milestones/checkpoint_000401728_102842368.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b59d7141b193bc7f71bc3084d5fe159dbb90f2255f8ad20803d409d6273aedac
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000414752_106176512.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c88ee4e3408229ba4a4a7602882f2859d5d3c4fac2aff1f758abdc9e5ea378d
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000427808_109518848.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3809a26225729572d246f8d1ddea8409f6b8a8534214f3c07256052335894b50
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000440768_112836608.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9dfb88def96f931a4b72ecc21fc8b4d938cb138246fe635474f2db211f677fcb
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000453728_116154368.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:981823c5e0a73cbc66e335ab10b3531de38e827350fde5d5662cea09cd86baf2
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000466688_119472128.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:772937695ab663a1b8e097445edcedea91be274fd04eeed3b6dcc598ebd7ccbf
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000479680_122798080.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f615091486fed779829e3f7958b2abf790872bac25a3517d5c6e964534c10eb
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000492640_126115840.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:14eb46c34e1b49b3d8525a2729496cb19a92ab6f6f27090902b609a35e3b3c53
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000505568_129425408.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:252539a1e9dbef6bd2b0ab08f1f8a0a11a2874705b25ad07d0d83821ea88b323
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000518560_132751360.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60d5800c36a6eff7e2027da9dd8a9d1011b530b3b14b24b35a1aa8220599adad
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000531552_136077312.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa2c0f34c7d9017750afb2379f70eeaf04c775a4fde3ebed4beeeb1e33cc685a
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000544544_139403264.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8326a6a1423c11272aca10e59dcaa071c25ba26750aa959ccedcb2f19c63181e
3
+ size 20723147
checkpoint_p0/milestones/checkpoint_000557440_142704640.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a874efee483b6f722635c6d35807a9c603c35daf7f5cb2e6bea095bbe60b162
3
+ size 20723147