Hugging Face's logo

MattStammers
/
appo-mujoco_standup

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files Metrics Community
MattStammers commited on
Commit
2cf3ad6
•
1 Parent(s): fc78da2

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +2 -0
  2. .summary/0/events.out.tfevents.1704031528.rhmmedcatt-proliant-ml350-gen10 +3 -0
  3. .summary/1/events.out.tfevents.1704031528.rhmmedcatt-proliant-ml350-gen10 +3 -0
  4. README.md +189 -0
  5. checkpoint_p0/best_001625216_416079872_reward_254903.875.pth +3 -0
  6. checkpoint_p0/checkpoint_001952896_499965952.pth +3 -0
  7. checkpoint_p0/checkpoint_001953056_500006912.pth +3 -0
  8. checkpoint_p0/milestones/checkpoint_000023648_6053888.pth +3 -0
  9. checkpoint_p0/milestones/checkpoint_000044672_11436032.pth +3 -0
  10. checkpoint_p0/milestones/checkpoint_000062112_15900672.pth +3 -0
  11. checkpoint_p0/milestones/checkpoint_000079296_20299776.pth +3 -0
  12. checkpoint_p0/milestones/checkpoint_000096352_24666112.pth +3 -0
  13. checkpoint_p0/milestones/checkpoint_000112928_28909568.pth +3 -0
  14. checkpoint_p0/milestones/checkpoint_000129824_33234944.pth +3 -0
  15. checkpoint_p0/milestones/checkpoint_000146816_37584896.pth +3 -0
  16. checkpoint_p0/milestones/checkpoint_000163416_41836544.pth +3 -0
  17. checkpoint_p0/milestones/checkpoint_000180056_46096384.pth +3 -0
  18. checkpoint_p0/milestones/checkpoint_000196792_50380800.pth +3 -0
  19. checkpoint_p0/milestones/checkpoint_000214200_54837248.pth +3 -0
  20. checkpoint_p0/milestones/checkpoint_000231224_59195392.pth +3 -0
  21. checkpoint_p0/milestones/checkpoint_000248504_63619072.pth +3 -0
  22. checkpoint_p0/milestones/checkpoint_000265784_68042752.pth +3 -0
  23. checkpoint_p0/milestones/checkpoint_000283736_72638464.pth +3 -0
  24. checkpoint_p0/milestones/checkpoint_000301720_77242368.pth +3 -0
  25. checkpoint_p0/milestones/checkpoint_000319896_81895424.pth +3 -0
  26. checkpoint_p0/milestones/checkpoint_000337976_86523904.pth +3 -0
  27. checkpoint_p0/milestones/checkpoint_000355896_91111424.pth +3 -0
  28. checkpoint_p0/milestones/checkpoint_000373464_95608832.pth +3 -0
  29. checkpoint_p0/milestones/checkpoint_000390616_99999744.pth +3 -0
  30. checkpoint_p0/milestones/checkpoint_000407832_104407040.pth +3 -0
  31. checkpoint_p0/milestones/checkpoint_000424632_108707840.pth +3 -0
  32. checkpoint_p0/milestones/checkpoint_000442008_113156096.pth +3 -0
  33. checkpoint_p0/milestones/checkpoint_000459384_117604352.pth +3 -0
  34. checkpoint_p0/milestones/checkpoint_000476632_122019840.pth +3 -0
  35. checkpoint_p0/milestones/checkpoint_000493784_126410752.pth +3 -0
  36. checkpoint_p0/milestones/checkpoint_000511448_130932736.pth +3 -0
  37. checkpoint_p0/milestones/checkpoint_000528792_135372800.pth +3 -0
  38. checkpoint_p0/milestones/checkpoint_000546096_139804672.pth +3 -0
  39. checkpoint_p0/milestones/checkpoint_000562992_144130048.pth +3 -0
  40. checkpoint_p0/milestones/checkpoint_000579856_148447232.pth +3 -0
  41. checkpoint_p0/milestones/checkpoint_000596560_152723456.pth +3 -0
  42. checkpoint_p0/milestones/checkpoint_000613448_157048832.pth +3 -0
  43. checkpoint_p0/milestones/checkpoint_000630440_161398784.pth +3 -0
  44. checkpoint_p0/milestones/checkpoint_000647176_165683200.pth +3 -0
  45. checkpoint_p0/milestones/checkpoint_000664168_170033152.pth +3 -0
  46. checkpoint_p0/milestones/checkpoint_000681256_174407680.pth +3 -0
  47. checkpoint_p0/milestones/checkpoint_000698152_178733056.pth +3 -0
  48. checkpoint_p0/milestones/checkpoint_000714792_182992896.pth +3 -0
  49. checkpoint_p0/milestones/checkpoint_000731496_187269120.pth +3 -0
  50. checkpoint_p0/milestones/checkpoint_000748248_191561728.pth +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ replay.mp4 filter=lfs diff=lfs merge=lfs -text
37
+ sf_log.txt filter=lfs diff=lfs merge=lfs -text
.summary/0/events.out.tfevents.1704031528.rhmmedcatt-proliant-ml350-gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae365091044c903c3879c491c8fbec765cd2f497f1c7827c2139f4e4e9dcb3df
3
+ size 57547101
.summary/1/events.out.tfevents.1704031528.rhmmedcatt-proliant-ml350-gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad677703ce4eea591d84a6e8c0ec9b1d864c109e58d5f494cf2c16cf454c2dbe
3
+ size 26903276
README.md ADDED
@@ -0,0 +1,189 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sample-factory
3
+ tags:
4
+ - deep-reinforcement-learning
5
+ - reinforcement-learning
6
+ - sample-factory
7
+ model-index:
8
+ - name: APPO
9
+ results:
10
+ - task:
11
+ type: reinforcement-learning
12
+ name: reinforcement-learning
13
+ dataset:
14
+ name: mujoco_standup
15
+ type: mujoco_standup
16
+ metrics:
17
+ - type: mean_reward
18
+ value: 252155.88 +/- 3218.55
19
+ name: mean_reward
20
+ verified: false
21
+ ---
22
+
23
+ ## About the Project
24
+
25
+ This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist).
26
+
27
+ In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
28
+
29
+ ## Project Aims
30
+
31
+ This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance.
32
+
33
+ I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota.
34
+
35
+ The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels.
36
+
37
+ After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
38
+
39
+
40
+ ## About the Model
41
+
42
+ The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
43
+ ```
44
+ hyperparameters = {
45
+ "help": false,
46
+ "algo": "APPO",
47
+ "env": "atari_asteroid",
48
+ "experiment": "atari_asteroid_APPO",
49
+ "train_dir": "./train_atari",
50
+ "restart_behavior": "restart",
51
+ "device": "gpu",
52
+ "seed": 1234,
53
+ "num_policies": 2,
54
+ "async_rl": true,
55
+ "serial_mode": false,
56
+ "batched_sampling": true,
57
+ "num_batches_to_accumulate": 2,
58
+ "worker_num_splits": 1,
59
+ "policy_workers_per_policy": 1,
60
+ "max_policy_lag": 1000,
61
+ "num_workers": 16,
62
+ "num_envs_per_worker": 2,
63
+ "batch_size": 1024,
64
+ "num_batches_per_epoch": 8,
65
+ "num_epochs": 4,
66
+ "rollout": 128,
67
+ "recurrence": 1,
68
+ "shuffle_minibatches": false,
69
+ "gamma": 0.99,
70
+ "reward_scale": 1.0,
71
+ "reward_clip": 1000.0,
72
+ "value_bootstrap": false,
73
+ "normalize_returns": true,
74
+ "exploration_loss_coeff": 0.0004677351413,
75
+ "value_loss_coeff": 0.5,
76
+ "kl_loss_coeff": 0.0,
77
+ "exploration_loss": "entropy",
78
+ "gae_lambda": 0.95,
79
+ "ppo_clip_ratio": 0.1,
80
+ "ppo_clip_value": 1.0,
81
+ "with_vtrace": true,
82
+ "vtrace_rho": 1.0,
83
+ "vtrace_c": 1.0,
84
+ "optimizer": "adam",
85
+ "adam_eps": 1e-05,
86
+ "adam_beta1": 0.9,
87
+ "adam_beta2": 0.999,
88
+ "max_grad_norm": 0.0,
89
+ "learning_rate": 0.0003033891184,
90
+ "lr_schedule": "linear_decay",
91
+ "lr_schedule_kl_threshold": 0.008,
92
+ "lr_adaptive_min": 1e-06,
93
+ "lr_adaptive_max": 0.01,
94
+ "obs_subtract_mean": 0.0,
95
+ "obs_scale": 255.0,
96
+ "normalize_input": true,
97
+ "normalize_input_keys": [
98
+ "obs"
99
+ ],
100
+ "decorrelate_experience_max_seconds": 0,
101
+ "decorrelate_envs_on_one_worker": true,
102
+ "actor_worker_gpus": [],
103
+ "set_workers_cpu_affinity": true,
104
+ "force_envs_single_thread": false,
105
+ "default_niceness": 0,
106
+ "log_to_file": true,
107
+ "experiment_summaries_interval": 3,
108
+ "flush_summaries_interval": 30,
109
+ "stats_avg": 100,
110
+ "summaries_use_frameskip": true,
111
+ "heartbeat_interval": 10,
112
+ "heartbeat_reporting_interval": 60,
113
+ "train_for_env_steps": 100000000,
114
+ "train_for_seconds": 10000000000,
115
+ "save_every_sec": 120,
116
+ "keep_checkpoints": 2,
117
+ "load_checkpoint_kind": "latest",
118
+ "save_milestones_sec": 1200,
119
+ "save_best_every_sec": 5,
120
+ "save_best_metric": "reward",
121
+ "save_best_after": 100000,
122
+ "benchmark": false,
123
+ "encoder_mlp_layers": [
124
+ 512,
125
+ 512
126
+ ],
127
+ "encoder_conv_architecture": "convnet_atari",
128
+ "encoder_conv_mlp_layers": [
129
+ 512
130
+ ],
131
+ "use_rnn": false,
132
+ "rnn_size": 512,
133
+ "rnn_type": "gru",
134
+ "rnn_num_layers": 1,
135
+ "decoder_mlp_layers": [],
136
+ "nonlinearity": "relu",
137
+ "policy_initialization": "orthogonal",
138
+ "policy_init_gain": 1.0,
139
+ "actor_critic_share_weights": true,
140
+ "adaptive_stddev": false,
141
+ "continuous_tanh_scale": 0.0,
142
+ "initial_stddev": 1.0,
143
+ "use_env_info_cache": false,
144
+ "env_gpu_actions": false,
145
+ "env_gpu_observations": true,
146
+ "env_frameskip": 4,
147
+ "env_framestack": 4,
148
+ "pixel_format": "CHW"
149
+ }
150
+
151
+ ```
152
+
153
+
154
+
155
+ A(n) **APPO** impala model trained on the **mujoco_standup** environment.
156
+
157
+ This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a
158
+ high throughput on-policy RL framework. I have been using
159
+ Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
160
+
161
+
162
+ ## Downloading the model
163
+
164
+ After installing Sample-Factory, download the model with:
165
+ ```
166
+ python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-mujoco_standup
167
+ ```
168
+
169
+
170
+ ## Using the model
171
+
172
+ To run the model after download, use the `enjoy` script corresponding to this environment:
173
+ ```
174
+ python -m sf_examples.mujoco.enjoy_mujoco --algo=APPO --env=mujoco_standup --train_dir=./train_dir --experiment=APPO-mujoco_standup
175
+ ```
176
+
177
+
178
+ You can also upload models to the Hugging Face Hub using the same script with the `--push_to_hub` flag.
179
+ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
180
+
181
+ ## Training with this model
182
+
183
+ To continue training with this model, use the `train` script corresponding to this environment:
184
+ ```
185
+ python -m sf_examples.mujoco.train_mujoco --algo=APPO --env=mujoco_standup --train_dir=./train_dir --experiment=APPO-mujoco_standup --restart_behavior=resume --train_for_env_steps=10000000000
186
+ ```
187
+
188
+ Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
189
+
checkpoint_p0/best_001625216_416079872_reward_254903.875.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d20941864b35ca74604fc7e9baec0e94d49d63781d2b550e3e70bb91765e234
3
+ size 373375
checkpoint_p0/checkpoint_001952896_499965952.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f472c04fbee1ff980435f9f1dadaba300f461649d64f948864cf6fbb60c8e88
3
+ size 373639
checkpoint_p0/checkpoint_001953056_500006912.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95613780cc095d05fdd0ca791e71149df3ee0c8421f61ef254431cab8aa00a87
3
+ size 373639
checkpoint_p0/milestones/checkpoint_000023648_6053888.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6cf199fb915ffc5f3b24d2feac31428d8408b2db23d0e833125cc8693bcfa7dc
3
+ size 374275
checkpoint_p0/milestones/checkpoint_000044672_11436032.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c399e757d8d37452f353d7018912db26765b02235e065e6d6ac85285a6d73264
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000062112_15900672.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c47eaf503d41474549115491e145481e1e32de0ca6f829437c9425a3dd78db42
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000079296_20299776.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e62d5d695530864ac109c5d718d34df3b486cdf5951510bb1a2d3454db42b89
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000096352_24666112.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a23adbc76830c8d5992b938d74c0e0387d021f22196d9effebd3a04b6b35aa5a
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000112928_28909568.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:180fbbc395c37168691be5d90dbce654d276a6687a2d535695642bb5e1cea92d
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000129824_33234944.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3cbc51a6c231ecbbc1bd2e64a0baad339d5bf781d8b126dc662725787c45604
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000146816_37584896.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:82cbe88c1d887e6e562900e23489e138ae6a7f1c7b5d40f30b82922f21074631
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000163416_41836544.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2478141d6312379269dd9a4e903d5091950381ebb072b7681f4d15396746b54f
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000180056_46096384.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2835643c444abdb4f980c20f579e8c32119920278970ef280359ac18f43f5647
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000196792_50380800.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2cfb3bf9efd22372c8d73a1d61c7330697a9dc590e1eab1ffc81611a247b992b
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000214200_54837248.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aab9939576bf92cfa14ac39e14b9a7d551cee1b2a184c4714ebe55e7ca182c03
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000231224_59195392.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f201aea54d551a489be237d5596121b89b0682bca03a12e5129c5a93fd648fe
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000248504_63619072.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4450cb5808b1489bbc227ae1af58f282b8bb66db84c5c2ef3e3d44bffbfbc876
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000265784_68042752.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4475c0f63bbaae6a36b6d5b636619917a8237fee9acb41c3073785f909d679b1
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000283736_72638464.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5b65c5fd96c506009d8eaf0024438527e55c8189b7a659c4c7a1d5a7a8618442
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000301720_77242368.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:394edceddd196453402205c86992797556d268b2f400e7c18fe687d74c255be9
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000319896_81895424.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9cda36d168ad7e485827219ef420b527bc47cde896f2880059d2d34540730fc4
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000337976_86523904.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bbd14da6178738edf16f8f3aea59f3a4d519d682022bfdee5480240f9e208a5b
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000355896_91111424.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f635cb87713ceafbf04e0472e9b8c1ebe729a8d0586f27567e56e6053feda26
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000373464_95608832.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a99a90c39731919e89831b0c5edeccedaf1d5440d273384566b6578d733519d7
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000390616_99999744.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:871914a7c3aeb3c2f4866617b349ccda2f5fd00bc0e110ac8f5ae9e45885e758
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000407832_104407040.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c65bb673689c70e544b00bf9a6e89380adac30eeef1a0a3d309d1900a9fa865e
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000424632_108707840.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6d5d4b085fe833adc1c8926bb76d448a8ced208709e7c1b5de3dfe9777e17651
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000442008_113156096.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1095e9ca9efd2eb9b25f3447c706bab72858e8920596e7a94048570bf6841ce4
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000459384_117604352.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:438e3ea3cc3024674651338a196ec3deba6f490dc111184720fa78e0bd0f4503
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000476632_122019840.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ab445fc376f98a834d6e1cffb104ddfd0410712384a8c8c764dad531988db3b
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000493784_126410752.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc11f6f5b565a4bc92382df70b7f1b91765da95bd66c6de70c5a857817f50ce8
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000511448_130932736.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f36322b690b3091f1fb4ff7ba3764c7be918ef4cdf1a7122aa6049dc71fa1ea5
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000528792_135372800.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:778a4b0342c8aa694d6c1ec3be0b29bec6e7bcdd6e25b6d6ee7c0e6dbb35fa5e
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000546096_139804672.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6a6a92d147b4d77748caa79f4c126b26fb859517554f986d34dc8afe0f6ce574
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000562992_144130048.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54560cf95f654a9905244fd76d6ee63e6350635feb1f1eca3e376c38639e71aa
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000579856_148447232.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3dee67c74296a113473c0215df7717bebd30decab01693090a939db066b2f1ee
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000596560_152723456.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b61db6b8cfd948b2a34a68d7843c4e2a46485b15f2f7a76bb3cbb82bd8adc555
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000613448_157048832.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1093dd43cbeb52534394569480baff4984309d5269f66cf667df0af32c8a463e
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000630440_161398784.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1585bf8a6e35a4c6cd390a814de3a8784b24423cb3e8981eba6231f3ac4e029
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000647176_165683200.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:371207c6a36e0f5796916e738df869735ea62734d2a323a6604fb7437c3ea028
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000664168_170033152.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09081c58774ed26f9a2fbde2888aa7866519bf2c67a23381cdf0e872476e4efd
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000681256_174407680.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91a0d2237d3560bfb766e12050b569bb2133ae1507227489fdbd6c4c63ae70a6
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000698152_178733056.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d42fbfcc8e437dc360d93ffd5001f56b0a718539e3055f2b4b4a772a3a7a3932
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000714792_182992896.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87ea236f53b1be07c6429ca9637da07ee7e0d14aaca8e16828e1e795dfe59aba
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000731496_187269120.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5855b7de68e5aa64ce1913b62adf015b02cbfd6250e3b109fcd1f6b7c69dfa1
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000748248_191561728.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:25ab060261e82b49b717e2f0ec61cb3a0ff62a52d51d5931f89f760f22e82587
3
+ size 374427