Hugging Face's logo

MattStammers
/
appo-mujoco_reacher-approaching_sota

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files Metrics Community
MattStammers commited on
Commit
8970264
•
1 Parent(s): 7e20b2c

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +1 -0
  2. .summary/0/events.out.tfevents.1703704421.rhmmedcatt-proliant-ml350-gen10 +3 -0
  3. .summary/1/events.out.tfevents.1703704421.rhmmedcatt-proliant-ml350-gen10 +3 -0
  4. README.md +189 -0
  5. checkpoint_p0/best_001267192_324460544_reward_-3.848.pth +3 -0
  6. checkpoint_p0/checkpoint_001952264_499859456.pth +3 -0
  7. checkpoint_p0/checkpoint_001952840_500006912.pth +3 -0
  8. checkpoint_p0/milestones/checkpoint_000046024_11788288.pth +3 -0
  9. checkpoint_p0/milestones/checkpoint_000092584_23707648.pth +3 -0
  10. checkpoint_p0/milestones/checkpoint_000138976_35586048.pth +3 -0
  11. checkpoint_p0/milestones/checkpoint_000185280_47439872.pth +3 -0
  12. checkpoint_p0/milestones/checkpoint_000231848_59367424.pth +3 -0
  13. checkpoint_p0/milestones/checkpoint_000278168_71237632.pth +3 -0
  14. checkpoint_p0/milestones/checkpoint_000324464_83091456.pth +3 -0
  15. checkpoint_p0/milestones/checkpoint_000370952_94994432.pth +3 -0
  16. checkpoint_p0/milestones/checkpoint_000416456_106643456.pth +3 -0
  17. checkpoint_p0/milestones/checkpoint_000462152_118341632.pth +3 -0
  18. checkpoint_p0/milestones/checkpoint_000508536_130220032.pth +3 -0
  19. checkpoint_p0/milestones/checkpoint_000554808_142065664.pth +3 -0
  20. checkpoint_p0/milestones/checkpoint_000600912_153870336.pth +3 -0
  21. checkpoint_p0/milestones/checkpoint_000647328_165756928.pth +3 -0
  22. checkpoint_p0/milestones/checkpoint_000692216_177258496.pth +3 -0
  23. checkpoint_p0/milestones/checkpoint_000738520_189112320.pth +3 -0
  24. checkpoint_p0/milestones/checkpoint_000784776_200957952.pth +3 -0
  25. checkpoint_p0/milestones/checkpoint_000831400_212893696.pth +3 -0
  26. checkpoint_p0/milestones/checkpoint_000877704_224747520.pth +3 -0
  27. checkpoint_p0/milestones/checkpoint_000923848_236560384.pth +3 -0
  28. checkpoint_p0/milestones/checkpoint_000970208_248430592.pth +3 -0
  29. checkpoint_p0/milestones/checkpoint_001016256_260218880.pth +3 -0
  30. checkpoint_p0/milestones/checkpoint_001062560_272072704.pth +3 -0
  31. checkpoint_p0/milestones/checkpoint_001108888_283934720.pth +3 -0
  32. checkpoint_p0/milestones/checkpoint_001155192_295788544.pth +3 -0
  33. checkpoint_p0/milestones/checkpoint_001201784_307716096.pth +3 -0
  34. checkpoint_p0/milestones/checkpoint_001248056_319561728.pth +3 -0
  35. checkpoint_p0/milestones/checkpoint_001294328_331407360.pth +3 -0
  36. checkpoint_p0/milestones/checkpoint_001340480_343228416.pth +3 -0
  37. checkpoint_p0/milestones/checkpoint_001387024_355147776.pth +3 -0
  38. checkpoint_p0/milestones/checkpoint_001433328_367001600.pth +3 -0
  39. checkpoint_p0/milestones/checkpoint_001479544_378839040.pth +3 -0
  40. checkpoint_p0/milestones/checkpoint_001525848_390692864.pth +3 -0
  41. checkpoint_p0/milestones/checkpoint_001572504_402636800.pth +3 -0
  42. checkpoint_p0/milestones/checkpoint_001618904_414515200.pth +3 -0
  43. checkpoint_p0/milestones/checkpoint_001665080_426336256.pth +3 -0
  44. checkpoint_p0/milestones/checkpoint_001711592_438247424.pth +3 -0
  45. checkpoint_p0/milestones/checkpoint_001757832_450084864.pth +3 -0
  46. checkpoint_p0/milestones/checkpoint_001804136_461938688.pth +3 -0
  47. checkpoint_p0/milestones/checkpoint_001850376_473776128.pth +3 -0
  48. checkpoint_p0/milestones/checkpoint_001896680_485629952.pth +3 -0
  49. checkpoint_p0/milestones/checkpoint_001943080_497508352.pth +3 -0
  50. checkpoint_p1/best_001902208_487022592_reward_-4.084.pth +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ sf_log.txt filter=lfs diff=lfs merge=lfs -text
.summary/0/events.out.tfevents.1703704421.rhmmedcatt-proliant-ml350-gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2febc9488351e317549a78a764fe414a955c04011f96e0cae6db581c4e538f4c
3
+ size 22296707
.summary/1/events.out.tfevents.1703704421.rhmmedcatt-proliant-ml350-gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4720fc77ace2105d899dc244d62137f592fa27419688316fdd01c47b0a288e75
3
+ size 10513946
README.md ADDED
@@ -0,0 +1,189 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sample-factory
3
+ tags:
4
+ - deep-reinforcement-learning
5
+ - reinforcement-learning
6
+ - sample-factory
7
+ model-index:
8
+ - name: APPO
9
+ results:
10
+ - task:
11
+ type: reinforcement-learning
12
+ name: reinforcement-learning
13
+ dataset:
14
+ name: mujoco_reacher
15
+ type: mujoco_reacher
16
+ metrics:
17
+ - type: mean_reward
18
+ value: -4.70 +/- 1.34
19
+ name: mean_reward
20
+ verified: false
21
+ ---
22
+
23
+ ## About the Project
24
+
25
+ This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist).
26
+
27
+ In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
28
+
29
+ ## Project Aims
30
+
31
+ This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance.
32
+
33
+ I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota.
34
+
35
+ The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels.
36
+
37
+ After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
38
+
39
+
40
+ ## About the Model
41
+
42
+ The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
43
+ ```
44
+ hyperparameters = {
45
+ "help": false,
46
+ "algo": "APPO",
47
+ "env": "atari_asteroid",
48
+ "experiment": "atari_asteroid_APPO",
49
+ "train_dir": "./train_atari",
50
+ "restart_behavior": "restart",
51
+ "device": "gpu",
52
+ "seed": 1234,
53
+ "num_policies": 2,
54
+ "async_rl": true,
55
+ "serial_mode": false,
56
+ "batched_sampling": true,
57
+ "num_batches_to_accumulate": 2,
58
+ "worker_num_splits": 1,
59
+ "policy_workers_per_policy": 1,
60
+ "max_policy_lag": 1000,
61
+ "num_workers": 16,
62
+ "num_envs_per_worker": 2,
63
+ "batch_size": 1024,
64
+ "num_batches_per_epoch": 8,
65
+ "num_epochs": 4,
66
+ "rollout": 128,
67
+ "recurrence": 1,
68
+ "shuffle_minibatches": false,
69
+ "gamma": 0.99,
70
+ "reward_scale": 1.0,
71
+ "reward_clip": 1000.0,
72
+ "value_bootstrap": false,
73
+ "normalize_returns": true,
74
+ "exploration_loss_coeff": 0.0004677351413,
75
+ "value_loss_coeff": 0.5,
76
+ "kl_loss_coeff": 0.0,
77
+ "exploration_loss": "entropy",
78
+ "gae_lambda": 0.95,
79
+ "ppo_clip_ratio": 0.1,
80
+ "ppo_clip_value": 1.0,
81
+ "with_vtrace": true,
82
+ "vtrace_rho": 1.0,
83
+ "vtrace_c": 1.0,
84
+ "optimizer": "adam",
85
+ "adam_eps": 1e-05,
86
+ "adam_beta1": 0.9,
87
+ "adam_beta2": 0.999,
88
+ "max_grad_norm": 0.0,
89
+ "learning_rate": 0.0003033891184,
90
+ "lr_schedule": "linear_decay",
91
+ "lr_schedule_kl_threshold": 0.008,
92
+ "lr_adaptive_min": 1e-06,
93
+ "lr_adaptive_max": 0.01,
94
+ "obs_subtract_mean": 0.0,
95
+ "obs_scale": 255.0,
96
+ "normalize_input": true,
97
+ "normalize_input_keys": [
98
+ "obs"
99
+ ],
100
+ "decorrelate_experience_max_seconds": 0,
101
+ "decorrelate_envs_on_one_worker": true,
102
+ "actor_worker_gpus": [],
103
+ "set_workers_cpu_affinity": true,
104
+ "force_envs_single_thread": false,
105
+ "default_niceness": 0,
106
+ "log_to_file": true,
107
+ "experiment_summaries_interval": 3,
108
+ "flush_summaries_interval": 30,
109
+ "stats_avg": 100,
110
+ "summaries_use_frameskip": true,
111
+ "heartbeat_interval": 10,
112
+ "heartbeat_reporting_interval": 60,
113
+ "train_for_env_steps": 100000000,
114
+ "train_for_seconds": 10000000000,
115
+ "save_every_sec": 120,
116
+ "keep_checkpoints": 2,
117
+ "load_checkpoint_kind": "latest",
118
+ "save_milestones_sec": 1200,
119
+ "save_best_every_sec": 5,
120
+ "save_best_metric": "reward",
121
+ "save_best_after": 100000,
122
+ "benchmark": false,
123
+ "encoder_mlp_layers": [
124
+ 512,
125
+ 512
126
+ ],
127
+ "encoder_conv_architecture": "convnet_atari",
128
+ "encoder_conv_mlp_layers": [
129
+ 512
130
+ ],
131
+ "use_rnn": false,
132
+ "rnn_size": 512,
133
+ "rnn_type": "gru",
134
+ "rnn_num_layers": 1,
135
+ "decoder_mlp_layers": [],
136
+ "nonlinearity": "relu",
137
+ "policy_initialization": "orthogonal",
138
+ "policy_init_gain": 1.0,
139
+ "actor_critic_share_weights": true,
140
+ "adaptive_stddev": false,
141
+ "continuous_tanh_scale": 0.0,
142
+ "initial_stddev": 1.0,
143
+ "use_env_info_cache": false,
144
+ "env_gpu_actions": false,
145
+ "env_gpu_observations": true,
146
+ "env_frameskip": 4,
147
+ "env_framestack": 4,
148
+ "pixel_format": "CHW"
149
+ }
150
+
151
+ ```
152
+
153
+
154
+
155
+ A(n) **APPO** impala model trained on the **mujoco_reacher** environment.
156
+
157
+ This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a
158
+ high throughput on-policy RL framework. I have been using
159
+ Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
160
+
161
+
162
+ ## Downloading the model
163
+
164
+ After installing Sample-Factory, download the model with:
165
+ ```
166
+ python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-mujoco_reacher
167
+ ```
168
+
169
+
170
+ ## Using the model
171
+
172
+ To run the model after download, use the `enjoy` script corresponding to this environment:
173
+ ```
174
+ python -m sf_examples.mujoco.enjoy_mujoco --algo=APPO --env=mujoco_reacher --train_dir=./train_dir --experiment=APPO-mujoco_reacher
175
+ ```
176
+
177
+
178
+ You can also upload models to the Hugging Face Hub using the same script with the `--push_to_hub` flag.
179
+ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
180
+
181
+ ## Training with this model
182
+
183
+ To continue training with this model, use the `train` script corresponding to this environment:
184
+ ```
185
+ python -m sf_examples.mujoco.train_mujoco --algo=APPO --env=mujoco_reacher --train_dir=./train_dir --experiment=APPO-mujoco_reacher --restart_behavior=resume --train_for_env_steps=10000000000
186
+ ```
187
+
188
+ Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
189
+
checkpoint_p0/best_001267192_324460544_reward_-3.848.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03832db38d05e2856dcbb4fb47dbca78d64035003e10184fc8495a41f809e503
3
+ size 75263
checkpoint_p0/checkpoint_001952264_499859456.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:407be72801f2743a8f9b4b370e532e43ddd71b3344cbaa7776afa8516c869eea
3
+ size 75527
checkpoint_p0/checkpoint_001952840_500006912.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f06cd41ca087fc506da554bfd09fce66875ea29dfab97da0e166aced2fec78da
3
+ size 75527
checkpoint_p0/milestones/checkpoint_000046024_11788288.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:421e0cfadc70861936c972a89b4ef2f735b9157c02761bc8cfb1cfbb56387111
3
+ size 76207
checkpoint_p0/milestones/checkpoint_000092584_23707648.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd3b26b33fe07edc82162cbe705032e5786c5d415b38d34674a90d378a53567d
3
+ size 76207
checkpoint_p0/milestones/checkpoint_000138976_35586048.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:912093d9597a4222b4142be4dae60dc61ac8807af9c349995245fdcbedc90a2d
3
+ size 76207
checkpoint_p0/milestones/checkpoint_000185280_47439872.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af83501c6672119391c61449d853571e72a2acdc11373e64b0a8a7137e607f4a
3
+ size 76207
checkpoint_p0/milestones/checkpoint_000231848_59367424.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec4315d79001c02ea6f6b27699bb723d2e641e079dd0cfd885c6e8e7dfcd5356
3
+ size 76207
checkpoint_p0/milestones/checkpoint_000278168_71237632.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65013027844e7dd5331582ea44ac0c9ae4b1f7b2d15740ebdf98f2b3e6a428e9
3
+ size 76207
checkpoint_p0/milestones/checkpoint_000324464_83091456.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:941f9cda7c74a70a1eb3beb9c45598e6b1c58f5fb099c993b579bace3516d18e
3
+ size 76207
checkpoint_p0/milestones/checkpoint_000370952_94994432.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b81feb0012b4f6be4bafb6f4d94b251c44cb5b55b81bba3cb2eca9fe031c99b7
3
+ size 76207
checkpoint_p0/milestones/checkpoint_000416456_106643456.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:440fbc2879c6491fb138258edebbacdede3fc49ef3238e89590ab98d46969c53
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000462152_118341632.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e83620a65c35d98ae3e7ebedfd8d94d12b2321267768bad181b431addf235eac
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000508536_130220032.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0fcdf7d45319daf21f5eb1cc62779c69de265c2821899fe7af002eaa92b20f74
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000554808_142065664.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c6b0db1b9800d68d3cff804230ad2b518e505ff53324f6299e2d7621a753a0e
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000600912_153870336.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c9f7f732c42731b40a3cd06b882716720d4a7124b8c5fa18c8aaab9fd3b02501
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000647328_165756928.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4c0b0dad022171c270a9337d30da4deaa7821092b8289efe1273d5aab73baff
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000692216_177258496.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e0a9cb8631efcd9a1f17223f0040082e75c523331c894747d12ddf09a70794f6
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000738520_189112320.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30d5b9eb1ddb6c52186ef7b468016e42f2da2db2293c977f2d0a4ffba9758cb6
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000784776_200957952.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91c326ef8c9655e8190ac4c673bedf12f1ed93ce96a05856afa84c537af8e183
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000831400_212893696.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d999a86897209291b83a059520b8a5094f769b52f48912c2e0d23be3f7195916
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000877704_224747520.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d92be2b8c29f025d92b28f15870672152febde60b1944ba7ead52346cca3b22
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000923848_236560384.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65e16f9459487c963e7d68445c921714d87d92a4294aaa6735486fd1fe167e69
3
+ size 76251
checkpoint_p0/milestones/checkpoint_000970208_248430592.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b28b7c52b4685b0ebf25fdba4c6301ccdea329e9c4a766961115db60f444ee70
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001016256_260218880.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fcde031585cae527fbbb4c08aa47cc7b033f2ce32913bf5a7e84f4710959e818
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001062560_272072704.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f118daad157822d628f44c3acb0d533cf8a7707cb9e7d7d3cbfeb14a511a75d0
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001108888_283934720.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4f126ef067868e45c5bcac57114fd88934b1901911551d6baae96078834fc74
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001155192_295788544.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84b5e9a7581e8c56736ca053b2c32fb954a784039d3598a7cc33110ff9beedac
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001201784_307716096.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4da8a6daa078fdbef35ad06dd312f4093559b91dc9ac95ed2b837f13fce8ad56
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001248056_319561728.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c7dac6f8402e3ef522134b7f4e436e208021721aacd84e72ea4663ed3641d4d5
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001294328_331407360.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c6e2f1bfc32940489e81b1c3efc1c5a8a6af913353e0a65e9db5dc663d0cf5e1
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001340480_343228416.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f69f6519bfed721cc43dc5d86ed70dd6702088f24bf6b710cfb4cd352fca28e
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001387024_355147776.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56212a97bb6ca8bbf3af93527eafd25f2e1a4e1d62723ec9d0531143111aada9
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001433328_367001600.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e75c7524eb658fef7458086c88927471e7cd49844dd49456e955ac7068c8073
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001479544_378839040.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0b59e789358f89a9c9b4db0764b54deedd825a15c38e05cc9765b598f3ea791f
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001525848_390692864.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6ed68aa60ee5830c43f43746cb0da03cd4b0cad8592c47da25b621b3819767c
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001572504_402636800.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c9352c8a5a8f5201c4b5968b6955c2e8a1be3e3edff857ae465d4fc98ded35de
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001618904_414515200.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f965f1953c35e8bb63afbe0a696ad5698cca9266bf4c520f60c019d8c7d8eccb
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001665080_426336256.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c4ed6045df1d34eaed9b591a9e6069ca207a78f49ce86cbadf5b2fe97779ecf
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001711592_438247424.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb4fbef5cf9442420f950cd89e5856564a83b87bc2e362962f5692444bb216e7
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001757832_450084864.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:161e4ca43e2a6f203d6e992bd4ce0e526a052e83c3c809ddc6ce432a1f6ae863
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001804136_461938688.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2bc8250a144a0071ec4cd9d3ba64b16c07209a7aab708ee8740e61c7ec2cb715
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001850376_473776128.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50325433435f738441bb844afa605ae1db643664df80e75b43cdfb7779077a86
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001896680_485629952.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b253cf6683ff563566ac71d1fc0bc5317926ef05523b3968bb8f2d55eddc2ea3
3
+ size 76251
checkpoint_p0/milestones/checkpoint_001943080_497508352.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1137d7c55e8792c16c8f826e9c9aefa720dd734619b3b26014cbc78f7031c77d
3
+ size 76251
checkpoint_p1/best_001902208_487022592_reward_-4.084.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a94aff8fa757f5414d947dcfa1ceaaa9d8741d9571ac58a05ddd4de417497516
3
+ size 75263