Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1702099890.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9d5e7ca225f1bb178ea10da7bcf3a598de410533c0e4aca8f03bc4d7c30ac31
+size 114143277

.summary/1/events.out.tfevents.1702099890.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c6e2496644e7274a754ab7b628587f6b49dfa3369f9c6a4362072a22b938c16
+size 59939872

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_upndown
     metrics:
     - type: mean_reward
-      value: 285811.00 +/- 36947.23
+      value: 353048.00 +/- 5475.06
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_upndown** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_upndown
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_upndown** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_upndown
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001951936_499695616_reward_748.830.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:739bb5cdd9a28c365a74b6ef0198baa5d7ad5731368d65d4777d9c5aa1bcbcf7
+size 20721843

checkpoint_p0/checkpoint_001955968_501465088.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e68d669b2e990003b9ea8be286944f1a3a524032df2ad77aa72601304d671133
+size 20722179

checkpoint_p0/checkpoint_001956304_501637120.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:737f67deb5f0adc6db3bef0026592e1e44169ae915c91fecfd7fe24fec7f8f13
+size 20722179

checkpoint_p0/milestones/checkpoint_000009824_2514944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0c799076f100cb1aebb6c9753f98cfebc39d3077b60a8514738a7ea1c158846
+size 20723035

checkpoint_p0/milestones/checkpoint_000019936_5103616.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:251697839efd3d2596f34ca063252106d35604d87ba4c8eac9772af8c5d0cdb4
+size 20723035

checkpoint_p0/milestones/checkpoint_000030080_7700480.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:631042591a92e6b6d8f58f327f602ac97604f8a6c6f5d1fbf86af259c6bfc54e
+size 20723035

checkpoint_p0/milestones/checkpoint_000040256_10305536.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0cf90622c3f06699455083193a37d3962f98d418be2516660a767c5821f7a875
+size 20723091

checkpoint_p0/milestones/checkpoint_000050400_12902400.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b21751d652bf788fc43c634c49e02056f90b6daf224845ceba1085e2bae1e5fc
+size 20723091

checkpoint_p0/milestones/checkpoint_000060544_15499264.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:255127f74d1e9a8b4a940c9f28411201b13ae16e3a5abd20c6534dbaa2876f64
+size 20723091

checkpoint_p0/milestones/checkpoint_000070656_18087936.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb356d6f3dc6feab056b75e90b7a21b0c9cbfd3bfe3646de8ec4a0b7d0e0b967
+size 20723091

checkpoint_p0/milestones/checkpoint_000080672_20652032.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdf2c234008c300ed97f29eb63d0bc1102c7c46fb6e85f5a1b19414258db9dfc
+size 20723091

checkpoint_p0/milestones/checkpoint_000090848_23257088.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8ab3360d3e8d0e0ac1974f6a1d5f16c0f5d554a8dcd5da35e8bf883219c1c3e
+size 20723091

checkpoint_p0/milestones/checkpoint_000101024_25862144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad3ecae444ee6db6d4ae5a77b5ebb0afe4881686d076f9187c7f9a5520a7d1cd
+size 20723091

checkpoint_p0/milestones/checkpoint_000111232_28475392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eeee1df30a0e855a65bad7e21a6017b17a38e90060e50c86ced5d4c6399cb768
+size 20723091

checkpoint_p0/milestones/checkpoint_000121280_31047680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d638fc5f13b2ee483981a432b7e8419b326309cbd955d88ccdf4cde7f61cbc6f
+size 20723091

checkpoint_p0/milestones/checkpoint_000131360_33628160.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5dd9e155e990c7ba1dba62a6feb731ae5b228ba21dea036ed37b32cfdf1bd6cd
+size 20723091

checkpoint_p0/milestones/checkpoint_000141504_36225024.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:356361f89952a741f431d42c5641ed7873754eea558f9a16498b514853f9cfd2
+size 20723091

checkpoint_p0/milestones/checkpoint_000151584_38805504.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b17b2b4d8efef3c4f23d7b7318be1638b7b284bf96e6344619e0876a00fbc16b
+size 20723091

checkpoint_p0/milestones/checkpoint_000161760_41410560.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2149408467dd823af84ec7d659db90d4a03739f2ed333e09118d8101978caaaa
+size 20723091

checkpoint_p0/milestones/checkpoint_000171904_44007424.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cc854ad490867f6c946544143e34b0037ecd1c1bb01a5a1f6d831216523ea39
+size 20723091

checkpoint_p0/milestones/checkpoint_000181920_46571520.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:447373ed6b923a3059dd3f37c2dffc815d2e4445ccca6217ce2e21758906238b
+size 20723091

checkpoint_p0/milestones/checkpoint_000192032_49160192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c8d43ed65bf76026e5d9fc99f5e646a3758650395d28c54b2221f6d63ae7249
+size 20723091

checkpoint_p0/milestones/checkpoint_000202208_51765248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f008f12a694c121398d9431c65b57b451cce39944d4efebad18d9e4565bc8389
+size 20723091

checkpoint_p0/milestones/checkpoint_000212256_54337536.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:153a6ef0ee40054533dd0b531dae886985cc9b8801927b63d66c1fbcbbca7ee8
+size 20723091

checkpoint_p0/milestones/checkpoint_000222304_56909824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8a88b3a83b228b50ae18c942c24c59851d54504b8bb33d701c704299db436d4
+size 20723091

checkpoint_p0/milestones/checkpoint_000232416_59498496.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3584ec569af16a5f88742a24e3d21211cf5d71ac8e6eceeeaa56fa0b0a9c99e7
+size 20723091

checkpoint_p0/milestones/checkpoint_000242432_62062592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b1bcdf592970dbfcf660b0f00d6d8dc9d32680a5fd8735f6cc254282149a8c4
+size 20723091

checkpoint_p0/milestones/checkpoint_000252576_64659456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b8f33d8b4960651cd01ba82127061d78b9a04ebdc10f4dcbcdd18a8058d2f77
+size 20723091

checkpoint_p0/milestones/checkpoint_000262688_67248128.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d73ee25c4ad7cf8f9bc7c281b2acd9fe12547af000d8c9e77488a78ca3e9747d
+size 20723091

checkpoint_p0/milestones/checkpoint_000272704_69812224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c7f700d2b745fff9dd2bca8a99074ec29b924e1e363a373dc6ad076deb17d93
+size 20723091

checkpoint_p0/milestones/checkpoint_000282816_72400896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:514a207f1c2a0ec345c95166384bce11d225d5e66fc5cfb94c26fa97ca91e6c0
+size 20723091

checkpoint_p0/milestones/checkpoint_000292992_75005952.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98db2e91688b7c80540d7de96d35c38ea0100a7db4f916920e2ec8183da1d990
+size 20723091

checkpoint_p0/milestones/checkpoint_000303136_77602816.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f4d2755b6ecc36f900676ff17060e03b3130527cf673269af475dd7e427c3a6
+size 20723091

checkpoint_p0/milestones/checkpoint_000313280_80199680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d028d75681f5a3959dffd2c2a6c79f0a2d76dfc46102fbd22a31462963997e84
+size 20723091

checkpoint_p0/milestones/checkpoint_000323328_82771968.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f32d6212dac8fa907dd006cfed922e0778c58c52a3ae2017636ba1b3d8283b4
+size 20723091

checkpoint_p0/milestones/checkpoint_000333504_85377024.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c00b46502175fbda059768ef21e79cc256dc30f6ba2b382d8f4eb0849e13109c
+size 20723091

checkpoint_p0/milestones/checkpoint_000343616_87965696.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d99487c45d7bcb3acbfa94a09bdd9365d8fde1e83ea75090306d5bb43bb92f4f
+size 20723091

checkpoint_p0/milestones/checkpoint_000353728_90554368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7958be337439b4c1aca99c4e428247d8460aae14b8a10662762af5081a98091
+size 20723091

checkpoint_p0/milestones/checkpoint_000363904_93159424.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee6196fefc8a4bae477f02c6229a9979d96d411f0167037de01889fa0cd7aea4
+size 20723091

checkpoint_p0/milestones/checkpoint_000373952_95731712.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdead161e6f6ddcb61baa32336bd4a04a59faa7b3fcca1576e899a97a0347964
+size 20723091

checkpoint_p0/milestones/checkpoint_000384128_98336768.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ee225c4526216fc41b625723a7a207c2b0920d76402b6f37fc2101adb4202ef
+size 20723091

checkpoint_p0/milestones/checkpoint_000394208_100917248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba9b9ee68b67ee619ccd042858e04d2d20a10cb0e887fe67483f7e981fc1ad70
+size 20723147

checkpoint_p0/milestones/checkpoint_000404384_103522304.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de41d5eed52bc374172e739439cc50bf88b490d4be88a49d61cf15f9b1c178e4
+size 20723147

checkpoint_p0/milestones/checkpoint_000414432_106094592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94aa02b06ffa0170d7ac1975debc94e6ccfeac2703b5aeddb271fb2fe1fa07d8
+size 20723147

checkpoint_p0/milestones/checkpoint_000424512_108675072.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8eaadb49da6c8443034c9b7b8410f6f566b5377a13b25da2a2f84af1c5291f88
+size 20723147

checkpoint_p0/milestones/checkpoint_000434656_111271936.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cf836d54504ba9f41efb3922fe7f66195df395ff0be8e9be23c7fcc6d53cf01
+size 20723147