Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1697950142.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79fb13e6cd328f0c188a21164935ea28285721b379ba6d825adf4b2ff5a6b352
+size 87458900

.summary/1/events.out.tfevents.1697950142.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f35d7d48b27cb48a8c05e15010db17f73c6951acd385a9f83adad8f1b860e55
+size 45922976

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_beamrider
     metrics:
     - type: mean_reward
-      value: 7105.40 +/- 2931.13
+      value: 25759.80 +/- 12833.80
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_beamrider** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_beamrider
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_beamrider** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_beamrider
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001944896_497893376_reward_92.950.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37cf89658d98086e5509b0770d703715905cf2818345fab085e898580c241099
+size 20740275

checkpoint_p0/checkpoint_001952000_499712000.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5025e2ca3d122fc69185e8d59153121b68665b8780750ce7f4edf4f6d7755e20
+size 20740611

checkpoint_p0/checkpoint_001953120_500006912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ba47587d43237442d46328dc420d70e2faf091f24964bfbb885759333dbb1af
+size 20740611

checkpoint_p0/milestones/checkpoint_000013056_3342336.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e9c4fed560749fc75ce59fce59fb49c3066c4a57c3e1fedfedfc47908de39b5
+size 20741467

checkpoint_p0/milestones/checkpoint_000026144_6692864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0502c9583f97cb387db4b27bdef8aecbf2575b496b47b9481b4213cb552e918c
+size 20741467

checkpoint_p0/milestones/checkpoint_000039296_10059776.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca4a0ee7f9c4c3033c07f091920b8917b7fd5e7e8d18ee80a9855359631e74ae
+size 20741523

checkpoint_p0/milestones/checkpoint_000052480_13434880.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a7eed12d6d2e9c9928e474747c2f9977dce920b6cf9464ba96a02170dcdc312
+size 20741523

checkpoint_p0/milestones/checkpoint_000065696_16818176.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4649c195ba687c1d659dc001392e4e1be9da6e0cbbe687b531a89f8aaf6afe57
+size 20741523

checkpoint_p0/milestones/checkpoint_000078880_20193280.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:628ee29a98d7becfc2652765198d927383208bbfcf2101c87e616b7d684bc1ea
+size 20741523

checkpoint_p0/milestones/checkpoint_000092032_23560192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ead6e4b2e32d88595910e8ba09c8508b85bd14a5b638eb0b29a9a9ac581a8a0d
+size 20741523

checkpoint_p0/milestones/checkpoint_000105184_26927104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e195a85d7f37daa236986c8142d986e494ee3e645510aeedb8a4ccb8ddb847f
+size 20741523

checkpoint_p0/milestones/checkpoint_000118240_30269440.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c78aaf8713cf49ec843836cac947bb488cec08a0c12811b52638699ce726768b
+size 20741523

checkpoint_p0/milestones/checkpoint_000131456_33652736.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:daee77d686e2e49d47c6668580bc5ba3ff637a8bc815b614d13d59480943dca8
+size 20741523

checkpoint_p0/milestones/checkpoint_000144640_37027840.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:262ce35988580eca76c8ea1781690cfe18b96681af6f0f9ce9e9a964d47f7ff2
+size 20741523

checkpoint_p0/milestones/checkpoint_000157728_40378368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9cea9cdfa501565a82ce00980463a9b3235e906f1c61e6c6325cec11da769289
+size 20741523

checkpoint_p0/milestones/checkpoint_000170912_43753472.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7dc6d7605e2e6ae64ea5a499e0e1c43c03d22a9a34c6d10639af18d1444a8da5
+size 20741523

checkpoint_p0/milestones/checkpoint_000183968_47095808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87262944042825cecf8b7f1afd95b1d1a9af7be7a43fb77b01206349abece154
+size 20741523

checkpoint_p0/milestones/checkpoint_000197120_50462720.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f38e809d88da761bc22d844cf7d80f886cda48ee8d764f63ff67da60a5cd3f6
+size 20741523

checkpoint_p0/milestones/checkpoint_000210272_53829632.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5057c117910d589b99f972f44d3d2278bb2d0160ed8e0812cd80172bb7732051
+size 20741523

checkpoint_p0/milestones/checkpoint_000223520_57221120.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b1e8d809e60d5bf0247cbbf0d29b4538ac9e43694f0c6ff0f3691f622e65745
+size 20741523

checkpoint_p0/milestones/checkpoint_000236640_60579840.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c66b24075898f6e3b51c54c68dd68e876e3568bc7245930fecdda1a6f1d5ab7b
+size 20741523

checkpoint_p0/milestones/checkpoint_000249856_63963136.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:339256ce94df9784e1a91b08870d06a4eb1049cd5e0b8e4a28f813d4f85faa99
+size 20741523

checkpoint_p0/milestones/checkpoint_000263008_67330048.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa3616a191783b69514b4f96eac575e08e29f12cc5d5cc6e33ecee2d8a7084f4
+size 20741523

checkpoint_p0/milestones/checkpoint_000276160_70696960.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75b1891cedc2743f9e68f22a2948ac31aa751a82cfa5562d7d1f97904074f2b8
+size 20741523

checkpoint_p0/milestones/checkpoint_000289280_74055680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a795534531312dd18972748781a116fd9743f1703d4056148e5a4af5d3067239
+size 20741523

checkpoint_p0/milestones/checkpoint_000302464_77430784.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf5eac01e7b499517c591a7fd4a13e66d082215af492c586f8c87642a54e04f1
+size 20741523

checkpoint_p0/milestones/checkpoint_000315648_80805888.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a8df2e75c51286742622d7e335ff8a1478c021a98bb4e6883d7cba5b7d7c4db
+size 20741523

checkpoint_p0/milestones/checkpoint_000328768_84164608.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c95ed660b70ef640af59ed22195ec40ee1f3761ee91049a3fda50a475a910ec7
+size 20741523

checkpoint_p0/milestones/checkpoint_000341952_87539712.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd4300d0f5510248a7027777b4fa46454577b309b30d20b7efe62b73ba1907f9
+size 20741523

checkpoint_p0/milestones/checkpoint_000355104_90906624.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6aa48c16eadd84288de79ca9e409b55532e393bf10b9d14360ae7e997d69184b
+size 20741523

checkpoint_p0/milestones/checkpoint_000368256_94273536.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70d29372096f2afa6fb8d1b0efc759855d6092dd46823bee4854ee8b0425eb06
+size 20741523

checkpoint_p0/milestones/checkpoint_000381376_97632256.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e35c07c306de8e1c79fd0a2348d7e34fecf94af3c115d25bf907d6487a2d698
+size 20741523

checkpoint_p0/milestones/checkpoint_000394496_100990976.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95058cfd0fbfb9c0dbfceda2e0038437c3fa923e5a0b5c872fac438098034dc1
+size 20741579

checkpoint_p0/milestones/checkpoint_000407712_104374272.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a15743714ee002c106a1cf6a2ca9a1dc5bda36411f8a56fddfdaa4f36f4d7284
+size 20741579

checkpoint_p0/milestones/checkpoint_000420768_107716608.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d9e75a366c5f84dc7b432b808e53a8941bc0dfd1c3a14bd455fcab4d180ea6f
+size 20741579

checkpoint_p0/milestones/checkpoint_000433888_111075328.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:336b857d78bc356117ac49f5e6a72bab5b0b269df4cf922d808da082edf575f9
+size 20741579

checkpoint_p0/milestones/checkpoint_000447104_114458624.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:258ea40399b20faa119eaee78f92e4925059e863e7cf5e0b1353c15347e31dea
+size 20741579

checkpoint_p0/milestones/checkpoint_000460384_117858304.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3448400a28969a169cd8856b8512cbc0ff6958f45cad9002c87a4ef636cf6efb
+size 20741579

checkpoint_p0/milestones/checkpoint_000473600_121241600.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5861643de0aafbf66722d7e9b3415e83981ac3b7cfb9b2d1562c44b800ec8dcb
+size 20741579

checkpoint_p0/milestones/checkpoint_000486848_124633088.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7dd5365c644e147e8e904795059e644187217f880c058b55d8098cb714dce31
+size 20741579

checkpoint_p0/milestones/checkpoint_000500032_128008192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65095e07093f0282ccddf62d7a12eed12d5c86a71b2f2083afb8955f39fff816
+size 20741579

checkpoint_p0/milestones/checkpoint_000513248_131391488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:767e481835b5f8d9092184ef32f8de0c6f65adfaa4df10b59bb0ae4400c552a3
+size 20741579

checkpoint_p0/milestones/checkpoint_000526368_134750208.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c891eded9783039d2cf16cab2d28539f002e9ea455bf0972944ed877cd673d22
+size 20741579

checkpoint_p0/milestones/checkpoint_000539456_138100736.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91e33ff72f099ea2cf103bc540cedda10af126221728ad61370fbbf1cf615a3e
+size 20741579

checkpoint_p0/milestones/checkpoint_000552576_141459456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efbbd8d3c122bcbbdcba195b5c75532f3e0ece4eba183912eb913976a25e6a42
+size 20741579

checkpoint_p0/milestones/checkpoint_000565696_144818176.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39c307f66809ba50df3898f793fd15d92c00a7586d1efd250c4c05bd29c20f9a
+size 20741579