better

README.md

@@ -34,7 +34,7 @@ and merged in
     
 ### 9-partition merge
 
-All of the layers were partitioned in to 9 random bins.  Alternating models were slerped at [1...0] (inputs), and [0...1] (outputs) gradients; except attention, which was slerped at 0.03 (with a drop rate of .28).
+All of the layers were partitioned in to 9 random bins.  Alternating models were slerped at [1...0.5] (inputs), and [0.5...1] (outputs) gradients; except attention, which was slerped at 0.97 (with a drop rate of .28).
 
 This means that the model is still predominantly ordered around the base model - including half of the input and output layers, and 28% of attention.
 

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "OpenHermes-2.5-Mistral-7B",
+  "_name_or_path": "../../llama/raw/merge/math/OpenHermes-2.5-Mistral-7B",
   "architectures": [
     "MistralForCausalLM"
   ],

model-00001-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:211843c235e9ae16fd3c08312bab5cda2fca88c352a8b51799e46f556cedfe4c
+oid sha256:f115c7c3446b2920cea1c486e7f33f27c7cd311a2c13addf26daf79f0a2ce84e
 size 4943178624

model-00002-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e68b03c876d29ab6c638b0fd87f8a7305e0d630975ddd98a85e69918971fd8f1
+oid sha256:c25c26b425b2c9a3ed0e5279610168e17ee2c828d4971ff6408b5ed3d2ce8a2d
 size 4999819232

model-00003-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b02db52b535ea433c47201eda7cf935d5785a56b02f806fe9ada0344d9f0aa20
+oid sha256:9b74362b58b5698b5b57ce87a3a84152ac55b22df0558fd1491c58e73afc08c5
 size 4540532640