Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,91 @@
+---
+base_model: []
+library_name: transformers
+tags:
+- mergekit
+- merge
+
+---
+# 2x_bagel-34b-v0.2
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+## Merge Details
+### Merge Method
+
+This model was merged using the passthrough merge method.
+
+### Models Merged
+
+The following models were included in the merge:
+* ./jondurbin_bagel-34b-v0.2
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+dtype: float32
+merge_method: passthrough
+slices:
+- sources:
+  - layer_range: [0, 20]
+    model: "jondurbin_bagel-34b-v0.2"
+    parameters:
+      scale:
+        - filter: q_proj
+          value: 0.7071067812
+        - filter: k_proj
+          value: 0.7071067812
+        - value: 1
+- sources:
+  - layer_range: [10, 30]
+    model: "jondurbin_bagel-34b-v0.2"
+    parameters:
+      scale:
+        - filter: q_proj
+          value: 0.7071067812
+        - filter: k_proj
+          value: 0.7071067812
+        - value: 1
+- sources:
+  - layer_range: [20, 40]
+    model: "jondurbin_bagel-34b-v0.2"
+    parameters:
+      scale:
+        - filter: q_proj
+          value: 0.7071067812
+        - filter: k_proj
+          value: 0.7071067812
+        - value: 1
+- sources:
+  - layer_range: [30, 50]
+    model: "jondurbin_bagel-34b-v0.2"
+    parameters:
+      scale:
+        - filter: q_proj
+          value: 0.7071067812
+        - filter: k_proj
+          value: 0.7071067812
+        - value: 1
+- sources:
+  - layer_range: [40, 60]
+    model: "jondurbin_bagel-34b-v0.2"
+    parameters:
+      scale:
+        - filter: q_proj
+          value: 0.7071067812
+        - filter: k_proj
+          value: 0.7071067812
+        - value: 1
+name: 2xbagel_fp32
+---
+dtype: bfloat16
+merge_method: passthrough
+slices:
+- sources:
+  - layer_range: [0, 100]
+    model: 2xbagel_fp32
+name: bagel_new
+
+```