rmdhirr commited on
Commit
0a6670b
1 Parent(s): e3dcc3b

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -0
README.md ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - merge
4
+ - mergekit
5
+ - mistral
6
+ - MTSAIR/multi_verse_model
7
+ - ResplendentAI/Paradigm_7B
8
+ base_model:
9
+ - MTSAIR/multi_verse_model
10
+ - ResplendentAI/Paradigm_7B
11
+ language:
12
+ - en
13
+ library_name: transformers
14
+ license: apache-2.0
15
+ ---
16
+ # 🌠 Multiparadigm_7B
17
+
18
+ Multiparadigm_7B is a merge of the following models:
19
+ * [MTSAIR/multi_verse_model](https://huggingface.co/MTSAIR/multi_verse_model)
20
+ * [ResplendentAI/Paradigm_7B](https://huggingface.co/ResplendentAI/Paradigm_7B)
21
+
22
+ # Configuration
23
+ ```yaml
24
+ slices:
25
+ - sources:
26
+ - model: MTSAIR/multi_verse_model
27
+ layer_range: [0, 32]
28
+ - model: ResplendentAI/Paradigm_7B
29
+ layer_range: [0, 32]
30
+ merge_method: slerp
31
+ base_model: MTSAIR/multi_verse_model
32
+ parameters:
33
+ t:
34
+ - filter: self_attn
35
+ value: [0, 0.6, 0.3, 0.7, 1]
36
+ - filter: mlp
37
+ value: [1, 0.6, 0.7, 0.3, 0]
38
+ - value: 0.6
39
+ dtype: bfloat16
40
+ ```