khrisintw commited on
Commit
4e61fac
1 Parent(s): 6b4aa2b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +47 -47
README.md CHANGED
@@ -1,47 +1,47 @@
1
- ---
2
- base_model:
3
- - NousResearch/Llama-2-7b-chat-hf
4
- - NousResearch/Llama-2-7b-hf
5
- - taide/TAIDE-LX-7B
6
- library_name: transformers
7
- tags:
8
- - mergekit
9
- - merge
10
-
11
- ---
12
- # merged
13
-
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf) as a base.
20
-
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [NousResearch/Llama-2-7b-chat-hf](https://huggingface.co/NousResearch/Llama-2-7b-chat-hf)
25
- * [taide/TAIDE-LX-7B](https://huggingface.co/taide/TAIDE-LX-7B)
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- base_model: NousResearch/Llama-2-7b-hf
33
- dtype: bfloat16
34
- merge_method: task_arithmetic
35
- slices:
36
- - sources:
37
- - layer_range: [0, 32]
38
- model: taide/TAIDE-LX-7B
39
- parameters:
40
- weight: 1.0
41
- - layer_range: [0, 32]
42
- model: NousResearch/Llama-2-7b-chat-hf
43
- parameters:
44
- weight: 1.0
45
- - layer_range: [0, 32]
46
- model: NousResearch/Llama-2-7b-hf
47
- ```
 
1
+ ---
2
+ base_model:
3
+ - NousResearch/Llama-2-7b-chat-hf
4
+ - NousResearch/Llama-2-7b-hf
5
+ - taide/TAIDE-LX-7B
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+ license: mit
11
+ ---
12
+ # merged
13
+
14
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
+
16
+ ## Merge Details
17
+ ### Merge Method
18
+
19
+ This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf) as a base.
20
+
21
+ ### Models Merged
22
+
23
+ The following models were included in the merge:
24
+ * [NousResearch/Llama-2-7b-chat-hf](https://huggingface.co/NousResearch/Llama-2-7b-chat-hf)
25
+ * [taide/TAIDE-LX-7B](https://huggingface.co/taide/TAIDE-LX-7B)
26
+
27
+ ### Configuration
28
+
29
+ The following YAML configuration was used to produce this model:
30
+
31
+ ```yaml
32
+ base_model: NousResearch/Llama-2-7b-hf
33
+ dtype: bfloat16
34
+ merge_method: task_arithmetic
35
+ slices:
36
+ - sources:
37
+ - layer_range: [0, 32]
38
+ model: taide/TAIDE-LX-7B
39
+ parameters:
40
+ weight: 1.0
41
+ - layer_range: [0, 32]
42
+ model: NousResearch/Llama-2-7b-chat-hf
43
+ parameters:
44
+ weight: 1.0
45
+ - layer_range: [0, 32]
46
+ model: NousResearch/Llama-2-7b-hf
47
+ ```