jsfs11 commited on
Commit
467e26d
1 Parent(s): 134170c

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -0
README.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Sao10K/L3-8B-Stheno-v3.2
4
+ - Sao10K/L3-8B-Stheno-v3.1
5
+ tags:
6
+ - merge
7
+ - mergekit
8
+ - lazymergekit
9
+ - Sao10K/L3-8B-Stheno-v3.2
10
+ - Sao10K/L3-8B-Stheno-v3.1
11
+ ---
12
+
13
+ # L3-8B-Stheno-slerp-GGUF
14
+
15
+ L3-8B-Stheno-slerp is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
16
+ * [Sao10K/L3-8B-Stheno-v3.2](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2)
17
+ * [Sao10K/L3-8B-Stheno-v3.1](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.1)
18
+
19
+ ## 🧩 Configuration
20
+
21
+ ```yaml
22
+ slices:
23
+ - sources:
24
+ - model: Sao10K/L3-8B-Stheno-v3.2
25
+ layer_range: [0, 32]
26
+ - model: Sao10K/L3-8B-Stheno-v3.1
27
+ layer_range: [0, 32]
28
+ merge_method: slerp
29
+ base_model: Sao10K/L3-8B-Stheno-v3.2
30
+ parameters:
31
+ t:
32
+ - filter: self_attn
33
+ value: [0, 0.5, 0.3, 0.7, 1]
34
+ - filter: mlp
35
+ value: [1, 0.5, 0.7, 0.3, 0]
36
+ - value: 0.5
37
+ dtype: bfloat16
38
+ ```