RDson commited on
Commit
8d12568
1 Parent(s): 574ff66

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -1
README.md CHANGED
@@ -5,4 +5,29 @@ license_link: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/raw/mai
5
  ---
6
  Merged [cognitivecomputations/dolphin-2.9-llama3-8b](https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b) and [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) because I thought the Dolphin finetune was a bit too 'robot-y' in the answers.
7
 
8
- GGUF files can be found here: [RDson/Dolphin-less-Llama-3-Instruct-8B-GGUF](https://huggingface.co/RDson/Dolphin-less-Llama-3-Instruct-8B-GGUF).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  ---
6
  Merged [cognitivecomputations/dolphin-2.9-llama3-8b](https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b) and [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) because I thought the Dolphin finetune was a bit too 'robot-y' in the answers.
7
 
8
+ GGUF files can be found here: [RDson/Dolphin-less-Llama-3-Instruct-8B-GGUF](https://huggingface.co/RDson/Dolphin-less-Llama-3-Instruct-8B-GGUF).
9
+
10
+ Mergekit yaml:
11
+ ```
12
+ tokenizer_source: union
13
+ slices:
14
+ - sources:
15
+ - model: ollama/llama3/Meta-Llama-3-8B-Instruct
16
+ layer_range: [0, 32]
17
+ - model: dolphin-2.9-llama3-8b
18
+ layer_range: [0, 32]
19
+ parameters:
20
+ weight: 0.75
21
+ merge_method: slerp
22
+ base_model: ollama/llama3/Meta-Llama-3-8B-Instruct
23
+ parameters:
24
+ normalize: true
25
+ embed_slerp: true
26
+ t:
27
+ - filter: self_attn
28
+ value: [0, 0.5, 0.3, 0.7, 1]
29
+ - filter: mlp
30
+ value: [1, 0.5, 0.7, 0.3, 0]
31
+ - value: 0.5
32
+ dtype: bfloat16
33
+ ```