v000000 commited on
Commit
776ed42
1 Parent(s): dc563e9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -34,9 +34,9 @@ Healed Llama-3 15B Frankenmerge
34
 
35
  ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/MCdGdalCCtOVPn8X7rqha.jpeg)
36
 
37
- This is the third iteration "Emerald" of the final four and the one I liked the most. It has had limited testing though, but seems decent.
38
 
39
- Findings: o_proj and down_proj can be stolen from Aethora-v2 so new 15B frankenmerges don't seem to need finetuning to "heal" the layers at all.
40
 
41
  # <h1>merge</h1>
42
 
@@ -45,7 +45,7 @@ This is a merge of pre-trained language models created using [mergekit](https://
45
  # <h1>Merge Details</h1>
46
  # <h1>Merge Method</h1>
47
 
48
- This model was merged using an iterative merging process. (Probably ~15 models got thrown away in the process.)
49
 
50
  # <h1>Models Merged</h1>
51
 
 
34
 
35
  ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/MCdGdalCCtOVPn8X7rqha.jpeg)
36
 
37
+ This is the third iteration "Emerald" of the final four and the one I liked the most. It has had limited testing though, but seems relatively decent.
38
 
39
+ Findings: o_proj and down_proj can be stolen from Aethora-v2 so new 15B frankenmerges don't seem to really ***need*** finetuning to "heal" the layers at all.
40
 
41
  # <h1>merge</h1>
42
 
 
45
  # <h1>Merge Details</h1>
46
  # <h1>Merge Method</h1>
47
 
48
+ This model was merged using an iterative merging process. (Probably ~10 models got thrown away in the process.)
49
 
50
  # <h1>Models Merged</h1>
51