|
--- |
|
base_model: |
|
- crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- crestf411/L3-8B-daybreak-stheno-v3.1 |
|
library_name: transformers |
|
tags: |
|
- mergekit |
|
- merge |
|
--- |
|
|
|
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/667eea5cdebd46a5ec4dcc3d/cnRSK9WuGMYGvlbyRGEE7.jpeg) |
|
|
|
Well, this was hell to make. Passthrough merges aren't fun to start but this was another level of fuck you if the configs say anything. |
|
Crepus Horizon was an attempt at an expanded merge of Sunfall and Daybreak Stheno by [crestf4ll](https://huggingface.co/crestf411), mainly for the verbosity I was able to get out of it. |
|
This and Blackened Sunfall are what I'm calling 'cracked starters', models that were purposefully broken in ways to force certain attributes at the cost of coherency, context following, etc. |
|
These models are not stand alone, at all; they're merge fodder. This one especially can misspell and struggles with anatomy, settings, and dialogue. |
|
|
|
So, be careful. Do not use this by itself. |
|
|
|
### Quants |
|
|
|
I don't recommend quanting this just to save your GPUs, but I can't stop yall either. |
|
|
|
[OG Q8 GGUF] |
|
|
|
### Details & Recommended Settings |
|
|
|
|
|
|
|
Rec. Settings: None, there is no making this model work. |
|
|
|
### Models Merged & Merge Theory |
|
|
|
* [crestf411/L3-8B-daybreak-stheno-v3.1](https://huggingface.co/crestf411/L3-8B-daybreak-stheno-v3.1) |
|
* [crestf411/L3-8B-sunfall-v0.4-stheno-v3.2](https://huggingface.co/crestf411/L3-8B-sunfall-v0.4-stheno-v3.2) |
|
|
|
|
|
|
|
### Configs |
|
|
|
```yaml |
|
models: |
|
slices: |
|
- sources: |
|
- layer_range: [0, 9] # top cap |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [7, 9] # top bit |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- sources: |
|
- layer_range: [11, 13] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [8, 13] |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- sources: |
|
- layer_range: [15, 16] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [10, 16] # mids |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- sources: |
|
- layer_range: [16, 24] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
parameters: |
|
scale: |
|
- filter: k_proj |
|
value: 0.8 |
|
- filter: down_proj |
|
value: 0.8 |
|
- sources: |
|
- layer_range: [16, 24] |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- sources: |
|
- layer_range: [24, 26] # end bit |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
parameters: |
|
scale: |
|
- filter: v_proj |
|
value: 0.8 |
|
- sources: |
|
- layer_range: [27, 29] |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
parameters: |
|
scale: |
|
- filter: v_proj |
|
value: 0.8 |
|
- sources: |
|
- layer_range: [29, 31] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [23, 32] # end cap |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
parameters: |
|
int8_mask: true |
|
merge_method: passthrough |
|
name: horizon1 |
|
--- |
|
models: |
|
slices: |
|
- sources: |
|
- layer_range: [0, 12] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [3, 8] |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- sources: |
|
- layer_range: [5, 10] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [9, 13] |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- sources: |
|
- layer_range: [12, 15] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [16, 19] |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- sources: |
|
- layer_range: [20, 26] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [22, 29] |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
- sources: |
|
- layer_range: [26, 31] |
|
model: crestf411/L3-8B-daybreak-stheno-v3.1 |
|
- sources: |
|
- layer_range: [26, 32] |
|
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2 |
|
parameters: |
|
int8_mask: true |
|
merge_method: passthrough |
|
name: horizon3 |
|
--- |
|
models: |
|
- model: parts/horizon1 |
|
parameters: |
|
weight: [0.9, 1, 0.75] |
|
- model: parts/horizon3 |
|
parameters: |
|
weight: [0.1 , 0, 0.25] |
|
merge_method: dare_linear |
|
parameters: |
|
normalize: false |
|
int8_mask: true |
|
base_model: parts/horizon1 |
|
dtype: bfloat16 |
|
``` |