|
--- |
|
base_model: |
|
- jondurbin/nontoxic-bagel-34b-v0.2 |
|
tags: |
|
- mergekit |
|
- frankenstein |
|
- merge |
|
license: mit |
|
--- |
|
|
|
# Shqiponja-59 V1 |
|
|
|
 |
|
|
|
This is an untrained experimental 59B merged model. |
|
|
|
Picked these two models specifically to compliment each others strengths. |
|
### Models Merged |
|
* NousResearch/Nous-Hermes-2-Yi-34B |
|
* jondurbin/nontoxic-bagel-34b-v0.2 |
|
Merged using the Undi95 style passthrough merge method. |
|
|
|
|
|
### The secret sauce |
|
|
|
The following YAML configuration was used to produce this model: |
|
|
|
```yaml |
|
dtype: bfloat16 |
|
merge_method: passthrough |
|
slices: |
|
- sources: |
|
- layer_range: [0, 52] |
|
model: /home/admin/nv1/nontoxic-bagel-34b-v0.2 |
|
- sources: |
|
- layer_range: [8, 60] |
|
model: /home/admin/nv1/Nous-Hermes-2-Yi-34B |
|
``` |
|
# License MIT - Enjoy |
