metadata
base_model:
- unsloth/Mistral-Small-Instruct-2409
library_name: transformers
tags:
- mergekit
- merge
BA-Zephyria-39b [EXPERIMENTAL]
Model Information
Base Model: unsloth/Mistral-Small-Instruct-2409
Strategy: Balanced Approach
Total Layers: 55
Duplication Start: Layer 19 (34.5% of model)
Duplicated Layers: 23 (41.8% of model)
Unique Final Layers: 14 (25.5% of model)
Model Characteristics
- Models down_proj and o_proj layers have been nulled and will require healing
- Combines benefits of early and mid duplication strategies
- Balanced between unique initial layers, duplicated middle layers, and unique final layers
- Versatile approach suitable for a wide range of tasks
- Provides substantial unique layers at the end for task-specific adaptations
Configuration Visualization
[ Unique ][ Duplicated ][ Unique ]
0 ----------- 18 19 ------------ 41 42 ---------- 54
34.5% 41.8% 23.7%