final_merge
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the task arithmetic merge method using /home/evol_merge_storage/input_models/Mistral-7B-v0.1_8133861 as a base.
Models Merged
The following models were included in the merge:
- /home/evol_merge_storage/input_models/zephyr-7b-beta_2449712360
- /home/evol_merge_storage/input_models/Hermes-2-Pro-Mistral-7B_2793206805
- /home/evol_merge_storage/input_models/Dans-AdventurousWinds-Mk2-7b_1152917843
Configuration
The following YAML configuration was used to produce this model:
base_model: /home/evol_merge_storage/input_models/Mistral-7B-v0.1_8133861
dtype: bfloat16
merge_method: task_arithmetic
parameters:
int8_mask: 1.0
normalize: 0.0
slices:
- sources:
- layer_range: [0, 8]
model: /home/evol_merge_storage/input_models/Hermes-2-Pro-Mistral-7B_2793206805
parameters:
weight: 0.45632162073777516
- layer_range: [0, 8]
model: /home/evol_merge_storage/input_models/Dans-AdventurousWinds-Mk2-7b_1152917843
parameters:
weight: 0.6468013824217197
- layer_range: [0, 8]
model: /home/evol_merge_storage/input_models/zephyr-7b-beta_2449712360
parameters:
weight: 0.04013732886365818
- layer_range: [0, 8]
model: /home/evol_merge_storage/input_models/Mistral-7B-v0.1_8133861
- sources:
- layer_range: [8, 16]
model: /home/evol_merge_storage/input_models/Hermes-2-Pro-Mistral-7B_2793206805
parameters:
weight: 0.39991499422381294
- layer_range: [8, 16]
model: /home/evol_merge_storage/input_models/Dans-AdventurousWinds-Mk2-7b_1152917843
parameters:
weight: 0.3033908462486953
- layer_range: [8, 16]
model: /home/evol_merge_storage/input_models/zephyr-7b-beta_2449712360
parameters:
weight: 0.3250623744211742
- layer_range: [8, 16]
model: /home/evol_merge_storage/input_models/Mistral-7B-v0.1_8133861
- sources:
- layer_range: [16, 24]
model: /home/evol_merge_storage/input_models/Hermes-2-Pro-Mistral-7B_2793206805
parameters:
weight: 0.6202885707451035
- layer_range: [16, 24]
model: /home/evol_merge_storage/input_models/Dans-AdventurousWinds-Mk2-7b_1152917843
parameters:
weight: 0.235874597687366
- layer_range: [16, 24]
model: /home/evol_merge_storage/input_models/zephyr-7b-beta_2449712360
parameters:
weight: 0.27733993810853536
- layer_range: [16, 24]
model: /home/evol_merge_storage/input_models/Mistral-7B-v0.1_8133861
- sources:
- layer_range: [24, 32]
model: /home/evol_merge_storage/input_models/Hermes-2-Pro-Mistral-7B_2793206805
parameters:
weight: 0.5484088078896818
- layer_range: [24, 32]
model: /home/evol_merge_storage/input_models/Dans-AdventurousWinds-Mk2-7b_1152917843
parameters:
weight: 0.876059424739865
- layer_range: [24, 32]
model: /home/evol_merge_storage/input_models/zephyr-7b-beta_2449712360
parameters:
weight: 0.26196309089548436
- layer_range: [24, 32]
model: /home/evol_merge_storage/input_models/Mistral-7B-v0.1_8133861
- Downloads last month
- 32
Invalid base_model specified in model card
metadata. Needs to be a model id from
hf.co/models.