Home baked
Collection
Home-baked merges and tunes.
•
10 items
•
Updated
Several-staged merge this time. Actually decent, from my testings.
Use ChatML or Alpaca, both seemed to work though I liked the outputs from ChatML more.
models:
- model: mistralai/Mixtral-8x7B-v0.1+/ai/LLM/tmp/pefts/daybreak-peft/mixtral-8x7b
parameters:
weight: 0.65
- model: mistralai/Mixtral-8x7B-v0.1+retrieval-bar/Mixtral-8x7B-v0.1_case-briefs
parameters:
weight: 0.25
merge_method: task_arithmetic
base_model: mistralai/Mixtral-8x7B-v0.1
dtype: float16
models:
- model: mistralai/Mixtral-8x7B-Instruct-v0.1+SeanWu25/Mixtral_8x7b_Medicine
parameters:
weight: 0.33
- model: NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss
parameters:
weight: 0.15
merge_method: task_arithmetic
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
dtype: float16
This is a merge of pre-trained language models created using mergekit.
This model was merged using the SLERP merge method.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
models:
- model: ./temp-output-base
- model: ./temp-output-instruct
merge_method: slerp
base_model: ./temp-output-base
parameters:
t:
- value: 0.5
dtype: float16