biomistral_dare_ties_v1_7b

This model is the result of merge of the following models made with flow-merge:

• Base model:
  • mistralai/Mistral-7B-v0.1
• Models:
  • mistralai/Mistral-7B-Instruct-v0.1
  • BioMistral/BioMistral-Safetensors

flow-merge config

The following configuration was used to merge the models:

base_model: mistralai/Mistral-7B-v0.1
models:
- path_or_id: mistralai/Mistral-7B-v0.1
- path_or_id: mistralai/Mistral-7B-Instruct-v0.1
  weight: 0.5
- path_or_id: BioMistral/BioMistral-Safetensors
  weight: 0.5
method: dare-ties-merging
device:
- cuda
method_global_parameters:
  scaling_coefficient: 1.0
  p: 0.3
directory_settings:
  output_dir: ./biomistral/biomistral_dare_ties_7b/