twosmoothslateslabs/Flora-Mistral-7B-v1.0

temp

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the della merge method using a finetuned Mistral 7b v0.1 with a private dataset I curated as a base. The dataset I used specializes in reasoning, so gutenberg should add a touch of creativity.

Models Merged

The following models were included in the merge:

• nbeerbower/Hermes2-Gutenberg2-Mistral-7B

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: E:/models/dddd/output/intermediate/
    parameters:
      weight: 1.0
  - model: nbeerbower/Hermes2-Gutenberg2-Mistral-7B
    parameters:
      weight: 0.6
  - model: nbeerbower/Hermes2-Gutenberg2-Mistral-7B
    parameters:
      weight: 0.2
merge_method: della
base_model: E:/models/dddd/output/intermediate/
parameters:
  normalize: true
  int8_mask: true
  lambda: 1
  epsilon: 0.25
dtype: float16