README.md

---
base_model:
- NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
- NeuralNovel/Gecko-7B-v0.1-DPO
tags:
- mergekit
- merge
license: apache-2.0

datasets:
- Intel/orca_dpo_pairs
- NeuralNovel/Neural-Story-v1

---


![tiger](https://cdn-uploads.huggingface.co/production/uploads/645cfe4603fc86c46b3e46d1/a9GqRTNoGZQsRVU-C6XRO.jpeg)


# Tiger-7b-v0.1

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

[Join our Discord!](https://discord.gg/Qge8Ds9C)

## Metrics



![image/png](https://cdn-uploads.huggingface.co/production/uploads/645cfe4603fc86c46b3e46d1/Z58bB5sYr3pyE2Ilbk7Dk.png)



### Merge Method

This model was merged using the SLERP merge method.

### Models Merged

The following models were included in the merge:
* [NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story](https://huggingface.co/NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story)
* [NeuralNovel/Gecko-7B-v0.1-DPO](https://huggingface.co/NeuralNovel/Gecko-7B-v0.1-DPO)
# merge
### Configuration

The following YAML configuration was used to produce this model:

```yaml

slices:
  - sources:
      - model: NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
        layer_range: [0, 32]
      - model: NeuralNovel/Gecko-7B-v0.1-DPO
        layer_range: [0, 32]
merge_method: slerp
base_model: NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16



```