Nous-Hermes-2-SUS-Chat-34B-Slerp

This is the model for Nous-Hermes-2-SUS-Chat-34B-Slerp. I used mergekit to merge models.

Prompt Templates

You can use these prompt templates, but I recommend using ChatML.

ChatML (NousResearch/Nous-Hermes-2-Yi-34B):

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant
{asistant}<|im_end|>

Human - Asistant (SUSTech/SUS-Chat-34B):

### Human: {user}

### Assistant: {asistant}

Yaml Config


slices:
- sources:
    - model: Nous-Hermes-2-Yi-34B
      layer_range: [0, 60]
    - model: SUS-Chat-34B
      layer_range: [0, 60]

merge_method: slerp
base_model: Yi-34B

parameters:
t:
  - filter: self_attn
    value: [0, 0.5, 0.3, 0.7, 1]
  - filter: mlp
    value: [1, 0.5, 0.7, 0.3, 0]
  - value: 0.5
tokenizer_source: union
dtype: bfloat16

Quantizationed versions

Quantizationed versions of this model is available thanks to TheBloke.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	74.06
AI2 Reasoning Challenge (25-Shot)	66.72
HellaSwag (10-Shot)	84.97
MMLU (5-Shot)	77.00
TruthfulQA (0-shot)	59.23
Winogrande (5-shot)	83.58
GSM8k (5-shot)	72.86

Weyaxi
/

Nous-Hermes-2-SUS-Chat-34B-Slerp