AtakanTekparmak/hermes2-pro-llama3-14b-GGUF

Hermes-2-Pro-Llama-3-14B

This is the GGUF version of a naive (passthrough) self-merge of two NousResearch/Hermes-2-Pro-Llama-3-8B, using mergekit. The merge config was inpired by mlabonne/Meta-Llama-3-120B-Instruct.

Results

This gives better (and actually usable) results compared to my previous merges. I can clearly see a rise in capability as it responded with a more thought out answer to my function calling prompt, but failed to adhere to the format and instructions fully. Will read more and look at other merge methods. This is also the first GGUF that I converted myself which also might have contributed to the increase in quality.

Conversion Command

Using the convert-hf-to-gguf.py provided in llama.cpp:

python convert-hf-to-gguf.py --outfile hermes2-pro-llama3-14b-gguf --outtype f16 --use-temp-file --model-name Hermes2-Pro-Llama3-14b hermes2-pro-llama3-14b

Merge Details

Merge Method

This model was merged using the passthrough merge method.

Models Merged

The following models were included in the merge:

• NousResearch/Hermes-2-Pro-Llama-3-8B

Configuration

The following YAML configuration was used to produce this model:

slices:
- sources:
  - layer_range: [0, 8]
    model: NousResearch/Hermes-2-Pro-Llama-3-8B
- sources:
  - layer_range: [4, 12]
    model: NousResearch/Hermes-2-Pro-Llama-3-8B
- sources:
  - layer_range: [8, 16]
    model: NousResearch/Hermes-2-Pro-Llama-3-8B
- sources:
  - layer_range: [12, 20]
    model: NousResearch/Hermes-2-Pro-Llama-3-8B
- sources:
  - layer_range: [16, 24]
    model: NousResearch/Hermes-2-Pro-Llama-3-8B
- sources:
  - layer_range: [20, 28]
    model: NousResearch/Hermes-2-Pro-Llama-3-8B
- sources:
  - layer_range: [24, 32]
    model: NousResearch/Hermes-2-Pro-Llama-3-8B
merge_method: passthrough
dtype: float16