metadata

base_model:
  - NousResearch/Yarn-Mistral-7b-128k
  - Test157t/Kunocchini-1.1-7b
library_name: transformers
tags:
  - mistral
  - quantized
  - text-generation-inference
  - merge
  - mergekit
pipeline_tag: text-generation
inference: false

Quantazing and uploading...

GGUF-Imatrix quantizations for Test157t/Kunocchini-1.2-7b-longtext.

SillyTavern preset files for the previous version are located here.

If you want any specific quantization to be added, feel free to ask.

All credits belong to the creator.

Base⇢ GGUF(F16)⇢ Imatrix(F16)⇢ GGUF-Imatrix(Quants)

The new IQ3_S merged today has shown to be better than the old Q3_K_S, but will only be supported in koboldcpp-1.60 or newer.

Using llama.cpp-b2254.

For --imatrix data, imatrix-Kunocchini-1.2-7b-longtext-F16.dat was used.

Original model information:

Thanks to @Epiculous for the dope model/ help with llm backends and support overall.

Id like to also thank @kalomaze for the dope sampler additions to ST.

@SanjiWatsuki Thank you very much for the help, and the model!

This model was merged using the DARE TIES.

Models Merged

The following models were included in the merge:

NousResearch/Yarn-Mistral-7b-128k + Test157t/Kunocchini-1.1-7b

Configuration

The following YAML configuration was used to produce this model:

merge_method: dare_ties
base_model: Test157t/Kunocchini-1.1-7b
parameters:
  normalize: true
models:
  - model: NousResearch/Yarn-Mistral-7b-128k
    parameters:
      weight: 1
  - model: Test157t/Kunocchini-1.1-7b
    parameters:
      weight: 1
dtype: float16