athirdpath's picture
Update README.md
0cc62cf
metadata
license: cc-by-nc-4.0
language:
  - en
pipeline_tag: text-generation

An interesting experiment. Give it a shot.

Seems to perfer Alpaca for instruct, and ChatML for storytelling.

4-bit Examples with Alpaca

!!NSFW!! - Erotica Writing Example - !!NSFW!!

4-bit Examples with ChatML

!!NSFW!! - Erotica Writing Example - !!NSFW!!

Recipe

merge_method: dare_ties

  • base_model: athirdpath/BigMistral-11b

  • model: athirdpath/NeuralPivot-11b

    weight: [0.50, 0.45, 0.30] / density: 0.30

  • model: athirdpath/PianoMaid-11b

    weight: [0.22, 0.27, 0.42] / density: [0.55, 0.52, 0.35]

  • model: athirdpath/BigMistral-11b-GLUED

    weight: 0.28 / density: 0.30

int8_mask: true

dtype: bfloat16