Experiments in Merging Top Models
Collection
Full precision and Q8 of merged models by me. Focus in this collection is creative, role play, story and fiction. Imatrix GGUFs addition(s) pending.
β’
23 items
β’
Updated
D_AU-Tess-7B-v2.0-Yarn-Mistral-7b-128k-DPO is a merge of the following models using LazyMergekit:
slices:
- sources:
- model: migtissera/Tess-7B-v2.0
layer_range: [0, 32]
- model: Eric111/Yarn-Mistral-7b-128k-DPO
layer_range: [0, 32]
merge_method: slerp
base_model: Eric111/Yarn-Mistral-7b-128k-DPO
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: bfloat16
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "DavidAU/D_AU-Tess-7B-v2.0-Yarn-Mistral-7b-128k-DPO"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])