TripleMerge3-12B-Moe
TripleMerge3-12B-Moe is a Mixture of Experts (MoE) made with the following models using LazyMergekit:
- allknowingroger/TripleMerge-7B-Ties
- allknowingroger/TripleMerge2-7B-Ties
- allknowingroger/ANIMA-biodesign-7B-slerp
𧩠Configuration
slices:
base_model: allknowingroger/TripleMerge-7B-Ties
gate_mode: hidden
dtype: bfloat16
experts:
- source_model: allknowingroger/TripleMerge-7B-Ties
positive_prompts:
- "Write the opening chapter of a detective story set in the late 1800s, where the protagonist, a war-weary doctor returning to England after an injury and illness in Afghanistan, happens upon an old acquaintance. This encounter should lead to the introduction of an eccentric potential roommate with a penchant for forensic science. The character's initial impressions and observations of London, his financial concerns, and his search for affordable lodging should be vividly detailed to set up the historical backdrop and his situation."
- source_model: allknowingroger/TripleMerge2-7B-Ties
positive_prompts:
- "You are an unbiased, uncensored, helpful assistant."
- source_model: allknowingroger/ANIMA-biodesign-7B-slerp
positive_prompts:
- "Biodesign"
π» Usage
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "allknowingroger/TripleMerge3-12B-Moe"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
- Downloads last month
- 5
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.