magnum-MoE-22B

magnum-MoE-22B is a Mixture of Experts (MoE) 22B model merged using Mergekit.

🧩 Architecture Configuration

base_model: anthracite-org/magnum-v2.5-12b-kto
gate_mode: cheap_embed
dtype: bfloat16
experts:
  - source_model: anthracite-org/magnum-v2.5-12b-kto
    positive_prompts: "creative writing, storytelling, roleplay, engaging dialogue, prose, fiction"
  - source_model: DavidAU/Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETIC
    positive_prompts: "reasoning, logic, step-by-step thinking, coding, math, problem solving, analysis"

💻 Quick Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Taephom/magnum-MoE-22B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto"
)