Yo-Purple/Meditor-1-500M-Universal

Meditor 1 500M Universal — Universal Dense Engine

Created by Yo-Purple
494.0M Stored Parameters • 100% Bit-Exact Active Compute to Meditor 1 MoE • Standard Qwen2 Architecture
Compatible with Transformers (trust_remote_code=False), vLLM, Ollama, LM Studio, and llama.cpp

---

🌟 Overview

Meditor 1 Universal is the standardized, universal-architecture release of Meditor 1. In Meditor 1's foundational release, the model's active compute flowed through its 24 Grouped-Query Attention (GQA) layers and dedicated 4,864-dimensional SwiGLU shared expert (routed_scale = 0.0).

By re-encapsulating these active parameters into the industry-standard Qwen2ForCausalLM (qwen2) topology, Meditor 1 Universal delivers:

• 100.0% Bit-Exact Active Parity: Produces identical mathematical outputs to the original PyTorch model forward pass.
• Zero Custom Drivers Required: Runs natively across the entire open-source ecosystem without requiring custom Python files, external kernels, or trust_remote_code=True.
• Flawless GGUF & C++ Engine Support: Solves the missing-gating bias bug in C++ inference engines (llama.cpp, Ollama, LM Studio) that affected custom MoE weights, achieving blistering throughput up to 287+ tokens/sec on consumer GPUs.
• Ultra-Lightweight Footprint: Requires only ~0.93 GB of VRAM, running easily on laptops, Raspberry Pis, iPhones, and desktop GPUs with abundant headroom for long context.

---

📊 Model Specifications

Attribute	Specification
Model Name	Meditor-1-500M-Universal
Creator	Yo-Purple
Architecture	Standard Qwen2ForCausalLM (qwen2)
Foundation Origin	Derived from Yo-Purple/Meditor-1-2.2B-A500M-MoE
Total Stored Parameters	494,032,768 (~494M) (Tied Word Embeddings)
Active Parameters / Token	494,032,768 (100% dense active)
Layers	24 Transformer Layers
Hidden Size ($d_{model}$)	896
Intermediate FFN Size	4,864 (SwiGLU activation)
Attention Mechanism	Grouped-Query Attention (14 Q heads, 2 KV heads; GQA 7:1)
Max Context Length	32,768 tokens (Native RoPE theta: 1,000,000.0)
Formats Provided	Hugging Face SafeTensors (bfloat16) + Native GGUF (BF16, Q8_0, Q4_K_M)

---

🔬 Benchmark & Performance Results

Empirically evaluated locally on an NVIDIA GeForce RTX 4060 8GB Laptop GPU with PyTorch 2.6 and llama.cpp (CUDA build 4576).

Metric	Result	Benchmark Details
Reasoning Perplexity (PPL)	3.56	Cross-entropy loss: 1.2699 across 8,867 reasoning tokens (GSM8K)
ARC-Style Science Reasoning	86.7%	13/15 zero-shot log-likelihood across multiple-choice questions
Direct Arithmetic Accuracy	73.3%	11/15 raw autoregressive decoding
Tool-Augmented Arithmetic	100.0%	15/15 when paired with safe AST mental math coprocessor
Prefill Latency (256 tok)	37.7 ms	Ultra-responsive ingestion (~6,790 prompt tok/s)
PyTorch Decode Speed	29.4 tok/s	Native PyTorch FP16/BF16 generation
GGUF BF16 Throughput	157.5 tok/s	Full-precision llama.cpp CUDA decode
GGUF Q8_0 Throughput	249.3 tok/s	8-bit quantized llama.cpp CUDA decode (644 MB)
GGUF Q4_K_M Throughput	287.4 tok/s	4-bit medium quantized llama.cpp CUDA decode (468 MB)
Active VRAM Consumption	~0.93 GB	Leaves ~7.0 GB VRAM free for context & multi-agent concurrency

---

📦 Available Files & Formats

This repository contains both the official SafeTensors weights and pre-quantized GGUF binaries ready for immediate deployment:

File Name	Format	Size	Description
model.safetensors	SafeTensors (BF16)	1.18 GB	Official Hugging Face weights for PyTorch & vLLM
meditor-1-universal-bf16.gguf	GGUF (BF16)	1.18 GB	Full-precision unquantized GGUF binary
meditor-1-universal-q8_0.gguf	GGUF (Q8_0)	644 MB	High-fidelity 8-bit quantization for precision tasks
meditor-1-universal-q4_k_m.gguf	GGUF (Q4_K_M)	468 MB	Recommended balance of maximum speed (287 tok/s) and accuracy

---

💻 Quick Start & Deployment

1. Hugging Face transformers (Zero Remote Code)

Because Meditor 1 Universal uses the standard Qwen2 architecture, you do not need trust_remote_code=True:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Yo-Purple/Meditor-1-500M-Universal"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "<|im_start|>user\nExplain why leaves change color in autumn in two simple sentences.<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

output = model.generate(**inputs, max_new_tokens=100, temperature=0.7)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

2. LM Studio (One-Click Local GUI)

1. Open LM Studio.
2. In the search bar, enter: Yo-Purple/Meditor-1-500M-Universal.
3. Select and download meditor-1-universal-q4_k_m.gguf (468 MB).
4. Chat at full hardware speed with GPU acceleration!

---

3. vLLM High-Throughput Serving

Serve an OpenAI-compatible endpoint with vLLM:

vllm serve Yo-Purple/Meditor-1-500M-Universal --port 8000

---

4. llama.cpp CLI & Server

Run directly in your terminal at 280+ tokens per second:

llama-cli -m meditor-1-universal-q4_k_m.gguf -ngl 99 -c 4096 --temp 0.7 -p "User: Write a poem about space exploration.\nAssistant:"

---

📜 Citation & Credits

• Architecture Remapping & Quantization: Yo-Purple
• Foundation Pretrained Weights: Derived from Yo-Purple/Meditor-1-2.2B-A500M-MoE and Qwen/Qwen2.5-0.5B-Instruct
• License: Apache 2.0