Quartz-M1-100M-Instruct

Quartz-M1-100M-Instruct is a lightweight, ultra-fast 100M parameter language model based on the LLaMA architecture. It is designed for low-latency edge deployment, local chat, and high-throughput inference (reaching 400–485 tokens/second on consumer GPUs like the RTX 3060).

---

Model Files & Formats

File	Format / Quant	Size	Recommended Use
model.safetensors	Safetensors (BF16/FP16)	~202 MB	PyTorch, Hugging Face Transformers
Quartz-M1-100M-Instruct-f16.gguf	GGUF FP16	~202 MB	Full precision, llama.cpp, LM Studio, Ollama
Quartz-M1-100M-Instruct-Q8_0.gguf	GGUF Q8_0	~107 MB	Near-lossless 8-bit quantization
Quartz-M1-100M-Instruct-Q4_K_M.gguf	GGUF Q4_K_M	~65 MB	Ultra-compact 4-bit quantization

---

Performance Benchmarks

• RTX 3060 (12GB) with CUDA:
  • Text Generation: ~400 – 485 Tokens/Second (TPS)
  • Prompt Processing: ~23,000 – 33,000 Tokens/Second
  • VRAM Usage: < 250 MB

---

Quickstart

1. Using llama.cpp CLI

llama-cli -m Quartz-M1-100M-Instruct-f16.gguf -ngl 99 -p "Explain quantum computing simply." -n 128

2. Using transformers (Python)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "honvert/Quartz-M1-100M-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

messages = [
    {"role": "user", "content": "What are three key habits of effective developers?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

Architecture & Parameters

• Parameters: 101,343,744 (101M)
• Hidden Size: 512
• Layers: 24
• Attention Heads: 8 (KV Heads: 4 - Grouped Query Attention)
• Context Window: 1,024 tokens