VertexAGI/quartz-micro-preview-v3

Quartz Micro Preview V3 (instruction-tuned)

A tiny language model trained from scratch by Vertex AGI on a single GTX 1660 Ti: a 100.09M-parameter dense Llama-architecture decoder (12 layers, hidden 768, 12 heads / 4 KV heads, SwiGLU 2048, RoPE, tied 32K byte-level BPE, 1,024 context). This is the instruction-tuned chat model. The raw pretrained model is VertexAGI/quartz-micro-preview-v3-base. The previous generation is VertexAGI/quartz-micro-preview-v2. You can try it and compare it with V2, V1 and the base models in the chat Space.

Honest summary: at ~100M parameters this model writes short, fluent text but still states wrong facts and invents details, is weak at maths, reasoning and code, and ignores length limits. V3 is clearly better than V2 on our tests (below), but treat it as a small research / hobby model, not a source of truth.

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Format	File / folder	Precision	Size	Works with
Transformers (safetensors)	model.safetensors (repo root)	fp32 (32-bit)	400 MB	Hugging Face Transformers / PyTorch
MLX	mlx/fp16/	fp16 (16-bit)	200 MB	mlx-lm / MLX on Apple Silicon
MLX	mlx/q4/	4-bit (group size 64)	63 MB	mlx-lm / MLX on Apple Silicon
MLX	mlx/q8/	8-bit (group size 64)	113 MB	mlx-lm / MLX on Apple Silicon
GGUF	gguf/quartz-micro-preview-v3-f16.gguf	F16 (16-bit float)	201 MB	llama.cpp, Ollama, LM Studio
GGUF	gguf/quartz-micro-preview-v3-q4_k_m.gguf	Q4_K_M (4-bit)	67 MB	llama.cpp, Ollama, LM Studio
GGUF	gguf/quartz-micro-preview-v3-q8_0.gguf	Q8_0 (8-bit)	108 MB	llama.cpp, Ollama, LM Studio

Formats (one repo, three formats)

Where	Format
repo root	stock Hugging Face LlamaForCausalLM (fp32 safetensors), loads with plain transformers and plain mlx_lm
mlx/fp16, mlx/q8, mlx/q4	stock MLX (fp16, 8-bit, 4-bit)
gguf/	stock llama.cpp GGUF: f16, q8_0, q4_k_m

Every format was checked against the original training code on held-out text (perplexity; lower is better):

Format	Perplexity	Difference from original
transformers fp32 (this repo root)	13.878	0.000%
MLX fp16	13.875	0.020%
MLX 8-bit	13.878	0.001%
MLX 4-bit	14.739	6.205%
GGUF f16	13.878	0.001%
GGUF Q8_0	13.877	0.009%
GGUF Q4_K_M	13.955	0.557%

Usage

# stock transformers (no custom code, no trust_remote_code)
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v3")
model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v3")
messages = [{"role": "user", "content": "Why do cats purr?"}]
prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=400, do_sample=True, temperature=0.6, top_p=0.9, repetition_penalty=1.1)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

# stock MLX (Apple silicon)
from mlx_lm import load, generate
model, tok = load("VertexAGI/quartz-micro-preview-v3")
print(generate(model, tok, prompt=tok.apply_chat_template([{"role": "user", "content": "Why do cats purr?"}], add_generation_prompt=True), max_tokens=400))

# stock llama.cpp (GGUF in the gguf/ folder)
hf download VertexAGI/quartz-micro-preview-v3 gguf/quartz-micro-preview-v3-q8_0.gguf --local-dir .
llama-cli -m gguf/quartz-micro-preview-v3-q8_0.gguf -cnv

Benchmark: V3 vs V2 (instruct models, same prompts)

Both models answered the same 156 prompts with identical decoding (greedy, repetition penalty 1.1, same tokenizer and chat template), run on one GPU. Judge for the open-ended parts: nvidia/nemotron-3-ultra-550b-a55b, temperature 0; pairwise comparisons were judged in both orders and only count as a win when both orders agree. No judge call failed (all counted, none dropped).

Test	V2	V3
Identity, 100 held-out prompts (judge checks the reply against the model's fact sheet)	3/100 (3%)	71/100 (71%)
Identity, 24 hand-written prompts	5/24 (21%)	23/24 (96%)
Facts, V2's 15 original keyword prompts	5/15	11/15
Facts, 11 new prompts (grounded QA, honest limits, simple facts)	4/11	9/11
Facts, all 26	9/26	20/26
Head-to-head on 33 general prompts (judge, both orders agree)	4 wins	25 wins (4 ties)
Answers that end on their own (<eos>)	23/33	31/33
Repetition (distinct-4-gram, higher = less repetitive)	0.801	0.97
Average answer length (words)	126.8	51.5
Loss on assistant tokens, general validation set (lower is better)	2.383	1.719
Loss on held-out identity answers (lower is better)	4.298	1.304

How to read this

• Identity is where V3 was designed to improve (V3 adds the quartz-identity-data set; V2 had none). V2 usually names a different maker (it called itself "Open Assistant" in earlier tests). The held-out set is 100 prompts from a split that was never in V3's training data; V3 still fails 29 of them, mostly in multi-turn conversations (4/13 passed).
• Facts is a keyword check (the answer must contain the expected word), so it rewards plain correct answers and is easy to fool; it is a smoke test, not a knowledge benchmark. The 11 "new" prompts target what V3's extra data teaches (answer from a passage, say "the passage does not say", admit it cannot browse), so V2 was never trained for them: that row shows the added ability, not a like-for-like gain.
• Length: V2 rambles (127 words on average, 10 of 33 answers never ended within the 300-token budget); V3 is shorter (52 words) and mostly stops on its own.
• Still weak: V3 is a 100M-parameter model. It still invents facts (for example it described a made-up city as fictional and from a famous novel, and wrote a 16-line poem when asked for two). Treat answers as unreliable.
• The two runs are a single greedy sample per prompt; with 26 keyword prompts and 33 judged comparisons, differences of one or two items are noise. Judge scores moved by 1 item between two scoring passes of the same V3 answers (70 vs 71 of 100).
• V2 and V3 differ in more than the instruction data: V3 is pretrained on a much larger mix (about 6.5 B tokens vs 100 M for V2), so this compares the two releases, not the effect of the SFT data alone.

Every prompt and both models' replies are in eval_outputs_v3_vs_v2.md; the raw numbers are in eval_scores_v3_vs_v2.json.

Instruction tuning

• Base: VertexAGI/quartz-micro-preview-v3-base (pretrained on ~6.5B tokens, see that card).
• Data: 44,389 English conversations (12.3M tokens; 7.6M are assistant tokens that carry the loss): the V2 chat set (24,170 conversations, OpenAssistant OASST2 + smol-smoltalk slices), Dolly-15k (11,881), SQuAD 2.0 grounded questions with the unanswerable ones answered "the passage does not say" (3,908), CoCoNot "honest limits" examples (no browsing, no real-time information, false premises; 1,730) and 2,700 identity conversations that teach the model who it is. 300 further identity conversations were held out and never trained on.
• Method: full-parameter fine-tune, loss on assistant tokens only, 2 epochs (886 steps), lr 1e-4 cosine, fp32 on the same GPU (about 1.5 hours). Final validation loss on the held-out chat set: 1.719 (the pretrained model: 2.085).
• Chat format: <|user|>\n{message}\n<|assistant|>\n{reply}<eos> (built into the tokenizer's chat template; no system role, so put instructions in the user message).
• Suggested sampling: temperature 0.6, top-p 0.9, repetition penalty 1.1.

Identity

V3 was tuned to say that it is a small language model from Vertex AGI in the Quartz series, a preview that is not reliable, with no browsing, no memory between chats and no image or audio input, and that it is not ChatGPT, Claude, Gemini or another assistant. It is right about this in most single questions (71 of 100 held-out prompts, 23 of 24 hand-written ones), and weaker in longer conversations. It can still be argued out of it.

Limitations

• States wrong facts and invents details (for example it described a made-up city as fictional and from a famous novel, and wrote a 16-line poem when asked for two).
• Weak at maths, logic and code; follows length and format limits unreliably.
• 1,024-token context; English only; no internet, tools, images or audio.
• No safety tuning or refusal training (the CoCoNot examples teach it to admit what it cannot do, not to refuse harmful requests). Do not deploy it unsupervised.

Data and licences (read before commercial use)

The weights are released under Apache-2.0, but they were trained on third-party data with its own terms, and some of those terms are NonCommercial or ShareAlike (for example the Tulu-3 mixture includes no_robots, CC-BY-NC; Wikipedia, Stack Exchange, Dolly and SQuAD are CC-BY-SA; CoCoNot is ODC-By). Whether training-data terms extend to model weights is legally unsettled. This is a research / hobby model: if you need legal certainty for a commercial product, ask a lawyer or retrain on data you have cleared. The pretraining and identity datasets are not public.