ChapAF/intent-0.1-0.5b

This repository contains a custom decoder-only model trained by the first-order-dictionary-learning pipeline. It is an inference artifact for the project's pretraining.model.GPT implementation, rather than a native transformers.AutoModel checkpoint.

Files

• model.safetensors: model weights only (455933952 parameters).
• model-f16.gguf: llama.cpp/LM Studio compatible GGUF export.
• config.json: the exact model and training configuration used to build the model.
• tokenizer files: a Rust LlamaTokenizerFast/fast tokenizer saved with the model.
• pretraining/model.py and chat.py: the minimal custom inference code.
• checkpoint_metadata.json: source and conversion metadata.

Optimizer state, distributed rank state, raw training data, and W&B files are intentionally not included.

The GGUF uses llama.cpp's Arcee ReLU-squared graph. The exact training model also applies Q/K RMSNorm, which the upstream Arcee graph does not expose, so the GGUF is a tooling-compatible approximation. Use model.safetensors with the included chat.py for numerically faithful inference.

Model details

• Tokenizer source: meta-llama/Llama-2-7b-chat-hf
• Vocabulary size: 32000
• Context length: 2048
• Layers / hidden size: 24 / 1152
• Thinking: off by default; explicitly request <think>...</think> reasoning when needed.

Run locally

From the root of this repository (with PyTorch, transformers, safetensors, flash-attn/liger as available):

PYTHONPATH=. python chat.py \
  --checkpoint model.safetensors \
  --config config.json \
  --mode chat --prompt-format sft \
  --think true \
  --prompt "Explain what a steering vector is." \
  --max-new-tokens 256

The checkpoint uses the serialized SFT prompt format when it was produced by the SFT run: <|system|>, <|user|>, and <|assistant|> role markers.