Instructions to use russellbal/ANDREA-12M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use russellbal/ANDREA-12M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="russellbal/ANDREA-12M", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("russellbal/ANDREA-12M", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use russellbal/ANDREA-12M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "russellbal/ANDREA-12M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "russellbal/ANDREA-12M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/russellbal/ANDREA-12M

SGLang

How to use russellbal/ANDREA-12M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "russellbal/ANDREA-12M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "russellbal/ANDREA-12M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "russellbal/ANDREA-12M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "russellbal/ANDREA-12M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use russellbal/ANDREA-12M with Docker Model Runner:
```
docker model run hf.co/russellbal/ANDREA-12M
```

ANDREA-12M

A 12.8M parameter decoder-only language model grown on a single RTX 4090 with a bandit-controlled curriculum over open data. Part of a permacomputer project: open source, open data, open weights.

Weights here are bit-identical to the CUDA training checkpoint step_43587.bin at step 43,587 — the export is verified tensor by tensor, not assumed.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("russellbal/ANDREA-12M", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("russellbal/ANDREA-12M", trust_remote_code=True)

ids = tok.apply_chat_template(
    [{"role": "user", "content": "What is a permacomputer?"}],
    add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids, max_new_tokens=64, do_sample=True,
                                temperature=0.7)[0]))

trust_remote_code=True is required: ANDREA is its own architecture, and the three *_andrea.py files in this repo are its reference implementation.

Prompt format matters more than usual here

The corpus writes dialogue on a single line as > question / < answer, with turns joined by /. The chat template shipped with this repo produces exactly that, and staying inside it is the difference between coherent output and word salad — a bare prose prompt is out of distribution for this model.

Documents are also separated by BOS during training, and the tokenizer prepends it by default. Encoding with add_special_tokens=False drops it and starts the model mid-document, which measurably degrades generation. Sampling at temperature=0.7 matches how the model was sampled during training; greedy decoding at this scale degenerates into repetition.

An unconditional sample (input_ids=[[bos]], temperature=0.7) looks like:

> what is the most popular breed of dog? / < the most popular breed of dog is ...
> How does GPU architecture affect gaming performance? / < Modern GPU architectures
  like NVIDIA's Ada Lovelace and AMD's ...

Architecture

Property	Value
Parameters	12,780,288
Embedding dim	384
Heads	12
Layers	6
Context	1024 tokens
Vocab	2,305
Tokenizer	Harris successor-variety, 2,048 morpheme segments
Training steps	43,587
Precision	float32
License	AGPL-3.0

ANDREA is deliberately neither GPT-2 nor LLaMA. Reimplementing it means matching all of these:

RMSNorm with no learnable gain. There is no norm weight in this checkpoint because the architecture has none.
Learned absolute position embeddings. No RoPE, no ALiBi. The model cannot extrapolate past 1024 tokens and raises rather than guessing.
A norm on the embedding sum, applied once before layer 0.
No final norm before the LM head.
ReLU MLP at 4x. Not GELU, not SwiGLU.
No biases on any projection.
Untied LM head — lm_head.weight is its own tensor, not wte reused.

x = wte[tok] + wpe[pos]
x = rmsnorm(x)
per layer:
    x = x + o_proj(attn(rmsnorm(x)))
    x = x + down_proj(relu(up_proj(rmsnorm(x))))
logits = lm_head(x)

RMSNorm is x * (mean(x^2) + 1e-05) ** -0.5.

Tokenizer

Harris (1955) successor-variety segmentation: morpheme boundaries discovered from the distributional properties of text, with no language-specific rules. The vocabulary is 256 raw byte fallbacks, then 2,048 learned segments, then BOS at id 2304. Any byte sequence encodes and round-trips, so text the segment table has never seen degrades to bytes rather than to an unknown token.

Training

Trained on the megachat-v8 corpus through a phase-based UCB1 bandit curriculum that reweights sources as the model's per-source loss moves. Data was filtered for dirty text, non-English, and assistant refusals before training.

Limitations

A model this size is a research artifact. It produces fluent English and has absorbed real factual material, but it hallucinates, has no alignment tuning, and has no safety filtering beyond the corpus cleaning described above. It was not built for and should not be used for advice, moderation, or any decision affecting a person.

Provenance

Full training history, probe analyses, and reproduction instructions live with the source: https://git.unturf.com. Companion papers: ANDREA-WHITEPAPER (training) and ANDREA-PROBES (15 interpretability probes).

Downloads last month: 16

Safetensors

Model size

12.8M params

Tensor type

F32

russellbal
/

ANDREA-12M