atom-slm (ATOM-1)

A small, experimental Gemma-architecture causal language model, trained from scratch (not a fine-tune of a released checkpoint). Provided in GGUF format for use with llama.cpp / LM Studio.

Status: experimental / undertrained. This model does not yet produce meaningful text. In greedy decoding it tends to repeat the last input token (e.g. prompt The quick brown fox -> fox fox fox ...). It is shared for research and development purposes, not for production use.

Model details

Property	Value
Architecture	Gemma (decoder-only, causal LM)
Parameters	68,681,984 (~68.7M)
Layers (block_count)	4
Hidden size (embedding_length)	256
Feed-forward length	682
Attention heads	8 (key/value length 32)
Context length	512
Vocabulary size	256,000 (Gemma tokenizer, llama-format)
Embeddings	Tied input/output embeddings
Quantization	F16
File size	~137 MB
GGUF version	3

File

• atom_slm_f16.gguf — SHA256: 0e9f16da00580de210b2b1df1b931efcc9ec76601043064b0cd47f9a5fe33728

Usage

llama.cpp

llama-server -m atom_slm_f16.gguf -c 512

LM Studio

Place the .gguf file in your LM Studio models directory, or download this repo from the in-app model browser.

Limitations

• Undertrained: generations are degenerate (token repetition).
• Very small hidden size (256) and only 4 layers, combined with a large 256k vocabulary, make this a hard configuration to train to convergence.
• Context window is only 512 tokens.
• No chat template / instruction tuning.

Notes

The token-embedding table accounts for ~65.5M of the ~68.7M parameters, so the transformer body itself is very small.