genz-translator

A small (~17M parameter) decoder-only transformer trained entirely from scratch — random weight initialization, no pretrained base, no fine-tuning — to translate Gen Z slang into standard English.

Training data

~139,074 cleaned instruction/input/output rows (two fixed instruction types: single sentence vs. paragraph translation). Split 90/5/5 into train/val/test, seeded and stratified by instruction type.

Architecture

• Decoder-only transformer (LlamaConfig), 6 layers, hidden size 384, 6 attention heads, tied embeddings.
• Custom byte-level BPE tokenizer, vocab size 8000, trained only on this dataset.
• Max sequence length 384 (chosen from the p99 of training token lengths).

Limitation — small from-scratch dataset

~130k short instruction pairs is small for pretraining completely from random initialization. This model has not seen any English outside its training set, so its general grammar and fluency are narrower than a fine-tuned pretrained model, and it may overfit to phrasing patterns present in the training data. Treat outputs accordingly, especially for slang or phrasing far from the training distribution.

Intended use

Translating short Gen Z slang sentences/paragraphs into clear standard English. Not intended as a general-purpose chat or instruction-following model.

Example

Input: "bro that fit is so mid ngl" Output: (see the model's actual generation — varies by checkpoint/quantization)

Files

• pytorch_model.bin / tokenizer files — base HF-format model.
• genz-translator-f16.gguf — full-precision GGUF.
• genz-translator-q4_k_m.gguf — recommended default for local use.
• genz-translator-q8_0.gguf — higher-fidelity quantization.

Usage

llama.cpp:

llama-cli -m genz-translator-q4_k_m.gguf -p "<s><|instruction|>Translate the following Gen Z slang sentence into clear, standard English.<|input|>bro that fit is so mid ngl<|response|>" -n 96

Ollama (after ollama create, see the Modelfile in this repo / notebook):

ollama run genz-translator "bro that fit is so mid ngl"