ShitpostGPT-20M (GGUF)

A 20.5M-parameter language model trained from scratch on one consumer GPU (RTX 3060 12GB) to write Twitch chat, imageboard posts and Reddit shitposts. It is a toy: small, unfiltered and chaotic on purpose. Inspired by Codeically's "ChatGPT-2 on a potato" video.

This repo holds the 22 MB q8_0 GGUF for LM Studio and llama.cpp.

It is a text-continuation model, not an assistant. It does not follow instructions or answer questions. It continues whatever text it sees, and the style of that text decides what you get.

How to use it

LM Studio: load ShitpostGPT-20M-q8_0.gguf. LM Studio has no pure completion box, so set Prompt Template to Manual with empty prefixes and suffixes, and put a Twitch-style seed in the System Prompt, for example:

PepeLaugh chat is this real
KEKW no way
LULW

Sampling: temperature 1.1, top-k 100, repeat penalty 1.1, max tokens about 80. Without a seed, plain messages like hi drift into essay-style text.

llama.cpp:

llama-cli -m ShitpostGPT-20M-q8_0.gguf -p "PepeLaugh chat is this real\nKEKW" --temp 1.1 --top-k 100 -n 80

Which prompt gives which style:

text before the model	what you get
PepeLaugh, KEKW, chat is this real	Twitch chat spam (the best mode)
>be me	greentext / imageboard style
plain English (hello)	rambling Reddit or essay-style text

Example (temperature 1.15, top-k 100, started with pepehello):

PepeLaugh NO GAME OMEGALUL
PepeLaugh I hate you
PepeLaugh nice night chat

Model details

Architecture	Llama (RoPE, RMSNorm, SwiGLU, tied embeddings)
Parameters	20.45M
Layers / hidden size / heads / FFN	8 / 384 / 6 / 1024
Context length	1024
Tokenizer	SentencePiece BPE, 16,384 vocab, byte fallback (embedded in the GGUF)
Training tokens	300M (about 0.8 epochs of a 240M-token corpus)
Final validation loss	3.521 (perplexity 33.8)
Hardware / time	1x RTX 3060 12GB, about 1h40m
Quantization	q8_0 (K-quants don't work here: the 384 hidden size isn't divisible by 256)

Training data

Roughly 1 GB of text from public sources:

• Clean English web text: HuggingFaceFW/fineweb-edu (sample-10BT), about 45%
• Twitch chat: lparkourer10/twitch_chat
• Imageboard posts: simonko912/chan-shitpost-2.5
• Reddit shitposting comments: innovation-hacking2/shitposting_dataset0 to 5
• Greentext: DarwinAnim8or/greentext
• A small personal scrape of Twitch chat and Reddit comments

Cleaning: HTML unescaped, URLs and >> reply references removed, whitespace normalised, length filters, exact-duplicate removal. No toxicity filtering was applied.

Limitations

• It is tiny. Expect fluent-looking fragments with no real meaning, facts or reasoning.
• Output drifts and repeats after a few lines.
• Without a Twitch-style seed or prompt, replies fall into rambling essay mode.
• English only.

Content warning and intended use

The training data includes raw imageboard and Reddit text, so the model will produce slurs, sexual content, harassment, conspiracy talk and other offensive text, sometimes without being prompted. It is meant for entertainment, experimentation and learning how small language models are trained. Do not deploy it to users, use it for moderation or decisions, or treat anything it says as true.

License

The weights come with no warranty. The training data comes from several third-party datasets with their own licenses (and in places unclear ones), so check those terms before redistributing or using the model commercially.