UDGAM LABS · Language model · 1.21M parameters

PhoneGPT

A 1.21M-parameter character-level language model, trained on the CPU of a 2021 Samsung Galaxy F62, live on YouTube and X, on 5 November 2026. It started from random weights and learned from about 1 MB of Shakespeare, in flight mode, with no cloud.

We publish it as a record of that run, not as a model to use. It writes Shakespeare-shaped nonsense. PhoneGPT is only our project name, with no link to the offline-AI iPhone app of the same name.

Read and watch: the write-up and tutorial · code on GitHub · the stream recording

Quick start

These checkpoints are in the trainer's own format, so they need the small C programs from GitHub, not transformers. You need Zig. This writes text on your computer:

git clone https://github.com/udgamlabs/phonegpt
cd phonegpt
curl -LO https://huggingface.co/udgamlabs/PhoneGPT/resolve/main/checkpoints/live_step018000.bin
zig cc -O3 -ffast-math -o phone_sample phone_sample.c -lpthread -lm
./phone_sample live_step018000.bin 500

To train your own on a phone, follow the tutorial.

Results

The target was 1.4697, the best validation loss in the README of nanoGPT, on one A100 GPU. Lower is better. Our rule, set before the start: at or below 1.4697 at two checkpoints in a row.

It missed by 0.0047. The best score was 1.4663 at one checkpoint, and the closest two in a row were 1.4744 and 1.4663.

Step	16,000	17,000	18,000	19,000	20,000	24,000
Hours in	7.2	7.6	8.1	8.5	9.0	10.75
Score	1.4782	1.4744	1.4663	1.4748	1.4785	1.4817

Each score is on every window of the held-out 10% of Tiny Shakespeare, computed on a Mac from the phone's checkpoint. All 24 scores are in results/scores.csv.

This isn't a like-for-like comparison:

	nanoGPT's README	PhoneGPT
Hardware	One A100 GPU	Galaxy F62, 2 CPU cores
Time	About 3 minutes	10 h 45 min
Parameters	10.65M	1.21M
Context	256 characters	128 characters
Score	Best of its sampled checks	Every window of the same held-out split

Output at step 24,000, real and unedited, cut for length:

DION:
Say, where I may be made what I add
And to my lord, like a mercy than to my father.

LORD WILLOUGHBY:
As I am, in king, and so follow my uncle?

Where it falls short

• It missed the target we set. One checkpoint under the line doesn't meet the rule.
• It has no practical use. It reads and writes single characters, knows only one small text, and its words and sentences often make no sense.
• 1.4697 comes from a small teaching run, not a measure of a good language model, and a GPU reaches it in minutes.
• 1.4663 is the best of 24 scores on the same text, so it flatters the model a little. From step 18,000 to 24,000 the score rose by 0.0154, which looks like overfitting; we didn't test that with another run.
• This is one run on one phone.
• The numbers in results/live_log.txt and in the headers of results/live_samples.txt are the trainer's quick check on a small sample, and they read lower than the scores above. Use results/scores.csv to compare.

How it was trained

• Model: a nanoGPT-style GPT with 6 layers, 4 attention heads, width 128 and context 128: 1,214,976 parameters. It isn't nanoGPT.
• Data: the first 90% of Tiny Shakespeare, about 1 MB in all. No other data, and no other model's weights.
• Trainer: phone_gpt.c, adapted from llm.c by Andrej Karpathy, who isn't involved in this. Seed 20261105, 24,000 steps. The exact command is in the GitHub README.
• Compute: training ran on 2 big CPU cores of the phone's Exynos 9825, about 1.5 seconds a step, plugged in over USB.

Files

Path	What it is
checkpoints/live_stepNNNNNN.bin	All 25 checkpoints, from random weights at step 0 to step 24,000; about 5 MB each. Step 18,000 scored best
hashes.csv	SHA-256 of each checkpoint, made when it came off the phone
results/scores.csv	Step and whole-split score
results/live_log.txt	The trainer's log from the run
results/live_samples.txt	The text it wrote every 250 steps

Intended use and limitations

A record of the run, and a small object for learning: look at how a character model improves between checkpoints, or check our scores. Not for any real writing task.

Licence

MIT. The trainer it came from is also MIT.

Citation

@misc{phonegpt2026,
  title  = {PhoneGPT: a small language model trained live on a 5-year-old phone},
  author = {Udgam Labs},
  year   = {2026},
  url    = {https://huggingface.co/udgamlabs/PhoneGPT}
}

---

About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com