maddiedreese/tinystories-palm-tungsten

paLLM TinyStories for Palm OS

paLLM is a TinyStories-class transformer language model packaged for local inference on a Palm Tungsten E2. The model is embedded into a Palm OS application as row-wise INT8 resources and generated entirely on-device after installation.

This is a research/retro-computing artifact, not a general assistant model. Generation on the tested Palm Tungsten E2 is roughly one token every seven seconds, and output quality is intentionally limited by the tiny model size, fixed-point inference, and Palm-safe approximations.

Contents

• model/q8_chunk_*.bin: row-wise INT8 model payload split into Palm resource-sized chunks.
• model/q8_index.h: chunk index used by the Palm runtime.
• model/q8_resources.rcp: Palm resource declarations for the model chunks.
• model/palm_vocab.h: 512-token TinyStories tokenizer table compiled into the app.
• palm/: Palm OS C runtime and resources.
• source/Makefile: build entry point from the GitHub repository.

The embedded model payload is 282,584 bytes across 18 chunks. The original project builds a .prc of about 292 KiB.

Model Lineage

This package is based on the tiny Llama/TinyStories ecosystem:

• Andrej Karpathy's llama2.c
• karpathy/tinyllamas, especially the TinyStories 260K family
• The TinyStories dataset/paper by Ronen Eldan and Yuanzhi Li

The Palm build is not a direct llama2.c port. It adds a custom quantizer, Palm resource packing, a fixed-point runtime, and decoder guardrails for a constrained Palm OS environment.

Runtime

Target hardware:

• Palm Tungsten E2
• Palm OS Garnet 5.4.x
• 200 MHz Intel XScale-class CPU
• No hardware floating point unit
• Small dynamic heap and limited non-volatile storage

Runtime characteristics:

• Q12 fixed-point activations
• row-wise Q8 weights
• Palm resource chunk loading
• simplified sigmoid
• recency-based attention approximation rather than full softmax
• greedy decoding with small readability guardrails

Use

The full build flow lives in the GitHub project:

git clone https://github.com/maddiedreese/paLLM.git
cd paLLM
make

The GitHub Actions workflow builds the Palm .prc using a Palm OS build package, m68k-none-elf-gcc, PilRC, and Palm OS 5 SDK headers.

Limitations

• Output quality is fragile and prompt-dependent.
• The attention path is an approximation, not faithful transformer softmax.
• The decoder includes hand-written guardrails.
• This is tuned for a tiny TinyStories-style model, not assistant behavior.
• Build reproducibility depends on the Palm build package used by the GitHub workflow.

Citation and Links

• GitHub: https://github.com/maddiedreese/paLLM
• Base artifacts: https://huggingface.co/karpathy/tinyllamas
• TinyStories paper: https://arxiv.org/abs/2305.07759
• llama2.c: https://github.com/karpathy/llama2.c