Instructions to use ulmentflam/gpt2-774m-fineweb-nvfp4-mojo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ulmentflam/gpt2-774m-fineweb-nvfp4-mojo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ulmentflam/gpt2-774m-fineweb-nvfp4-mojo")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ulmentflam/gpt2-774m-fineweb-nvfp4-mojo")
model = AutoModelForCausalLM.from_pretrained("ulmentflam/gpt2-774m-fineweb-nvfp4-mojo", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ulmentflam/gpt2-774m-fineweb-nvfp4-mojo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ulmentflam/gpt2-774m-fineweb-nvfp4-mojo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ulmentflam/gpt2-774m-fineweb-nvfp4-mojo",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ulmentflam/gpt2-774m-fineweb-nvfp4-mojo

SGLang

How to use ulmentflam/gpt2-774m-fineweb-nvfp4-mojo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ulmentflam/gpt2-774m-fineweb-nvfp4-mojo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ulmentflam/gpt2-774m-fineweb-nvfp4-mojo",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ulmentflam/gpt2-774m-fineweb-nvfp4-mojo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ulmentflam/gpt2-774m-fineweb-nvfp4-mojo",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ulmentflam/gpt2-774m-fineweb-nvfp4-mojo with Docker Model Runner:
```
docker model run hf.co/ulmentflam/gpt2-774m-fineweb-nvfp4-mojo
```

GPT-2 774M (nvfp4), trained from scratch on FineWeb with llm.mojo

A GPT-2 774M (36 layers, 20 heads, 1280 channels (d36)) language model trained from scratch — random initialisation, no GPT-2 warm-start — for one epoch of the FineWeb classic 10B-token sample, using llm.mojo, a Mojo/MAX port of Andrej Karpathy's llm.c.

This is a training-run artifact and reproducibility reference, not a state-of-the-art model. At this scale and data budget GPT-2 produces locally-coherent but not highly capable text.

Why these weights were replaced

An earlier version of this repo held a checkpoint trained with a defect: a scratch-buffer overrun in the fused bias-gradient kernel meant several bias tensors received no gradient at any step. It was measurable directly from the published weights — GPT-2 initialises biases to exactly zero, so a bias still bit-exactly zero after 22,345 optimizer steps was never updated.

The kernel is fixed and this checkpoint is a complete re-run. The export now refuses to publish any checkpoint whose matmul biases are all zero. Full write-up: docs/ai/dbias_scratch_overrun_silent_zero_bug.md.

The previous weights remain reachable in this repo's commit history.

Results


Final validation loss	3.0128
HellaSwag (acc_norm)	3665/10042 = 36.50% (Wilson 95% CI [35.6%, 37.4%])
Final train loss	2.9945
Throughput	~109,535 tok/s (4188.3 ms/step, median over the last 500 steps)

Training

Precision: NVFP4 (e2m1, block-scaled) on the middle blocks' MLP forward GEMMs with stochastic rounding and a randomized Hadamard transform; backward stays bf16, parameters stay bf16, optimizer stays fp32.
Data: FineWeb classic sample-10BT, GPT-2 BPE, one epoch = 22,345 steps
Batch: 458,752 tokens/step (7 ranks x gradient accumulation)
LR: cosine, peak 2.5e-4, 700-step warmup, decayed to 0, weight decay 0.1
Parallelism: 7x RTX PRO 6000 Blackwell Max-Q, ZeRO-1 data parallel
Code: github.com/ulmentflam/llm.mojo

Files

model.safetensors — GPT2LMHeadModel-compatible export, loadable with transformers
model_22345.bin — the raw llm.mojo/llm.c-format checkpoint

infer_gpt2.mojo loads any of a local .bin, a local .safetensors, or this repo directly via --hf ulmentflam/gpt2-774m-fineweb-nvfp4-mojo.

Reproducing

git clone --recurse-submodules https://github.com/ulmentflam/llm.mojo.git
cd llm.mojo && make install-cuda
pixi run python data/fineweb.py -t classic -v 10B -m gpt-2
make build-bf16 WORLD_SIZE=7

See the launcher for this arm's exact flags.

Downloads last month: 25

Safetensors

Model size

0.8B params

Tensor type

BF16

Dataset used to train ulmentflam/gpt2-774m-fineweb-nvfp4-mojo

Collection including ulmentflam/gpt2-774m-fineweb-nvfp4-mojo

llm.mojo GPT-2 releases

Collection

Model checkpoints trained from scratch with llm.mojo (Mojo/MAX port of Karpathy's llm.c). https://github.com/ulmentflam/llm.mojo • 6 items • Updated 8 days ago