Instructions to use markab/rpn-gpt-lightning-Forth with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use markab/rpn-gpt-lightning-Forth with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="markab/rpn-gpt-lightning-Forth", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("markab/rpn-gpt-lightning-Forth", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use markab/rpn-gpt-lightning-Forth with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "markab/rpn-gpt-lightning-Forth"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "markab/rpn-gpt-lightning-Forth",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/markab/rpn-gpt-lightning-Forth

SGLang

How to use markab/rpn-gpt-lightning-Forth with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "markab/rpn-gpt-lightning-Forth" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "markab/rpn-gpt-lightning-Forth",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "markab/rpn-gpt-lightning-Forth" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "markab/rpn-gpt-lightning-Forth",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use markab/rpn-gpt-lightning-Forth with Docker Model Runner:
```
docker model run hf.co/markab/rpn-gpt-lightning-Forth
```

rpn-gpt-lightning-Forth

rpn-gpt-lightning with an exact Forth machine compiled into the network. The model writes a ```forth block; when it closes the block, a second, hand-built network (the kernel, 3.7B parameters whose weights were computed, not trained) writes the ```output block token by token in the same forward passes. No tool call, no client loop: a plain chat request gets the program, its exact output and the answer. Code and the full story: github.com/markab21/rpn-gpt, kernel/README.

This repository holds the kernel (7.4 GB), 453 embedding rows and 31 head rows for the words the model writes, and the modeling code. The language model itself is read from config.base_model = markab/rpn-gpt-lightning (63 GB, downloaded on first use) and stays byte-identical.

A real turn of the round-7 model (the adapter on the NVFP4 base) with the kernel in the weights (full example). The problem: "Calculate the remainder when 779951090558722902 is divided by 112974328."

After a short thinking part, the model writes:

779951090558722902 112974328 mod .

89002478

The remainder is 89002478.

\boxed{89002478}

The kernel inside the model wrote the output block. Before it, the kernel ran 348 hidden tokens. A hidden token is a reserved special token that decodes to an empty string. All 361 kernel-owned positions (348 hidden, 13 in the output block) were equal to the spec, kernel/text_vm.py. The answer is correct.

transformers (batch 1, reference)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("markab/rpn-gpt-lightning-Forth", trust_remote_code=True,
                                         dtype=torch.bfloat16, device_map="cuda").eval()
tok = AutoTokenizer.from_pretrained("markab/rpn-gpt-lightning-Forth")
card = open("reference_v4.txt").read()
msgs = [{"role": "system", "content": card}, {"role": "user", "content": "What is 123456789 * 987654321?"}]
x = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True).to("cuda")
out = m.generate(**x, max_new_tokens=6000, do_sample=True, temperature=1.0, top_p=0.95, pad_token_id=0)
print(tok.decode(out[0, x["input_ids"].shape[1]:], skip_special_tokens=True))

Tested (local copy, transformers 5.16.1, one RTX PRO 6000): 3 of 3 problems correct, and all 1431 kernel-written tokens equal to the spec, kernel/text_vm.py (one turn sent a block that failed, read the error and fixed it). About 35 tokens/s. skip_special_tokens=True hides the kernel's internal tokens (they decode to empty strings anyway).

vLLM (batched, OpenAI server)

The vLLM model class is a plugin in the GitHub repository, for vLLM 0.29.0 only:

git clone https://github.com/markab21/rpn-gpt && cd rpn-gpt
FORTH_VISIBLE=1 VLLM_DISABLE_COMPILE_CACHE=1 PYTHONPATH=$PWD/kernel/composite \
  vllm serve markab/rpn-gpt-lightning-Forth --served-model-name forth --max-model-len 65536 \
  --no-enable-flashinfer-autotune --no-enable-prefix-caching --no-async-scheduling -cc '{"cudagraph_mode": "FULL_DECODE_ONLY"}'

Then send an ordinary chat request ("model": "forth", the card as the system message, temperature 1.0, top-p 0.95, max_tokens 24000). Tested from this repository with an empty Hugging Face cache (vLLM 0.29.0, one RTX PRO 6000): the server downloaded this repository and its base, and 4 of 4 chat answers were correct with no client. Three of them used the machine: all 6 output blocks that the kernel wrote (one error message, then the fixed program) were equal to the spec. The fourth answered by hand. The flags are the ones the kernel was verified with; playbooks/deploy.md has the batched checker (kernel/composite/e2e_vllm.py).

Limits of the machine: integers only, 20 digits a number, at most 10 digits for the top operand of * and mod; the kernel is slow (one forward pass per kernel step: a / on 6-digit numbers takes about 115 hidden tokens). The Forth page lists every limit.

License

NVIDIA Nemotron 3.5 Lightning is under the OpenMDW License Agreement, version 1.1: full text and NVIDIA's notice in LICENSE. Our parts (the fine-tune, the card, the code) are MIT, Copyright (c) 2026 Mark Berry and rpn-gpt contributors (LICENSE-MIT). Please credit rpn-gpt with a link to github.com/markab21/rpn-gpt.