StarAtNyte1/superbeam-env

SUPERBEAM: an OpenEnv platformer

Demo (watch Beam play, play it yourself, look inside the model): https://staratnyte1-beam-boot.static.hf.space/

SUPERBEAM is an original side-scrolling platformer written in CHIP-8 assembly (64x32 monochrome screen). This package serves it as an OpenEnv environment: reset / step / state over HTTP and WebSocket, usable from TRL GRPO. The game runs on a pure-Python CHIP-8 emulator (CPU only, no torch).

Actions

noop, left, right, jump, right+jump, left+jump. One action lasts frame_skip frames (default 4). Direction keys are held; jump is a tap (starts only while standing, rises 10 px). Pipes are solid blocks (stand on top, sides stop you); a tall pipe is climbed from a pipe next to it. Coins (low / mid / high; high ones only from a pipe top) add reward. Walkers turn at pits and pipes.

Observation

field	meaning
screen	labelled ASCII frame, rows 0-25 (P you, E walker, # ground, H pipe, o coin, F flag, C castle)
facts	one line: your columns and footing, ground gaps, pipes, coins and walkers with px-ahead distances, coins collected, progress
frame	raw 32x64 display bits
reward, done	per-step reward (100 x progress gained, +5 per coin, +50 win, -25 death); episode end
progress, won, dead, truncated	outcome info
player_x/y, world_x, frames, episode_reward	numeric state (metadata: deaths, best_progress, coins, coins_total)
level, difficulty	level id (classic, int seed or random) when the game supports procedural levels

reset() accepts seed, level, difficulty, frame_skip, max_frames. level/difficulty are passed to the game only if the bundled SuperBeamEnv supports them. Defaults come from the env vars SUPERBEAM_LEVEL, SUPERBEAM_DIFFICULTY, SUPERBEAM_FRAME_SKIP, SUPERBEAM_MAX_FRAMES.

Hosted server

A public instance runs on Modal (scales to zero; the first request may take ~30 s): https://baptinjean20--superbeam-env-web.us-east.modal.run (API docs at /docs). Use it as base_url below.

Quick start

from superbeam_env import SuperBeamAction, SuperBeamEnv

URL = "https://baptinjean20--superbeam-env-web.us-east.modal.run"   # or http://localhost:8000
with SuperBeamEnv(base_url=URL).sync() as env:
    r = env.reset()
    while not r.done:
        r = env.step(SuperBeamAction(action="right"))
    print(r.observation.facts)

Async: async with SuperBeamEnv(base_url=...) as env: await env.reset(); await env.step(...).

Run locally

python scripts/prepare_superbeam_env.py        # from the beam repo: bundles the game into ./vendor
cd envs/superbeam_env
uvicorn server.app:app --port 8000             # needs: pip install openenv uvicorn numpy websockets
python play_scripted.py --url http://localhost:8000

Docker: python scripts/prepare_superbeam_env.py && docker build -f envs/superbeam_env/server/Dockerfile -t superbeam-env envs/superbeam_env.

Training with TRL GRPO

See examples/grpo_superbeam.py in this repo (multi-turn act tool, reward = episode reward). --smoke runs 2 real training steps on a 6 GB GPU.

The in-weights computer (source)

vendor/ also holds the CHIP-8 computer built into a Hugging Face BeamForCausalLM, the second backend of this environment (verified evaluation; needs torch and transformers with Beam):

file	what it is
vendor/beamboot/chip8_lm.py	builds the 7-layer, hidden-640, fp32 model by hand (no training): latest-write attention RAM, MLP ALU, the model emits its own clock. Docstring lists every layer and head.
vendor/beamboot/chip8_fast.py	per-frame checkpoint prompts + speculative verification: the emulator drafts a frame, one forward pass checks every token, and the model's own greedy decoding wins on any disagreement
vendor/scripts/run_chip8_fast.py	replays a key schedule through the model, frame by frame, with per-frame stats
vendor/scripts/attention_probe.py	reads the RAM circuit out of the real forward pass: layer 5 head 0 put >99% of its weight on the newest write to the register being read, on all 126 register reads of game frame 6
vendor/tests/	cd vendor && pytest tests/ (9 tests, token-for-token against the emulator)

cd vendor
python scripts/attention_probe.py --frames 6 --draft-tokens 1400     # CPU, about a minute
SuperBeamEnv(backend="beam-weights")                                  # in Python: the gym on the model