rei-piano: policies that play Kanye West's "Runaway"

Trained policies from rei-piano: a simulated Unitree G1 humanoid (and, as a baseline, two Shadow Hands on sliders) learns the opening of "Runaway" on a MuJoCo piano with Soft Actor-Critic.

Training dashboard (every run, with replay): https://ctxnn.github.io/rei-piano/

Models

Folder	Robot	Control	Steps	Note F1	Key presses / notes	Per-step F1
g1_humanoid/	Unitree G1 + two Shadow Hands	fingers + wrist positions (IK)	3.6M	0.905	51 / 44	0.985
hands/	Two Shadow Hands on sliders (RoboPianist)	fingers + sliders	1.0M	0.99	45 / 44	1.00

Note F1: each note must be struck on the right key within ±100 ms, and every extra key press counts against precision (1.0 = perfect). Scores are for one deterministic performance of the 60 s clip (44 notes). Replaying hands/ on macOS gave 0.978 (one extra press) because of small cross-platform simulation differences; g1_humanoid/ reproduced exactly.

g1_humanoid/ is the best evaluation of run r7_ik_p_s0 (IK arm control, extra-press penalty 0.5, discount 0.88). hands/ is run r1_g084_s0 (discount 0.84).

Files

g1_humanoid/actor.msgpack   policy weights (Flax params, msgpack)
g1_humanoid/config.json     environment + SAC settings, eval seed, reported metrics
hands/actor.msgpack
hands/config.json
play_pretrained.py          load a policy, play the song, save a recording

Each policy is an MLP with 3 × 256 GELU layers that outputs the mean and log-std of a tanh-squashed Gaussian. The scores use the deterministic action tanh(mean). Only the actor is included; the critics are not needed to play.

Usage

The policies need the rei-piano environment (RoboPianist + MuJoCo):

git clone https://github.com/ctxnn/rei-piano && cd rei-piano
uv venv -p 3.12 .venv && VIRTUAL_ENV=.venv uv pip install -e . soundfile
bash scripts/get_g1.sh                                # G1 model, for g1_humanoid/
hf download ctxnn1/rei-piano --local-dir pretrained

.venv/bin/python pretrained/play_pretrained.py pretrained/g1_humanoid
# note F1 0.905  (51 presses for 44 notes)  per-step F1 0.985

# Video with the robot's own piano audio (needs ffmpeg + fluidsynth)
.venv/bin/python -m runaway.render --recording pretrained/g1_humanoid/evals/performance.npz \
    --camera humanoid_front_left

Training

• Algorithm: SAC with DroQ-style critics (dropout 0.01 + LayerNorm), 1 update per env step, batch 256, learning rate 3e-4, automatic temperature.
• Reward: RoboPianist key-press reward + optimal-transport fingering reward (RP1M) + energy and forearm terms; the G1 run adds a penalty of 0.5 per extra key press.
• Observations: joint angles, key states and the next 0.5 s of the piano roll (1,118 numbers for the G1). Control at 20 Hz.
• Hardware: one H100 per run, 20 parallel simulators. The G1 run took ~0.9 h for 4M steps.

Full details, all seven training rounds and the maths: github.com/ctxnn/rei-piano.

Limitations

• These policies play only this 60 s clip of "Runaway"; they do not read arbitrary music.
• Simulation only, no sim-to-real work. The G1's legs and waist are locked, and the wrists are replaced by the Shadow Hands' own wrist joints.
• The G1 still makes ~7 extra key presses per minute (precision 0.84).

License and credits

Weights and code: MIT. "Runaway" © Kanye West / Roc-A-Fella / Def Jam; the song's notes live in the GitHub repo as a research transcription and are not covered by the MIT license. Built on RoboPianist, MuJoCo Menagerie (Shadow Hand, Unitree G1) and RP1M's fingering reward.