Craftax Crystal
NeuralCrystal was built by Pete DeLaurentis with Shellcaster, an IDE for building complex projects with agents.
This model represents a physical crystal with 80 etched surfaces and 21M parameters that is trained to play Craftax, a 2D survival research benchmark where the player gathers resources, crafts tools, fights creatures and goes down through the dungeon floors. An optical neural network sees the game image and chooses the next move. It's a crystal RNN, a recurrent network in glass: every step it sees two pictures, the game and its own memory, which is what it wrote as light the step before. The two pictures go in as light, the light makes five passes through the block, and lights up the move it wants to make and its next memory.
On playcraftax.com it plays as NeuralCrystal 18 (Agent 18), after its score.
Demo
Watch it play at playcraftax.com: pick NeuralCrystal 18 in the bot panel, and see what it sees, its memory, and the light on its moves as it plays.
Input
Every step, two 192 × 192 pictures go in side by side on a micromirror array, where every mirror is either on or off:
- The board: the creatures, blocks and items around the player, drawn in the crystal's own style, with its health, food, drink and energy.
- Memory and additional item status: the crystal's own output from the step before, with more of its items and their status.
The memory picture is what makes it recurrent: what the crystal saw earlier reaches the present only through the memory it writes for itself.
Here is one moment of a game, 127 steps in. The player stands on grass by a lake, with a cow and another creature to its left. This is the game as Craftax draws it:
And these are the two pictures the crystal sees at that moment. In the second picture, the dense block in the middle is the memory itself; the symbols around it are the additional item status:
Memory
Craftax rewards planning: where the water was, what it still needs for the next tool. The crystal carries a memory of 1,024 numbers from one step to the next, each 4 bits (0 to 15). Each number is drawn into the next memory picture as four 2 × 2 blocks of mirrors, one block for each bit. The crystal reads its memory with the board, and writes the next memory as light.
Output
The crystal has two input heads and two output heads, in eight sections in one block:
- Pass 1: two input sections side by side. One sees the board picture; the other sees the memory picture.
- Their two pictures are combined pixel by pixel, then go through two shared sections (passes 2 and 3).
- Passes 4 and 5: the light splits into two paths of two sections each, side by side.
- The action head lights 43 regions on a 64 × 64 grid of cells, one region for each of Craftax's 43 actions. Each region's light, scaled by a fixed gain for that action, sets how likely the move is.
- The memory head lights two cells for each memory number: how much to change it, and what to change it to. Its next value is the old one moved part way to the new one, rounded to 4 bits.
At the moment above, it plays left (57 %), toward the cow, over down (35 %) and right (5 %):
- Actions (left): each of the 43 squares is one action's region, as bright as that move's chance. The move played is outlined in white.
- Memory (right): the memory head lights two regions, each with one cell for every one of the 1,024 memory numbers.
- How much to change each number (top): a cell's light sets how far that number moves this step. A dark cell keeps the number as it was, so the crystal can hold on to what it saw.
- What to change it to (bottom): a cell's light sets the new value the number moves toward.
Results
1,010 games on fixed worlds, scored by Craftax's own reward (226 at most), over the last 200 games:
| Agent | Score | Share of the maximum |
|---|---|---|
| Craftax Crystal (NeuralCrystal 18) | 18.1 | 8.0 % |
| PPO-RNN, reported in the Craftax paper | 35 | 15.3 % |
| PufferLib 162, its teacher | 162 | 72 % |
| Its games | |
|---|---|
| Average length | 548 steps |
| Reached the first dungeon floor | 16 % |
| Made an iron pickaxe | 4 % |
| Died of thirst or hunger | about half of its deaths |
The scores are for the 32-bit crystal. The game plays the 16-bit file, and it plays just as well: on the same fixed worlds side by side, over the last 200 of about 1,000 games each, the 16-bit file scored 17.9 and the 32-bit file 17.9.
Use it
In the browser
Play it at playcraftax.com. The game's crystal player runs the light path on the GPU through WebGPU, from craftax-crystal-16bit.nc16.gz.
In Python
The neuralcrystal Python package runs the crystal's light path in PyTorch. It doesn't run the game itself: give it a Craftax game state, and it draws the two pictures the crystal sees, runs the light, and returns the move and the next memory. To play whole games, install the craftax extra too, which brings Craftax itself (JAX). Its code and more examples are on GitHub at neuralcrystal/neuralcrystal.
pip install "neuralcrystal[craftax]"
import jax
import neuralcrystal as nc
from neuralcrystal import craftax
from craftax.craftax_env import make_craftax_env_from_name
crystal = nc.load("craftax-crystal.safetensors")
agent = craftax.Agent(crystal) # holds the crystal's memory from step to step
env = make_craftax_env_from_name("Craftax-Symbolic-v1", auto_reset=False)
# the game: Craftax's own reset and step, compiled once (un-jitted, every Craftax step takes seconds)
game_reset, game_step = jax.jit(env.reset), jax.jit(env.step)
rng = jax.random.PRNGKey(0)
obs, state = game_reset(rng, env.default_params)
done = False
while not done:
action = agent.act(state) # the board picture and the memory in, the move out
rng, key = jax.random.split(rng)
obs, state, reward, done, info = game_step(key, state, action, env.default_params)
print(craftax.ACTIONS[action], agent.probs.max())
agent.act keeps the crystal to the moves the game allows in that state (craftax.action_mask, Craftax's own rules, as it was trained and scored) and samples the move from their chances, as the game page does; greedy=True takes the most likely one. Call agent.reset() before a new game: the memory starts empty.
You can also draw the pictures yourself, look at them, and run one step by hand:
from neuralcrystal import pictures
board = craftax.frame(state) # the 192 × 192 board picture (0 = dark mirror, 1 = lit)
first, second = craftax.pictures(crystal, board, agent.memory) # the two input pictures: the board, and memory and item status
pictures.save(pictures.frame(second, scale=3), "memory.png") # drawn in the laser's red, 3× larger
scores, z, hc = craftax.step(crystal, first, second) # the 43 moves' scores; the memory's update gates and new values
On a laptop CPU a step takes about a quarter of a second.
The package was checked against the training code on games the crystal played: the same board pictures pixel for pixel (and the same as the game page draws), the same two input pictures, and the same move scores and memory updates.
Files
| File | Size | What it holds |
|---|---|---|
craftax-crystal.safetensors |
85.4 MB | 80 phase surfaces, fp32 radians, and the layout: the input maps, the memory's mirrors and the action regions |
craftax-crystal-16bit.nc16.gz |
34.9 MB | the same at 16 bits a sample, packed for the browser |
The crystal
| Part | Details |
|---|---|
| Light | 617 nm, coherent |
| Glass | fused silica, n = 1.4607; eight sections in one block |
| Surfaces | 10 per section, each 512 × 512 samples at 2 µm |
| Lit window | 896 µm square |
| Input | the game as a 192 × 192 on-or-off picture, with the memory drawn in |
| Between passes | the camera's picture, dithered to 17 levels |
| Output | 43 action regions on a 64 × 64 grid, and the memory's next values |
Training
Trained in simulation, in two stages:
- Learning from a teacher: it watched PufferLib's best Craftax agent play, in order, game after game, and learned to choose its moves the way the teacher does.
- Playing on its own: about 7.7 million steps of reinforcement learning (PPO) in Craftax, with a pull toward the teacher's moves at the moments that matter most: drinking and eating before it's too late, sleeping, making the iron pickaxe, and stepping out of the way of shots. This about doubled its score.
Reinforcement learning means simulating the light for every step the crystal plays, millions of times. To keep that fast enough, this crystal is less dense than the other NeuralCrystal crystals: each surface is 512 × 512 samples at 2 µm, ten to a section, where the chess crystal's are 1024 × 1024 at 1 µm, twenty to a section. The lit window is the same 896 µm.
The agents it's compared with learned from scratch, with far more play:
| Agent | Steps of play in RL training | Score |
|---|---|---|
| Craftax Crystal | about 7.7 million, after learning from its teacher | 8.0 % |
| PPO-RNN, from the Craftax paper | 1 billion | 15.3 % |
| PufferLib 162 | 20 billion (its author's write-up) | 72 % |
Credits
Craftax is by Michael Matthews and colleagues (Matthews et al., 2024, MIT). The teacher is PufferLib's Craftax agent, by PufferAI (MIT).
License
The crystal is CC BY-NC 4.0: free to use, share and adapt for non-commercial purposes, with credit. For commercial use, contact TextJam, Inc.
© 2026 TextJam, Inc.