ccnets/causal-gpt-rl-unity

Causal GPT-RL — Pyramids

A Causal GPT-RL policy trained for the Unity ML-Agents release-23 Pyramids environment.

This repository contains one self-contained ONNX policy:

pyramids/pyramids-b16.onnx

The graph includes observation normalization and can run with ONNX Runtime or Unity Sentis without PyTorch at inference time.

Training data

This policy was trained using:

• unity/pyramids/simple-v0
• unity/pyramids/medium-v0

The unity/pyramids/expert-v0 dataset was not used for training and is included below only as an evaluation reference.

Evaluation

Policy	Context	Batch	Observation	Action	Return	Completed
pyramids/pyramids-b16.onnx	32	16	172	Discrete(5)	1.7738 ± 0.1278	128/128

The result is the mean ± population standard deviation over 128 episodes: 16 agents across eight fresh Unity launches using environment seeds 100..107.

All episodes completed successfully on the matching model-removed ML-Agents release-23 Pyramids build.

Dataset reference

Dataset	Episodes	Mean return	Training use
unity/pyramids/simple-v0	3,042	1.6256 ± 0.4154	Used
unity/pyramids/medium-v0	4,159	1.7432 ± 0.2601	Used
unity/pyramids/expert-v0	5,348	1.7953 ± 0.2661	Not used

The ONNX policy exceeds the mean returns of both training datasets. It is also approximately 1.2% below the expert dataset mean, despite never being trained on the expert dataset.

Dataset returns are sums of the recorded per-step rewards for each Minari episode.

ONNX interface

Input/output	Shape
states	[16, 32, 172]
actions	[16, 32, 5]
is_bos	[16, 32, 1]
mask	[16, 32]
action	[16, 5]

The four observation sensors must be concatenated in this order:

56 + 56 + 56 + 4

action contains five logits. Select the environment action with argmax, then feed its five-way one-hot representation into the next action window. Maintain one independent 32-step rolling context per agent.

The graph uses the checkpoint's bos_cache_mode=discard serving convention.

Companion resources

• Unity environment build
• Recorded datasets
• Evaluation code
• Causal GPT-RL models for Gymnasium and MuJoCo

License

Released under PolyForm Noncommercial License 1.0.0. See LICENSE for details. For commercial licensing, contact the maintainers via ccnets.org.