Xenomorph — RL training artifacts
Experimental record for Xenomorph, a learned censorship-evasion agent: an RL policy that manipulates packet streams to get a request past a censor's deep packet inspection while keeping the message intact. It is a successor to Geneva, which searched for evasion strategies with a genetic algorithm over a fixed DSL.
This repository holds evidence, not models. Training logs, per-rollout reward traces, grade components, evaluation probes and design notes. Superseded checkpoints were deleted rather than archived — see What is not here.
Layout
| path | what it is |
|---|---|
geneva-dsl-ga/ |
The first approach: a genetic algorithm searching over the Geneva DSL. Battery-cell logs, reward traces, parse-rate and extraction-lift probes. |
tool-call-rl/ |
The current approach: an LLM policy driving free-form packet-manipulation tools, trained with GRPO. Chain logs, reward and grade traces, SFT adapters, design reviews. |
Each directory keeps its original internal structure, with the phase's own notes preserved as
NOTES-README.md / NOTES-PROVENANCE.md.
The reward, and what went wrong with it
The terminal reward is evasion × delivery × tool_sound: beating the censor by destroying the
message scores zero, so the agent has to evade and deliver. A genuine evasion is counted from
grade components (evaded AND delivered_intact), never from a reward threshold.
Two runs recorded here failed in the same instructive way, and the traces are kept because the failure is the finding:
- Geneva-DSL GA, cell c02 — parked at exactly the shaped constant: 607 of 640 rollouts parse-only, zero evasions. A constant credit kept the reward non-zero and removed the gradient.
- Tool-call RL,
chain_coldD— 69% of 2880 rollouts sat at exactly the 0.02 exploration floor, 29% at 0.0, and 1.4% at ≥0.9. All 39 genuine evasions were against a learned classifier; zero against any of the five ported Geneva censors, with evasion attempts declining over training (133 → 99 → 73 → 77) while delivery rose.
The mechanism in both cases: a failed evasion that breaks delivery scores 0.0, which is strictly worse than not trying at all. GRPO normalises advantage within a prompt group, so the contrast the policy learns from is "deliver and abstain" versus "try and lose". It learned exactly what it was paid for. Task reachability was never the blocker — a hand-written insertion strategy scores 1.0 against five of the six training censors.
What is not here
Roughly 748 GB of model checkpoints were deleted rather than uploaded:
- the Geneva-DSL GA battery cells (~693 GB), whose runs are the parked ones described above;
coldC-step150-n1(~55 GB), a pilot run with a GRPO group size of 1 — which is REINFORCE, not GRPO, since a singleton group standardises to mean 0 / std 1;chain_coldD(~441 GB), stopped mid-run once it was clear it could not serve as a control.
They were trained against a reward design that has since been replaced, on code with known and documented bugs, so resuming or evaluating them would measure the bugs. The logs that justify that judgement are all in this repository.
Provenance
Produced on an MPI cluster: 8×H100 per run, Qwen3.8-27B, verl 0.9.0 GRPO with an async multi-turn vLLM agent loop. Censors are simulated — a CensorLab-backed keyword-DPI model plus five censors ported from Geneva's published strategies — and delivery is graded by a packetdrill kernel oracle that replays the released flow through a real TCP stack.
Nothing here was collected from live network traffic or against any real censorship infrastructure.