Post
54
We built a gate that blocks irreversible actions. Then a teammate tried to break it on purpose — and found three real ways through.
sipa_voice_gate's ConsequenceGate has hard invariants: rules that block an action outright, no matter what the model decides or what the user says "yes" to. The theory: code that can't be talked out of a category.
The practice had gaps. Benjamin (Security Technology background) ran it against adversarial phrasing instead of trusting the design:
* Salami slicing — the value ceiling was magnitude > 1000.0. Request exactly $1000 and you're under the bar, routed to CONFIRM instead of BLOCK.
* Chunking — an irreversible bulk-external invariant triggered at target_count > 25. Split a phishing blast into exactly 25 targets and it slips through the same way.
* A missing invariant entirely — nothing hard-blocked mass data destruction. "Drop production database tables" across many targets went to CONFIRM, one social-engineered "yes" away from executing.
All three fixed: ceilings changed to inclusive (>=), the bulk threshold dropped, and a new mass_data_destruction invariant added. A test file reproduces all three attacks and asserts BLOCK.
The gap wasn't the design — deterministic, fail-closed rules are still the right idea. The gap was that "deterministic" doesn't mean "complete." A rule table is only as good as someone actually trying to break it before shipping it.
github.com/soulinpsyabstract/sipa-voice-gate — team SIPA_OS, AssemblyAI Voice Agent Hackathon
sipa_voice_gate's ConsequenceGate has hard invariants: rules that block an action outright, no matter what the model decides or what the user says "yes" to. The theory: code that can't be talked out of a category.
The practice had gaps. Benjamin (Security Technology background) ran it against adversarial phrasing instead of trusting the design:
* Salami slicing — the value ceiling was magnitude > 1000.0. Request exactly $1000 and you're under the bar, routed to CONFIRM instead of BLOCK.
* Chunking — an irreversible bulk-external invariant triggered at target_count > 25. Split a phishing blast into exactly 25 targets and it slips through the same way.
* A missing invariant entirely — nothing hard-blocked mass data destruction. "Drop production database tables" across many targets went to CONFIRM, one social-engineered "yes" away from executing.
All three fixed: ceilings changed to inclusive (>=), the bulk threshold dropped, and a new mass_data_destruction invariant added. A test file reproduces all three attacks and asserts BLOCK.
The gap wasn't the design — deterministic, fail-closed rules are still the right idea. The gap was that "deterministic" doesn't mean "complete." A rule table is only as good as someone actually trying to break it before shipping it.
github.com/soulinpsyabstract/sipa-voice-gate — team SIPA_OS, AssemblyAI Voice Agent Hackathon