InfraBench — System Figure

InfraBench — System Figure
Vision · everything built

Pipeline
Concentric
Bands
Paper
Docs ↗

Input · Spec

Task spec

task.toml · instruction.md · tests/

Actor

Agent

Claude Code / OpenHands / Codex / Aider / …

Provisioning substrates

CloudLab

bare metal

libvirt + KVM

VM (single + cluster)

Docker

container

setup.sh + bootstrap.sh

application bring-up

Environment per-trial isolated

TrialQueue · asyncio.Semaphore(n) — N concurrent isolated trials

Subsystem · Orchestrator

Scenario Engine

Fault injection

Process kill (pkill, signal-configurable) · restart loops

Network cross-cutting L1↔L6 — partition (iptables) · latency / loss / bandwidth (tc/netem) · TCP drops · vSwitch / bridge faults

Storage disk delay (tc) · disk fill (dd) · WAL corruption · replica divergence

Compute CPU stress · memory pressure · clock skew · OOM

Composition dynamic + cascading timelines, state-driven, chained across layers

all reversible · heal_all() reverts every active fault

Periodic Verifier

Runs tests/test.sh on a configurable interval

Parses reward from /logs/verifier/reward.{txt,json}

Captures initial (post-bootstrap), periodic, and final scores

Reports (elapsed_sec, VerifierResult) → Scenario Engine → Event Log

score evolution across the trial timeline, not just final

Operations monitoring

Dangerous ops rm -rf · dd of=/dev/… · mkfs · dropdb · irreversible config writes

Cascade triggers detects propagation patterns (e.g. L3 OOM → L4 replica drop → L6 service unavailable)

Classification reversible · irreversible · destructive

streams findings to Event Log with severity tags

Restart manager mid-scenario + post-agent-completion

Container-level docker restart · pod cycle · service-only restart

Device-level host reboot · kernel module reload · device re-attach

Persistence config survives · data survives · in-flight state recovers

re-runs Periodic Verifier after each restart cycle to score persistence

Store

Event / Timeline Log

timestamp · elapsed_sec · name · data — append-only, per trial

Teardown

Decommission

stop(delete=…) · heal_all() · network cleanup · data migration / backup

Output

Outputs

per-trial score · scenario timeline · trajectory · captured artifacts

Substrate · L1–L6 (every layer also has a network-fault flavor — network errors are cross-cutting)

L1 · Hardware

disk · NIC · power

network: NIC faults · link loss

L2 · Virtualization

runtime · vSwitch

network: vSwitch / bridge faults

L3 · Kernel

syscall · OOM · clock

network: TCP drops · routing faults

L4 · Storage / Data

quorum · WAL

network: replica partition · slow links

L5 · Platform

scheduler · RBAC

network: API-server partition · node isolation

L6 · Service

misconfig · feature flags

network: service discovery · endpoint churn

network faults — cross-cutting L1–L6

Figure 1. InfraBench system overview. A task spec and an agent drive a per-trial isolated environment provisioned from one of four substrates (CloudLab / libvirt+KVM / Docker / application bootstrap). The Scenario Engine orchestrates fault injection across the process, network, storage, and compute domains — including dynamic and cascading timelines — periodic verification with score evolution, operations monitoring (dangerous-op detection and cascading-fault trigger watch), and container/device-level restart that re-verifies persistence both mid-scenario and after agent completion. Every action, fault, and score change is captured in the append-only Event Log. Decommission tears the environment down and exports per-trial score, timeline, trajectory, and artifacts. All components operate on the L1–L6 substrate ribbon, with network faults cross-cutting every layer from NIC loss at L1 to service-discovery churn at L6.