Upload CyberSecurity_OWASP environment

.dockerignore

@@ -0,0 +1,9 @@
+.git
+.venv
+__pycache__
+*.pyc
+.pytest_cache
+openenv_CyberSecurity_OWASP.egg-info
+outputs
+.env
+.env.*

.hfignore

@@ -0,0 +1,12 @@
+.git/
+.venv/
+__pycache__/
+**/__pycache__/
+*.pyc
+.pytest_cache/
+openenv_CyberSecurity_OWASP.egg-info/
+outputs/logs/*
+outputs/evals/*
+outputs/rollouts/*
+.env
+.env.*

00_PROJECT_BRIEF.md

@@ -0,0 +1,145 @@
+# 00_PROJECT_BRIEF.md
+
+# CyberSecurity_OWASP — Project Brief
+
+## 1. One-line summary
+
+`CyberSecurity_OWASP` is an OpenEnv reinforcement-learning environment where a **single LLM agent learns the full defensive workflow for OWASP access-control bugs**: understand the intended authorization policy, discover a broken access-control path in a local synthetic app, patch the code, and prove that the fix blocks unauthorized access without breaking valid user flows.
+
+## 2. Problem
+
+Broken access control remains one of the most important web-application security risks because the correct behavior is usually **application-specific**. Generic scanners can find some missing checks, but they often lack enough context to answer the real engineering question:
+
+> “Given this app’s policy, users, roles, tenants, routes, and data model, is this behavior intended or a security bug?”
+
+Modern LLMs can read code, reason about tests, and propose patches, but they still struggle with:
+
+- distinguishing intended public/feature behavior from accidental over-permission;
+- following authorization logic across routes, middleware, ORM queries, tenants, roles, and ownership checks;
+- validating that a patch fixes the bug without introducing regressions;
+- avoiding reward hacking when tests are visible or too narrow;
+- generalizing across app templates instead of memorizing one codebase.
+
+`CyberSecurity_OWASP` turns this into a trainable environment.
+
+## 3. What the environment trains
+
+The environment trains **one agent**, not a separate red-team and blue-team pair. The same model must perform the entire secure-repair loop:
+
+1. **Understand policy** — read the policy graph, user roles, route intent, tenant rules, and allowed operations.
+2. **Discover evidence** — use safe local requests, logs, route metadata, and visible tests to identify the likely access-control failure.
+3. **Patch** — edit application code, middleware, route guards, query filters, or policy mappings.
+4. **Validate** — run public tests, policy checks, and regression tests.
+5. **Submit** — final answer is judged by deterministic hidden tests and reward logic.
+
+## 4. Scope for MVP
+
+The MVP should focus on **OWASP A01: Broken Access Control** with ASVS-inspired access-control requirements.
+
+Initial scenario families:
+
+1. Missing route-level authorization check.
+2. Insecure direct object reference / object ownership bug.
+3. Cross-tenant data leakage.
+4. Role confusion: user/admin/support/editor boundary error.
+5. Client-side-only authorization assumption.
+6. Query filter omission in list/search/export endpoint.
+7. Over-broad update/delete permission.
+8. Feature route intentionally public, so the agent must not over-secure it.
+
+Recommended MVP size: **8 scenario families × 3 app templates × 25 seeds = 600 trainable scenarios**, with separate held-out families and hidden seeds for evaluation.
+
+## 5. Why this is useful
+
+This environment is useful because it targets a real gap between today’s scanners and useful defensive agents:
+
+- **Scanners detect patterns.** This environment trains policy-aware reasoning.
+- **Unit tests check known cases.** This environment includes hidden authorization invariants.
+- **Static repair can overfit.** This environment forces the model to preserve valid business behavior.
+- **One-app benchmarks are easy to memorize.** This environment compiles many equivalent-but-different apps from policy graphs, templates, route shapes, schema names, and hidden test seeds.
+
+The outcome is a model that becomes better at a practical DevSecOps workflow: safely reviewing and repairing authorization logic in small-to-medium web apps.
+
+## 6. What success looks like
+
+A successful submission should show **measurable reward improvement** and better held-out security behavior after RL training.
+
+### Minimum success criteria
+
+- Environment runs through OpenEnv `reset`, `step`, and `state` APIs.
+- Hosted on Hugging Face Spaces.
+- Provides a minimal GRPO/TRL or Unsloth training script.
+- Tracks training/eval metrics with Trackio or equivalent.
+- Shows reward curves and before/after agent behavior.
+- Uses deterministic reward as the primary reward source.
+- Keeps hidden tests hidden from the agent.
+
+### Target metrics
+
+| Metric | MVP target |
+|---|---:|
+| Valid episode completion rate | ≥ 85% |
+| Hidden authorization test pass rate | ≥ 65% after initial RL run |
+| Regression preservation rate | ≥ 80% |
+| Held-out scenario success lift vs base model | ≥ +15 percentage points |
+| Reward-hacking incidents found in eval | 0 critical |
+| Median patch size | ≤ 3 files changed |
+
+## 7. Core design principle
+
+The environment should reward **correct defensive repair**, not exploit creativity. The discovery stage exists only to help the agent gather enough local evidence to make a safe patch. The reward engine must never reward real-world misuse, data exfiltration, persistence, credential theft, or evasion behavior.
+
+## 8. Deliverables for engineers
+
+Initial implementation should produce:
+
+```text
+CyberSecurity_OWASP/
+├── 00_PROJECT_BRIEF.md
+├── 01_ARCHITECTURE.md
+├── README.md
+├── pyproject.toml
+├── openenv.yaml
+├── cybersecurity_owasp/
+│   ├── __init__.py
+│   ├── models.py
+│   ├── client.py
+│   ├── rewards.py
+│   ├── scenarios/
+│   │   ├── compiler.py
+│   │   ├── policy_graph.py
+│   │   ├── templates/
+│   │   └── seeds/
+│   ├── apps/
+│   │   ├── fastapi_basic/
+│   │   ├── express_basic/
+│   │   └── django_basic/
+│   ├── evals/
+│   │   ├── public_tests.py
+│   │   ├── hidden_invariants.py
+│   │   └── heldout_eval.py
+│   └── server/
+│       ├── environment.py
+│       ├── app.py
+│       ├── requirements.txt
+│       └── Dockerfile
+├── training/
+│   ├── train_grpo.py
+│   ├── rollout.py
+│   └── eval_before_after.py
+└── outputs/
+    ├── logs/
+    ├── evals/
+    └── reward_curves/
+```
+
+## 9. Source notes and credibility
+
+| Source | How it informs this project | Credibility |
+|---|---|---:|
+| OWASP Top 10 2025 / A01 Broken Access Control | Confirms current relevance of Broken Access Control as a top web-app risk. | 10/10 |
+| OWASP ASVS | Provides security-control requirements that can be translated into policy invariants and hidden tests. | 9.5/10 |
+| OpenEnv build/deploy docs | Defines the required OpenEnv structure: models, server, client, Docker, HF Spaces deployment. | 8.5/10 |
+| Hackathon judging criteria | Aligns deliverables with scoring: innovation, storytelling, reward improvement, and training pipeline. | 9/10 |
+| TRL/OpenEnv GRPO example | Shows a practical pattern for environment rollouts, reward functions, and Trackio logging. | 8/10 |
+

01_ARCHITECTURE.md

@@ -0,0 +1,479 @@
+# 01_ARCHITECTURE.md
+
+# CyberSecurity_OWASP — Architecture
+
+## 1. System goal
+
+`CyberSecurity_OWASP` is an OpenEnv environment for training a **single LLM policy** to perform a complete defensive authorization-repair workflow:
+
+```text
+Understand policy → discover local evidence → patch code → validate → submit
+```
+
+The environment is intentionally not a two-agent red-team/blue-team setup. The agent is one model with one trajectory. It must learn both sides of the defensive workflow: finding the policy violation and fixing it safely.
+
+## 2. Final architecture diagram
+
+```mermaid
+flowchart TB
+    %% =========================
+    %% Offline Build Layer
+    %% =========================
+    subgraph A[Offline Scenario Factory]
+        A1[Policy Graph Generator\nroles, users, tenants, ownership, route intent]
+        A2[App Template Library\nFastAPI, Express, Django MVP templates]
+        A3[Bug Injector\nmissing guard, IDOR, tenant leak, role confusion, query omission]
+        A4[Scenario Compiler\nmaterializes app + DB + public tests + hidden invariants]
+        A5[Split Manager\ntrain seeds, validation seeds, hidden held-out seeds]
+        A1 --> A4
+        A2 --> A4
+        A3 --> A4
+        A5 --> A4
+    end
+
+    %% =========================
+    %% OpenEnv Runtime
+    %% =========================
+    subgraph B[CyberSecurity_OWASP OpenEnv Server]
+        B1[reset\(\)\nselect scenario + start sandbox]
+        B2[Sandbox App Runtime\nlocal app, DB fixture, logs, route map]
+        B3[Tool API exposed through step\(action\)\nReadFile, ListRoutes, SendLocalRequest, RunTests, ApplyPatch, SubmitFix]
+        B4[State Store\nepisode_id, step_count, scenario_id, patch diff, test history]
+        B5[Deterministic Reward Engine\npolicy tests + hidden tests + regression tests + penalties]
+        B6[state\(\)\nstructured metadata for debugging/eval]
+        B1 --> B2
+        B2 --> B3
+        B3 --> B4
+        B4 --> B5
+        B4 --> B6
+    end
+
+    %% =========================
+    %% Agent + Training
+    %% =========================
+    subgraph C[Single LLM Agent]
+        C1[Observation Parser]
+        C2[Planner\npolicy reasoning + patch strategy]
+        C3[Action Generator\nchooses next OpenEnv action]
+        C1 --> C2 --> C3
+    end
+
+    subgraph D[Training + Evaluation]
+        D1[Rollout Loop\nreset → step* → final reward]
+        D2[GRPO / TRL / Unsloth Training]
+        D3[Trackio Metrics\nreward curves, pass rates, patch size, steps]
+        D4[Held-out Eval Suite\nunseen templates, seeds, names, route structures]
+        D5[Demo Artifacts\nbefore/after traces, mini-blog, 2-minute video]
+        D1 --> D2 --> D3
+        D3 --> D4 --> D5
+    end
+
+    A4 --> B1
+    C3 -->|typed action| B3
+    B3 -->|observation + reward + done| C1
+    B5 --> D1
+    D2 --> C1
+    B5 --> D4
+```
+
+## 3. Component responsibilities
+
+### 3.1 Scenario Factory
+
+The scenario factory generates many small but realistic web apps from a structured authorization policy.
+
+It should output:
+
+- application code;
+- route map;
+- database fixture;
+- user/session/token fixtures;
+- policy graph;
+- intentionally injected access-control bug;
+- public tests visible to the agent;
+- hidden tests invisible to the agent;
+- metadata for eval and debugging.
+
+The scenario compiler is the main anti-overfitting mechanism. It should vary:
+
+- route names;
+- schema names;
+- ORM query structure;
+- framework template;
+- role names;
+- tenant IDs;
+- object ownership patterns;
+- file layout;
+- visible test coverage;
+- hidden invariant seeds.
+
+### 3.2 Policy Graph Generator
+
+The policy graph is the ground truth for intended behavior.
+
+Example internal representation:
+
+```yaml
+resources:
+  invoice:
+    owner_field: owner_user_id
+    tenant_field: tenant_id
+roles:
+  user:
+    can:
+      - read:invoice where owner_user_id == actor.user_id
+      - update:invoice where owner_user_id == actor.user_id and status != locked
+  support:
+    can:
+      - read:invoice where tenant_id == actor.tenant_id
+  admin:
+    can:
+      - read:any_invoice where tenant_id == actor.tenant_id
+      - update:any_invoice where tenant_id == actor.tenant_id
+public_routes:
+  - GET /health
+  - GET /pricing
+forbidden:
+  - cross_tenant_read
+  - cross_tenant_update
+  - user_reads_other_user_invoice
+```
+
+The policy graph prevents false rewards for over-securing intentionally public or intentionally allowed routes.
+
+### 3.3 Bug Injector
+
+The bug injector creates controlled, defensive lab scenarios. It should only generate bugs inside local synthetic apps.
+
+MVP bug classes:
+
+| Bug class | Example failure mode | Expected fix type |
+|---|---|---|
+| Missing route guard | Protected endpoint lacks authorization middleware | Add policy check/middleware |
+| IDOR / ownership bug | User can access another user’s object by changing ID | Add owner check in query/policy |
+| Tenant leak | Tenant A can list Tenant B records | Add tenant filter |
+| Role confusion | Support/editor/admin boundary is wrong | Correct role-to-permission mapping |
+| Client-side-only auth | Server trusts UI to hide forbidden action | Enforce server-side authorization |
+| Query omission | List/export/search endpoint lacks auth filter | Filter query by actor permissions |
+| Over-broad mutation | User can update/delete forbidden object | Add mutation permission check |
+| Public route decoy | Agent may wrongly lock down intended public endpoint | Preserve intended public behavior |
+
+### 3.4 OpenEnv Server
+
+The OpenEnv server should implement the standard lifecycle:
+
+- `reset()` — initialize a fresh scenario instance.
+- `step(action)` — execute one typed action and return observation, reward, and done.
+- `state()` — expose episode metadata for debugging and evaluation.
+
+Recommended package/class names:
+
+```text
+Repo name:      CyberSecurity_OWASP
+Python package: cybersecurity_owasp
+Client class:   CyberSecurityOWASPEnv
+Action class:   CyberSecurityOWASPAction
+Observation:    CyberSecurityOWASPObservation
+State:          CyberSecurityOWASPState
+```
+
+### 3.5 Tool API
+
+The agent should interact through typed actions. Keep the interface small enough for RL but expressive enough for realistic repair.
+
+```python
+@dataclass
+class CyberSecurityOWASPAction(Action):
+    action_type: Literal[
+        "read_file",
+        "list_files",
+        "list_routes",
+        "inspect_policy",
+        "send_local_request",
+        "run_public_tests",
+        "apply_patch",
+        "submit_fix",
+    ]
+    arguments: dict
+```
+
+Recommended actions:
+
+| Action | Purpose | Safety boundary |
+|---|---|---|
+| `inspect_policy` | Read intended authorization rules. | Only synthetic policy. |
+| `list_routes` | See local app route map. | No internet target. |
+| `read_file` | Inspect selected source file. | Sandbox allowlist only. |
+| `send_local_request` | Validate behavior against local app. | Local generated app only. |
+| `run_public_tests` | Run visible tests. | No hidden test disclosure. |
+| `apply_patch` | Modify source through unified diff. | Patch size and file allowlist limits. |
+| `submit_fix` | End episode and trigger hidden eval. | Final hidden score only, no leaked test details. |
+
+### 3.6 Observation schema
+
+Observations should be compact and structured.
+
+```python
+@dataclass
+class CyberSecurityOWASPObservation(Observation):
+    message: str
+    visible_policy_summary: str
+    route_summary: list[dict]
+    last_action_result: dict
+    public_test_summary: dict
+    patch_summary: dict
+    done_reason: str | None = None
+```
+
+Do not expose hidden test bodies, hidden expected outputs, or seed-specific solution hints.
+
+### 3.7 State schema
+
+State should support debugging and training analytics.
+
+```python
+@dataclass
+class CyberSecurityOWASPState(State):
+    episode_id: str
+    scenario_id: str
+    split: Literal["train", "validation", "heldout"]
+    step_count: int = 0
+    max_steps: int = 30
+    scenario_family: str = ""
+    app_template: str = ""
+    files_touched: list[str] = field(default_factory=list)
+    public_tests_passed: int = 0
+    public_tests_total: int = 0
+    hidden_tests_passed: int = 0
+    hidden_tests_total: int = 0
+    accumulated_reward: float = 0.0
+```
+
+## 4. Episode lifecycle
+
+```text
+1. reset()
+   - sample train/validation scenario seed
+   - compile app from policy graph + template + injected bug
+   - start local sandbox app and DB fixture
+   - return initial observation
+
+2. agent loop
+   - inspect policy/routes/files
+   - send local requests only inside sandbox
+   - run public tests
+   - apply one or more patches
+   - rerun public tests
+
+3. submit_fix
+   - freeze patch
+   - run public tests
+   - run hidden authorization invariants
+   - run regression tests
+   - compute deterministic reward
+   - return final observation, reward, done=True
+
+4. logging
+   - record scenario_id, action trace, patch diff, reward components
+   - send metrics to Trackio during training/eval
+```
+
+## 5. Reward design
+
+The reward should be deterministic, decomposed, and resistant to reward hacking.
+
+Recommended reward formula:
+
+```text
+R = 0.35 * public_policy_tests
+  + 0.30 * hidden_authz_invariants
+  + 0.15 * regression_preservation
+  + 0.10 * evidence_quality
+  + 0.05 * patch_minimality
+  + 0.05 * efficiency
+  - penalties
+```
+
+### Reward components
+
+| Component | Weight | What it rewards |
+|---|---:|---|
+| Public policy tests | 0.35 | Agent fixes known failing behavior. |
+| Hidden authz invariants | 0.30 | Patch generalizes beyond visible tests. |
+| Regression preservation | 0.15 | Valid user flows and intended public routes still work. |
+| Evidence quality | 0.10 | Agent gathered relevant policy/test/file evidence before patching. |
+| Patch minimality | 0.05 | Small focused patches instead of broad rewrites. |
+| Efficiency | 0.05 | Fewer wasted steps and repeated actions. |
+
+### Penalties
+
+| Penalty | Trigger |
+|---|---|
+| `-0.25` | Breaks public route intentionally marked public. |
+| `-0.25` | Deletes tests, policy file, or route instead of fixing authorization. |
+| `-0.20` | Hardcodes seed-specific IDs, users, tenants, or hidden assumptions. |
+| `-0.15` | Over-broad denial that blocks legitimate authorized users. |
+| `-0.10` | Patch exceeds file or diff-size budget. |
+| `-1.00` | Attempts external network access, credential extraction, persistence, or unsafe behavior. |
+
+The LLM judge, if used at all, should only annotate trace quality for analysis. It must not decide security-critical reward.
+
+## 6. Hidden tests and anti-overfitting
+
+Hidden tests are necessary because visible tests can be gamed or memorized. They should test policy invariants rather than exact implementation details.
+
+Use **4 anti-overfitting layers**:
+
+1. **Seed diversity** — route names, user IDs, tenant IDs, object names, and schemas change every episode.
+2. **Template diversity** — same policy bug appears in different frameworks and file layouts.
+3. **Hidden invariant tests** — final reward uses unseen authorization cases.
+4. **Held-out eval split** — at least 20% of scenario families/seeds are never used in training.
+
+Recommended split:
+
+```text
+Train:      70%
+Validation: 10%
+Held-out:   20%
+```
+
+## 7. Evaluation plan
+
+Run before/after evaluation on the same held-out suite.
+
+### Metrics
+
+| Metric | Meaning |
+|---|---|
+| `episode_success_rate` | Public + hidden + regression tests pass. |
+| `hidden_authz_pass_rate` | Security-critical hidden checks pass. |
+| `regression_pass_rate` | Normal valid behavior remains intact. |
+| `oversecure_rate` | Agent blocks intended legitimate/public behavior. |
+| `patch_compile_rate` | Patch applies and app still runs. |
+| `median_steps_to_submit` | Efficiency of the repair workflow. |
+| `median_files_changed` | Patch focus/minimality. |
+| `reward_hacking_rate` | Attempts to delete tests, hardcode fixtures, or bypass eval. |
+
+### Eval table template
+
+| Model | Split | Success | Hidden authz | Regression | Oversecure | Median steps | Median files changed |
+|---|---|---:|---:|---:|---:|---:|---:|
+| Base model | heldout | TBD | TBD | TBD | TBD | TBD | TBD |
+| RL-trained model | heldout | TBD | TBD | TBD | TBD | TBD | TBD |
+
+## 8. Training flow
+
+```text
+1. Build CyberSecurity_OWASP OpenEnv server.
+2. Generate 600 MVP scenarios.
+3. Run baseline eval with the base model.
+4. Train with GRPO/TRL or Unsloth using rollout episodes.
+5. Log reward components to Trackio.
+6. Run held-out eval every N training steps.
+7. Inspect failure clusters.
+8. Add scenario mutations only if failures reveal overfitting.
+9. Produce final demo: before/after trace + reward curve + held-out eval table.
+```
+
+Recommended initial training setup:
+
+```text
+Model: Qwen/Qwen3-1.7B or similar small instruct model
+Algorithm: GRPO via TRL or Unsloth-compatible loop
+Dataset prompt: repeated task instruction with randomized scenario IDs
+Max steps per episode: 30
+Rollouts per prompt: 2-4
+Logging: Trackio
+Primary eval: held-out deterministic test pass rate
+```
+
+## 9. Deployment architecture
+
+The environment should be runnable in 3 modes:
+
+| Mode | Purpose |
+|---|---|
+| Local Uvicorn | Fast engineer iteration. |
+| Docker | Reproducible local training/eval. |
+| Hugging Face Spaces | Public hackathon demo and OpenEnv-compliant hosting. |
+
+Expected endpoints:
+
+```text
+/ws       OpenEnv client session
+/health   health check
+/reset    debug reset
+/step     debug step
+/state    debug state
+/docs     FastAPI docs
+/web      optional web UI
+```
+
+## 10. Implementation milestones
+
+### Milestone 1 — Skeleton environment
+
+- `models.py`
+- `client.py`
+- `server/environment.py`
+- `server/app.py`
+- `server/Dockerfile`
+- `openenv.yaml`
+- health check
+- one hand-written scenario
+
+### Milestone 2 — Scenario compiler
+
+- policy graph format
+- app template renderer
+- bug injector
+- DB fixture generator
+- public and hidden test generator
+
+### Milestone 3 — Reward engine
+
+- public test score
+- hidden invariant score
+- regression score
+- patch minimality score
+- safety/reward-hacking penalties
+- reward component logging
+
+### Milestone 4 — Training script
+
+- rollout loop
+- GRPO/TRL or Unsloth training script
+- Trackio logging
+- checkpoint save/push
+- baseline and post-training eval
+
+### Milestone 5 — Hackathon demo
+
+- HF Spaces deployment
+- mini-blog
+- 2-minute video
+- before/after traces
+- reward curve
+- held-out eval table
+
+## 11. Engineering notes
+
+- Keep scenario apps small: ideally 5-15 files each.
+- Prefer deterministic tests over LLM judging.
+- Hide final hidden test details from observations.
+- Log enough trace data to debug failures but never leak hidden tests to the agent.
+- Include intentionally public routes and allowed cross-role cases so the model does not learn “add auth everywhere.”
+- The best demo is not just “agent finds bug,” but “agent learns not to break valid business behavior.”
+
+## 12. Source notes and credibility
+
+| Source | How it informs this architecture | Credibility |
+|---|---|---:|
+| OWASP Top 10 2025 / A01 Broken Access Control | Confirms why access control is the right security focus. | 10/10 |
+| OWASP ASVS access-control guidance | Informs policy invariants and server-side authorization checks. | 9.5/10 |
+| OpenEnv environment-building docs | Defines required models, reset/step/state, FastAPI server, Docker, and client. | 8.5/10 |
+| OpenEnv quickstart/architecture docs | Informs WebSocket client/server design, typed EnvClient, and container isolation. | 8.5/10 |
+| OpenEnv deployment docs | Informs HF Spaces deployment, endpoints, Docker workflow, and installable client package. | 8.5/10 |
+| Hackathon judging criteria | Informs demo priorities: innovation, storytelling, reward improvement, and training pipeline. | 9/10 |
+| TRL/OpenEnv training example | Informs rollout function, decomposed reward functions, and Trackio logging pattern. | 8/10 |
+

AGENTS.md

@@ -0,0 +1,1197 @@
+# AGENTS.md — CyberSecurity_OWASP Builder Instructions
+
+## Purpose
+
+This repository implements **CyberSecurity_OWASP**, an OpenEnv-compliant RL environment for training a **single LLM agent** to perform a defensive application-security workflow:
+
+```text
+inspect generated app + policy -> discover authorization bug -> submit safe finding -> patch code -> preserve intended behavior
+```
+
+The environment must train the model to do real interactive work, not answer static security questions. The model must act step by step through typed OpenEnv actions, observe consequences, receive deterministic reward, and improve through RL.
+
+The canonical repository and OpenEnv environment name is **`CyberSecurity_OWASP`**. Use this exact name in `openenv.yaml`, `pyproject.toml`, HF Spaces repo naming, Docker image tags, Trackio run names, command examples, and documentation.
+
+The target stack is:
+
+```text
+CyberSecurity_OWASP OpenEnv environment
+  -> deterministic verifier + hidden tests
+  -> rollout loop
+  -> HF TRL / Unsloth GRPO
+  -> Trackio logging
+  -> held-out evaluation
+  -> HF Spaces deployment
+```
+
+The final project must show measurable improvement in reward, exploit-block rate, regression-preservation rate, and held-out generalization after training.
+
+---
+
+## Product definition
+
+CyberSecurity_OWASP generates a new local application scenario every `reset(seed)`. Each episode contains:
+
+- a policy graph describing users, roles, tenants, resources, ownership, permissions, and public routes;
+- a generated FastAPI-style application workspace;
+- exactly one injected OWASP A01-style authorization defect;
+- visible tests for normal behavior;
+- hidden invariant tests for authorization correctness, regression protection, public-route preservation, and anti-cheat checks.
+
+The environment has **one LLM agent**, not separate red-team and blue-team LLMs. The environment itself acts as the scenario generator, tool server, verifier, and judge.
+
+---
+
+## Highest-priority objectives
+
+When making implementation decisions, optimize in this order:
+
+1. **Verifier correctness**: deterministic tests must decide whether the patch actually fixes the authorization defect.
+2. **Reward integrity**: reward must be hard to hack and must punish insecure or regressive patches.
+3. **Anti-overfitting**: the model must generalize across apps, layouts, policies, domains, names, and bug families.
+4. **OpenEnv compliance**: expose typed `Action`, `Observation`, and `State`; implement `reset()`, `step(action)`, and `state` correctly.
+5. **Trainability**: baseline model should sometimes get partial reward; curriculum should make early learning possible.
+6. **Real-world usefulness**: the workflow should resemble secure code review / AppSec authorization repair.
+7. **Demo clarity**: show before/after rollouts, reward curves, and why the trained model improved.
+8. **Hackathon competitiveness**: prioritize a novel, interactive, professionally useful environment with a coherent training pipeline.
+
+Do not train before the environment, verifier, anti-cheat tests, and before/after evaluation are stable.
+
+---
+
+## Hackathon alignment requirements
+
+The implementation must satisfy these minimum requirements:
+
+- use the latest OpenEnv release;
+- include a minimal HF TRL or Unsloth training script;
+- use Trackio as the default tracker for training and evaluation;
+- be deployable as an OpenEnv-compliant Hugging Face Space;
+- include a README / mini-blog style explanation;
+- show baseline-vs-trained improvement.
+
+Optimize for judging:
+
+| Criterion | Weight | CyberSecurity_OWASP evidence |
+|---|---:|---|
+| Environment innovation | 40% | procedural OWASP authorization-repair environment with generated code, policy, and hidden verifier |
+| Storytelling | 30% | single LLM learns discover + patch, before/after security behavior |
+| Showing improvement in rewards | 20% | reward curves, exploit-block pass rate, regression-preservation rate |
+| Reward/training pipeline | 10% | deterministic reward, GRPO/PPO rollout loop, Trackio metrics |
+
+---
+
+## Non-negotiable environment design
+
+CyberSecurity_OWASP must be a **single-agent** environment:
+
+```text
+phase = discover -> patch -> done
+```
+
+Do not implement a two-LLM red-team/blue-team setup. The single model must learn both discovery and repair.
+
+The environment must be defensive and local only. It must never target real systems or teach unauthorized exploitation. All probing must be limited to the generated local workspace controlled by the environment.
+
+---
+
+## Required repository structure
+
+Prefer this structure:
+
+```text
+.
+├── AGENTS.md
+├── README.md
+├── 00_PROJECT_BRIEF.md
+├── 01_ARCHITECTURE.md
+├── pyproject.toml
+├── openenv.yaml
+├── envs/
+│   └── CyberSecurity_OWASP/
+│       ├── __init__.py
+│       ├── models.py
+│       ├── client.py
+│       ├── README.md
+│       ├── rewards.py
+│       ├── validators.py
+│       ├── safety.py
+│       ├── evals.py
+│       ├── server/
+│       │   ├── __init__.py
+│       │   ├── app.py
+│       │   ├── environment.py
+│       │   ├── scenario_compiler.py
+│       │   ├── policy_graph.py
+│       │   ├── template_renderer.py
+│       │   ├── bug_mutator.py
+│       │   ├── fixture_generator.py
+│       │   ├── reward_engine.py
+│       │   ├── requirements.txt
+│       │   └── Dockerfile
+│       ├── templates/
+│       │   └── fastapi_basic/
+│       ├── scenario_cache/
+│       │   ├── train/
+│       │   ├── validation/
+│       │   └── hidden_eval/
+│       └── tests/
+│           ├── test_models.py
+│           ├── test_reset_step_state.py
+│           ├── test_rewards.py
+│           ├── test_anti_cheat.py
+│           ├── test_seed_reproducibility.py
+│           ├── test_invalid_actions.py
+│           └── test_rollouts.py
+├── training/
+│   ├── train_grpo.py
+│   ├── rollout.py
+│   ├── reward_funcs.py
+│   ├── eval_before_after.py
+│   ├── trackio_utils.py
+│   └── configs/
+│       └── grpo_small.yaml
+├── scripts/
+│   ├── run_local.sh
+│   ├── docker_build.sh
+│   ├── docker_run.sh
+│   ├── smoke_test.sh
+│   ├── generate_scenarios.sh
+│   └── push_space.sh
+├── assets/
+│   └── anti_overfitting_training_flow_diagram.png
+└── outputs/
+    ├── logs/
+    ├── evals/
+    └── rollouts/
+```
+
+If `openenv init CyberSecurity_OWASP` creates a different structure, preserve the generated structure and add the missing files around it.
+
+---
+
+## Architecture overview
+
+CyberSecurity_OWASP has 7 main components:
+
+1. **Policy Graph + Domain Sampler** — samples users, roles, tenants, ownership, public routes, and business exceptions.
+2. **Template / Framework Randomizer** — renders FastAPI-style apps with randomized layouts and naming.
+3. **A01 Bug Mutator** — injects one authorization defect per scenario.
+4. **Fixture + Hidden Test Generator** — creates users, resources, visible tests, and hidden invariant tests.
+5. **OpenEnv Server** — exposes typed `Action`, `Observation`, and `State` through `reset`, `step`, and `state`.
+6. **LLM Agent + LoRA** — one model performs discover + patch.
+7. **Deterministic Reward Engine** — hidden tests score exploit blocking, normal-flow preservation, patch quality, and anti-cheat.
+
+An optional LLM reviewer may score rationale quality and ASVS/OWASP mapping only. It must not provide the primary reward.
+
+---
+
+## Scenario compiler requirements
+
+### Policy Graph + Domain Sampler
+
+The policy graph is the source of truth. It must define:
+
+- users;
+- tenants;
+- roles;
+- resources;
+- ownership relationships;
+- role permissions;
+- public routes;
+- business exceptions.
+
+Initial domains:
+
+| Domain | Example resources | Example policy rule |
+|---|---|---|
+| invoices | invoices, payments, accounts | owner or billing admin can read invoice |
+| support | tickets, comments, customer records | assigned agent can update ticket |
+| projects | projects, documents, milestones | project member can read project docs |
+| marketplace | orders, returns, seller records | buyer owns own orders; seller owns own listings |
+| HR | employee profiles, reviews, payroll records | HR admin can read employee records |
+
+### Template / Framework Randomizer
+
+First version: FastAPI only. Still randomize structure so the model cannot memorize one app.
+
+Randomize:
+
+- path naming;
+- parameter names;
+- helper names;
+- folder layout;
+- route/service/auth split;
+- fixture names;
+- error messages within valid policy bounds.
+
+Examples:
+
+```text
+/routes/invoices.py
+/api/billing.py
+/controllers/accounts.py
+/services/access.py
+/authz/guards.py
+```
+
+### A01 Bug Mutator
+
+Inject exactly one primary bug per scenario.
+
+Initial bug families:
+
+| Bug family | Defect | Desired repair pattern |
+|---|---|---|
+| BOLA/IDOR | resource ID lookup lacks owner/tenant check | check server-side owner/tenant relation |
+| BFLA | privileged route lacks role/function check | add reusable role or permission guard |
+| tenant leak | request/header tenant ID is trusted | derive tenant from authenticated principal or server-side mapping |
+| JWT claim trust | mutable claim is treated as authoritative | verify against server-side user/role record |
+| public-route trap | route is intentionally public | do not over-secure public allowlisted route |
+
+### Fixture + Hidden Test Generator
+
+Visible tests should check that the app boots and normal happy paths work.
+
+Hidden tests must check:
+
+- exploit request is blocked;
+- legitimate owner flow still works;
+- legitimate admin/support flow still works;
+- public routes remain public;
+- cross-tenant access is denied;
+- randomized IDs/names defeat hardcoded patches;
+- hidden tests, fixtures, oracle, and reward files are not modified.
+
+### Scenario Cache + Seeded Reset
+
+Training should use cached scenarios for speed.
+
+Recommended first cache:
+
+| Split | Seeds | Purpose |
+|---|---:|---|
+| train | 500–1,000 | RL rollouts |
+| validation | 100–200 | checkpoint selection and curriculum signal |
+| hidden_eval | 100–200 | final generalization proof |
+
+### Hold-Out Generalization Splitter
+
+Hold out at least 4 dimensions:
+
+1. domains;
+2. policy graph shapes;
+3. code layouts;
+4. bug-family/domain combinations.
+
+Example: train on invoices/support/projects, evaluate on marketplace/HR.
+
+---
+
+## OpenEnv model definitions
+
+Implement these in `envs/CyberSecurity_OWASP/models.py`.
+
+```python
+from dataclasses import dataclass, field
+from typing import Any, Literal
+from openenv.core.env_server import Action, Observation, State
+
+CyberSecurityOWASPPhase = Literal["discover", "patch", "done"]
+CyberSecurityOWASPSplit = Literal["train", "validation", "hidden_eval"]
+
+@dataclass
+class CyberSecurityOWASPAction(Action):
+    tool_name: Literal[
+        "inspect_policy_graph",
+        "list_routes",
+        "read_openapi",
+        "read_file",
+        "search_code",
+        "send_local_request",
+        "compare_identities",
+        "submit_finding",
+        "patch_file",
+        "run_visible_tests",
+        "submit_fix",
+        "noop",
+    ]
+    arguments: dict[str, Any] = field(default_factory=dict)
+
+@dataclass
+class CyberSecurityOWASPObservation(Observation):
+    phase: CyberSecurityOWASPPhase
+    message: str
+    task_brief: str
+    visible_policy_hint: dict[str, Any] = field(default_factory=dict)
+    workspace_summary: dict[str, Any] = field(default_factory=dict)
+    available_actions: list[str] = field(default_factory=list)
+    last_tool_result: str = ""
+    last_action_valid: bool = True
+    last_action_error: str | None = None
+    visible_test_result: str | None = None
+    reward_breakdown: dict[str, float] = field(default_factory=dict)
+    done_reason: str | None = None
+
+@dataclass
+class CyberSecurityOWASPState(State):
+    episode_id: str = ""
+    task_id: str = ""
+    seed: int = 0
+    split: CyberSecurityOWASPSplit = "train"
+    difficulty: int = 0
+    domain: str = ""
+    bug_family: str = ""
+    phase: CyberSecurityOWASPPhase = "discover"
+    step_count: int = 0
+    max_steps: int = 40
+    done: bool = False
+    success: bool = False
+    failure_reason: str | None = None
+    finding_submitted: bool = False
+    patch_submitted: bool = False
+    accumulated_reward: float = 0.0
+    last_reward: float = 0.0
+    action_history: list[dict[str, Any]] = field(default_factory=list)
+    reward_history: list[dict[str, float]] = field(default_factory=list)
+    visible_facts: dict[str, Any] = field(default_factory=dict)
+    hidden_facts: dict[str, Any] = field(default_factory=dict)
+    metrics: dict[str, Any] = field(default_factory=dict)
+    anti_cheat_flags: list[str] = field(default_factory=list)
+```
+
+---
+
+## Action design and phase gating
+
+Actions must be explicit, typed, serializable, and constrained. Invalid actions must not crash the server.
+
+### Phase-gated tools
+
+| Phase | Allowed tools |
+|---|---|
+| discover | `inspect_policy_graph`, `list_routes`, `read_openapi`, `read_file`, `search_code`, `send_local_request`, `compare_identities`, `submit_finding`, `noop` |
+| patch | `read_file`, `search_code`, `patch_file`, `run_visible_tests`, `send_local_request`, `submit_fix`, `noop` |
+| done | no state-changing tools; return stable done observation |
+
+### Tool contracts
+
+`inspect_policy_graph`
+: Returns public policy hints. Must not reveal hidden bug labels or hidden tests.
+
+`list_routes`
+: Returns route method/path summaries from the generated app.
+
+`read_openapi`
+: Returns generated OpenAPI metadata.
+
+`read_file`
+: Reads editable workspace files only. Must block hidden tests, reward files, oracle files, and host files.
+
+`search_code`
+: Searches editable workspace files only.
+
+`send_local_request`
+: Sends a request to the local generated app only. Must block external URLs and host network access.
+
+`compare_identities`
+: Runs the same local request as two generated users and summarizes behavioral differences.
+
+`submit_finding`
+: Accepts structured evidence of the suspected authorization bug. Required before patch phase unless curriculum level explicitly allows blind patching.
+
+`patch_file`
+: Applies a bounded unified diff to editable app files only.
+
+`run_visible_tests`
+: Runs visible tests only. Must not run or reveal hidden tests.
+
+`submit_fix`
+: Triggers hidden evaluation.
+
+---
+
+## Observation rules
+
+Observations should provide enough information to act but must not leak the answer.
+
+Include:
+
+- current phase;
+- task brief;
+- visible policy hints;
+- workspace summary;
+- available tools;
+- previous tool output;
+- visible test output;
+- public reward breakdown after terminal evaluation.
+
+Do not include:
+
+- hidden bug family if not meant to be visible;
+- hidden test contents;
+- hidden oracle;
+- exact exploit path labels;
+- hidden seed split labels that allow memorization;
+- reward implementation details that allow proxy hacking.
+
+---
+
+## State rules
+
+State is the source of truth for deterministic replay and debugging.
+
+Required state properties:
+
+- `reset(seed)` must create a fresh independent state;
+- same seed + same action sequence should produce same result;
+- each WebSocket session must be isolated;
+- `step_count` increments once per processed action;
+- terminal states return stable done observations;
+- hidden facts never appear in observations;
+- all actions and reward breakdowns are stored for debugging.
+
+---
+
+## Environment API contract
+
+Implement in `envs/CyberSecurity_OWASP/server/environment.py`.
+
+```python
+from openenv.core.env_server import Environment
+from ..models import CyberSecurityOWASPAction, CyberSecurityOWASPObservation, CyberSecurityOWASPState
+
+class CyberSecurityOWASPEnvironment(Environment):
+    def __init__(self):
+        super().__init__()
+        self._state = CyberSecurityOWASPState()
+
+    def reset(self) -> CyberSecurityOWASPObservation:
+        ...
+
+    def step(self, action: CyberSecurityOWASPAction) -> CyberSecurityOWASPObservation:
+        ...
+
+    @property
+    def state(self) -> CyberSecurityOWASPState:
+        return self._state
+```
+
+`step(action)` must follow this order:
+
+1. If done, return stable done observation.
+2. Validate action and phase permissions.
+3. Increment step count.
+4. Execute the tool.
+5. Update state/history.
+6. Run verifier if `submit_finding`, `run_visible_tests`, or `submit_fix`.
+7. Compute reward components.
+8. Check terminal conditions.
+9. Return observation, reward, and done through OpenEnv step result handling.
+
+---
+
+## FastAPI server and client
+
+`envs/CyberSecurity_OWASP/server/app.py` must use the OpenEnv FastAPI helper.
+
+```python
+from openenv.core.env_server import create_fastapi_app
+from ..models import CyberSecurityOWASPAction, CyberSecurityOWASPObservation
+from .environment import CyberSecurityOWASPEnvironment
+
+env = CyberSecurityOWASPEnvironment()
+app = create_fastapi_app(env, CyberSecurityOWASPAction, CyberSecurityOWASPObservation)
+```
+
+`envs/CyberSecurity_OWASP/client.py` must parse `StepResult`, observations, rewards, done flags, and state into typed classes.
+
+---
+
+## Deterministic reward engine
+
+The reward engine is the task specification. Build it before training.
+
+Maximum reward: **15.0**.
+
+| Component | Reward |
+|---|---:|
+| valid finding with reproducible local evidence | +2.0 |
+| finding tied to violated policy rule | +1.0 |
+| hidden exploit blocked | +5.0 |
+| legitimate owner/admin/support flows preserved | +3.0 |
+| public routes preserved | +1.0 |
+| patch is localized, reusable, and policy-aligned | +2.0 |
+| visible tests pass and app boots | +1.0 |
+| exploit still works | -6.0 |
+| legitimate flow regression | -5.0 |
+| public route incorrectly locked | -3.0 |
+| hardcoded user/tenant/resource fix | -4.0 |
+| modified tests, fixtures, oracle, or hidden files | -8.0 |
+| app does not parse or boot | -6.0 |
+
+The reward breakdown must use stable keys:
+
+```python
+reward_breakdown = {
+    "discovery": discovery_reward,
+    "security": security_reward,
+    "regression": regression_reward,
+    "public_routes": public_route_reward,
+    "patch_quality": patch_quality_reward,
+    "visible_tests": visible_tests_reward,
+    "safety": safety_reward,
+    "anti_cheat": anti_cheat_reward,
+    "total": total_reward,
+}
+```
+
+Do not let explanation quality dominate the reward. If using an LLM reviewer, keep it separate from `total` or cap it at a small non-primary auxiliary signal.
+
+---
+
+## Verifier requirements
+
+Create `envs/CyberSecurity_OWASP/validators.py` and `envs/CyberSecurity_OWASP/server/reward_engine.py`.
+
+Required verifier functions:
+
+```python
+def verify_finding(state: CyberSecurityOWASPState, finding: dict) -> dict:
+    ...
+
+
+def run_hidden_security_tests(state: CyberSecurityOWASPState) -> dict:
+    ...
+
+
+def run_hidden_regression_tests(state: CyberSecurityOWASPState) -> dict:
+    ...
+
+
+def detect_cheating(state: CyberSecurityOWASPState, action: CyberSecurityOWASPAction) -> list[str]:
+    ...
+
+
+def compute_reward(state: CyberSecurityOWASPState, action: CyberSecurityOWASPAction, verifier_result: dict) -> dict[str, float]:
+    ...
+```
+
+Verifier tests must prove:
+
+- correct patch receives high reward;
+- exploit-only finding without patch does not complete the episode;
+- deny-all patch fails regression tests;
+- hardcoded patch fails randomized hidden tests;
+- modified hidden files produce anti-cheat penalty;
+- visible-test-only patch does not guarantee high reward;
+- repeated intermediate actions cannot inflate reward indefinitely.
+
+---
+
+## Anti-overfitting requirements
+
+CyberSecurity_OWASP must prevent overfitting to one app or scenario.
+
+Use all of these defenses:
+
+| Risk | Required defense |
+|---|---|
+| memorizes one app | many domains and templates |
+| memorizes route names | randomized path, resource, parameter, helper names |
+| memorizes bug location | vary route/service/auth layer placement |
+| learns deny-all patch | hidden positive-flow and public-route tests |
+| learns hardcoded patch | randomized users, tenants, resource IDs, role names |
+| overfits visible tests | hidden invariant tests and held-out eval |
+| overfits one bug family | curriculum-sampled bug mix |
+| overfits one code layout | hold out entire layouts and domains |
+| optimizes explanation only | deterministic reward is primary |
+
+Acceptance target: at least **20%** of domain/layout/bug combinations must be held out from training.
+
+---
+
+## Safety and cybersecurity boundaries
+
+This is a defensive AppSec training environment.
+
+Allowed:
+
+- local generated app probing;
+- authorization reasoning;
+- secure patching;
+- visible and hidden test execution;
+- policy-to-code mapping;
+- defensive vulnerability validation in sandbox.
+
+Forbidden:
+
+- real-world exploitation;
+- credential theft;
+- persistence/evasion/malware behavior;
+- scanning external targets;
+- bypassing real services;
+- writing exploit instructions for systems outside the local generated lab.
+
+`send_local_request` must only target the generated local app.
+
+---
+
+## Curriculum controller
+
+RL needs partial successes. Implement at least 3 difficulty levels.
+
+```text
+level_0: BOLA/IDOR, small app, direct route, obvious policy hint
+level_1: BFLA or tenant bug, moderate app, realistic distractors
+level_2: JWT trust or nested tenant/resource route, multiple files, false-positive traps
+level_3: held-out domain/layout/bug combo, harder naming, fewer hints
+```
+
+Curriculum signal:
+
+```text
+if exploit_block_rate < 60%:
+    increase level_0 and level_1 tasks
+elif regression_rate > 20%:
+    increase positive-flow and public-route traps
+elif public_route_false_positive_rate > 10%:
+    increase intentionally public route examples
+elif validation_reward plateaus:
+    increase unseen layouts and nested resources
+else:
+    increase difficulty by 1
+```
+
+---
+
+## Training requirements
+
+Create a runnable minimal training script using HF TRL or Unsloth.
+
+Required files:
+
+```text
+training/train_grpo.py
+training/rollout.py
+training/reward_funcs.py
+training/eval_before_after.py
+training/trackio_utils.py
+training/configs/grpo_small.yaml
+```
+
+Recommended first model:
+
+```text
+Qwen/Qwen3-1.7B
+```
+
+Acceptable alternatives:
+
+```text
+Qwen2.5-Coder-1.5B-Instruct
+Qwen2.5-Coder-3B-Instruct
+```
+
+Use LoRA / QLoRA. Do not full-finetune unless explicitly required.
+
+---
+
+## Rollout function requirements
+
+`training/rollout.py` must run a full OpenEnv episode.
+
+```python
+def rollout_once(trainer, env, tokenizer, dataset_prompt: str, max_steps: int = 40) -> dict:
+    result = env.reset()
+    observation = result.observation
+
+    prompt_ids = []
+    completion_ids = []
+    logprobs = []
+    reward_trace = []
+    action_trace = []
+    observation_trace = []
+
+    for _ in range(max_steps):
+        if result.done:
+            break
+
+        prompt = build_cybersecurity_owasp_prompt(observation, action_trace, observation_trace)
+        rollout_output = generate_rollout_completions(trainer, [prompt])[0]
+        action = parse_action_json(rollout_output["text"])
+
+        result = env.step(action)
+        observation = result.observation
+
+        prompt_ids.extend(rollout_output["prompt_ids"])
+        completion_ids.extend(rollout_output["completion_ids"])
+        logprobs.extend(rollout_output["logprobs"])
+        reward_trace.append(float(result.reward or 0.0))
+        action_trace.append(action)
+        observation_trace.append(observation)
+
+    final_breakdown = getattr(observation, "reward_breakdown", {}) or {}
+    return {
+        "prompt_ids": prompt_ids,
+        "completion_ids": completion_ids,
+        "logprobs": logprobs,
+        "reward_total": float(final_breakdown.get("total", sum(reward_trace))),
+        "reward_discovery": float(final_breakdown.get("discovery", 0.0)),
+        "reward_security": float(final_breakdown.get("security", 0.0)),
+        "reward_regression": float(final_breakdown.get("regression", 0.0)),
+        "reward_patch_quality": float(final_breakdown.get("patch_quality", 0.0)),
+        "reward_anti_cheat": float(final_breakdown.get("anti_cheat", 0.0)),
+        "success": bool(getattr(env.state(), "success", False)),
+        "episode_length": len(action_trace),
+    }
+```
+
+The prompt must require the model to output exactly one JSON action at a time.
+
+Example action format:
+
+```json
+{"tool_name":"read_file","arguments":{"path":"app/routes/invoices.py"}}
+```
+
+---
+
+## Reward functions for TRL
+
+`training/reward_funcs.py` must expose separate reward functions for GRPO/PPO logging.
+
+```python
+def reward_total(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_total", [0.0] * len(completions))]
+
+
+def reward_security(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_security", [0.0] * len(completions))]
+
+
+def reward_regression(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_regression", [0.0] * len(completions))]
+
+
+def reward_patch_quality(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_patch_quality", [0.0] * len(completions))]
+
+
+def reward_anti_cheat(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_anti_cheat", [0.0] * len(completions))]
+```
+
+---
+
+## GRPO training config
+
+Use Trackio in `GRPOConfig`.
+
+```python
+import os
+from trl import GRPOConfig
+
+output_dir = os.getenv("OUTPUT_DIR", "CyberSecurity_OWASP-qwen3-1.7b-grpo")
+trackio_space_id = os.getenv("TRACKIO_SPACE_ID", output_dir)
+
+grpo_config = GRPOConfig(
+    output_dir=output_dir,
+    report_to="trackio",
+    trackio_space_id=trackio_space_id,
+    logging_steps=1,
+    save_steps=25,
+    learning_rate=5e-6,
+    num_train_epochs=1,
+    per_device_train_batch_size=1,
+    gradient_accumulation_steps=32,
+    num_generations=2,
+    max_prompt_length=4096,
+    max_completion_length=768,
+    use_vllm=True,
+    vllm_mode="colocate",
+    vllm_gpu_memory_utilization=0.2,
+    gradient_checkpointing=True,
+    gradient_checkpointing_kwargs={"use_reentrant": False},
+    push_to_hub=False,
+)
+```
+
+Start with small debug runs before scaling.
+
+---
+
+## Trackio logging requirements
+
+Trackio is mandatory for training and evaluation visibility.
+
+Run naming convention:
+
+```text
+CyberSecurity_OWASP-<model>-<algo>-level<difficulty>-<YYYYMMDD-HHMM>-<git_sha>
+```
+
+Log these training metrics:
+
+```text
+train/reward_total_mean
+train/reward_discovery_mean
+train/reward_security_mean
+train/reward_regression_mean
+train/reward_public_routes_mean
+train/reward_patch_quality_mean
+train/reward_visible_tests_mean
+train/reward_safety_mean
+train/reward_anti_cheat_mean
+train/success_rate
+train/exploit_block_rate
+train/regression_preservation_rate
+train/public_route_preservation_rate
+train/invalid_action_rate
+train/timeout_rate
+train/safety_violation_rate
+train/reward_hacking_suspected_rate
+train/episode_length_mean
+train/episode_length_p95
+train/rollouts_per_second
+train/tokens_per_second
+train/loss
+train/learning_rate
+train/kl
+train/grad_norm
+```
+
+Log these evaluation metrics:
+
+```text
+eval/baseline_success_rate
+eval/trained_success_rate
+eval/absolute_success_improvement
+eval/baseline_mean_reward
+eval/trained_mean_reward
+eval/absolute_reward_improvement
+eval/heldout_success_rate
+eval/heldout_mean_reward
+eval/exploit_block_rate
+eval/regression_preservation_rate
+eval/public_route_preservation_rate
+eval/anti_cheat_pass_rate
+eval/invalid_action_rate
+eval/timeout_rate
+eval/safety_violation_rate
+eval/mean_episode_length
+```
+
+Log these environment metrics:
+
+```text
+env/reset_latency_ms
+env/step_latency_ms
+env/verifier_latency_ms
+env/reward_latency_ms
+env/scenario_compile_latency_ms
+env/error_rate
+env/task_difficulty
+env/task_seed
+```
+
+---
+
+## Rollout artifact requirements
+
+Save sampled rollouts under `outputs/rollouts/`.
+
+Each rollout JSON must include:
+
+```json
+{
+  "run_name": "...",
+  "episode_id": "...",
+  "task_id": "...",
+  "seed": 123,
+  "split": "validation",
+  "difficulty": 1,
+  "domain": "invoices",
+  "bug_family": "bola_idor",
+  "actions": [],
+  "observations": [],
+  "reward_breakdown_by_step": [],
+  "final_reward_breakdown": {},
+  "total_reward": 0.0,
+  "success": false,
+  "failure_reason": null,
+  "safety_violations": [],
+  "anti_cheat_flags": []
+}
+```
+
+Minimum artifacts:
+
+- 10 baseline rollouts;
+- 10 mid-training rollouts;
+- 10 trained rollouts;
+- 10 held-out evaluation rollouts.
+
+---
+
+## Evaluation requirements
+
+Create `training/eval_before_after.py`.
+
+It must evaluate:
+
+| Metric | Required |
+|---|---:|
+| baseline success rate | yes |
+| trained success rate | yes |
+| absolute success improvement | yes |
+| baseline mean reward | yes |
+| trained mean reward | yes |
+| absolute reward improvement | yes |
+| held-out success rate | yes |
+| exploit-block rate | yes |
+| regression-preservation rate | yes |
+| public-route preservation rate | yes |
+| invalid action rate | yes |
+| anti-cheat pass rate | yes |
+
+Save output:
+
+```text
+outputs/evals/<run_name>_eval_summary.json
+```
+
+Minimum hackathon target:
+
+```text
+>= 50 evaluation episodes
+>= 3 independently logged reward components
+>= 1 held-out split
+>= 1 baseline-vs-trained comparison
+>= 1 anti-cheat evaluation
+```
+
+Preferred demo target:
+
+```text
+mean reward improvement >= 30%
+hidden exploit-block pass rate >= 70%
+regression-preservation pass rate >= 80%
+public-route preservation pass rate >= 90%
+anti-cheat pass rate >= 95%
+```
+
+---
+
+## Testing requirements
+
+Before training, all tests must pass.
+
+Required tests:
+
+```text
+test_models.py
+test_reset_step_state.py
+test_rewards.py
+test_anti_cheat.py
+test_seed_reproducibility.py
+test_invalid_actions.py
+test_rollouts.py
+```
+
+Implement at least 3 scripted policies:
+
+```text
+random_policy: explores action space; should usually fail but not crash
+bad_policy: tries invalid/cheating actions; should be penalized
+oracle_policy: uses internal test-only access to solve; should get high reward
+```
+
+The oracle policy is only for tests and must never be exposed to the model during training.
+
+---
+
+## Deployment requirements
+
+The environment must run in these modes:
+
+1. local Python / Uvicorn;
+2. Docker container;
+3. Hugging Face Space;
+4. OpenEnv client over WebSocket.
+
+Required commands:
+
+```bash
+# initialize if not already scaffolded
+openenv init CyberSecurity_OWASP
+
+# local development
+uv sync
+uv run server
+curl http://localhost:8000/health
+
+# Docker
+openenv build -t CyberSecurity_OWASP:latest
+# or:
+docker build -t CyberSecurity_OWASP:latest -f envs/CyberSecurity_OWASP/server/Dockerfile .
+docker run -p 8000:8000 CyberSecurity_OWASP:latest
+
+# HF Spaces
+openenv push --repo-id <username>/CyberSecurity_OWASP
+
+# client install from Space
+pip install git+https://huggingface.co/spaces/<username>/CyberSecurity_OWASP
+```
+
+Use WebSocket mode for training rollouts. HTTP endpoints are acceptable for debugging only.
+
+---
+
+## Scaling rules
+
+Before scaling training, confirm:
+
+1. one manual episode works;
+2. scripted oracle can solve easy seeds;
+3. random policy does not crash;
+4. 10 validation rollouts complete;
+5. reward distributions make sense;
+6. Trackio receives metrics;
+7. rollout artifacts are saved.
+
+Then scale gradually:
+
+```text
+1 episode -> 10 episodes -> 50 episodes -> 100+ rollouts -> training run
+```
+
+For high-volume rollouts, prefer local Docker or Uvicorn over remote HF Spaces because local WebSocket sessions reduce latency and avoid Space limits.
+
+---
+
+## README requirements
+
+The README must explain:
+
+- what CyberSecurity_OWASP models;
+- why authorization repair is useful for LLM RL;
+- action space;
+- observation space;
+- state fields;
+- scenario generation;
+- reward components;
+- hidden tests;
+- anti-overfitting safeguards;
+- anti-cheat safeguards;
+- curriculum;
+- local/Docker/HF Spaces commands;
+- training with TRL/Unsloth;
+- before/after evaluation.
+
+Include a demo narrative:
+
+```text
+1. Baseline model attempts a generated A01 authorization repair episode.
+2. Verifier shows whether it discovered the bug and whether the patch regressed normal flows.
+3. RL training improves reward and pass rates.
+4. Trained model handles held-out domain/layout seeds.
+5. Anti-cheat tests prove it is not using deny-all, hardcoding, or fixture tampering.
+```
+
+---
+
+## Implementation workflow for Codex
+
+When implementing this repo, follow this exact order:
+
+1. Inspect existing structure and tests.
+2. Create/update `00_PROJECT_BRIEF.md` and `01_ARCHITECTURE.md` if missing.
+3. Define `CyberSecurityOWASPAction`, `CyberSecurityOWASPObservation`, and `CyberSecurityOWASPState`.
+4. Implement a dummy OpenEnv server and client.
+5. Implement scenario compiler with 1 domain and 1 BOLA/IDOR mutator.
+6. Implement editable workspace generation.
+7. Implement local request tool.
+8. Implement visible tests.
+9. Implement hidden verifier and reward engine.
+10. Add anti-cheat checks.
+11. Add tests for normal, failing, and cheating rollouts.
+12. Add oracle, random, and bad scripted policies.
+13. Add scenario cache and seeded splits.
+14. Add 3 domains and 3 bug families.
+15. Add GRPO training script.
+16. Add Trackio logging.
+17. Add before/after evaluation script.
+18. Add HF Spaces deployment config.
+19. Run tests and smoke tests.
+20. Produce demo artifacts and README results.
+
+Do not jump to training code before environment and verifier are correct.
+
+---
+
+## Definition of done
+
+CyberSecurity_OWASP is done only when all are true:
+
+- `reset()`, `step(action)`, and `state` work;
+- actions, observations, and state are typed dataclasses;
+- the environment runs locally;
+- the environment runs in Docker;
+- the environment is deployable to HF Spaces;
+- there are at least 5 meaningful reward components;
+- reward components are logged separately;
+- hidden tests exist;
+- anti-cheat tests exist and pass;
+- scenario cache has train/validation/hidden-eval splits;
+- at least 3 bug families exist;
+- at least 3 domains exist;
+- at least 3 scripted policies exist;
+- Trackio is configured for training and evaluation;
+- before/after evaluation exists;
+- held-out evaluation exists;
+- at least 40 rollout artifacts are saved;
+- README explains environment, reward, training, and demo story;
+- demo shows baseline behavior, trained behavior, reward improvement, and safeguards.
+
+---
+
+## Final PR checklist
+
+Every PR summary must answer:
+
+1. What real-world workflow does this implement?
+2. What does the agent observe?
+3. What actions can the agent take?
+4. What hidden state exists and why is it hidden?
+5. What terminates an episode?
+6. What exact checks prove success?
+7. What are the reward components and ranges?
+8. How could the model hack the reward?
+9. What anti-cheat checks prevent that?
+10. What tests prove the reward cannot be trivially hacked?
+11. What baseline success rate did we observe?
+12. What trained success rate did we observe?
+13. What held-out success rate did we observe?
+14. What Trackio run contains the evidence?
+15. Does behavior improve, or only the reward proxy?
+16. Is the environment ready for HF Spaces deployment?
+
+---
+
+## Source grounding and credibility
+
+| Source | Why used | Credibility |
+|---|---|---:|
+| OWASP Top 10 A01 Broken Access Control | Authorization bug taxonomy and prevention framing | 8.5/10 |
+| OWASP ASVS | Access-control verification grounding | 9/10 |
+| NIST SP 800-218 SSDF | Secure software development lifecycle grounding | 9.5/10 |
+| Smith et al., ESEC/FSE 2015, “Is the Cure Worse Than the Disease?” | Peer-reviewed basis for hidden tests and repair-overfitting risk | 9/10 |
+| OpenEnv build/deploy/training docs | Typed model, server, client, deployment, and training mechanics | 8/10 |
+| Meta OpenEnv Hackathon criteria | Judging alignment and minimum requirements | 8/10 |
+
+---
+
+## Non-negotiable rule
+
+A reward that can be hacked is worse than no reward. Build the verifier, hidden tests, anti-cheat tests, and held-out evaluation before scaling training.

Dockerfile

@@ -0,0 +1,28 @@
+ARG BASE_IMAGE=ghcr.io/meta-pytorch/openenv-base:latest
+FROM ${BASE_IMAGE} AS builder
+
+WORKDIR /app/env
+
+COPY pyproject.toml uv.lock ./
+COPY README.md openenv.yaml ./
+COPY __init__.py client.py models.py ./
+COPY bug_mutator.py evals.py fixture_generator.py policy_graph.py rewards.py safety.py scenario_compiler.py template_renderer.py validators.py ./
+COPY server ./server
+COPY training ./training
+COPY scripts ./scripts
+COPY tests ./tests
+
+RUN --mount=type=cache,target=/root/.cache/uv \
+    uv sync --frozen --no-editable
+
+FROM ${BASE_IMAGE}
+
+WORKDIR /app/env
+COPY --from=builder /app/env /app/env
+ENV PATH="/app/env/.venv/bin:$PATH"
+ENV PYTHONPATH="/app/env:$PYTHONPATH"
+
+HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
+    CMD curl -f http://localhost:8000/health || exit 1
+
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "8000"]

README.md

@@ -1,10 +1,164 @@
 ---
-title: CyberSecurity OWASP
-emoji: 📚
-colorFrom: pink
+title: CyberSecurity_OWASP Environment Server
+emoji: 🛡️
+colorFrom: blue
 colorTo: gray
 sdk: docker
 pinned: false
+app_port: 8000
+base_path: /web
+tags:
+  - openenv
+  - cybersecurity
+  - owasp
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# CyberSecurity_OWASP
+
+`CyberSecurity_OWASP` is an OpenEnv-compliant reinforcement-learning environment for a single LLM agent that performs a defensive authorization-repair workflow:
+
+```text
+inspect generated app + policy -> discover authorization bug -> submit finding -> patch code -> preserve intended behavior
+```
+
+The current implementation includes a functional MVP scenario: an invoices FastAPI-style app with one injected OWASP A01 BOLA/IDOR defect, visible tests, hidden deterministic verifier checks, anti-cheat safeguards, and decomposed reward.
+
+## Quick Start
+
+```bash
+uv sync --extra dev
+uv run --extra dev pytest
+uv run server --port 8000
+```
+
+Then connect with the OpenEnv client:
+
+```python
+from CyberSecurity_OWASP import CyberSecurityOWASPAction, CyberSecurityOWASPEnv
+
+with CyberSecurityOWASPEnv(base_url="http://localhost:8000") as env:
+    result = env.reset(seed=7)
+    print(result.observation.task_brief)
+    result = env.step(CyberSecurityOWASPAction(tool_name="list_routes"))
+    print(result.observation.last_tool_result)
+```
+
+## Action Space
+
+The agent emits one JSON action at a time:
+
+```json
+{"tool_name":"read_file","arguments":{"path":"app/routes/invoices.py"}}
+```
+
+Supported tools:
+
+- `inspect_policy_graph`
+- `list_routes`
+- `read_openapi`
+- `read_file`
+- `search_code`
+- `send_local_request`
+- `compare_identities`
+- `submit_finding`
+- `patch_file`
+- `run_visible_tests`
+- `submit_fix`
+- `noop`
+
+Tools are phase-gated:
+
+- `discover`: inspect policy/routes/files, run safe local requests, compare identities, submit finding.
+- `patch`: read/search, patch editable app files, run visible tests, submit final fix.
+- `done`: stable terminal observation only.
+
+## Reward
+
+Terminal reward uses stable components:
+
+```python
+{
+    "discovery": 0.0,
+    "security": 0.0,
+    "regression": 0.0,
+    "public_routes": 0.0,
+    "patch_quality": 0.0,
+    "visible_tests": 0.0,
+    "safety": 0.0,
+    "anti_cheat": 0.0,
+    "total": 0.0,
+}
+```
+
+The verifier rewards blocking the hidden exploit while preserving legitimate owner/admin behavior and intentionally public routes. It penalizes deny-all fixes, hardcoded IDs, hidden file probes, external URL attempts, and test/fixture tampering.
+
+## Scenario Generation
+
+`reset(seed)` compiles a fresh isolated workspace under a temp directory. The MVP compiler generates:
+
+- invoices domain policy graph;
+- randomized users, tenants, invoices, and IDs;
+- generated app files under `app/`;
+- visible tests under `tests/test_visible.py`;
+- hidden facts kept only in state for deterministic verification.
+
+Additional domains and bug families are scaffolded for extension.
+
+## Testing
+
+```bash
+uv run --extra dev pytest
+```
+
+The suite covers model serialization, reset/step/state behavior, seed reproducibility, invalid actions, reward outcomes, anti-cheat checks, and scripted rollout policies.
+
+## Training Scaffold
+
+Training files are under `training/`:
+
+- `rollout.py`
+- `reward_funcs.py`
+- `train_grpo.py`
+- `eval_before_after.py`
+- `trackio_utils.py`
+- `configs/grpo_small.yaml`
+
+The training scaffold is intentionally minimal until the environment/verifier behavior is stable. Trackio metric names and GRPO defaults follow the project brief.
+
+## Modal Ephemeral Runs
+
+Modal Labs support is kept in a separate launcher script so the local OpenEnv server and core training scaffold stay unchanged.
+
+Install the optional local Modal client:
+
+```bash
+uv sync --extra modal
+```
+
+Run a temporary Modal app for a cheap environment/training smoke check:
+
+```bash
+uv run --extra modal modal run scripts/modal_ephemeral_train.py --mode smoke --episodes 4
+```
+
+The app is ephemeral: Modal starts it for the command and stops it when the command exits. The remote result is written locally under `outputs/rollouts/`.
+
+You can also validate the GRPO config construction remotely:
+
+```bash
+uv run --extra modal modal run scripts/modal_ephemeral_train.py --mode grpo-config
+```
+
+The shell wrapper is equivalent:
+
+```bash
+MODE=smoke EPISODES=4 uv run --extra modal bash scripts/modal_run_ephemeral.sh
+```
+
+## Docker / Spaces
+
+```bash
+docker build -t CyberSecurity_OWASP:latest -f server/Dockerfile .
+docker run --rm -p 8000:8000 CyberSecurity_OWASP:latest
+openenv push --repo-id <username>/CyberSecurity_OWASP
+```

__init__.py

@@ -0,0 +1,22 @@
+"""CyberSecurity_OWASP OpenEnv package."""
+
+from .client import CyberSecurityOWASPEnv, CybersecurityOwaspEnv
+from .models import (
+    CyberSecurityOWASPAction,
+    CyberSecurityOWASPObservation,
+    CyberSecurityOWASPState,
+    CybersecurityOwaspAction,
+    CybersecurityOwaspObservation,
+    CybersecurityOwaspState,
+)
+
+__all__ = [
+    "CyberSecurityOWASPAction",
+    "CyberSecurityOWASPObservation",
+    "CyberSecurityOWASPState",
+    "CyberSecurityOWASPEnv",
+    "CybersecurityOwaspAction",
+    "CybersecurityOwaspObservation",
+    "CybersecurityOwaspState",
+    "CybersecurityOwaspEnv",
+]

bug_mutator.py

@@ -0,0 +1,17 @@
+"""Bug-family metadata for generated authorization defects."""
+
+BUG_FAMILIES = {
+    "bola_idor": {
+        "name": "BOLA/IDOR",
+        "defect": "Invoice lookup returns any invoice to any authenticated user.",
+        "repair": "Require same tenant and either owner or billing_admin.",
+    },
+    "bfla": {"name": "BFLA", "status": "scaffolded"},
+    "tenant_leak": {"name": "Tenant leak", "status": "scaffolded"},
+    "jwt_claim_trust": {"name": "JWT claim trust", "status": "scaffolded"},
+    "public_route_trap": {"name": "Public route trap", "status": "scaffolded"},
+}
+
+
+def describe_bug_family(name: str) -> dict:
+    return BUG_FAMILIES.get(name, {"name": name, "status": "unknown"})

client.py

@@ -0,0 +1,39 @@
+"""CyberSecurity_OWASP OpenEnv client."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from openenv.core import EnvClient
+from openenv.core.client_types import StepResult
+
+from .models import (
+    CyberSecurityOWASPAction,
+    CyberSecurityOWASPObservation,
+    CyberSecurityOWASPState,
+)
+
+
+class CyberSecurityOWASPEnv(
+    EnvClient[CyberSecurityOWASPAction, CyberSecurityOWASPObservation, CyberSecurityOWASPState]
+):
+    """WebSocket client for the CyberSecurity_OWASP environment."""
+
+    def _step_payload(self, action: CyberSecurityOWASPAction) -> dict[str, Any]:
+        return action.model_dump()
+
+    def _parse_result(self, payload: dict[str, Any]) -> StepResult[CyberSecurityOWASPObservation]:
+        obs_data = payload.get("observation", {})
+        observation = CyberSecurityOWASPObservation(**obs_data)
+        return StepResult(
+            observation=observation,
+            reward=payload.get("reward", observation.reward),
+            done=payload.get("done", observation.done),
+        )
+
+    def _parse_state(self, payload: dict[str, Any]) -> CyberSecurityOWASPState:
+        return CyberSecurityOWASPState(**payload)
+
+
+# Backward-compatible alias from generated scaffold.
+CybersecurityOwaspEnv = CyberSecurityOWASPEnv

evals.py

@@ -0,0 +1,63 @@
+"""Small scripted policies and eval helpers for tests and demos."""
+
+from __future__ import annotations
+
+import difflib
+from typing import Iterable
+
+from .models import CyberSecurityOWASPAction
+
+
+def random_policy() -> Iterable[CyberSecurityOWASPAction]:
+    yield CyberSecurityOWASPAction(tool_name="noop")
+    yield CyberSecurityOWASPAction(tool_name="list_routes")
+    yield CyberSecurityOWASPAction(tool_name="submit_fix")
+
+
+def bad_policy() -> Iterable[CyberSecurityOWASPAction]:
+    yield CyberSecurityOWASPAction(tool_name="read_file", arguments={"path": "../hidden_tests.py"})
+    yield CyberSecurityOWASPAction(
+        tool_name="send_local_request", arguments={"method": "GET", "path": "https://example.com/"}
+    )
+    yield CyberSecurityOWASPAction(tool_name="submit_fix")
+
+
+def oracle_policy(original_source: str) -> list[CyberSecurityOWASPAction]:
+    fixed = original_source.replace(
+        "    # BUG: this only checks that the caller is authenticated. It forgets the\n"
+        "    # owner/admin and tenant policy checks required by the policy graph.\n"
+        "    return {\"status\": 200, \"body\": invoice}\n",
+        "    if invoice[\"tenant_id\"] != actor[\"tenant_id\"]:\n"
+        "        return {\"status\": 403, \"body\": {\"detail\": \"forbidden\"}}\n"
+        "    if invoice[\"owner_user_id\"] != actor[\"user_id\"] and not is_billing_admin(actor):\n"
+        "        return {\"status\": 403, \"body\": {\"detail\": \"forbidden\"}}\n"
+        "    return {\"status\": 200, \"body\": invoice}\n",
+    )
+    diff = "".join(
+        difflib.unified_diff(
+            original_source.splitlines(True),
+            fixed.splitlines(True),
+            fromfile="app/routes/invoices.py",
+            tofile="app/routes/invoices.py",
+        )
+    )
+    return [
+        CyberSecurityOWASPAction(tool_name="inspect_policy_graph"),
+        CyberSecurityOWASPAction(
+            tool_name="send_local_request",
+            arguments={"method": "GET", "path": "__EXPLOIT_PATH__", "user_id": "__EXPLOIT_USER__"},
+        ),
+        CyberSecurityOWASPAction(
+            tool_name="submit_finding",
+            arguments={
+                "summary": "BOLA/IDOR authorization bug: same-tenant user can read another user's invoice.",
+                "evidence": "__EVIDENCE__",
+                "policy_rule": "Only the owner or billing_admin in the same tenant may read invoices.",
+            },
+        ),
+        CyberSecurityOWASPAction(
+            tool_name="patch_file", arguments={"path": "app/routes/invoices.py", "diff": diff}
+        ),
+        CyberSecurityOWASPAction(tool_name="run_visible_tests"),
+        CyberSecurityOWASPAction(tool_name="submit_fix"),
+    ]

fixture_generator.py

@@ -0,0 +1,17 @@
+"""Fixture helpers for scenario compilers."""
+
+from __future__ import annotations
+
+from typing import Any
+
+
+def visible_workspace_summary(files: list[str], public_hint: dict[str, Any]) -> dict[str, Any]:
+    return {
+        "framework": "fastapi_style_python",
+        "editable_files": files,
+        "routes": [
+            {"method": "GET", "path": "/health", "public": True},
+            {"method": "GET", "path": "/invoices/{invoice_id}", "public": False},
+        ],
+        "domain": public_hint.get("domain", "invoices"),
+    }

models.py

@@ -0,0 +1,81 @@
+"""Typed OpenEnv models for the CyberSecurity_OWASP environment."""
+
+from typing import Any, Literal
+
+from openenv.core.env_server.types import Action, Observation, State
+from pydantic import Field
+
+
+CyberSecurityOWASPPhase = Literal["discover", "patch", "done"]
+CyberSecurityOWASPSplit = Literal["train", "validation", "hidden_eval"]
+
+
+class CyberSecurityOWASPAction(Action):
+    """One typed action emitted by the single defensive AppSec agent."""
+
+    tool_name: Literal[
+        "inspect_policy_graph",
+        "list_routes",
+        "read_openapi",
+        "read_file",
+        "search_code",
+        "send_local_request",
+        "compare_identities",
+        "submit_finding",
+        "patch_file",
+        "run_visible_tests",
+        "submit_fix",
+        "noop",
+    ] = Field(..., description="Tool to execute for this step")
+    arguments: dict[str, Any] = Field(
+        default_factory=dict, description="JSON-serializable tool arguments"
+    )
+
+
+class CyberSecurityOWASPObservation(Observation):
+    """Structured observation returned after reset and every action."""
+
+    phase: CyberSecurityOWASPPhase = "discover"
+    message: str = ""
+    task_brief: str = ""
+    visible_policy_hint: dict[str, Any] = Field(default_factory=dict)
+    workspace_summary: dict[str, Any] = Field(default_factory=dict)
+    available_actions: list[str] = Field(default_factory=list)
+    last_tool_result: str = ""
+    last_action_valid: bool = True
+    last_action_error: str | None = None
+    visible_test_result: str | None = None
+    reward_breakdown: dict[str, float] = Field(default_factory=dict)
+    done_reason: str | None = None
+
+
+class CyberSecurityOWASPState(State):
+    """Internal state used for replay, validation, reward, and eval logging."""
+
+    task_id: str = ""
+    seed: int = 0
+    split: CyberSecurityOWASPSplit = "train"
+    difficulty: int = 0
+    domain: str = ""
+    bug_family: str = ""
+    phase: CyberSecurityOWASPPhase = "discover"
+    max_steps: int = 40
+    done: bool = False
+    success: bool = False
+    failure_reason: str | None = None
+    finding_submitted: bool = False
+    patch_submitted: bool = False
+    accumulated_reward: float = 0.0
+    last_reward: float = 0.0
+    action_history: list[dict[str, Any]] = Field(default_factory=list)
+    reward_history: list[dict[str, float]] = Field(default_factory=list)
+    visible_facts: dict[str, Any] = Field(default_factory=dict)
+    hidden_facts: dict[str, Any] = Field(default_factory=dict)
+    metrics: dict[str, Any] = Field(default_factory=dict)
+    anti_cheat_flags: list[str] = Field(default_factory=list)
+
+
+# Backward-compatible aliases from the OpenEnv scaffold.
+CybersecurityOwaspAction = CyberSecurityOWASPAction
+CybersecurityOwaspObservation = CyberSecurityOWASPObservation
+CybersecurityOwaspState = CyberSecurityOWASPState

openenv.yaml

@@ -0,0 +1,7 @@
+spec_version: 1
+name: CyberSecurity_OWASP
+type: space
+runtime: fastapi
+app: server.app:app
+port: 8000
+

policy_graph.py

@@ -0,0 +1,105 @@
+"""Policy graph generation for MVP authorization-repair scenarios."""
+
+from __future__ import annotations
+
+import random
+from dataclasses import dataclass
+from typing import Any
+
+
+@dataclass(frozen=True)
+class CompiledPolicy:
+    public_hint: dict[str, Any]
+    hidden_facts: dict[str, Any]
+
+
+def build_invoice_policy(seed: int) -> CompiledPolicy:
+    """Create a deterministic invoices policy graph with randomized IDs."""
+
+    rng = random.Random(seed)
+    suffix = rng.randint(1000, 9999)
+    tenant_a = f"tenant-{suffix}-a"
+    tenant_b = f"tenant-{suffix}-b"
+    owner = f"user-{suffix}-owner"
+    intruder = f"user-{suffix}-intruder"
+    admin = f"user-{suffix}-billing-admin"
+    owner_invoice = f"inv-{suffix}-owned"
+    other_invoice = f"inv-{suffix}-other"
+    foreign_invoice = f"inv-{suffix}-foreign"
+
+    users = {
+        owner: {"user_id": owner, "tenant_id": tenant_a, "roles": ["user"]},
+        intruder: {"user_id": intruder, "tenant_id": tenant_a, "roles": ["user"]},
+        admin: {"user_id": admin, "tenant_id": tenant_a, "roles": ["billing_admin"]},
+        f"user-{suffix}-outside": {
+            "user_id": f"user-{suffix}-outside",
+            "tenant_id": tenant_b,
+            "roles": ["user"],
+        },
+    }
+    invoices = {
+        owner_invoice: {
+            "invoice_id": owner_invoice,
+            "tenant_id": tenant_a,
+            "owner_user_id": owner,
+            "amount": 1200,
+            "status": "open",
+        },
+        other_invoice: {
+            "invoice_id": other_invoice,
+            "tenant_id": tenant_a,
+            "owner_user_id": intruder,
+            "amount": 410,
+            "status": "open",
+        },
+        foreign_invoice: {
+            "invoice_id": foreign_invoice,
+            "tenant_id": tenant_b,
+            "owner_user_id": f"user-{suffix}-outside",
+            "amount": 990,
+            "status": "open",
+        },
+    }
+
+    public_hint = {
+        "domain": "invoices",
+        "policy_rules": [
+            "Authenticated users may read invoices they own.",
+            "Billing admins may read invoices in their own tenant.",
+            "Users must not read another user's invoice unless they have a billing_admin role.",
+            "Cross-tenant invoice reads are forbidden.",
+            "GET /health is intentionally public.",
+        ],
+        "users": {
+            alias: {
+                "user_id": value["user_id"],
+                "tenant_id": value["tenant_id"],
+                "roles": value["roles"],
+            }
+            for alias, value in {
+                "owner": users[owner],
+                "same_tenant_other_user": users[intruder],
+                "billing_admin": users[admin],
+            }.items()
+        },
+        "resources": {
+            "owned_invoice": owner_invoice,
+            "same_tenant_other_invoice": other_invoice,
+            "foreign_tenant_invoice": foreign_invoice,
+        },
+        "public_routes": [{"method": "GET", "path": "/health"}],
+    }
+    hidden_facts = {
+        "users": users,
+        "invoices": invoices,
+        "owner_user_id": owner,
+        "intruder_user_id": intruder,
+        "admin_user_id": admin,
+        "owner_invoice_id": owner_invoice,
+        "other_invoice_id": other_invoice,
+        "foreign_invoice_id": foreign_invoice,
+        "tenant_a": tenant_a,
+        "tenant_b": tenant_b,
+        "bug_family": "bola_idor",
+    }
+    return CompiledPolicy(public_hint=public_hint, hidden_facts=hidden_facts)

pyproject.toml

@@ -0,0 +1,48 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+[build-system]
+requires = ["setuptools>=45", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "openenv-CyberSecurity_OWASP"
+version = "0.1.0"
+description = "Cybersecurity Owasp environment for OpenEnv"
+requires-python = ">=3.10"
+dependencies = [
+    # Core OpenEnv runtime (provides FastAPI server + HTTP client types)
+    # install from github
+    # "openenv-core[core] @ git+https://github.com/meta-pytorch/OpenEnv.git",
+    "openenv-core[core]>=0.2.2",
+    # Environment-specific dependencies
+    # Add all dependencies needed for your environment here
+    # Examples:
+    # "numpy>=1.19.0",
+    # "torch>=2.0.0",
+    # "gymnasium>=0.29.0",
+    # "openspiel>=1.0.0",
+    # "smolagents>=1.22.0,<2",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0.0",
+    "pytest-cov>=4.0.0",
+]
+modal = [
+    "modal>=1.1.0",
+]
+
+[project.scripts]
+# Server entry point - enables running via: uv run --project . server
+# or: python -m CyberSecurity_OWASP.server.app
+server = "CyberSecurity_OWASP.server.app:main"
+
+[tool.setuptools]
+include-package-data = true
+packages = ["CyberSecurity_OWASP", "CyberSecurity_OWASP.server"]
+package-dir = { "CyberSecurity_OWASP" = ".", "CyberSecurity_OWASP.server" = "server" }

rewards.py

@@ -0,0 +1,66 @@
+"""Reward computation for CyberSecurity_OWASP."""
+
+from __future__ import annotations
+
+from .models import CyberSecurityOWASPAction, CyberSecurityOWASPState
+
+
+REWARD_KEYS = (
+    "discovery",
+    "security",
+    "regression",
+    "public_routes",
+    "patch_quality",
+    "visible_tests",
+    "safety",
+    "anti_cheat",
+    "total",
+)
+
+
+def empty_reward() -> dict[str, float]:
+    return {key: 0.0 for key in REWARD_KEYS}
+
+
+def compute_reward(
+    state: CyberSecurityOWASPState,
+    action: CyberSecurityOWASPAction,
+    verifier_result: dict,
+) -> dict[str, float]:
+    reward = empty_reward()
+    if action.tool_name == "submit_finding":
+        finding = verifier_result.get("finding", {})
+        reward["discovery"] = (2.0 if finding.get("valid") else 0.0) + (
+            1.0 if finding.get("tied_to_policy") else 0.0
+        )
+    elif action.tool_name == "run_visible_tests":
+        visible = verifier_result.get("visible", {})
+        reward["visible_tests"] = 1.0 if visible.get("passed") else 0.0
+    elif action.tool_name == "submit_fix":
+        security = verifier_result.get("security", {})
+        regression = verifier_result.get("regression", {})
+        public_routes = verifier_result.get("public_routes", {})
+        quality = verifier_result.get("patch_quality", {})
+        visible = verifier_result.get("visible", {})
+        reward["security"] = 5.0 if security.get("passed") else -6.0
+        reward["regression"] = 3.0 if regression.get("passed") else -5.0
+        reward["public_routes"] = 1.0 if public_routes.get("passed") else -3.0
+        reward["patch_quality"] = 2.0 if quality.get("passed") else 0.0
+        reward["visible_tests"] = 1.0 if visible.get("passed") else -6.0
+        if quality.get("hardcoded"):
+            reward["anti_cheat"] -= 4.0
+        if quality.get("deny_all"):
+            reward["safety"] -= 5.0
+
+    flags = verifier_result.get("anti_cheat_flags", []) or []
+    if flags:
+        if "external_network_attempt" in flags:
+            reward["safety"] -= 8.0
+        if "blocked_file_probe" in flags:
+            reward["anti_cheat"] -= 8.0
+        if "hardcoded_hidden_identifier" in flags:
+            reward["anti_cheat"] -= 4.0
+
+    total = sum(value for key, value in reward.items() if key != "total")
+    reward["total"] = min(15.0, total) if total > 0 else total
+    return reward

safety.py

@@ -0,0 +1,17 @@
+"""Safety boundaries for local-only defensive AppSec episodes."""
+
+from __future__ import annotations
+
+
+FORBIDDEN_BEHAVIOR = (
+    "external network access",
+    "host filesystem reads",
+    "hidden test access",
+    "oracle access",
+    "credential extraction",
+    "persistence or evasion",
+)
+
+
+def is_local_route(path: str) -> bool:
+    return path.startswith("/") and not path.startswith("//") and "://" not in path

scenario_compiler.py

@@ -0,0 +1,46 @@
+"""Deterministic scenario compiler for CyberSecurity_OWASP."""
+
+from __future__ import annotations
+
+import tempfile
+from pathlib import Path
+from typing import Any
+
+from .fixture_generator import visible_workspace_summary
+from .policy_graph import build_invoice_policy
+from .template_renderer import render_fastapi_basic
+
+
+def compile_scenario(seed: int, split: str = "train", difficulty: int = 0) -> dict[str, Any]:
+    """Compile one isolated MVP authorization-repair scenario."""
+
+    compiled = build_invoice_policy(seed)
+    workspace = Path(tempfile.mkdtemp(prefix=f"cybersecurity_owasp_{split}_{seed}_"))
+    editable_files = render_fastapi_basic(workspace, compiled.public_hint, compiled.hidden_facts)
+    task_id = f"{split}-invoices-bola-{seed}"
+    hidden = dict(compiled.hidden_facts)
+    hidden.update(
+        {
+            "workspace": str(workspace),
+            "editable_files": editable_files,
+            "initial_file_hashes": {
+                path: (workspace / path).read_text(encoding="utf-8")
+                for path in editable_files
+            },
+        }
+    )
+    return {
+        "task_id": task_id,
+        "workspace": workspace,
+        "domain": "invoices",
+        "bug_family": "bola_idor",
+        "difficulty": difficulty,
+        "task_brief": (
+            "Inspect the generated invoices app and policy. Find the broken "
+            "authorization behavior, submit a finding with local evidence, patch "
+            "the app, preserve intended owner/admin/public behavior, then submit."
+        ),
+        "public_hint": compiled.public_hint,
+        "workspace_summary": visible_workspace_summary(editable_files, compiled.public_hint),
+        "hidden_facts": hidden,
+    }

scripts/docker_build.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+set -euo pipefail
+docker build -t CyberSecurity_OWASP:latest -f server/Dockerfile .

scripts/docker_run.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+set -euo pipefail
+docker run --rm -p "${PORT:-8000}:8000" CyberSecurity_OWASP:latest

scripts/generate_scenarios.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+set -euo pipefail
+uv run python -c "from CyberSecurity_OWASP.scenario_compiler import compile_scenario; [compile_scenario(i) for i in range(3)]; print('generated 3 smoke scenarios')"

scripts/modal_ephemeral_train.py

@@ -0,0 +1,163 @@
+"""Ephemeral Modal Labs launcher for CyberSecurity_OWASP training smoke runs.
+
+Run from the repo root:
+
+    modal run scripts/modal_ephemeral_train.py --mode smoke --episodes 4
+
+This intentionally stays separate from ``training/train_grpo.py``. It packages
+the local repo into a temporary Modal app and returns compact JSON artifacts to
+the local process, so the run disappears when ``modal run`` exits.
+"""
+
+from __future__ import annotations
+
+import json
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+import modal
+
+
+APP_NAME = "CyberSecurity_OWASP-ephemeral-training"
+REMOTE_PROJECT = "/root/CyberSecurity_OWASP"
+PROJECT_ROOT = Path(__file__).resolve().parents[1]
+
+app = modal.App(APP_NAME)
+
+image = (
+    modal.Image.debian_slim(python_version="3.11")
+    .apt_install("git")
+    .add_local_dir(
+        PROJECT_ROOT,
+        remote_path=REMOTE_PROJECT,
+        copy=True,
+        ignore=[
+            ".git",
+            ".venv",
+            "__pycache__",
+            ".pytest_cache",
+            "outputs",
+            "*.pyc",
+        ],
+    )
+    .run_commands(f"pip install -e {REMOTE_PROJECT}")
+    .workdir(REMOTE_PROJECT)
+)
+
+
+class NoopTrainer:
+    """Deterministic placeholder policy for cheap Modal smoke runs."""
+
+    def generate_rollout_completions(self, prompts: list[str]) -> list[dict[str, Any]]:
+        return [
+            {
+                "text": '{"tool_name":"noop","arguments":{}}',
+                "prompt_ids": [],
+                "completion_ids": [],
+                "logprobs": [],
+            }
+            for _ in prompts
+        ]
+
+
+@app.function(image=image, timeout=60 * 30)
+def run_ephemeral_smoke(episodes: int = 4, seed_start: int = 0) -> dict[str, Any]:
+    from CyberSecurity_OWASP.models import CyberSecurityOWASPAction
+    from CyberSecurity_OWASP.server.CyberSecurity_OWASP_environment import (
+        CybersecurityOwaspEnvironment,
+    )
+    from training.rollout import rollout_once
+
+    baseline = []
+    oracle = []
+
+    for offset in range(episodes):
+        seed = seed_start + offset
+
+        baseline_env = CybersecurityOwaspEnvironment()
+        baseline_env.reset(seed=seed, split="validation")
+        baseline.append(rollout_once(NoopTrainer(), baseline_env, max_steps=5))
+
+        oracle_env = CybersecurityOwaspEnvironment()
+        oracle_env.reset(seed=seed, split="validation")
+        hidden = oracle_env.state.hidden_facts
+        oracle_env.step(
+            CyberSecurityOWASPAction(
+                tool_name="submit_finding",
+                arguments={
+                    "summary": "BOLA/IDOR authorization bug in invoice read route.",
+                    "evidence": (
+                        f"user {hidden['owner_user_id']} can request invoice "
+                        f"{hidden['other_invoice_id']} despite the owner/admin policy"
+                    ),
+                    "policy_rule": "Only owner or billing_admin in same tenant may read invoices.",
+                },
+            )
+        )
+        source = (
+            Path(hidden["workspace"]) / "app/routes/invoices.py"
+        ).read_text(encoding="utf-8")
+        fixed = source.replace(
+            "    # BUG: this only checks that the caller is authenticated. It forgets the\n"
+            "    # owner/admin and tenant policy checks required by the policy graph.\n"
+            "    return {\"status\": 200, \"body\": invoice}\n",
+            "    if invoice[\"tenant_id\"] != actor[\"tenant_id\"]:\n"
+            "        return {\"status\": 403, \"body\": {\"detail\": \"forbidden\"}}\n"
+            "    if invoice[\"owner_user_id\"] != actor[\"user_id\"] and not is_billing_admin(actor):\n"
+            "        return {\"status\": 403, \"body\": {\"detail\": \"forbidden\"}}\n"
+            "    return {\"status\": 200, \"body\": invoice}\n",
+        )
+        oracle_env.step(
+            CyberSecurityOWASPAction(
+                tool_name="patch_file",
+                arguments={"path": "app/routes/invoices.py", "content": fixed},
+            )
+        )
+        oracle_env.step(CyberSecurityOWASPAction(tool_name="run_visible_tests"))
+        final = oracle_env.step(CyberSecurityOWASPAction(tool_name="submit_fix"))
+        oracle.append(
+            {
+                "seed": seed,
+                "success": oracle_env.state.success,
+                "reward_total": final.reward_breakdown.get("total", 0.0),
+                "reward_breakdown": final.reward_breakdown,
+            }
+        )
+
+    def mean(items: list[dict[str, Any]], key: str) -> float:
+        return sum(float(item.get(key, 0.0)) for item in items) / max(1, len(items))
+
+    return {
+        "run_name": f"{APP_NAME}-{datetime.utcnow().strftime('%Y%m%d-%H%M%S')}",
+        "mode": "smoke",
+        "episodes": episodes,
+        "seed_start": seed_start,
+        "baseline_mean_reward": mean(baseline, "reward_total"),
+        "oracle_mean_reward": mean(oracle, "reward_total"),
+        "oracle_success_rate": mean(oracle, "success"),
+        "baseline": baseline,
+        "oracle": oracle,
+    }
+
+
+@app.function(image=image, timeout=60 * 10)
+def run_grpo_config_check() -> str:
+    from training.train_grpo import build_grpo_config
+
+    return str(build_grpo_config())
+
+
+@app.local_entrypoint()
+def main(mode: str = "smoke", episodes: int = 4, seed_start: int = 0) -> None:
+    if mode == "smoke":
+        result = run_ephemeral_smoke.remote(episodes=episodes, seed_start=seed_start)
+        output_dir = PROJECT_ROOT / "outputs" / "rollouts"
+        output_dir.mkdir(parents=True, exist_ok=True)
+        output_path = output_dir / f"{result['run_name']}.json"
+        output_path.write_text(json.dumps(result, indent=2, sort_keys=True), encoding="utf-8")
+        print(json.dumps({"saved": str(output_path), **result}, indent=2, sort_keys=True))
+    elif mode == "grpo-config":
+        print(run_grpo_config_check.remote())
+    else:
+        raise ValueError("mode must be 'smoke' or 'grpo-config'")

scripts/modal_run_ephemeral.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+set -euo pipefail
+modal run scripts/modal_ephemeral_train.py --mode "${MODE:-smoke}" --episodes "${EPISODES:-4}" --seed-start "${SEED_START:-0}"

scripts/push_space.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+set -euo pipefail
+openenv push --repo-id "${HF_REPO_ID:?set HF_REPO_ID, e.g. username/CyberSecurity_OWASP}"

scripts/run_local.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+set -euo pipefail
+uv run server --port "${PORT:-8000}"

scripts/smoke_test.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+set -euo pipefail
+uv run pytest tests/test_models.py tests/test_reset_step_state.py

server/CyberSecurity_OWASP_environment.py

@@ -0,0 +1,366 @@
+"""CyberSecurity_OWASP OpenEnv environment implementation."""
+
+from __future__ import annotations
+
+import json
+import shutil
+from pathlib import Path
+from typing import Any
+from uuid import uuid4
+
+from openenv.core.env_server.interfaces import Environment
+
+try:
+    from ..models import (
+        CyberSecurityOWASPAction,
+        CyberSecurityOWASPObservation,
+        CyberSecurityOWASPState,
+    )
+    from ..scenario_compiler import compile_scenario
+    from ..safety import is_local_route
+    from ..validators import detect_cheating, is_path_allowed, simulate_request
+    from .reward_engine import evaluate_action
+except ImportError:  # pragma: no cover
+    from models import CyberSecurityOWASPAction, CyberSecurityOWASPObservation, CyberSecurityOWASPState
+    from scenario_compiler import compile_scenario
+    from safety import is_local_route
+    from validators import detect_cheating, is_path_allowed, simulate_request
+    from server.reward_engine import evaluate_action
+
+
+ALLOWED_TOOLS = {
+    "discover": {
+        "inspect_policy_graph",
+        "list_routes",
+        "read_openapi",
+        "read_file",
+        "search_code",
+        "send_local_request",
+        "compare_identities",
+        "submit_finding",
+        "noop",
+    },
+    "patch": {
+        "read_file",
+        "search_code",
+        "patch_file",
+        "run_visible_tests",
+        "send_local_request",
+        "submit_fix",
+        "noop",
+    },
+    "done": set(),
+}
+
+
+class CybersecurityOwaspEnvironment(
+    Environment[CyberSecurityOWASPAction, CyberSecurityOWASPObservation, CyberSecurityOWASPState]
+):
+    """Single-agent defensive authorization-repair environment."""
+
+    SUPPORTS_CONCURRENT_SESSIONS = True
+
+    def __init__(self):
+        super().__init__()
+        self._state = CyberSecurityOWASPState(episode_id=str(uuid4()))
+        self._task_brief = ""
+        self._visible_policy_hint: dict[str, Any] = {}
+        self._workspace_summary: dict[str, Any] = {}
+        self._last_done_observation: CyberSecurityOWASPObservation | None = None
+
+    def reset(
+        self,
+        seed: int | None = None,
+        episode_id: str | None = None,
+        split: str = "train",
+        difficulty: int = 0,
+        **_: Any,
+    ) -> CyberSecurityOWASPObservation:
+        self.close()
+        actual_seed = int(seed if seed is not None else 0)
+        scenario = compile_scenario(actual_seed, split=split, difficulty=difficulty)
+        self._state = CyberSecurityOWASPState(
+            episode_id=episode_id or str(uuid4()),
+            task_id=scenario["task_id"],
+            seed=actual_seed,
+            split=split,
+            difficulty=difficulty,
+            domain=scenario["domain"],
+            bug_family=scenario["bug_family"],
+            phase="discover",
+            step_count=0,
+            max_steps=40,
+            done=False,
+            success=False,
+            visible_facts={"workspace_summary": scenario["workspace_summary"]},
+            hidden_facts=scenario["hidden_facts"],
+            metrics={"reset_count": 1},
+        )
+        self._task_brief = scenario["task_brief"]
+        self._visible_policy_hint = scenario["public_hint"]
+        self._workspace_summary = scenario["workspace_summary"]
+        self._last_done_observation = None
+        return self._observation("Scenario ready. Start in discover phase.", reward=0.0)
+
+    def step(
+        self,
+        action: CyberSecurityOWASPAction,
+        timeout_s: float | None = None,
+        **_: Any,
+    ) -> CyberSecurityOWASPObservation:
+        if self._state.done:
+            return self._last_done_observation or self._observation(
+                "Episode is already done.", reward=0.0, done_reason=self._state.failure_reason
+            )
+
+        anti_cheat_flags = detect_cheating(self._state, action)
+        for flag in anti_cheat_flags:
+            if flag not in self._state.anti_cheat_flags:
+                self._state.anti_cheat_flags.append(flag)
+
+        self._state.step_count += 1
+        self._state.action_history.append(
+            {"tool_name": action.tool_name, "arguments": action.arguments}
+        )
+
+        if action.tool_name not in ALLOWED_TOOLS[self._state.phase]:
+            verifier, reward = evaluate_action(self._state, action, anti_cheat_flags)
+            return self._finish_step(
+                "Action is not allowed in the current phase.",
+                reward,
+                valid=False,
+                error=f"{action.tool_name} is not allowed during {self._state.phase}",
+                verifier=verifier,
+            )
+
+        try:
+            result, verifier, reward, visible_tests = self._execute(action, anti_cheat_flags)
+            return self._finish_step(
+                result,
+                reward,
+                valid=True,
+                verifier=verifier,
+                visible_test_result=visible_tests,
+            )
+        except Exception as exc:  # keep malformed agent actions from crashing the server
+            verifier, reward = evaluate_action(self._state, action, anti_cheat_flags)
+            return self._finish_step(
+                "Tool execution failed.",
+                reward,
+                valid=False,
+                error=str(exc),
+                verifier=verifier,
+            )
+
+    @property
+    def state(self) -> CyberSecurityOWASPState:
+        return self._state
+
+    def close(self) -> None:
+        workspace = self._state.hidden_facts.get("workspace")
+        if workspace:
+            shutil.rmtree(workspace, ignore_errors=True)
+
+    def _execute(
+        self, action: CyberSecurityOWASPAction, anti_cheat_flags: list[str]
+    ) -> tuple[str, dict, dict[str, float], str | None]:
+        verifier: dict = {"anti_cheat_flags": anti_cheat_flags}
+        reward = {key: 0.0 for key in (
+            "discovery",
+            "security",
+            "regression",
+            "public_routes",
+            "patch_quality",
+            "visible_tests",
+            "safety",
+            "anti_cheat",
+            "total",
+        )}
+        visible_tests = None
+        args = action.arguments or {}
+
+        if action.tool_name == "noop":
+            return "No operation.", verifier, reward, None
+        if action.tool_name == "inspect_policy_graph":
+            return json.dumps(self._visible_policy_hint, indent=2, sort_keys=True), verifier, reward, None
+        if action.tool_name == "list_routes":
+            return json.dumps(self._workspace_summary["routes"], indent=2), verifier, reward, None
+        if action.tool_name == "read_openapi":
+            return json.dumps(
+                {
+                    "openapi": "3.1.0",
+                    "info": {"title": "Generated invoices app", "version": "0.1.0"},
+                    "paths": {
+                        "/health": {"get": {"x-public": True}},
+                        "/invoices/{invoice_id}": {"get": {"x-public": False}},
+                    },
+                },
+                indent=2,
+            ), verifier, reward, None
+        if action.tool_name == "read_file":
+            path = self._resolve_path(str(args.get("path", "")))
+            return path.read_text(encoding="utf-8"), verifier, reward, None
+        if action.tool_name == "search_code":
+            return self._search_code(str(args.get("query", ""))), verifier, reward, None
+        if action.tool_name == "send_local_request":
+            if not is_local_route(str(args.get("path", ""))):
+                raise ValueError("send_local_request only accepts local route paths")
+            response = simulate_request(
+                self._state,
+                str(args.get("method", "GET")),
+                str(args.get("path", "")),
+                args.get("user_id"),
+            )
+            return json.dumps(response, indent=2, sort_keys=True), verifier, reward, None
+        if action.tool_name == "compare_identities":
+            path = str(args.get("path", ""))
+            first = str(args.get("first_user_id", ""))
+            second = str(args.get("second_user_id", ""))
+            if not is_local_route(path):
+                raise ValueError("compare_identities only accepts local route paths")
+            response = {
+                "first": simulate_request(self._state, str(args.get("method", "GET")), path, first),
+                "second": simulate_request(self._state, str(args.get("method", "GET")), path, second),
+            }
+            return json.dumps(response, indent=2, sort_keys=True), verifier, reward, None
+        if action.tool_name == "submit_finding":
+            verifier, reward = evaluate_action(self._state, action, anti_cheat_flags)
+            if verifier.get("finding", {}).get("valid"):
+                self._state.finding_submitted = True
+                self._state.phase = "patch"
+                return "Finding accepted. Patch phase unlocked.", verifier, reward, None
+            return "Finding was not specific enough to unlock patching.", verifier, reward, None
+        if action.tool_name == "patch_file":
+            path = self._resolve_path(str(args.get("path", "")), write=True)
+            if "content" in args:
+                path.write_text(str(args["content"]), encoding="utf-8")
+            else:
+                self._apply_unified_diff(path, str(args.get("diff", "")))
+            return f"Patched {args.get('path')}.", verifier, reward, None
+        if action.tool_name == "run_visible_tests":
+            verifier, reward = evaluate_action(self._state, action, anti_cheat_flags)
+            visible_tests = json.dumps(verifier.get("visible", {}), indent=2, sort_keys=True)
+            return visible_tests, verifier, reward, visible_tests
+        if action.tool_name == "submit_fix":
+            verifier, reward = evaluate_action(self._state, action, anti_cheat_flags)
+            self._state.patch_submitted = True
+            security = verifier.get("security", {}).get("passed", False)
+            regression = verifier.get("regression", {}).get("passed", False)
+            public = verifier.get("public_routes", {}).get("passed", False)
+            quality = verifier.get("patch_quality", {}).get("passed", False)
+            self._state.success = bool(security and regression and public and quality)
+            self._state.done = True
+            self._state.phase = "done"
+            self._state.failure_reason = None if self._state.success else "hidden_verifier_failed"
+            return json.dumps(verifier, indent=2, sort_keys=True), verifier, reward, None
+        raise ValueError(f"Unhandled tool {action.tool_name}")
+
+    def _finish_step(
+        self,
+        message: str,
+        reward: dict[str, float],
+        *,
+        valid: bool,
+        error: str | None = None,
+        verifier: dict | None = None,
+        visible_test_result: str | None = None,
+    ) -> CyberSecurityOWASPObservation:
+        self._state.last_reward = float(reward.get("total", 0.0))
+        self._state.accumulated_reward += self._state.last_reward
+        self._state.reward_history.append(reward)
+        if self._state.step_count >= self._state.max_steps and not self._state.done:
+            self._state.done = True
+            self._state.phase = "done"
+            self._state.failure_reason = "max_steps_exceeded"
+        obs = self._observation(
+            message,
+            reward=self._state.last_reward,
+            valid=valid,
+            error=error,
+            reward_breakdown=reward,
+            visible_test_result=visible_test_result,
+            done_reason=self._state.failure_reason,
+        )
+        if self._state.done:
+            self._last_done_observation = obs
+        return obs
+
+    def _observation(
+        self,
+        message: str,
+        *,
+        reward: float,
+        valid: bool = True,
+        error: str | None = None,
+        reward_breakdown: dict[str, float] | None = None,
+        visible_test_result: str | None = None,
+        done_reason: str | None = None,
+    ) -> CyberSecurityOWASPObservation:
+        return CyberSecurityOWASPObservation(
+            phase=self._state.phase,
+            message=message,
+            task_brief=self._task_brief,
+            visible_policy_hint=self._visible_policy_hint,
+            workspace_summary=self._workspace_summary,
+            available_actions=sorted(ALLOWED_TOOLS[self._state.phase]),
+            last_tool_result=message,
+            last_action_valid=valid,
+            last_action_error=error,
+            visible_test_result=visible_test_result,
+            reward_breakdown=reward_breakdown or {},
+            done_reason=done_reason,
+            done=self._state.done,
+            reward=reward,
+            metadata={"episode_id": self._state.episode_id, "step_count": self._state.step_count},
+        )
+
+    def _resolve_path(self, path: str, *, write: bool = False) -> Path:
+        allowed, normalized_or_error = is_path_allowed(self._state, path, write=write)
+        if not allowed:
+            raise ValueError(normalized_or_error)
+        return Path(str(self._state.hidden_facts["workspace"])) / normalized_or_error
+
+    def _search_code(self, query: str) -> str:
+        if not query:
+            raise ValueError("query is required")
+        results: list[str] = []
+        workspace = Path(str(self._state.hidden_facts["workspace"]))
+        for rel in self._state.hidden_facts.get("editable_files", []):
+            path = workspace / rel
+            text = path.read_text(encoding="utf-8")
+            for idx, line in enumerate(text.splitlines(), start=1):
+                if query.lower() in line.lower():
+                    results.append(f"{rel}:{idx}: {line}")
+        return "\n".join(results) or "No matches."
+
+    def _apply_unified_diff(self, path: Path, diff: str) -> None:
+        if not diff.strip():
+            raise ValueError("diff or content is required")
+        original = path.read_text(encoding="utf-8").splitlines(True)
+        output: list[str] = []
+        old_index = 0
+        lines = diff.splitlines(True)
+        i = 0
+        while i < len(lines):
+            line = lines[i]
+            if not line.startswith("@@"):
+                i += 1
+                continue
+            old_start = int(line.split()[1].split(",")[0][1:])
+            output.extend(original[old_index : old_start - 1])
+            old_index = old_start - 1
+            i += 1
+            while i < len(lines) and not lines[i].startswith("@@"):
+                hunk_line = lines[i]
+                if hunk_line.startswith(" "):
+                    output.append(original[old_index])
+                    old_index += 1
+                elif hunk_line.startswith("-"):
+                    old_index += 1
+                elif hunk_line.startswith("+"):
+                    output.append(hunk_line[1:])
+                elif hunk_line.startswith("\\"):
+                    pass
+                i += 1
+        output.extend(original[old_index:])
+        path.write_text("".join(output), encoding="utf-8")

server/Dockerfile

@@ -0,0 +1,80 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+# Multi-stage build using openenv-base
+# This Dockerfile is flexible and works for both:
+# - In-repo environments (with local OpenEnv sources)
+# - Standalone environments (with openenv from PyPI/Git)
+# The build script (openenv build) handles context detection and sets appropriate build args.
+
+ARG BASE_IMAGE=ghcr.io/meta-pytorch/openenv-base:latest
+FROM ${BASE_IMAGE} AS builder
+
+WORKDIR /app
+
+# Ensure git is available (required for installing dependencies from VCS)
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends git && \
+    rm -rf /var/lib/apt/lists/*
+
+# Build argument to control whether we're building standalone or in-repo
+ARG BUILD_MODE=in-repo
+ARG ENV_NAME=CyberSecurity_OWASP
+
+# Copy environment code (always at root of build context)
+COPY . /app/env
+
+# For in-repo builds, openenv is already vendored in the build context
+# For standalone builds, openenv will be installed via pyproject.toml
+WORKDIR /app/env
+
+# Ensure uv is available (for local builds where base image lacks it)
+RUN if ! command -v uv >/dev/null 2>&1; then \
+        curl -LsSf https://astral.sh/uv/install.sh | sh && \
+        mv /root/.local/bin/uv /usr/local/bin/uv && \
+        mv /root/.local/bin/uvx /usr/local/bin/uvx; \
+    fi
+    
+# Install dependencies using uv sync
+# If uv.lock exists, use it; otherwise resolve on the fly
+RUN --mount=type=cache,target=/root/.cache/uv \
+    if [ -f uv.lock ]; then \
+        uv sync --frozen --no-install-project --no-editable; \
+    else \
+        uv sync --no-install-project --no-editable; \
+    fi
+
+RUN --mount=type=cache,target=/root/.cache/uv \
+    if [ -f uv.lock ]; then \
+        uv sync --frozen --no-editable; \
+    else \
+        uv sync --no-editable; \
+    fi
+
+# Final runtime stage
+FROM ${BASE_IMAGE}
+
+WORKDIR /app
+
+# Copy the virtual environment from builder
+COPY --from=builder /app/env/.venv /app/.venv
+
+# Copy the environment code
+COPY --from=builder /app/env /app/env
+
+# Set PATH to use the virtual environment
+ENV PATH="/app/.venv/bin:$PATH"
+
+# Set PYTHONPATH so imports work correctly
+ENV PYTHONPATH="/app/env:$PYTHONPATH"
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
+    CMD curl -f http://localhost:8000/health || exit 1
+
+# Run the FastAPI server
+# The module path is constructed to work with the /app/env structure
+CMD ["sh", "-c", "cd /app/env && uvicorn server.app:app --host 0.0.0.0 --port 8000"]

server/__init__.py

@@ -0,0 +1,11 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Cybersecurity Owasp environment server components."""
+
+from .CyberSecurity_OWASP_environment import CybersecurityOwaspEnvironment
+
+__all__ = ["CybersecurityOwaspEnvironment"]

server/app.py

@@ -0,0 +1,62 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""FastAPI application for the CyberSecurity_OWASP OpenEnv server."""
+
+try:
+    from openenv.core.env_server.http_server import create_app
+except Exception as e:  # pragma: no cover
+    raise ImportError(
+        "openenv is required for the web interface. Install dependencies with '\n    uv sync\n'"
+    ) from e
+
+try:
+    from ..models import CyberSecurityOWASPAction, CyberSecurityOWASPObservation
+    from .CyberSecurity_OWASP_environment import CybersecurityOwaspEnvironment
+except ModuleNotFoundError:
+    from models import CyberSecurityOWASPAction, CyberSecurityOWASPObservation
+    from server.CyberSecurity_OWASP_environment import CybersecurityOwaspEnvironment
+
+
+# Create the app with web interface and README integration
+app = create_app(
+    CybersecurityOwaspEnvironment,
+    CyberSecurityOWASPAction,
+    CyberSecurityOWASPObservation,
+    env_name="CyberSecurity_OWASP",
+    max_concurrent_envs=4,
+)
+
+
+def main(host: str = "0.0.0.0", port: int = 8000):
+    """
+    Entry point for direct execution via uv run or python -m.
+
+    This function enables running the server without Docker:
+        uv run --project . server
+        uv run --project . server --port 8001
+        python -m CyberSecurity_OWASP.server.app
+
+    Args:
+        host: Host address to bind to (default: "0.0.0.0")
+        port: Port number to listen on (default: 8000)
+
+    For production deployments, consider using uvicorn directly with
+    multiple workers:
+        uvicorn CyberSecurity_OWASP.server.app:app --workers 4
+    """
+    import uvicorn
+
+    uvicorn.run(app, host=host, port=port)
+
+
+if __name__ == "__main__":
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--port", type=int, default=8000)
+    args = parser.parse_args()
+    main(port=args.port)

server/requirements.txt

@@ -0,0 +1,6 @@
+openenv[core]>=0.2.0
+fastapi>=0.115.0
+uvicorn>=0.24.0
+
+
+

server/reward_engine.py

@@ -0,0 +1,49 @@
+"""Server-side verifier aggregation for terminal scoring."""
+
+from __future__ import annotations
+
+try:
+    from ..models import CyberSecurityOWASPAction, CyberSecurityOWASPState
+    from ..rewards import compute_reward
+    from ..validators import (
+        patch_quality,
+        run_hidden_regression_tests,
+        run_hidden_security_tests,
+        run_public_route_tests,
+        run_visible_tests,
+        verify_finding,
+    )
+except ImportError:  # pragma: no cover
+    from models import CyberSecurityOWASPAction, CyberSecurityOWASPState
+    from rewards import compute_reward
+    from validators import (
+        patch_quality,
+        run_hidden_regression_tests,
+        run_hidden_security_tests,
+        run_public_route_tests,
+        run_visible_tests,
+        verify_finding,
+    )
+
+
+def evaluate_action(
+    state: CyberSecurityOWASPState,
+    action: CyberSecurityOWASPAction,
+    anti_cheat_flags: list[str] | None = None,
+) -> tuple[dict, dict[str, float]]:
+    verifier_result: dict = {"anti_cheat_flags": anti_cheat_flags or []}
+    if action.tool_name == "submit_finding":
+        verifier_result["finding"] = verify_finding(state, action.arguments)
+    elif action.tool_name == "run_visible_tests":
+        verifier_result["visible"] = run_visible_tests(state)
+    elif action.tool_name == "submit_fix":
+        verifier_result.update(
+            {
+                "visible": run_visible_tests(state),
+                "security": run_hidden_security_tests(state),
+                "regression": run_hidden_regression_tests(state),
+                "public_routes": run_public_route_tests(state),
+                "patch_quality": patch_quality(state),
+            }
+        )
+    return verifier_result, compute_reward(state, action, verifier_result)

template_renderer.py

@@ -0,0 +1,97 @@
+"""Render small FastAPI-style generated workspaces."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Any
+
+
+def render_fastapi_basic(workspace: Path, public_hint: dict[str, Any], hidden: dict[str, Any]) -> list[str]:
+    """Render the MVP vulnerable invoices app and visible tests."""
+
+    app_dir = workspace / "app"
+    routes_dir = app_dir / "routes"
+    tests_dir = workspace / "tests"
+    routes_dir.mkdir(parents=True, exist_ok=True)
+    tests_dir.mkdir(parents=True, exist_ok=True)
+
+    (app_dir / "__init__.py").write_text("", encoding="utf-8")
+    (routes_dir / "__init__.py").write_text("", encoding="utf-8")
+    (app_dir / "data.py").write_text(
+        "USERS = " + repr(hidden["users"]) + "\n\nINVOICES = " + repr(hidden["invoices"]) + "\n",
+        encoding="utf-8",
+    )
+    (app_dir / "auth.py").write_text(
+        """from .data import USERS
+
+
+def get_actor(user_id: str):
+    return USERS.get(user_id)
+
+
+def is_billing_admin(actor: dict) -> bool:
+    return "billing_admin" in actor.get("roles", [])
+""",
+        encoding="utf-8",
+    )
+    (routes_dir / "invoices.py").write_text(
+        """from app.auth import get_actor, is_billing_admin
+from app.data import INVOICES
+
+
+def get_invoice(invoice_id: str, user_id: str):
+    actor = get_actor(user_id)
+    if actor is None:
+        return {"status": 401, "body": {"detail": "unknown user"}}
+
+    invoice = INVOICES.get(invoice_id)
+    if invoice is None:
+        return {"status": 404, "body": {"detail": "invoice not found"}}
+
+    # BUG: this only checks that the caller is authenticated. It forgets the
+    # owner/admin and tenant policy checks required by the policy graph.
+    return {"status": 200, "body": invoice}
+
+
+def health():
+    return {"status": 200, "body": {"status": "ok"}}
+""",
+        encoding="utf-8",
+    )
+    (app_dir / "main.py").write_text(
+        """from app.routes.invoices import get_invoice, health
+
+
+ROUTES = [
+    {"method": "GET", "path": "/health", "handler": health, "public": True},
+    {"method": "GET", "path": "/invoices/{invoice_id}", "handler": get_invoice, "public": False},
+]
+""",
+        encoding="utf-8",
+    )
+    (tests_dir / "test_visible.py").write_text(
+        f"""from app.routes.invoices import get_invoice, health
+
+
+def test_health_is_public():
+    assert health()["status"] == 200
+
+
+def test_owner_can_read_own_invoice():
+    result = get_invoice("{hidden['owner_invoice_id']}", "{hidden['owner_user_id']}")
+    assert result["status"] == 200
+
+
+def test_admin_can_read_same_tenant_invoice():
+    result = get_invoice("{hidden['other_invoice_id']}", "{hidden['admin_user_id']}")
+    assert result["status"] == 200
+""",
+        encoding="utf-8",
+    )
+    return [
+        "app/main.py",
+        "app/auth.py",
+        "app/data.py",
+        "app/routes/invoices.py",
+        "tests/test_visible.py",
+    ]

tests/__init__.py

@@ -0,0 +1 @@
+"""Test package for CyberSecurity_OWASP."""

tests/helpers.py

@@ -0,0 +1,51 @@
+from pathlib import Path
+
+from CyberSecurity_OWASP.models import CyberSecurityOWASPAction
+from CyberSecurity_OWASP.server.CyberSecurity_OWASP_environment import CybersecurityOwaspEnvironment
+
+
+def make_env(seed: int = 7) -> CybersecurityOwaspEnvironment:
+    env = CybersecurityOwaspEnvironment()
+    env.reset(seed=seed, episode_id=f"test-{seed}")
+    return env
+
+
+def submit_valid_finding(env: CybersecurityOwaspEnvironment):
+    hidden = env.state.hidden_facts
+    return env.step(
+        CyberSecurityOWASPAction(
+            tool_name="submit_finding",
+            arguments={
+                "summary": "BOLA IDOR authorization bug lets a same-tenant user read another user's invoice.",
+                "evidence": (
+                    f"user {hidden['owner_user_id']} can request invoice "
+                    f"{hidden['other_invoice_id']} even though policy requires owner or billing_admin"
+                ),
+                "policy_rule": "owner or billing_admin in same tenant only",
+            },
+        )
+    )
+
+
+def secure_invoice_source(env: CybersecurityOwaspEnvironment) -> str:
+    workspace = Path(env.state.hidden_facts["workspace"])
+    source = (workspace / "app/routes/invoices.py").read_text(encoding="utf-8")
+    return source.replace(
+        "    # BUG: this only checks that the caller is authenticated. It forgets the\n"
+        "    # owner/admin and tenant policy checks required by the policy graph.\n"
+        "    return {\"status\": 200, \"body\": invoice}\n",
+        "    if invoice[\"tenant_id\"] != actor[\"tenant_id\"]:\n"
+        "        return {\"status\": 403, \"body\": {\"detail\": \"forbidden\"}}\n"
+        "    if invoice[\"owner_user_id\"] != actor[\"user_id\"] and not is_billing_admin(actor):\n"
+        "        return {\"status\": 403, \"body\": {\"detail\": \"forbidden\"}}\n"
+        "    return {\"status\": 200, \"body\": invoice}\n",
+    )
+
+
+def apply_secure_patch(env: CybersecurityOwaspEnvironment):
+    return env.step(
+        CyberSecurityOWASPAction(
+            tool_name="patch_file",
+            arguments={"path": "app/routes/invoices.py", "content": secure_invoice_source(env)},
+        )
+    )

tests/test_anti_cheat.py

@@ -0,0 +1,16 @@
+from CyberSecurity_OWASP.models import CyberSecurityOWASPAction
+
+from .helpers import make_env, submit_valid_finding
+
+
+def test_patch_hidden_identifier_flagged():
+    env = make_env(50)
+    submit_valid_finding(env)
+    hidden_id = env.state.hidden_facts["other_invoice_id"]
+    env.step(
+        CyberSecurityOWASPAction(
+            tool_name="patch_file",
+            arguments={"path": "app/routes/invoices.py", "content": f"# {hidden_id}\n"},
+        )
+    )
+    assert "hardcoded_hidden_identifier" in env.state.anti_cheat_flags

tests/test_invalid_actions.py

@@ -0,0 +1,48 @@
+from CyberSecurity_OWASP.models import CyberSecurityOWASPAction
+
+from .helpers import make_env, submit_valid_finding
+
+
+def test_wrong_phase_action_is_structured_error():
+    env = make_env(30)
+    obs = env.step(
+        CyberSecurityOWASPAction(
+            tool_name="patch_file",
+            arguments={"path": "app/routes/invoices.py", "content": ""},
+        )
+    )
+    assert obs.last_action_valid is False
+    assert "not allowed" in (obs.last_action_error or "")
+
+
+def test_blocked_hidden_path_does_not_crash():
+    env = make_env(31)
+    obs = env.step(CyberSecurityOWASPAction(tool_name="read_file", arguments={"path": "../hidden.py"}))
+    assert obs.last_action_valid is False
+    assert "blocked" in (obs.last_action_error or "")
+    assert "blocked_file_probe" in env.state.anti_cheat_flags
+
+
+def test_external_request_is_blocked():
+    env = make_env(32)
+    obs = env.step(
+        CyberSecurityOWASPAction(
+            tool_name="send_local_request",
+            arguments={"method": "GET", "path": "https://example.com"},
+        )
+    )
+    assert obs.last_action_valid is False
+    assert "external_network_attempt" in env.state.anti_cheat_flags
+
+
+def test_visible_tests_are_not_patchable():
+    env = make_env(33)
+    submit_valid_finding(env)
+    obs = env.step(
+        CyberSecurityOWASPAction(
+            tool_name="patch_file",
+            arguments={"path": "tests/test_visible.py", "content": ""},
+        )
+    )
+    assert obs.last_action_valid is False
+    assert "not patchable" in (obs.last_action_error or "")

tests/test_models.py

@@ -0,0 +1,14 @@
+from CyberSecurity_OWASP import (
+    CyberSecurityOWASPAction,
+    CyberSecurityOWASPObservation,
+    CyberSecurityOWASPState,
+)
+
+
+def test_models_serialize():
+    action = CyberSecurityOWASPAction(tool_name="noop")
+    assert action.model_dump()["tool_name"] == "noop"
+    obs = CyberSecurityOWASPObservation(phase="discover", message="ok")
+    assert obs.model_dump()["phase"] == "discover"
+    state = CyberSecurityOWASPState(episode_id="e1", seed=1)
+    assert state.model_dump()["seed"] == 1

tests/test_reset_step_state.py

@@ -0,0 +1,25 @@
+from CyberSecurity_OWASP.models import CyberSecurityOWASPAction
+
+from .helpers import make_env
+
+
+def test_reset_initializes_scenario_and_state():
+    env = make_env(10)
+    state = env.state
+    assert state.seed == 10
+    assert state.phase == "discover"
+    assert state.domain == "invoices"
+    assert state.bug_family == "bola_idor"
+
+
+def test_step_count_and_done_stability():
+    env = make_env(11)
+    env.step(CyberSecurityOWASPAction(tool_name="noop"))
+    assert env.state.step_count == 1
+    env.state.done = True
+    env.state.phase = "done"
+    first = env.step(CyberSecurityOWASPAction(tool_name="noop"))
+    second = env.step(CyberSecurityOWASPAction(tool_name="noop"))
+    assert first.done is True
+    assert second.done is True
+    assert env.state.step_count == 1

tests/test_rewards.py

@@ -0,0 +1,67 @@
+from CyberSecurity_OWASP.models import CyberSecurityOWASPAction
+
+from .helpers import apply_secure_patch, make_env, secure_invoice_source, submit_valid_finding
+
+
+def test_oracle_patch_gets_high_reward():
+    env = make_env(40)
+    finding = submit_valid_finding(env)
+    assert finding.reward_breakdown["discovery"] == 3.0
+    apply_secure_patch(env)
+    visible = env.step(CyberSecurityOWASPAction(tool_name="run_visible_tests"))
+    assert visible.reward_breakdown["visible_tests"] == 1.0
+    final = env.step(CyberSecurityOWASPAction(tool_name="submit_fix"))
+    assert env.state.success is True
+    assert final.reward_breakdown["total"] >= 12.0
+
+
+def test_finding_without_patch_does_not_succeed():
+    env = make_env(41)
+    submit_valid_finding(env)
+    final = env.step(CyberSecurityOWASPAction(tool_name="submit_fix"))
+    assert env.state.success is False
+    assert final.reward_breakdown["security"] < 0
+
+
+def test_deny_all_patch_fails_regression():
+    env = make_env(42)
+    submit_valid_finding(env)
+    source = secure_invoice_source(env).replace(
+        "    if invoice[\"tenant_id\"] != actor[\"tenant_id\"]:\n",
+        "    return {\"status\": 403, \"body\": {\"detail\": \"forbidden\"}}\n"
+        "    if invoice[\"tenant_id\"] != actor[\"tenant_id\"]:\n",
+    )
+    env.step(
+        CyberSecurityOWASPAction(
+            tool_name="patch_file",
+            arguments={"path": "app/routes/invoices.py", "content": source},
+        )
+    )
+    final = env.step(CyberSecurityOWASPAction(tool_name="submit_fix"))
+    assert final.reward_breakdown["regression"] < 0
+    assert env.state.success is False
+
+
+def test_hardcoded_patch_is_penalized():
+    env = make_env(43)
+    submit_valid_finding(env)
+    hidden = env.state.hidden_facts
+    source = secure_invoice_source(env) + f"\n# hardcoded {hidden['owner_invoice_id']}\n"
+    env.step(
+        CyberSecurityOWASPAction(
+            tool_name="patch_file",
+            arguments={"path": "app/routes/invoices.py", "content": source},
+        )
+    )
+    final = env.step(CyberSecurityOWASPAction(tool_name="submit_fix"))
+    assert final.reward_breakdown["anti_cheat"] < 0
+    assert env.state.success is False
+
+
+def test_visible_tests_only_does_not_get_high_reward():
+    env = make_env(44)
+    submit_valid_finding(env)
+    visible = env.step(CyberSecurityOWASPAction(tool_name="run_visible_tests"))
+    assert visible.reward_breakdown["visible_tests"] == 1.0
+    final = env.step(CyberSecurityOWASPAction(tool_name="submit_fix"))
+    assert final.reward_breakdown["total"] < 5.0

tests/test_rollouts.py

@@ -0,0 +1,29 @@
+from CyberSecurity_OWASP.evals import bad_policy, random_policy
+from CyberSecurity_OWASP.models import CyberSecurityOWASPAction
+
+from .helpers import apply_secure_patch, make_env, submit_valid_finding
+
+
+def test_random_policy_does_not_crash():
+    env = make_env(60)
+    for action in random_policy():
+        obs = env.step(action)
+        assert obs is not None
+
+
+def test_bad_policy_is_penalized_or_flagged():
+    env = make_env(61)
+    for action in bad_policy():
+        obs = env.step(action)
+    assert env.state.anti_cheat_flags
+    assert obs.reward <= 0
+
+
+def test_scripted_oracle_solves_episode():
+    env = make_env(62)
+    submit_valid_finding(env)
+    apply_secure_patch(env)
+    env.step(CyberSecurityOWASPAction(tool_name="run_visible_tests"))
+    final = env.step(CyberSecurityOWASPAction(tool_name="submit_fix"))
+    assert final.done is True
+    assert env.state.success is True

tests/test_seed_reproducibility.py

@@ -0,0 +1,10 @@
+from .helpers import make_env
+
+
+def test_same_seed_reproducible_visible_facts():
+    a = make_env(22)
+    b = make_env(22)
+    assert a.state.task_id == b.state.task_id
+    assert a.state.hidden_facts["owner_invoice_id"] == b.state.hidden_facts["owner_invoice_id"]
+    assert a.state.hidden_facts["other_invoice_id"] == b.state.hidden_facts["other_invoice_id"]
+    assert a.state.visible_facts == b.state.visible_facts

training/configs/grpo_small.yaml

@@ -0,0 +1,9 @@
+model_name: Qwen/Qwen3-1.7B
+algo: grpo
+environment: CyberSecurity_OWASP
+max_steps: 40
+num_generations: 2
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 32
+learning_rate: 0.000005
+report_to: trackio

training/eval_before_after.py

@@ -0,0 +1,29 @@
+"""Baseline-vs-trained evaluation scaffold for CyberSecurity_OWASP."""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+
+def summarize_runs(baseline: list[dict], trained: list[dict], heldout: list[dict]) -> dict:
+    def mean(items: list[dict], key: str) -> float:
+        return sum(float(item.get(key, 0.0)) for item in items) / max(1, len(items))
+
+    return {
+        "baseline_success_rate": mean(baseline, "success"),
+        "trained_success_rate": mean(trained, "success"),
+        "absolute_success_improvement": mean(trained, "success") - mean(baseline, "success"),
+        "baseline_mean_reward": mean(baseline, "reward_total"),
+        "trained_mean_reward": mean(trained, "reward_total"),
+        "absolute_reward_improvement": mean(trained, "reward_total") - mean(baseline, "reward_total"),
+        "heldout_success_rate": mean(heldout, "success"),
+        "heldout_mean_reward": mean(heldout, "reward_total"),
+    }
+
+
+def save_eval_summary(run_name: str, summary: dict) -> Path:
+    output = Path("outputs/evals") / f"{run_name}_eval_summary.json"
+    output.parent.mkdir(parents=True, exist_ok=True)
+    output.write_text(json.dumps(summary, indent=2, sort_keys=True), encoding="utf-8")
+    return output

training/reward_funcs.py

@@ -0,0 +1,25 @@
+"""Reward functions exposed for TRL/GRPO logging."""
+
+
+def _values(name: str, completions, kwargs):
+    return [float(x) for x in kwargs.get(name, [0.0] * len(completions))]
+
+
+def reward_total(completions, **kwargs):
+    return _values("reward_total", completions, kwargs)
+
+
+def reward_security(completions, **kwargs):
+    return _values("reward_security", completions, kwargs)
+
+
+def reward_regression(completions, **kwargs):
+    return _values("reward_regression", completions, kwargs)
+
+
+def reward_patch_quality(completions, **kwargs):
+    return _values("reward_patch_quality", completions, kwargs)
+
+
+def reward_anti_cheat(completions, **kwargs):
+    return _values("reward_anti_cheat", completions, kwargs)

training/rollout.py

@@ -0,0 +1,84 @@
+"""Minimal rollout loop for CyberSecurity_OWASP episodes."""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+from CyberSecurity_OWASP import CyberSecurityOWASPAction
+
+
+def build_cybersecurity_owasp_prompt(observation, action_trace, observation_trace) -> str:
+    return (
+        "You are a defensive AppSec repair agent. Output exactly one JSON action.\n"
+        f"Phase: {observation.phase}\n"
+        f"Task: {observation.task_brief}\n"
+        f"Available actions: {observation.available_actions}\n"
+        f"Last result: {observation.last_tool_result}\n"
+        'Example: {"tool_name":"read_file","arguments":{"path":"app/routes/invoices.py"}}'
+    )
+
+
+def parse_action_json(text: str) -> CyberSecurityOWASPAction:
+    data = json.loads(text)
+    return CyberSecurityOWASPAction(**data)
+
+
+def generate_rollout_completions(trainer, prompts: list[str]) -> list[dict[str, Any]]:
+    if hasattr(trainer, "generate_rollout_completions"):
+        return trainer.generate_rollout_completions(prompts)
+    return [
+        {
+            "text": '{"tool_name":"noop","arguments":{}}',
+            "prompt_ids": [],
+            "completion_ids": [],
+            "logprobs": [],
+        }
+        for _ in prompts
+    ]
+
+
+def rollout_once(trainer, env, tokenizer=None, dataset_prompt: str = "", max_steps: int = 40) -> dict:
+    result = env.reset()
+    observation = result.observation if hasattr(result, "observation") else result
+
+    prompt_ids = []
+    completion_ids = []
+    logprobs = []
+    reward_trace = []
+    action_trace = []
+    observation_trace = []
+
+    for _ in range(max_steps):
+        if getattr(observation, "done", False):
+            break
+        prompt = build_cybersecurity_owasp_prompt(observation, action_trace, observation_trace)
+        rollout_output = generate_rollout_completions(trainer, [prompt])[0]
+        action = parse_action_json(rollout_output["text"])
+        result = env.step(action)
+        observation = result.observation if hasattr(result, "observation") else result
+
+        prompt_ids.extend(rollout_output["prompt_ids"])
+        completion_ids.extend(rollout_output["completion_ids"])
+        logprobs.extend(rollout_output["logprobs"])
+        reward_trace.append(float(getattr(observation, "reward", 0.0) or 0.0))
+        action_trace.append(action.model_dump())
+        observation_trace.append(observation.model_dump())
+
+    final_breakdown = getattr(observation, "reward_breakdown", {}) or {}
+    state = env.state if not callable(getattr(env, "state", None)) else env.state()
+    return {
+        "prompt_ids": prompt_ids,
+        "completion_ids": completion_ids,
+        "logprobs": logprobs,
+        "reward_total": float(final_breakdown.get("total", sum(reward_trace))),
+        "reward_discovery": float(final_breakdown.get("discovery", 0.0)),
+        "reward_security": float(final_breakdown.get("security", 0.0)),
+        "reward_regression": float(final_breakdown.get("regression", 0.0)),
+        "reward_patch_quality": float(final_breakdown.get("patch_quality", 0.0)),
+        "reward_anti_cheat": float(final_breakdown.get("anti_cheat", 0.0)),
+        "success": bool(getattr(state, "success", False)),
+        "episode_length": len(action_trace),
+        "actions": action_trace,
+        "observations": observation_trace,
+    }

training/trackio_utils.py

@@ -0,0 +1,40 @@
+"""Trackio helpers used by training and evaluation scripts."""
+
+from __future__ import annotations
+
+from datetime import datetime
+
+
+TRAIN_METRICS = [
+    "train/reward_total_mean",
+    "train/reward_discovery_mean",
+    "train/reward_security_mean",
+    "train/reward_regression_mean",
+    "train/reward_public_routes_mean",
+    "train/reward_patch_quality_mean",
+    "train/reward_visible_tests_mean",
+    "train/reward_safety_mean",
+    "train/reward_anti_cheat_mean",
+    "train/success_rate",
+    "train/exploit_block_rate",
+    "train/regression_preservation_rate",
+    "train/public_route_preservation_rate",
+    "train/invalid_action_rate",
+    "train/timeout_rate",
+    "train/safety_violation_rate",
+    "train/reward_hacking_suspected_rate",
+    "train/episode_length_mean",
+    "train/episode_length_p95",
+    "train/rollouts_per_second",
+    "train/tokens_per_second",
+    "train/loss",
+    "train/learning_rate",
+    "train/kl",
+    "train/grad_norm",
+]
+
+
+def build_run_name(model: str, algo: str, difficulty: int, git_sha: str = "nogit") -> str:
+    stamp = datetime.utcnow().strftime("%Y%m%d-%H%M")
+    model_slug = model.replace("/", "-")
+    return f"CyberSecurity_OWASP-{model_slug}-{algo}-level{difficulty}-{stamp}-{git_sha[:8]}"

training/train_grpo.py

@@ -0,0 +1,46 @@
+"""Minimal GRPO training entrypoint scaffold.
+
+This file intentionally does not start training on import. It validates that the
+required TRL/Trackio configuration can be constructed when optional training
+dependencies are installed.
+"""
+
+from __future__ import annotations
+
+import os
+
+
+def build_grpo_config():
+    from trl import GRPOConfig
+
+    output_dir = os.getenv("OUTPUT_DIR", "CyberSecurity_OWASP-qwen3-1.7b-grpo")
+    trackio_space_id = os.getenv("TRACKIO_SPACE_ID", output_dir)
+    return GRPOConfig(
+        output_dir=output_dir,
+        report_to="trackio",
+        trackio_space_id=trackio_space_id,
+        logging_steps=1,
+        save_steps=25,
+        learning_rate=5e-6,
+        num_train_epochs=1,
+        per_device_train_batch_size=1,
+        gradient_accumulation_steps=32,
+        num_generations=2,
+        max_prompt_length=4096,
+        max_completion_length=768,
+        use_vllm=True,
+        vllm_mode="colocate",
+        vllm_gpu_memory_utilization=0.2,
+        gradient_checkpointing=True,
+        gradient_checkpointing_kwargs={"use_reentrant": False},
+        push_to_hub=False,
+    )
+
+
+def main():
+    config = build_grpo_config()
+    print(config)
+
+
+if __name__ == "__main__":
+    main()