Upload folder using huggingface_hub

README.md

@@ -21,7 +21,7 @@ Built for the Meta PyTorch OpenEnv Hackathon x Scaler School of Technology 2026.
 
 ## Infinite Unique Scenarios
 
-Unlike fixed-fixture environments where every episode presents the same scenario, this environment generates a unique broken request each episode. With 30 API spec templates across 6 domains and 10 error injection functions, the environment produces thousands of distinct training scenarios. An agent cannot memorize answers after one run - it must learn a generalizable debugging strategy. This is critical for real RL training value: agents learn transferable skills rather than dataset-specific shortcuts.
+Unlike fixed-fixture environments where every episode presents the same scenario, this environment generates a unique broken request each episode. With 45 API spec templates across 9 domains and 15 error injection functions (including chained multi-step errors), the environment produces tens of thousands of distinct training scenarios. An agent cannot memorize answers after one run - it must learn a generalizable debugging strategy. This is critical for real RL training value: agents learn transferable skills rather than dataset-specific shortcuts.
 
 ## Why This Domain
 
@@ -29,7 +29,7 @@ Developers spend significant time debugging API contract mismatches. Research fr
 
 ## How It Works
 
-1. On `reset()`, the environment picks a random API spec from 30 templates and injects 1-3 errors
+1. On `reset()`, the environment picks a random API spec from 45 templates and injects 1-3 errors
 2. The agent receives the broken request, headers, and the API specification
 3. The agent submits a fix attempt via `step()`
 4. The environment grades the attempt and returns structured feedback
@@ -42,7 +42,10 @@ Each episode allows multiple attempts. Perfect answers on early steps earn full
 | Task | Difficulty | Max Steps | Errors | Grading |
 |------|-----------|-----------|--------|---------|
 | easy | Identify error type and affected fields | 3 | 1 | Deterministic: 0.6 x type_match + 0.4 x fields_match |
+| classify | Identify ALL error types across multiple errors | 4 | 2-3 | Deterministic: 0.6 x Jaccard(types) + 0.4 x Jaccard(fields) |
 | medium | Fix the broken request | 5 | 1 | Deterministic: per-field validation against spec |
+| headers | Fix header-level errors (auth, content-type, tokens) | 4 | 1 | Deterministic: 0.7 x header_fix + 0.3 x type_match |
+| response | Validate API response for issues | 4 | 1-2 | Deterministic: 0.5 x Jaccard(issues) + 0.3 x Jaccard(fields) + 0.2 x status_code |
 | hard | Fix request + explain for developers | 7 | 2-3 | 70% deterministic fix + 30% LLM-as-judge explanation (gpt-4o-mini) |
 
 ## Error Types
@@ -59,8 +62,13 @@ Each episode allows multiple attempts. Perfect answers on early steps earn full
 | malformed_json_value | Corrupted field value | `{broken` as a value |
 | invalid_enum_value | Value not in allowed list | `currency: "xyz"` |
 | datetime_format_error | Wrong date format | `04/01/2026` instead of ISO 8601 |
+| wrong_content_type | Wrong Content-Type header | `text/plain` instead of `application/json` |
+| expired_auth_token | Expired or invalid auth token | `Bearer expired_token_2024` |
+| wrong_status_code | Wrong HTTP status code in response | `200` instead of `201` for resource creation |
+| redirect_loop | Redirect configuration error | Version upgrade redirect loop |
+| rate_limit_headers | Rate limit exceeded headers | `X-RateLimit-Remaining: 0` |
 
-## API Spec Domains (30 templates)
+## API Spec Domains (45 templates)
 
 | Domain | Count | Examples |
 |--------|-------|---------|
@@ -70,6 +78,9 @@ Each episode allows multiple attempts. Perfect answers on early steps earn full
 | Messaging (Twilio-like) | 5 | Send SMS, Send Email, Create Webhook |
 | E-Commerce | 5 | Create Order, Process Payment, Create Shipping Label |
 | Calendar and Auth | 5 | Create Event, OAuth Token, Create API Key |
+| Analytics/Monitoring | 5 | Create Dashboard, Add Metric, Create Alert |
+| DevOps/Infrastructure | 5 | Create Deployment, Scale Service, Create DNS Record |
+| AI/ML APIs | 5 | Submit Inference, Create Fine-tune Job, Upload Dataset |
 
 ## Action Space
 
@@ -77,11 +88,14 @@ The agent sends an `APIDebugAction` with these fields (all optional, submit what
 
 | Field | Type | Used In | Description |
 |-------|------|---------|-------------|
-| error_type | string | easy, hard | Diagnosed error type |
-| affected_fields | list[string] | easy, hard | Fields affected by the error |
+| error_type | string | easy, headers, hard | Diagnosed error type |
+| error_types | list[string] | classify | All diagnosed error types (multi-error) |
+| affected_fields | list[string] | easy, classify, response, hard | Fields affected by the error |
 | fixed_request | string (JSON) | medium, hard | Corrected request body |
-| fixed_headers | dict | medium, hard | Corrected HTTP headers |
+| fixed_headers | dict | medium, headers, hard | Corrected HTTP headers |
 | explanation | string | hard | Developer-facing explanation |
+| response_issues | list[string] | response | Issue types found in API response |
+| expected_status_code | int | response | Correct HTTP status code |
 
 ## Observation Space
 
@@ -89,13 +103,15 @@ The environment returns an `APIDebugObservation` with:
 
 | Field | Type | Description |
 |-------|------|-------------|
-| task | string | Current difficulty: easy, medium, hard |
+| task | string | Current task: easy, classify, medium, headers, response, hard |
 | api_name | string | Name of the API (e.g. "Create Customer") |
 | http_method | string | HTTP method of the request |
 | endpoint | string | API endpoint path |
 | broken_request | string (JSON) | The malformed request body |
 | broken_headers | dict | HTTP headers sent with the request |
 | api_spec | string (JSON) | API specification with required fields and types |
+| response_body | string (JSON) | Server response body (response task only) |
+| response_status_code | int | HTTP status code of response (response task only) |
 | error_count | int | Number of errors injected |
 | step_number | int | Current step in the episode |
 | max_steps | int | Maximum steps allowed |
@@ -138,12 +154,48 @@ Scores from running inference.py against the live HF Space (3 episodes per task,
 | Task | Episodes | Qwen2.5-72B-Instruct | gpt-4o-mini |
 |------|----------|----------------------|-------------|
 | easy | 3 | 0.999 | 0.667 |
+| classify | 3 | -- | -- |
 | medium | 3 | 0.999 | 0.999 |
+| headers | 3 | -- | -- |
+| response | 3 | -- | -- |
 | hard | 3 | 0.780 | 0.760 |
-| **overall** | **9** | **0.926** | **0.809** |
 
 Hard task uses LLM-as-judge (gpt-4o-mini) for explanation quality scoring, which is stricter than a heuristic baseline. The agent must fix 2-3 simultaneous errors and provide a developer-facing explanation to score high. Larger models perform better on the hard task, showing meaningful difficulty progression.
 
+## Chained Multi-Step Errors
+
+The hard task supports chained error scenarios where errors depend on each other. Fixing one error reveals the next, simulating real-world API debugging:
+
+| Chain Pattern | Gate Error | Body Errors |
+|---------------|-----------|-------------|
+| auth_gate | missing_auth_header, expired_auth_token | (any body error) |
+| content_type_gate | wrong_content_type | wrong_field_type, malformed_json_value, invalid_enum_value |
+| method_chain | wrong_http_method | missing_required_field, extra_unknown_field, null_value_in_required |
+| rate_limit_chain | rate_limit_headers | expired_auth_token, missing_required_field |
+| redirect_chain | redirect_loop | wrong_field_type, datetime_format_error, invalid_email_format |
+
+## GRPO Training with Curriculum Learning
+
+The `training/` directory contains a GRPO training script that trains a small LLM (Qwen 0.5B) using reward signals from the live environment:
+
+```bash
+pip install -r training/requirements.txt
+python training/train.py
+```
+
+The training auto-promotes through 6 difficulty levels based on rolling average reward:
+
+| Level | Task | Threshold | Max Turns |
+|-------|------|-----------|-----------|
+| 1 | easy | 0.7 | 3 |
+| 2 | classify | 0.6 | 4 |
+| 3 | medium | 0.6 | 5 |
+| 4 | headers | 0.5 | 4 |
+| 5 | response | 0.5 | 4 |
+| 6 | hard | -- | 7 |
+
+The environment also supports `task="auto"` which lets the environment itself manage curriculum progression based on session history.
+
 ## Setup
 
 ### Prerequisites
@@ -213,13 +265,19 @@ api-debug-env/
 ├── __init__.py
 ├── tests/
 │   ├── __init__.py
-│   └── test_environment.py  # 79 unit tests
+│   └── test_environment.py  # 109 unit tests
+├── training/
+│   ├── __init__.py
+│   ├── train.py            # GRPO training with 6-level curriculum
+│   ├── requirements.txt
+│   └── README.md
 └── server/
     ├── __init__.py
     ├── app.py              # FastAPI app via create_app()
-    ├── environment.py      # Core logic: reset(), step(), graders + LLM judge
-    ├── api_specs.py        # 30 API spec templates
-    ├── error_injectors.py  # 10 error injection functions
+    ├── environment.py      # Core logic: 6 tasks, graders, LLM judge, auto-curriculum
+    ├── api_specs.py        # 45 API spec templates across 9 domains
+    ├── error_injectors.py  # 15 error types + 5 chain patterns
+    ├── response_specs.py   # 8 response templates + 5 issue injection types
     └── validators.py       # Field type validation helpers
 ```
 

TECHNICAL_DEEP_DIVE.md

@@ -0,0 +1,642 @@
+# API Debug Environment - Technical Deep Dive
+
+A complete behind-the-scenes breakdown of the environment: what it does, how every piece works at the code level, what makes it unique, the bugs that almost killed the submission, and where it can go next.
+
+---
+
+## Table of Contents
+
+1. [What We Built](#what-we-built)
+2. [Terminology Reference](#terminology-reference)
+3. [Architecture Overview](#architecture-overview)
+4. [Code-Level Walkthrough](#code-level-walkthrough)
+5. [The Grading System](#the-grading-system)
+6. [Reward Shaping and Step Decay](#reward-shaping-and-step-decay)
+7. [LLM-as-Judge: How It Actually Works](#llm-as-judge-how-it-actually-works)
+8. [Error Injection System](#error-injection-system)
+9. [Infinite Unique Scenarios](#infinite-unique-scenarios)
+10. [The Bugs That Almost Killed It](#the-bugs-that-almost-killed-it)
+11. [Unique Features and Benefits](#unique-features-and-benefits)
+12. [Baseline Results](#baseline-results)
+13. [Scope for Advancement (Round 2)](#scope-for-advancement-round-2)
+14. [Practical Applications: Where the Trained LLM Sits](#practical-applications-where-the-trained-llm-sits)
+
+---
+
+## What We Built
+
+An RL (Reinforcement Learning) environment where an LLM agent learns to debug malformed API requests and validate API responses. The agent receives a broken HTTP request along with its API specification, and must handle 6 progressively harder tasks:
+
+- **Easy**: Identify the error type and affected fields (single error)
+- **Classify**: Identify ALL error types and fields across multiple simultaneous errors
+- **Medium**: Fix the broken request body to match the spec
+- **Headers**: Fix header-level errors (auth, content-type, expired tokens)
+- **Response**: Validate API responses for wrong status codes, missing fields, data leaks
+- **Hard**: Fix multi-error requests (including chained dependencies) and explain the fix
+
+Built on the **OpenEnv framework** (by Meta/PyTorch), with 45 API specs across 9 domains, 15 error types, 5 chained error patterns, GRPO training pipeline, and 6-level curriculum learning.
+
+---
+
+## Terminology Reference
+
+| Term | Meaning in this project |
+|------|------------------------|
+| **OpenEnv** | Meta/PyTorch framework for building RL environments. Provides `Environment` base class, `create_app()` for FastAPI, `EnvClient` for agents. |
+| **Episode** | One complete debugging session. Starts with `reset()`, ends when `done=True`. |
+| **Step** | One agent attempt within an episode. Agent submits an action, gets observation + reward + feedback. |
+| **Observation** | What the environment returns to the agent: broken request, spec, feedback, reward, done flag. |
+| **Action** | What the agent sends: error type, affected fields, fixed request, headers, explanation. |
+| **Ground truth** | The correct answer stored internally. Includes the original valid request, error type, and affected fields. Never exposed to the agent. |
+| **Reward** | Float in (0, 1) representing how good the agent's attempt was. Higher = better. |
+| **Step decay** | Multiplier that reduces reward for later steps: 1.0x at step 1, down to 0.3x floor at step 7+. |
+| **best_reward** | Highest reward achieved across all steps in an episode. Returned as final score. |
+| **Jaccard similarity** | Set similarity metric: size of intersection divided by size of union. Used for partial credit on field identification. |
+| **LLM-as-judge** | Using an LLM (gpt-4o-mini) to score the quality of the agent's explanation. Only for hard task. |
+| **Heuristic fallback** | Keyword + length scoring when the LLM judge is unavailable. Ensures hard task never blocks. |
+| **Reward clamping** | `max(0.001, min(0.999, reward))` - keeps rewards in open interval (0,1) because the evaluator rejects boundary values. |
+| **create_app()** | OpenEnv function that takes your Environment class and generates a FastAPI app with all required endpoints. |
+| **EnvClient** | OpenEnv SDK class that agents use to connect. Handles WebSocket, auto-connect via `_ensure_connected()`. |
+| **GRPO** | Group Relative Policy Optimization. An RL algorithm suitable for training LLMs with environment reward signals. |
+| **Deterministic grading** | Scoring that produces the same result every time for the same input. No randomness, no LLM calls. |
+| **Procedural generation** | Creating scenarios algorithmically (random spec + random error + random field) rather than from a fixed dataset. |
+| **HF Space** | HuggingFace Spaces - cloud deployment for the environment. Runs Docker container on port 7860. |
+| **Phase 1** | Automated check: Docker builds, `openenv validate` passes, health endpoint responds. |
+| **Phase 2** | Automated check: evaluator's agent runs against the environment, scores must be in (0, 1), no crashes. |
+| **Phase 3** | Human judges review environment quality, RL training value, code quality, documentation. |
+
+---
+
+## Architecture Overview
+
+```
+                    +--------------------+
+                    |   LLM Agent        |
+                    |  (Qwen2.5-72B /    |
+                    |   any model)       |
+                    +--------+-----------+
+                             |
+                     WebSocket / HTTP
+                             |
+                    +--------v-----------+
+                    |   OpenEnv SDK       |
+                    |   create_app()      |
+                    | POST /reset         |
+                    | POST /step          |
+                    | WS /ws             |
+                    | GET /health        |
+                    +--------+-----------+
+                             |
+              +--------------+---------------+
+              |              |               |
+    +---------v--+  +--------v---+  +--------v--------+
+    | api_specs  |  | error_     |  | environment.py  |
+    | .py        |  | injectors  |  | (core logic)    |
+    | 45 specs   |  | .py        |  | reset(), step() |
+    | 9 domains  |  | 15 types   |  | 6 graders       |
+    +------------+  +------------+  +---------+-------+
+                                              |
+                                    +---------v-------+
+                                    | validators.py   |
+                                    | Field type      |
+                                    | checking,       |
+                                    | spec validation |
+                                    +-----------------+
+```
+
+**Key files:**
+
+| File | Lines | Purpose |
+|------|-------|---------|
+| `server/environment.py` | 472 | Core environment: reset(), step(), 3 graders, LLM judge, reward clamping |
+| `server/api_specs.py` | 640 | 30 API spec templates across 6 domains |
+| `server/error_injectors.py` | 389 | 10 error injection functions + multi-error injector |
+| `server/validators.py` | 151 | 12 field type validators + request/header validation |
+| `server/app.py` | 84 | FastAPI app via OpenEnv's `create_app()` + `/tasks` endpoint |
+| `models.py` | 97 | Pydantic models: APIDebugAction (5 optional fields), APIDebugObservation (13 fields) |
+| `client.py` | 80 | EnvClient SDK: `_step_payload`, `_parse_result`, `_parse_state` |
+| `inference.py` | 331 | Baseline agent: LLM prompting, JSON parsing, episode runner, structured logging |
+| `tests/test_environment.py` | 579 | 79 unit tests covering all graders, edge cases, reward bounds |
+
+---
+
+## Code-Level Walkthrough
+
+### 1. Server Startup (`server/app.py`)
+
+```python
+app = create_app(
+    APIDebugEnvironment,    # Our environment class
+    APIDebugAction,         # What the agent sends
+    APIDebugObservation,    # What the environment returns
+    env_name="api_debug",
+    max_concurrent_envs=10, # Supports 10 parallel sessions
+)
+```
+
+`create_app()` is from OpenEnv SDK. It auto-generates all endpoints: `/reset`, `/step`, `/state`, `/schema`, `/ws`, `/health`. We just hand it our classes and it wires everything up. The WebSocket endpoint at `/ws` is what the evaluator's agent connects to.
+
+We also added a custom `/tasks` endpoint that returns all task configs, error types, and spec count. This helps external agents understand what the environment offers without reading docs.
+
+### 2. Episode Start: `reset()` (`server/environment.py:77-158`)
+
+When an agent calls `reset(task="medium")`:
+
+1. **Initialize RNG**: If a seed is provided, the episode is reproducible. Otherwise random.
+2. **Load task config**: `TASK_CONFIG["medium"]` gives `{"max_steps": 5, "error_count": 1}`
+3. **Pick random spec**: `get_random_spec(self.rng)` picks one of 30 API templates
+4. **Deep copy the valid example**: We never mutate the template itself
+5. **Inject errors**: For easy/medium, picks 1 random error type. For hard, picks 2-3 distinct error types using `rng.sample()` (no duplicates)
+6. **Build observation**: Returns the broken request, headers, API spec (field types + required fields), error count, and a message like "Debug this POST /v1/customers request. It contains 1 error(s). You have 5 steps."
+
+The agent never sees the ground truth. It only sees the broken request and the spec.
+
+### 3. Agent Action: `step()` (`server/environment.py:160-213`)
+
+When the agent submits a fix attempt:
+
+1. **Increment step counter**: Tracks which step we're on
+2. **Guard against stepping after done**: Returns 0 reward if episode already ended
+3. **Route to correct grader**: `_grade_easy()`, `_grade_medium()`, or `_grade_hard()` based on task
+4. **Apply step decay**: `multiplier = max(1.0 - 0.1 * (step - 1), 0.3)` - step 1 gets full reward, step 7+ gets 30% floor
+5. **Clamp reward**: `max(0.001, min(0.999, reward))` - open interval (0, 1) because the evaluator rejects exactly 0.0 or 1.0
+6. **Track best reward**: `self.best_reward = max(self.best_reward, reward)` - at episode end, returns the best reward across all steps
+7. **Check termination**: Episode ends if `raw_score >= 0.95` (near-perfect) or all steps exhausted
+
+### 4. Pydantic Models (`models.py`)
+
+```python
+class APIDebugAction(Action):
+    error_type: Optional[str]        # "missing_required_field"
+    affected_fields: Optional[List[str]]  # ["email"]
+    fixed_request: Optional[str]     # JSON string of corrected body
+    fixed_headers: Optional[Dict[str, str]]  # {"Authorization": "Bearer ..."}
+    explanation: Optional[str]       # Developer-facing text
+```
+
+All fields are Optional. The agent submits only what's needed for the current task:
+- Easy: `error_type` + `affected_fields`
+- Medium: `fixed_request` + `fixed_headers`
+- Hard: All five fields
+
+```python
+class APIDebugObservation(Observation):
+    task, api_name, http_method, endpoint,     # Context
+    broken_request, broken_headers, api_spec,  # The problem
+    error_count, step_number, max_steps,       # Episode state
+    feedback, message,                          # Grader output
+    done, reward                                # From Observation base
+```
+
+### 5. Client SDK (`client.py`)
+
+The client implements three abstract methods from `EnvClient`:
+
+- **`_step_payload(action)`**: Converts `APIDebugAction` to a JSON dict, only including non-None fields
+- **`_parse_result(payload)`**: Converts the server's JSON response into a `StepResult[APIDebugObservation]`
+- **`_parse_state(payload)`**: Converts server state to an `episode_id` + `step_count`
+
+The `EnvClient` base class handles WebSocket connection management. It has `_ensure_connected()` which auto-calls `connect()` if the WebSocket is not yet open. This means you can call `env.reset()` directly without explicitly opening a connection first.
+
+---
+
+## The Grading System
+
+### Easy Grader (`_grade_easy`, line 223)
+
+Fully deterministic. Two components:
+
+| Component | Weight | How it works |
+|-----------|--------|-------------|
+| Error type match | 0.6 | Exact string match against ground truth error type(s) |
+| Affected fields | 0.4 | Jaccard similarity: `|intersection| / |union|` of predicted vs actual fields |
+
+Jaccard similarity gives partial credit. If the ground truth is `["email", "name"]` and the agent says `["email", "phone"]`, the intersection is `{"email"}`, union is `{"email", "name", "phone"}`, Jaccard = 1/3, so fields score = 0.4 * 0.33 = 0.13.
+
+### Medium Grader (`_grade_medium`, line 264)
+
+Fully deterministic per-field validation:
+
+1. Parse the agent's `fixed_request` as JSON (fail = 0.0)
+2. Check every required field is present and non-null
+3. Check every present field has the correct type (using `validators.py`)
+4. Check no unknown fields are present
+5. Score = `passed_checks / total_checks`
+
+If the original error was `missing_auth_header`, headers are also validated (80% body + 20% headers blend).
+
+### Hard Grader (`_grade_hard`, line 307)
+
+70% deterministic + 30% LLM-judged:
+
+```python
+total = 0.7 * fix_score + 0.3 * explain_score
+```
+
+The fix portion reuses the medium grader exactly. The explanation goes through `_score_explanation()` which tries the LLM judge first, then falls back to a heuristic.
+
+---
+
+## Reward Shaping and Step Decay
+
+The reward formula:
+
+```
+reward = raw_score * max(1.0 - 0.1 * (step - 1), 0.3)
+```
+
+| Step | Multiplier | Effect |
+|------|-----------|--------|
+| 1 | 1.0x | Full reward for first-try solutions |
+| 2 | 0.9x | Slight penalty |
+| 3 | 0.8x | |
+| 4 | 0.7x | |
+| 5 | 0.6x | |
+| 6 | 0.5x | |
+| 7+ | 0.3x | Floor - agent still gets credit for late fixes |
+
+**Why this matters for RL training:**
+- The agent is incentivized to solve problems quickly (higher reward on step 1)
+- But it's not punished to zero for needing multiple attempts (0.3x floor)
+- The multi-turn feedback loop means the agent can learn from structured feedback between steps
+- At episode end, `best_reward` is returned - the best score across all attempts
+
+**Reward clamping** (`environment.py:194`):
+```python
+reward = max(0.001, min(0.999, reward))
+```
+
+This was added after Submission #3 failed. The evaluator's score range check requires strictly open interval (0, 1). Without this, a completely wrong answer scored 0.0 and a perfect first-step answer scored 1.0, both of which the evaluator rejected.
+
+---
+
+## LLM-as-Judge: How It Actually Works
+
+### The Judge Call (`_llm_judge_explanation`, line 349)
+
+The judge receives:
+- The API name, method, and endpoint
+- The **actual ground truth** errors (type + affected fields) - the judge knows the right answer
+- The agent's explanation text
+
+The judge scores on three weighted criteria:
+
+| Criterion | Max Score | What it evaluates |
+|-----------|----------|-------------------|
+| Root cause identification | 0.4 | Did the agent correctly name the error types and affected fields? |
+| Fix guidance | 0.3 | Does the explanation describe the correct remediation? |
+| Developer clarity | 0.3 | Is the explanation actionable and clear for a real developer? |
+
+The judge returns a single JSON object: `{"score": 0.85}`.
+
+**Key design decisions:**
+- **10-second timeout** (`timeout=10`): Prevents blocking `step()` if the LLM is slow. The grader must respond quickly.
+- **temperature=0.0**: Deterministic judge output for consistency
+- **max_tokens=50**: The judge only needs to return a score, not a long response
+
+### Heuristic Fallback (`_heuristic_score_explanation`, line 398)
+
+If the LLM judge fails (network error, timeout, bad response), we fall back to:
+
+```python
+keyword_score = min(keyword_hits / 6.0, 1.0)  # 23 debugging keywords
+length_score = based on len(explanation)        # Sweet spot: 50-500 chars
+final = 0.5 * keyword_score + 0.5 * length_score
+```
+
+Keywords include: "because", "should", "missing", "type", "format", "expected", "invalid", "authorization", "schema", "endpoint", "method", "payload", etc.
+
+This ensures the hard task never gets stuck. Even if the LLM judge is completely unavailable, agents still get meaningful (if less precise) scores for reasonable explanations.
+
+### How multiple valid paths are handled
+
+This is the question Alexa asked. The answer:
+
+- **For the fix portion (70%)**: Grading validates against the **spec**, not against a single golden answer. Any request that has all required fields with correct types and no unknown fields gets full credit. Two completely different valid fixes both score 1.0 on the fix portion.
+- **For the explanation (30%)**: The LLM judge evaluates whether the agent **identified the actual injected errors**, not whether it matched specific phrasing. An explanation that says "the email field was missing" and one that says "a required field (email) was not included in the request body" both get credit for root cause identification.
+
+---
+
+## Error Injection System
+
+### The 10 Error Types (`server/error_injectors.py`)
+
+Each injector is a pure function: `(request, headers, spec, rng) -> (broken_request, broken_headers, ground_truth)`
+
+| # | Error Type | What it does | Code location |
+|---|-----------|-------------|---------------|
+| 1 | `missing_required_field` | Removes a random required field from the request body | Line 38 |
+| 2 | `wrong_field_type` | Changes a field's value to the wrong type (int to string, etc.) | Line 62 |
+| 3 | `invalid_email_format` | Corrupts an email field (e.g. `user@` or `@domain.com`) | Line 103 |
+| 4 | `missing_auth_header` | Removes the Authorization header | Line 131 |
+| 5 | `extra_unknown_field` | Adds a field not in the spec (`debug_mode: true`, `_private`, etc.) | Line 159 |
+| 6 | `null_value_in_required` | Sets a required field to `null` | Line 185 |
+| 7 | `wrong_http_method` | Records that the wrong HTTP method was shown | Line 209 |
+| 8 | `malformed_json_value` | Replaces a field's value with broken JSON fragments (`{broken`, `NaN`) | Line 235 |
+| 9 | `invalid_enum_value` | Uses a value not in the allowed enum list | Line 270 |
+| 10 | `datetime_format_error` | Replaces ISO 8601 datetime with wrong format (`04/01/2026`) | Line 297 |
+
+### Multi-Error Injection (`inject_multiple_errors`, line 370)
+
+For hard tasks, we inject 2-3 errors simultaneously:
+
+```python
+chosen_types = rng.sample(ERROR_TYPES, min(count, len(ERROR_TYPES)))
+for err_type in chosen_types:
+    broken_req, broken_hdrs, gt = injector(broken_req, broken_hdrs, spec, rng)
+    all_truths.append(gt)
+```
+
+`rng.sample()` picks without replacement, so the agent never sees two of the same error type in one episode. Errors are applied sequentially to the same request, so they can compound (e.g., a field gets removed AND a type gets changed on another field).
+
+### Fallback Handling
+
+Some injectors need specific field types that might not exist in the chosen spec. For example, `inject_invalid_email_format` needs an email field. If the spec has no email fields, it falls back to `inject_missing_required_field` instead. Same for `inject_invalid_enum_value` (falls back to `inject_wrong_field_type`) and `inject_datetime_format_error`.
+
+---
+
+## Infinite Unique Scenarios
+
+The combinatorial space:
+
+- **30 API specs** across 6 domains
+- **10 error types** (each with random field selection within the spec)
+- **Multiple bad values per error type** (e.g., 5 bad email formats, 5 malformed JSON fragments, 5 bad datetime formats)
+- **Random field selection** within each spec (which required field gets removed, which gets type-changed)
+- **Hard mode**: 2-3 errors from different types, applied sequentially
+
+Conservative estimate: 30 specs x 10 error types x 3 field choices x 5 bad values = **4,500+ unique easy/medium scenarios**. Hard mode with combinations: significantly more.
+
+**Why this matters for RL**: An agent cannot memorize answers after one training run. It must learn a generalizable debugging strategy. If your environment has fixed scenarios, the agent overfits to those specific cases. With procedural generation, the agent has to learn the underlying skill.
+
+---
+
+## The Bugs That Almost Killed It
+
+### Submission #2: "inference.py raised an unhandled exception"
+
+**What happened**: The evaluator runs `inference.py` in their own runtime. Our script had two unprotected lines in `main()`:
+
+1. `OpenAI(api_key=None)` - When `HF_TOKEN` is not set in the evaluator's environment, `os.getenv("HF_TOKEN")` returns `None`. The OpenAI client constructor crashes immediately with `OpenAIError: The api_key client option must be set` when given `None`. This was confirmed by running it locally.
+
+2. `await APIDebugEnv.from_docker_image(IMAGE_NAME)` - If Docker isn't available or the image doesn't exist, this throws an unhandled exception.
+
+**Fix**:
+- Added fallback chain: `API_KEY = HF_TOKEN or os.getenv("OPENAI_API_KEY") or os.getenv("API_KEY")`
+- Wrapped `from_docker_image()` in try/except with fallback to direct URL connection
+- Added try/except around each episode so one failure doesn't crash the whole run
+
+**Misdiagnosis**: Another AI agent suggested the bug was a missing `async with` context manager for the WebSocket connection. This was wrong. We verified by reading the OpenEnv SDK source: `EnvClient._ensure_connected()` auto-calls `connect()` when the WebSocket is None. No explicit context manager needed.
+
+### Submission #3: "One or more task scores are out of range"
+
+**What happened**: The evaluator requires scores in the strictly open interval (0, 1). Our graders returned:
+- Exactly `0.0` for completely wrong answers (empty action, invalid JSON, etc.)
+- Exactly `1.0` for perfect first-step answers (1.0 raw score x 1.0 step multiplier)
+
+Both boundary values were rejected.
+
+**Fix**: Added reward clamping in `environment.py` step() method:
+```python
+reward = max(0.001, min(0.999, reward))
+```
+
+Also added score clamping in `inference.py`:
+```python
+score = min(max(score, 0.001), 0.999)
+```
+
+Updated 7 unit tests that previously asserted `== 0.0` or `== 1.0` to use `<= 0.01` and `>= 0.99`.
+
+---
+
+## Unique Features and Benefits
+
+### 1. Procedural Generation (Not Fixed Datasets)
+Most RL environments have a fixed set of scenarios. Our environment generates new scenarios every episode. 30 specs x 10 error types x random field selection = thousands of unique episodes. The agent genuinely learns a skill, not a lookup table.
+
+### 2. Multi-Turn with Structured Feedback
+The agent doesn't just get "right" or "wrong". It receives structured feedback like:
+```
+Validation: 3/5 checks passed.
+  email: PRESENT
+  name: MISSING
+  email type: VALID (email)
+  amount type: INVALID (expected integer)
+  debug_mode: UNKNOWN FIELD (not in spec)
+```
+This feedback loop lets the agent iterate and improve within an episode.
+
+### 3. Progressive Difficulty
+Three clearly separated difficulty levels:
+- Easy: Classification only (identify the error)
+- Medium: Fixing (produce correct JSON)
+- Hard: Fixing + reasoning (explain like a developer)
+
+Each level builds on the previous, and the hard task genuinely tests whether the agent can reason about what went wrong, not just pattern-match.
+
+### 4. Deterministic + LLM Hybrid Grading
+Easy and medium tasks are 100% deterministic. No variance, no API calls, reproducible results. Hard tasks are 70% deterministic (the fix portion) + 30% LLM-judged (explanation quality). This hybrid approach means:
+- Most of the score is stable and reproducible
+- The subjective part (explanation) uses an actual LLM to judge quality
+- If the LLM judge fails, a keyword heuristic ensures the system never blocks
+
+### 5. Reward Shaping for Efficient Problem-Solving
+Step decay encourages solving on the first try. The 0.3x floor means late fixes are still rewarded. `best_reward` tracking means the agent's best attempt is what counts.
+
+### 6. Real-World Domain Relevance
+API debugging is a real developer pain point. Calendar Gym research showed malformed tool arguments caused >50% of agent failures. This environment directly trains agents to handle that failure mode.
+
+### 7. Concurrent Session Support
+`SUPPORTS_CONCURRENT_SESSIONS = True` + `max_concurrent_envs=10` means the environment can train 10 agents in parallel on the same deployment. Each session has its own state.
+
+### 8. 79 Unit Tests (579 lines)
+Comprehensive test coverage:
+- All three graders (easy, medium, hard)
+- Step decay multiplier values at each step
+- Reward bounds (never < 0.001, never > 0.999)
+- Episode termination conditions
+- Seeded reproducibility
+- Edge cases (empty actions, invalid JSON, non-dict JSON, extra fields)
+- Best reward tracking
+- Heuristic explanation scorer
+
+---
+
+## Baseline Results
+
+Scores from running `inference.py` against the live HF Space (3 episodes per task):
+
+| Task | Episodes | Qwen2.5-72B-Instruct | gpt-4o-mini |
+|------|----------|----------------------|-------------|
+| easy | 3 | 0.999 | 0.667 |
+| medium | 3 | 0.999 | 0.999 |
+| hard | 3 | 0.780 | 0.760 |
+| **overall** | **9** | **0.926** | **0.809** |
+
+**Key takeaway**: Larger models perform better on hard tasks (explanation quality + multi-error fixing), showing meaningful difficulty progression. The environment is not trivially solvable but also not impossibly hard.
+
+---
+
+## Implemented Advancements (Round 2)
+
+All five advancement items from the original roadmap have been implemented:
+
+### 1. GRPO Training Pipeline (IMPLEMENTED)
+**What**: Full GRPO training loop in `training/train.py` that trains Qwen 0.5B on the live environment using TRL's GRPOTrainer with vLLM colocate mode.
+**How it works**: The model generates JSON debugging attempts, the environment grades them via its deterministic graders, and GRPO updates the policy to prefer higher-scoring responses. The rollout function connects to the live HF Space via WebSocket, runs multi-turn episodes, and returns prompt_ids, completion_ids, logprobs, and env_reward.
+**Key config**: `max_completion_length=128`, `gradient_accumulation_steps=16`, `vllm_gpu_memory_utilization=0.3`. Runs on free Colab T4 GPU.
+
+### 2. Expanded API Specs and Domains (IMPLEMENTED)
+**What**: Expanded from 30 specs / 6 domains to 45 specs / 9 domains.
+**New domains**: Analytics/Monitoring (dashboards, metrics, alerts), DevOps/Infrastructure (deployments, DNS, load balancers), AI/ML APIs (inference, fine-tuning, embeddings).
+**Impact**: 50% more scenario diversity for training generalization. Each domain uses realistic field types, headers, and validation rules.
+
+### 3. Chained Multi-Step Error Scenarios (IMPLEMENTED)
+**What**: 5 chain patterns where fixing one error reveals the next, simulating real-world API debugging.
+**Chain patterns**:
+- **auth_gate**: Missing/expired auth blocks body error visibility
+- **content_type_gate**: Wrong content type masks type/value errors
+- **method_chain**: Wrong HTTP method hides field-level errors
+- **rate_limit_chain**: Rate limit headers combined with auth/field errors
+- **redirect_chain**: Redirect loops combined with type/format errors
+
+**How it works**: `inject_chained_errors()` picks a random chain pattern, applies the gate error first, then injects body errors from the pattern's pool. The hard task uses chained errors 50% of the time.
+
+### 4. Response Validation Task (IMPLEMENTED)
+**What**: 6th task where the agent receives a request-response pair and identifies response issues.
+**Issue types**: wrong_status_code, missing_response_field, wrong_response_type, extra_response_field (data leak detection), inconsistent_error_format.
+**Grading**: 0.5 x Jaccard(issue_types) + 0.3 x Jaccard(affected_fields) + 0.2 x status_code_match.
+**8 response templates** covering: Create, List, Update, Delete, Batch, Authentication, File Upload, Search operations.
+
+### 5. Curriculum Learning (IMPLEMENTED)
+**What**: Both training-side and environment-side curriculum learning.
+**Training side** (`training/train.py`): 6-level curriculum that auto-promotes through easy -> classify -> medium -> headers -> response -> hard based on rolling average reward exceeding thresholds (0.7, 0.6, 0.6, 0.5, 0.5).
+**Environment side** (`task="auto"`): The environment itself tracks per-session reward history and auto-promotes, so any client can benefit from adaptive difficulty without implementing curriculum logic.
+
+### Scope for Further Advancement
+
+- **GraphQL and gRPC protocols**: Add non-REST API specs for cross-protocol debugging
+- **OAuth flow simulation**: Multi-step auth flows with token refresh, scope validation
+- **Response body fixing**: Agent generates the correct response body, not just identifies issues
+- **Multi-agent debugging**: Two agents collaborate on different aspects (headers vs body)
+- **Real-world API replay**: Import failed requests from production logs for training data
+
+---
+
+## Practical Applications: Where the Trained LLM Sits
+
+An LLM trained on this environment learns a specific skill: given a broken API request and a spec, diagnose the error, fix the request, and explain what went wrong. Here's how that skill translates to real-world developer tooling:
+
+### 1. IDE Integration (Copilot-Style API Debugger)
+
+```
+Developer writes code          Trained LLM              API Server
+       |                           |                        |
+       |--- makes API call ------->|                        |
+       |                           |--- forwards request -->|
+       |                           |<-- 400/422 error ------|
+       |                           |                        |
+       |                     [LLM analyzes request          |
+       |                      vs API spec, identifies       |
+       |                      error, generates fix]         |
+       |                           |                        |
+       |<-- "Your 'amount' field   |                        |
+       |    is a string but the    |                        |
+       |    API expects integer.   |                        |
+       |    Here's the fix: ..."   |                        |
+```
+
+**Where it sits**: As a VS Code / JetBrains extension plugin. Intercepts failed API calls in the developer's HTTP client (like Postman, Thunder Client, or `fetch`/`requests` in code), compares the request against known API specs, and suggests fixes inline.
+
+**Developer experience**: Developer hits "Send" on an API request, gets a 400 error. Instead of reading the error response and manually debugging, the extension pops up: "Missing required field `email`. The spec requires it as type `email`. Here's the corrected request." One click to apply the fix.
+
+### 2. API Gateway Middleware (Pre-Request Validation Layer)
+
+```
+Client App        API Gateway + Trained LLM           Backend API
+    |                      |                              |
+    |--- POST /v1/users -->|                              |
+    |    {bad request}     |                              |
+    |                      |-- [LLM validates against     |
+    |                      |   API schema before          |
+    |                      |   forwarding]                |
+    |                      |                              |
+    |<-- 422 + fix hint ---|                              |
+    |    "Field 'email'    |                              |
+    |     is malformed.    |                              |
+    |     Expected format: |                              |
+    |     user@domain.com" |                              |
+```
+
+**Where it sits**: As a middleware layer in an API gateway (Kong, AWS API Gateway, Nginx). Before the request reaches the backend, the LLM validates it against the spec and returns human-readable fix suggestions instead of cryptic validation errors.
+
+**Developer experience**: Instead of getting `{"error": "validation_error", "detail": [{"loc": ["body", "email"], "msg": "value is not a valid email address"}]}`, the developer gets: "The `email` field contains `user@` which is not a valid email. A valid email must have a domain (e.g., `user@example.com`). The `amount` field is a string but should be an integer. Send `2500` instead of `\"2500\"`."
+
+### 3. CI/CD Pipeline Integration (Contract Testing)
+
+```
+Developer pushes code
+        |
+        v
+CI Pipeline runs
+        |
+        v
+API Contract Tests (using trained LLM)
+        |
+        |--- Replays recent API calls against updated spec
+        |--- LLM identifies breaking changes
+        |--- Generates migration guide
+        |
+        v
+PR Comment: "3 API calls in your test suite
+will break with the new spec. Here are the fixes..."
+```
+
+**Where it sits**: As a CI step that runs after API spec changes. The trained LLM compares existing API calls (from test suites, logs, or recorded traffic) against the updated spec and flags what will break.
+
+**Developer experience**: Developer updates an API schema (adds a required field). The CI pipeline catches that 15 existing test calls are now invalid and generates the exact fix for each one.
+
+### 4. Production Error Analysis (Log-Based Debugging)
+
+```
+Production System           Error Aggregator          Trained LLM
+      |                          |                        |
+      |--- 400/422 errors ------>|                        |
+      |--- request logs -------->|                        |
+      |                          |--- batch analysis ---->|
+      |                          |                        |
+      |                          |<-- "Top 3 error        |
+      |                          |    patterns:           |
+      |                          |    1. 40% of failures  |
+      |                          |       are datetime     |
+      |                          |       format errors    |
+      |                          |       in /v1/events    |
+      |                          |    2. ..."             |
+```
+
+**Where it sits**: Connected to error aggregation tools (Sentry, Datadog, PagerDuty). Analyzes batches of 4xx errors, groups them by root cause, and suggests API spec improvements or client-side fixes.
+
+**Developer experience**: Oncall engineer gets a Slack alert: "87 new 422 errors on POST /v1/subscriptions in the last hour. Root cause: mobile clients sending `start_date` as `MM/DD/YYYY` instead of ISO 8601. Suggested fix: add format hint to error response, or accept both formats in the endpoint."
+
+### 5. SDK/Documentation Generator
+
+```
+API Spec (OpenAPI/Swagger)
+        |
+        v
+Trained LLM analyzes common error patterns
+        |
+        v
+Auto-generated:
+  - "Common mistakes" section per endpoint
+  - Request validation examples
+  - Error handling code snippets
+  - Migration guides between API versions
+```
+
+**Where it sits**: As part of the API documentation pipeline. The LLM, having been trained on thousands of debugging scenarios, knows which errors are most common per endpoint type and generates preventive documentation.
+
+### Key Insight
+
+The environment trains a skill that's useful at **every layer of the API stack** - from the developer's IDE to the API gateway to production monitoring. The core capability (understand spec, diagnose broken request, suggest fix) is the same; only the integration point changes. A model trained on this environment could power any of these tools, because the underlying reasoning is identical: compare request against spec, find the mismatch, produce the fix.

client.py

@@ -28,6 +28,8 @@ class APIDebugEnv(EnvClient[APIDebugAction, APIDebugObservation, State]):
         payload = {}
         if action.error_type is not None:
             payload["error_type"] = action.error_type
+        if action.error_types is not None:
+            payload["error_types"] = action.error_types
         if action.affected_fields is not None:
             payload["affected_fields"] = action.affected_fields
         if action.fixed_request is not None:
@@ -36,6 +38,10 @@ class APIDebugEnv(EnvClient[APIDebugAction, APIDebugObservation, State]):
             payload["fixed_headers"] = action.fixed_headers
         if action.explanation is not None:
             payload["explanation"] = action.explanation
+        if action.response_issues is not None:
+            payload["response_issues"] = action.response_issues
+        if action.expected_status_code is not None:
+            payload["expected_status_code"] = action.expected_status_code
         return payload
 
     def _parse_result(self, payload: Dict[str, Any]) -> StepResult[APIDebugObservation]:
@@ -60,6 +66,8 @@ class APIDebugEnv(EnvClient[APIDebugAction, APIDebugObservation, State]):
             error_count=obs_data.get("error_count", 1),
             step_number=obs_data.get("step_number", 0),
             max_steps=obs_data.get("max_steps", 3),
+            response_body=obs_data.get("response_body", ""),
+            response_status_code=obs_data.get("response_status_code", 0),
             feedback=obs_data.get("feedback", ""),
             message=obs_data.get("message", ""),
             done=payload.get("done", False),

inference.py

@@ -34,9 +34,9 @@ ENV_URL = os.getenv("ENV_URL") or "https://avichauhan-api-debug-env.hf.space"
 IMAGE_NAME = os.getenv("LOCAL_IMAGE_NAME")
 
 # Task configuration
-TASKS = ["easy", "medium", "hard"]
+TASKS = ["easy", "classify", "medium", "headers", "response", "hard"]
 EPISODES_PER_TASK = 3
-MAX_STEPS = {"easy": 3, "medium": 5, "hard": 7}
+MAX_STEPS = {"easy": 3, "classify": 4, "medium": 5, "headers": 4, "response": 4, "hard": 7}
 BENCHMARK_NAME = "api_debug"
 
 
@@ -87,7 +87,21 @@ SYSTEM_PROMPTS = {
         missing_required_field, wrong_field_type, invalid_email_format,
         missing_auth_header, extra_unknown_field, null_value_in_required,
         wrong_http_method, malformed_json_value, invalid_enum_value,
-        datetime_format_error
+        datetime_format_error, wrong_content_type, expired_auth_token
+    """).strip(),
+
+    "classify": textwrap.dedent("""
+        You are an API debugging expert. You receive a broken API request with MULTIPLE errors.
+        Your job: identify ALL error types and ALL affected fields.
+
+        Respond with ONLY a JSON object in this format:
+        {"error_types": ["type1", "type2"], "affected_fields": ["field1", "field2"]}
+
+        Valid error types:
+        missing_required_field, wrong_field_type, invalid_email_format,
+        missing_auth_header, extra_unknown_field, null_value_in_required,
+        wrong_http_method, malformed_json_value, invalid_enum_value,
+        datetime_format_error, wrong_content_type, expired_auth_token
     """).strip(),
 
     "medium": textwrap.dedent("""
@@ -100,6 +114,33 @@ SYSTEM_PROMPTS = {
         The fixed_request must be a valid JSON string. Include all required fields with correct types.
     """).strip(),
 
+    "headers": textwrap.dedent("""
+        You are an API debugging expert. You receive a broken API request with header-level errors.
+        Your job: identify the header error type and provide the corrected headers.
+
+        Respond with ONLY a JSON object in this format:
+        {"error_type": "<type>", "fixed_headers": {"Header-Name": "correct-value"}}
+
+        Valid header error types:
+        missing_auth_header, wrong_content_type, expired_auth_token
+
+        Common headers: Authorization (Bearer token), Content-Type (application/json)
+    """).strip(),
+
+    "response": textwrap.dedent("""
+        You are an API response validation expert. You receive an API request, its specification,
+        and the server's response. Your job: identify issues in the response.
+
+        Respond with ONLY a JSON object in this format:
+        {"response_issues": ["issue_type1", "issue_type2"], "affected_fields": ["field1"], "expected_status_code": 200}
+
+        Valid response issue types:
+        wrong_status_code, missing_response_field, wrong_response_type,
+        extra_response_field, inconsistent_error_format
+
+        Only include expected_status_code if you detect a wrong_status_code issue.
+    """).strip(),
+
     "hard": textwrap.dedent("""
         You are an API debugging expert. You receive a broken API request with multiple errors.
         Your job: diagnose the errors, fix the request, and explain the fix for a developer.
@@ -126,10 +167,14 @@ def build_user_prompt(obs, step_num: int) -> str:
         f"API: {obs.http_method} {obs.endpoint} ({obs.api_name})",
         f"Error count: {obs.error_count}",
         f"Step {step_num}/{obs.max_steps}",
-        f"\nBroken request body:\n{obs.broken_request}",
+        f"\nRequest body:\n{obs.broken_request}",
         f"\nRequest headers: {json.dumps(obs.broken_headers)}",
         f"\nAPI Specification:\n{obs.api_spec}",
     ]
+    # Include response data for response validation task
+    if obs.response_body:
+        parts.append(f"\nResponse status code: {obs.response_status_code}")
+        parts.append(f"\nResponse body:\n{obs.response_body}")
     if obs.feedback:
         parts.append(f"\nFeedback from previous attempt:\n{obs.feedback}")
     return "\n".join(parts)
@@ -182,10 +227,13 @@ def build_action(data: dict) -> APIDebugAction:
 
     return APIDebugAction(
         error_type=data.get("error_type"),
+        error_types=data.get("error_types"),
         affected_fields=data.get("affected_fields"),
         fixed_request=fixed_req,
         fixed_headers=data.get("fixed_headers"),
         explanation=data.get("explanation"),
+        response_issues=data.get("response_issues"),
+        expected_status_code=data.get("expected_status_code"),
     )
 
 
@@ -264,9 +312,18 @@ def _action_summary(action: APIDebugAction, task: str) -> str:
     """Short summary of the action for logging."""
     if task == "easy":
         return f"diagnose:{action.error_type or 'none'}"
+    elif task == "classify":
+        types = action.error_types or [action.error_type or "none"]
+        return f"classify:{','.join(str(t) for t in types)}"
     elif task == "medium":
         fix_len = len(action.fixed_request or "")
         return f"fix:len={fix_len}"
+    elif task == "headers":
+        hdr_count = len(action.fixed_headers or {})
+        return f"headers:{action.error_type or 'none'}+fix:{hdr_count}"
+    elif task == "response":
+        issues = action.response_issues or []
+        return f"response:{','.join(issues) or 'none'}+status:{action.expected_status_code or 'none'}"
     else:
         fix_len = len(action.fixed_request or "")
         exp_len = len(action.explanation or "")

models.py

@@ -22,6 +22,10 @@ class APIDebugAction(Action):
         default=None,
         description="Diagnosed error type, e.g. 'missing_required_field'"
     )
+    error_types: Optional[List[str]] = Field(
+        default=None,
+        description="All diagnosed error types (for classify task with multiple errors)"
+    )
     affected_fields: Optional[List[str]] = Field(
         default=None,
         description="List of field names affected by the error"
@@ -38,6 +42,14 @@ class APIDebugAction(Action):
         default=None,
         description="Developer-facing explanation of the fix (hard task only)"
     )
+    response_issues: Optional[List[str]] = Field(
+        default=None,
+        description="Issues found in the API response (response task only)"
+    )
+    expected_status_code: Optional[int] = Field(
+        default=None,
+        description="Correct HTTP status code for the response (response task only)"
+    )
 
 
 class APIDebugObservation(Observation):
@@ -48,7 +60,7 @@ class APIDebugObservation(Observation):
 
     task: str = Field(
         default="easy",
-        description="Current task difficulty: easy, medium, hard"
+        description="Current task: easy, classify, medium, headers, hard, response"
     )
     api_name: str = Field(
         default="",
@@ -86,6 +98,14 @@ class APIDebugObservation(Observation):
         default=3,
         description="Maximum steps allowed for this task"
     )
+    response_body: str = Field(
+        default="",
+        description="JSON string of the API response to validate (response task only)"
+    )
+    response_status_code: int = Field(
+        default=0,
+        description="HTTP status code of the response (response task only)"
+    )
     feedback: str = Field(
         default="",
         description="Structured validation feedback from the last action"

server/api_specs.py

@@ -623,7 +623,319 @@ CALENDAR_AUTH_SPECS = [
 ]
 
 
-# All 30 specs in a single flat list
+# =========================================================================
+# Domain 7: Analytics and Monitoring
+# =========================================================================
+
+ANALYTICS_SPECS = [
+    _spec(
+        api_name="Create Dashboard",
+        http_method="POST",
+        endpoint="/api/dashboards",
+        required_fields=["name", "workspace_id"],
+        optional_fields=["description", "layout", "shared"],
+        field_types={
+            "name": "string",
+            "workspace_id": "string",
+            "description": "string",
+            "layout": "enum:grid,freeform,list",
+            "shared": "boolean",
+        },
+        valid_example={
+            "name": "API Latency Overview",
+            "workspace_id": "ws_prod_001",
+        },
+    ),
+    _spec(
+        api_name="Add Metric",
+        http_method="POST",
+        endpoint="/api/metrics",
+        required_fields=["name", "type", "value"],
+        optional_fields=["tags", "timestamp", "unit"],
+        field_types={
+            "name": "string",
+            "type": "enum:counter,gauge,histogram,summary",
+            "value": "float",
+            "tags": "array",
+            "timestamp": "datetime",
+            "unit": "string",
+        },
+        valid_example={
+            "name": "api.request.duration",
+            "type": "histogram",
+            "value": 245.7,
+        },
+    ),
+    _spec(
+        api_name="Create Alert Rule",
+        http_method="POST",
+        endpoint="/api/alerts/rules",
+        required_fields=["name", "metric", "threshold", "condition"],
+        optional_fields=["description", "severity", "notification_channels"],
+        field_types={
+            "name": "string",
+            "metric": "string",
+            "threshold": "float",
+            "condition": "enum:above,below,equals",
+            "description": "string",
+            "severity": "enum:critical,warning,info",
+            "notification_channels": "array",
+        },
+        valid_example={
+            "name": "High Latency Alert",
+            "metric": "api.request.duration",
+            "threshold": 500.0,
+            "condition": "above",
+        },
+    ),
+    _spec(
+        api_name="Log Event",
+        http_method="POST",
+        endpoint="/api/logs",
+        required_fields=["level", "message", "service"],
+        optional_fields=["timestamp", "trace_id", "metadata"],
+        field_types={
+            "level": "enum:debug,info,warn,error,fatal",
+            "message": "string",
+            "service": "string",
+            "timestamp": "datetime",
+            "trace_id": "string",
+            "metadata": "object",
+        },
+        valid_example={
+            "level": "error",
+            "message": "Connection timeout to database",
+            "service": "payment-service",
+        },
+    ),
+    _spec(
+        api_name="Query Logs",
+        http_method="POST",
+        endpoint="/api/logs/search",
+        required_fields=["query", "start_time", "end_time"],
+        optional_fields=["limit", "service_filter", "level_filter"],
+        field_types={
+            "query": "string",
+            "start_time": "datetime",
+            "end_time": "datetime",
+            "limit": "integer",
+            "service_filter": "string",
+            "level_filter": "enum:debug,info,warn,error,fatal",
+        },
+        valid_example={
+            "query": "timeout OR connection refused",
+            "start_time": "2026-04-01T00:00:00Z",
+            "end_time": "2026-04-01T23:59:59Z",
+        },
+    ),
+]
+
+# =========================================================================
+# Domain 8: DevOps and Infrastructure
+# =========================================================================
+
+DEVOPS_SPECS = [
+    _spec(
+        api_name="Create Deployment",
+        http_method="POST",
+        endpoint="/api/deployments",
+        required_fields=["service_name", "image", "environment"],
+        optional_fields=["replicas", "cpu_limit", "memory_limit", "env_vars"],
+        field_types={
+            "service_name": "string",
+            "image": "string",
+            "environment": "enum:staging,production,development",
+            "replicas": "integer",
+            "cpu_limit": "string",
+            "memory_limit": "string",
+            "env_vars": "object",
+        },
+        valid_example={
+            "service_name": "api-gateway",
+            "image": "registry.io/api-gateway:v2.1.0",
+            "environment": "production",
+        },
+    ),
+    _spec(
+        api_name="Scale Service",
+        http_method="PATCH",
+        endpoint="/api/services/{service_id}/scale",
+        required_fields=["service_id", "replicas"],
+        optional_fields=["min_replicas", "max_replicas"],
+        field_types={
+            "service_id": "string",
+            "replicas": "integer",
+            "min_replicas": "integer",
+            "max_replicas": "integer",
+        },
+        valid_example={
+            "service_id": "svc_api_gateway",
+            "replicas": 5,
+        },
+    ),
+    _spec(
+        api_name="Create DNS Record",
+        http_method="POST",
+        endpoint="/api/dns/records",
+        required_fields=["domain", "type", "value"],
+        optional_fields=["ttl", "priority"],
+        field_types={
+            "domain": "string",
+            "type": "enum:A,AAAA,CNAME,MX,TXT,NS",
+            "value": "string",
+            "ttl": "integer",
+            "priority": "integer",
+        },
+        valid_example={
+            "domain": "api.example.com",
+            "type": "A",
+            "value": "203.0.113.50",
+        },
+    ),
+    _spec(
+        api_name="Add SSL Certificate",
+        http_method="POST",
+        endpoint="/api/certificates",
+        required_fields=["domain", "certificate", "private_key"],
+        optional_fields=["chain", "auto_renew"],
+        field_types={
+            "domain": "string",
+            "certificate": "string",
+            "private_key": "string",
+            "chain": "string",
+            "auto_renew": "boolean",
+        },
+        valid_example={
+            "domain": "api.example.com",
+            "certificate": "-----BEGIN CERTIFICATE-----\nMIIB...\n-----END CERTIFICATE-----",
+            "private_key": "-----BEGIN PRIVATE KEY-----\nMIIE...\n-----END PRIVATE KEY-----",
+        },
+    ),
+    _spec(
+        api_name="Create Load Balancer",
+        http_method="POST",
+        endpoint="/api/load-balancers",
+        required_fields=["name", "algorithm", "targets"],
+        optional_fields=["health_check_path", "health_check_interval", "sticky_sessions"],
+        field_types={
+            "name": "string",
+            "algorithm": "enum:round_robin,least_connections,ip_hash,weighted",
+            "targets": "array",
+            "health_check_path": "string",
+            "health_check_interval": "integer",
+            "sticky_sessions": "boolean",
+        },
+        valid_example={
+            "name": "api-lb-prod",
+            "algorithm": "round_robin",
+            "targets": [
+                {"host": "10.0.1.1", "port": 8080},
+                {"host": "10.0.1.2", "port": 8080},
+            ],
+        },
+    ),
+]
+
+# =========================================================================
+# Domain 9: AI/ML APIs
+# =========================================================================
+
+AI_ML_SPECS = [
+    _spec(
+        api_name="Submit Inference",
+        http_method="POST",
+        endpoint="/api/inference",
+        required_fields=["model_id", "inputs"],
+        optional_fields=["parameters", "stream", "timeout"],
+        field_types={
+            "model_id": "string",
+            "inputs": "string",
+            "parameters": "object",
+            "stream": "boolean",
+            "timeout": "integer",
+        },
+        valid_example={
+            "model_id": "meta-llama/Llama-3-8B-Instruct",
+            "inputs": "Explain reinforcement learning in one sentence.",
+        },
+    ),
+    _spec(
+        api_name="Create Fine-tune Job",
+        http_method="POST",
+        endpoint="/api/fine-tune",
+        required_fields=["base_model", "dataset_id", "num_epochs"],
+        optional_fields=["learning_rate", "batch_size", "validation_split"],
+        field_types={
+            "base_model": "string",
+            "dataset_id": "string",
+            "num_epochs": "integer",
+            "learning_rate": "float",
+            "batch_size": "integer",
+            "validation_split": "float",
+        },
+        valid_example={
+            "base_model": "Qwen/Qwen2.5-0.5B",
+            "dataset_id": "ds_api_debug_v1",
+            "num_epochs": 3,
+        },
+    ),
+    _spec(
+        api_name="Upload Dataset",
+        http_method="POST",
+        endpoint="/api/datasets",
+        required_fields=["name", "format", "source_url"],
+        optional_fields=["description", "license", "tags"],
+        field_types={
+            "name": "string",
+            "format": "enum:json,csv,parquet,arrow",
+            "source_url": "url",
+            "description": "string",
+            "license": "string",
+            "tags": "array",
+        },
+        valid_example={
+            "name": "api-debug-training-v1",
+            "format": "json",
+            "source_url": "https://storage.example.com/datasets/api_debug.json",
+        },
+    ),
+    _spec(
+        api_name="Create Embedding",
+        http_method="POST",
+        endpoint="/api/embeddings",
+        required_fields=["model_id", "input"],
+        optional_fields=["encoding_format", "dimensions"],
+        field_types={
+            "model_id": "string",
+            "input": "string",
+            "encoding_format": "enum:float,base64",
+            "dimensions": "integer",
+        },
+        valid_example={
+            "model_id": "BAAI/bge-small-en-v1.5",
+            "input": "API debugging is a critical developer skill.",
+        },
+    ),
+    _spec(
+        api_name="List Models",
+        http_method="GET",
+        endpoint="/api/models",
+        required_fields=["task"],
+        optional_fields=["library", "sort", "limit"],
+        field_types={
+            "task": "enum:text-generation,text-classification,embeddings,image-classification",
+            "library": "string",
+            "sort": "enum:downloads,likes,trending",
+            "limit": "integer",
+        },
+        valid_example={
+            "task": "text-generation",
+        },
+    ),
+]
+
+
+# All 45 specs in a single flat list
 ALL_SPECS = (
     PAYMENT_SPECS
     + USER_SPECS
@@ -631,6 +943,9 @@ ALL_SPECS = (
     + MESSAGING_SPECS
     + ECOMMERCE_SPECS
     + CALENDAR_AUTH_SPECS
+    + ANALYTICS_SPECS
+    + DEVOPS_SPECS
+    + AI_ML_SPECS
 )
 
 

server/app.py

@@ -36,6 +36,13 @@ def list_tasks():
                 "grading": "deterministic",
                 "description": "Identify the error type and affected fields",
             },
+            {
+                "name": "classify",
+                "max_steps": 4,
+                "error_count": "2-3",
+                "grading": "deterministic",
+                "description": "Identify ALL error types and affected fields across multiple errors",
+            },
             {
                 "name": "medium",
                 "max_steps": 5,
@@ -43,6 +50,20 @@ def list_tasks():
                 "grading": "deterministic",
                 "description": "Fix the broken request to match the API spec",
             },
+            {
+                "name": "headers",
+                "max_steps": 4,
+                "error_count": 1,
+                "grading": "deterministic",
+                "description": "Fix request headers (auth, content-type, tokens)",
+            },
+            {
+                "name": "response",
+                "max_steps": 4,
+                "error_count": "1-2",
+                "grading": "deterministic",
+                "description": "Validate API response: identify wrong status codes, missing fields, type errors, data leaks",
+            },
             {
                 "name": "hard",
                 "max_steps": 7,
@@ -51,6 +72,13 @@ def list_tasks():
                 "description": "Fix the request and explain the fix for developers",
             },
         ],
+        "response_issue_types": [
+            "wrong_status_code",
+            "missing_response_field",
+            "wrong_response_type",
+            "extra_response_field",
+            "inconsistent_error_format",
+        ],
         "error_types": [
             "missing_required_field",
             "wrong_field_type",
@@ -62,8 +90,13 @@ def list_tasks():
             "malformed_json_value",
             "invalid_enum_value",
             "datetime_format_error",
+            "wrong_content_type",
+            "expired_auth_token",
+            "wrong_status_code",
+            "redirect_loop",
+            "rate_limit_headers",
         ],
-        "api_spec_count": 30,
+        "api_spec_count": 45,
     })
 
 

server/environment.py

@@ -2,10 +2,11 @@
 Core environment for the API Debug Environment.
 
 Implements the OpenEnv Environment interface with:
-- 3 task difficulty levels (easy, medium, hard)
+- 5 task difficulty levels (easy, classify, medium, headers, hard)
 - Multi-turn episodes with structured feedback
-- Deterministic grading for easy/medium, LLM-as-judge for hard
+- Deterministic grading for easy/classify/medium/headers, LLM-as-judge for hard
 - Step reward decay to encourage efficient debugging
+- Auto-curriculum (task="auto") that promotes based on rolling reward
 """
 
 import copy
@@ -26,9 +27,12 @@ except ImportError:
 from .api_specs import get_random_spec
 from .error_injectors import (
     ERROR_TYPES,
+    HEADER_ERROR_TYPES,
+    inject_chained_errors,
     inject_error,
     inject_multiple_errors,
 )
+from .response_specs import get_random_response_template, inject_response_issues
 from .validators import (
     validate_field_type,
     validate_headers_against_spec,
@@ -39,8 +43,11 @@ from .validators import (
 # Task configuration: max steps and error count per difficulty
 TASK_CONFIG = {
     "easy": {"max_steps": 3, "error_count": 1},
+    "classify": {"max_steps": 4, "min_errors": 2, "max_errors": 3},
     "medium": {"max_steps": 5, "error_count": 1},
+    "headers": {"max_steps": 4, "error_count": 1},
     "hard": {"max_steps": 7, "min_errors": 2, "max_errors": 3},
+    "response": {"max_steps": 4, "min_issues": 1, "max_issues": 2},
 }
 
 
@@ -58,6 +65,18 @@ class APIDebugEnvironment(Environment):
 
     SUPPORTS_CONCURRENT_SESSIONS: bool = True
 
+    # Curriculum thresholds for task="auto" mode
+    # When rolling avg reward exceeds threshold, promote to next task
+    AUTO_CURRICULUM = {
+        "easy":     {"next": "classify", "threshold": 0.7},
+        "classify": {"next": "medium",   "threshold": 0.6},
+        "medium":   {"next": "headers",  "threshold": 0.6},
+        "headers":  {"next": "response", "threshold": 0.5},
+        "response": {"next": "hard",     "threshold": 0.5},
+        "hard":     {"next": None,       "threshold": None},
+    }
+    AUTO_WINDOW = 10
+
     def __init__(self):
         super().__init__()
         self._state = State(episode_id=str(uuid4()), step_count=0)
@@ -73,6 +92,13 @@ class APIDebugEnvironment(Environment):
         self.rng = random.Random()
         # For wrong_http_method error: the method shown to the agent
         self.shown_http_method = ""
+        # Response task state
+        self.response_body: Dict[str, Any] = {}
+        self.response_status_code: int = 0
+        self.response_template: Dict[str, Any] = {}
+        # Curriculum state for task="auto"
+        self._auto_task = "easy"
+        self._auto_rewards: List[float] = []
 
     def reset(
         self,
@@ -86,7 +112,7 @@ class APIDebugEnvironment(Environment):
         Args:
             seed: Random seed for reproducible episodes.
             episode_id: Custom episode identifier.
-            task: Difficulty level (easy, medium, hard).
+            task: Difficulty level (easy, classify, medium, headers, hard, auto).
         """
         # Initialize RNG
         if seed is not None:
@@ -94,8 +120,11 @@ class APIDebugEnvironment(Environment):
         else:
             self.rng = random.Random()
 
-        # Validate task
-        self.task = task if task in TASK_CONFIG else "easy"
+        # Validate task -- "auto" uses curriculum to pick difficulty
+        if task == "auto":
+            self.task = self._auto_task
+        else:
+            self.task = task if task in TASK_CONFIG else "easy"
         config = TASK_CONFIG[self.task]
         self.max_steps = config["max_steps"]
         self.current_step = 0
@@ -113,14 +142,70 @@ class APIDebugEnvironment(Environment):
         valid_request = copy.deepcopy(self.spec["valid_example"])
         valid_headers = copy.deepcopy(self.spec["required_headers"])
 
+        # Response task has a completely different setup: broken response, not request
+        if self.task == "response":
+            issue_count = self.rng.randint(config["min_issues"], config["max_issues"])
+            self.response_template = get_random_response_template(self.rng)
+            self.response_body, self.response_status_code, self.ground_truths = (
+                inject_response_issues(self.response_template, self.rng, issue_count)
+            )
+            # For response task, the request is correct -- agent examines the response
+            self.broken_request = valid_request
+            self.broken_headers = valid_headers
+            self.shown_http_method = self.spec["http_method"]
+            error_count = len(self.ground_truths)
+            return APIDebugObservation(
+                task=self.task,
+                api_name=self.spec["api_name"],
+                http_method=self.shown_http_method,
+                endpoint=self.spec["endpoint"],
+                broken_request=json.dumps(self.broken_request, indent=2),
+                broken_headers=self.broken_headers,
+                api_spec=self._build_spec_string(),
+                response_body=json.dumps(self.response_body, indent=2),
+                response_status_code=self.response_status_code,
+                error_count=error_count,
+                step_number=0,
+                max_steps=self.max_steps,
+                feedback="",
+                message=(
+                    f"Validate the response from {self.shown_http_method} {self.spec['endpoint']}. "
+                    f"The response has {error_count} issue(s). "
+                    f"You have {self.max_steps} steps."
+                ),
+                done=False,
+                reward=0.0,
+            )
+
         # Inject errors based on difficulty
         if self.task == "hard":
+            error_count = self.rng.randint(config["min_errors"], config["max_errors"])
+            # 50% chance of chained errors (header gate + body errors)
+            if self.rng.random() < 0.5:
+                self.broken_request, self.broken_headers, self.ground_truths = (
+                    inject_chained_errors(
+                        valid_request, valid_headers, self.spec, self.rng, error_count
+                    )
+                )
+            else:
+                self.broken_request, self.broken_headers, self.ground_truths = (
+                    inject_multiple_errors(
+                        valid_request, valid_headers, self.spec, self.rng, error_count
+                    )
+                )
+        elif self.task == "classify":
             error_count = self.rng.randint(config["min_errors"], config["max_errors"])
             self.broken_request, self.broken_headers, self.ground_truths = (
                 inject_multiple_errors(
                     valid_request, valid_headers, self.spec, self.rng, error_count
                 )
             )
+        elif self.task == "headers":
+            error_type = self.rng.choice(HEADER_ERROR_TYPES)
+            self.broken_request, self.broken_headers, gt = inject_error(
+                error_type, valid_request, valid_headers, self.spec, self.rng
+            )
+            self.ground_truths = [gt]
         else:
             error_type = self.rng.choice(ERROR_TYPES)
             self.broken_request, self.broken_headers, gt = inject_error(
@@ -181,8 +266,14 @@ class APIDebugEnvironment(Environment):
         # Grade based on task type
         if self.task == "easy":
             raw_score, feedback = self._grade_easy(action)
+        elif self.task == "classify":
+            raw_score, feedback = self._grade_classify(action)
         elif self.task == "medium":
             raw_score, feedback = self._grade_medium(action)
+        elif self.task == "headers":
+            raw_score, feedback = self._grade_headers(action)
+        elif self.task == "response":
+            raw_score, feedback = self._grade_response(action)
         else:
             raw_score, feedback = self._grade_hard(action)
 
@@ -205,6 +296,9 @@ class APIDebugEnvironment(Environment):
             self.episode_done = True
             # Return best reward achieved during the episode
             reward = self.best_reward
+            # Track for auto-curriculum promotion
+            self._auto_rewards.append(reward)
+            self._maybe_auto_promote()
 
         return self._make_observation(
             feedback=feedback,
@@ -216,6 +310,18 @@ class APIDebugEnvironment(Environment):
     def state(self) -> State:
         return self._state
 
+    def _maybe_auto_promote(self):
+        """Check if auto-curriculum should promote to next difficulty."""
+        config = self.AUTO_CURRICULUM.get(self._auto_task)
+        if not config or config["next"] is None or config["threshold"] is None:
+            return
+        if len(self._auto_rewards) < self.AUTO_WINDOW:
+            return
+        avg = sum(self._auto_rewards[-self.AUTO_WINDOW:]) / self.AUTO_WINDOW
+        if avg >= config["threshold"]:
+            self._auto_task = config["next"]
+            self._auto_rewards.clear()
+
     # =====================================================================
     # Grading methods
     # =====================================================================
@@ -261,6 +367,61 @@ class APIDebugEnvironment(Environment):
 
         return round(score, 4), "; ".join(parts)
 
+    def _grade_classify(self, action: APIDebugAction) -> Tuple[float, str]:
+        """Grade multi-error classification. Fully deterministic.
+
+        Like easy but the agent must identify ALL error types and ALL
+        affected fields across multiple injected errors.
+
+        Scoring: 0.6 for error types (Jaccard) + 0.4 for affected fields (Jaccard).
+        Accepts either error_types (list) or error_type (single) from the agent.
+        """
+        score = 0.0
+        parts = []
+
+        gt_types = {gt["error_type"] for gt in self.ground_truths}
+        gt_fields: set = set()
+        for gt in self.ground_truths:
+            gt_fields.update(gt.get("affected_fields", []))
+
+        # Accept error_types (list) or fall back to error_type (single)
+        agent_types = set(action.error_types or [])
+        if not agent_types and action.error_type:
+            agent_types = {action.error_type}
+
+        # Error types Jaccard (0.6 weight)
+        if gt_types and agent_types:
+            intersection = gt_types & agent_types
+            union = gt_types | agent_types
+            jaccard = len(intersection) / len(union) if union else 0.0
+            score += 0.6 * jaccard
+            parts.append(
+                f"error_types: {len(intersection)}/{len(gt_types)} correct, "
+                f"{len(agent_types - gt_types)} extra"
+            )
+        elif not agent_types:
+            parts.append("error_types: MISSING (none provided)")
+        else:
+            parts.append("error_types: INCORRECT (0 matches)")
+
+        # Affected fields Jaccard (0.4 weight)
+        agent_fields = set(action.affected_fields or [])
+        if gt_fields and agent_fields:
+            intersection = gt_fields & agent_fields
+            union = gt_fields | agent_fields
+            jaccard = len(intersection) / len(union) if union else 0.0
+            score += 0.4 * jaccard
+            parts.append(
+                f"affected_fields: {len(intersection)}/{len(gt_fields)} correct, "
+                f"{len(agent_fields - gt_fields)} extra"
+            )
+        elif not agent_fields:
+            parts.append("affected_fields: MISSING (none provided)")
+        else:
+            parts.append("affected_fields: INCORRECT (0 matches)")
+
+        return round(score, 4), "; ".join(parts)
+
     def _grade_medium(self, action: APIDebugAction) -> Tuple[float, str]:
         """Grade request fix. Fully deterministic per-field validation.
 
@@ -304,6 +465,97 @@ class APIDebugEnvironment(Environment):
 
         return round(total_score, 4), feedback
 
+    def _grade_headers(self, action: APIDebugAction) -> Tuple[float, str]:
+        """Grade header fix. Fully deterministic.
+
+        The agent must provide corrected headers that match the spec.
+        Also awards partial credit for identifying the error type.
+
+        Scoring: 0.7 for correct headers + 0.3 for error type identification.
+        """
+        score = 0.0
+        parts = []
+
+        # Error type identification (0.3 weight)
+        gt_types = {gt["error_type"] for gt in self.ground_truths}
+        if action.error_type and action.error_type in gt_types:
+            score += 0.3
+            parts.append("error_type: CORRECT")
+        else:
+            given = action.error_type or "(none)"
+            parts.append(f"error_type: INCORRECT (you said '{given}')")
+
+        # Header fix validation (0.7 weight)
+        if action.fixed_headers:
+            header_score, header_feedback = validate_headers_against_spec(
+                action.fixed_headers, self.spec
+            )
+            score += 0.7 * header_score
+            parts.append(header_feedback)
+        else:
+            parts.append("Headers: NOT PROVIDED (header fix needed)")
+
+        return round(score, 4), "; ".join(parts)
+
+    def _grade_response(self, action: APIDebugAction) -> Tuple[float, str]:
+        """Grade response validation. Fully deterministic.
+
+        Agent must identify issue types and, for wrong_status_code, provide
+        the correct status code.
+
+        Scoring: 0.5 for issue type identification (Jaccard) +
+                 0.3 for affected field identification (Jaccard) +
+                 0.2 for correct status code (if applicable).
+        """
+        score = 0.0
+        parts = []
+
+        gt_issue_types = {gt["issue_type"] for gt in self.ground_truths}
+        gt_fields = {gt.get("affected_field", "") for gt in self.ground_truths} - {""}
+
+        # Issue type identification (0.5 weight)
+        predicted_issues = set(action.response_issues or [])
+        if predicted_issues and gt_issue_types:
+            intersection = predicted_issues & gt_issue_types
+            union = predicted_issues | gt_issue_types
+            jaccard = len(intersection) / len(union) if union else 0.0
+            score += 0.5 * jaccard
+            parts.append(f"Issue types: {len(intersection)}/{len(gt_issue_types)} correct (Jaccard={jaccard:.2f})")
+        else:
+            parts.append("Issue types: NOT PROVIDED" if not predicted_issues else "Issue types: NONE CORRECT")
+
+        # Affected field identification via error_type or affected_fields (0.3 weight)
+        predicted_fields = set(action.affected_fields or [])
+        if predicted_fields and gt_fields:
+            intersection = predicted_fields & gt_fields
+            union = predicted_fields | gt_fields
+            jaccard = len(intersection) / len(union) if union else 0.0
+            score += 0.3 * jaccard
+            parts.append(f"Affected fields: {len(intersection)}/{len(gt_fields)} correct")
+        else:
+            parts.append("Affected fields: NOT PROVIDED" if not predicted_fields else "Affected fields: NONE CORRECT")
+
+        # Status code check (0.2 weight) -- only if wrong_status_code is a ground truth
+        has_status_issue = any(gt["issue_type"] == "wrong_status_code" for gt in self.ground_truths)
+        if has_status_issue:
+            correct_status = None
+            for gt in self.ground_truths:
+                if gt["issue_type"] == "wrong_status_code":
+                    correct_status = int(gt.get("correct_value", 0))
+                    break
+            if action.expected_status_code and action.expected_status_code == correct_status:
+                score += 0.2
+                parts.append(f"Status code: CORRECT ({correct_status})")
+            else:
+                given = action.expected_status_code or "(none)"
+                parts.append(f"Status code: INCORRECT (you said {given}, expected {correct_status})")
+        else:
+            # No status code issue -- redistribute 0.2 to issue types
+            score += 0.2 * (len(predicted_issues & gt_issue_types) / len(gt_issue_types) if gt_issue_types else 0.0)
+            parts.append("Status code: N/A (no status code issue)")
+
+        return round(score, 4), "; ".join(parts)
+
     def _grade_hard(self, action: APIDebugAction) -> Tuple[float, str]:
         """Grade fix + explanation. 70% deterministic fix, 30% explanation.
 
@@ -453,7 +705,7 @@ class APIDebugEnvironment(Environment):
             remaining = self.max_steps - self.current_step
             msg = f"{remaining} step(s) remaining. Use the feedback to improve."
 
-        return APIDebugObservation(
+        obs = APIDebugObservation(
             task=self.task,
             api_name=self.spec.get("api_name", ""),
             http_method=self.shown_http_method,
@@ -469,3 +721,8 @@ class APIDebugEnvironment(Environment):
             done=done,
             reward=reward,
         )
+        # Include response data for response task
+        if self.task == "response":
+            obs.response_body = json.dumps(self.response_body, indent=2)
+            obs.response_status_code = self.response_status_code
+        return obs

server/error_injectors.py

@@ -324,10 +324,178 @@ def inject_datetime_format_error(
     )
 
 
+# =========================================================================
+# 11. wrong_content_type
+# =========================================================================
+
+def inject_wrong_content_type(
+    request: Dict[str, Any],
+    headers: Dict[str, str],
+    spec: Dict[str, Any],
+    rng: random_module.Random,
+) -> InjectorResult:
+    """Change Content-Type to an incorrect value."""
+    broken_headers = copy.deepcopy(headers)
+    wrong_types = [
+        "text/plain",
+        "application/xml",
+        "multipart/form-data",
+        "text/html",
+        "application/x-www-form-urlencoded",
+    ]
+    if "Content-Type" in broken_headers:
+        broken_headers["Content-Type"] = rng.choice(wrong_types)
+    else:
+        broken_headers["Content-Type"] = rng.choice(wrong_types)
+    return request, broken_headers, _ground_truth(
+        "wrong_content_type", ["Content-Type"], request, headers
+    )
+
+
+# =========================================================================
+# 12. expired_auth_token
+# =========================================================================
+
+def inject_expired_token(
+    request: Dict[str, Any],
+    headers: Dict[str, str],
+    spec: Dict[str, Any],
+    rng: random_module.Random,
+) -> InjectorResult:
+    """Replace the Authorization token with an expired/malformed one."""
+    broken_headers = copy.deepcopy(headers)
+    bad_tokens = [
+        "Bearer expired_token_abc123",
+        "Bearer ",
+        "Basic dXNlcjpwYXNz",
+        "Token invalid",
+        "Bearer eyJhbGciOiJub25lIn0.e30.",
+    ]
+    if "Authorization" in broken_headers:
+        broken_headers["Authorization"] = rng.choice(bad_tokens)
+        return request, broken_headers, _ground_truth(
+            "expired_auth_token", ["Authorization"], request, headers
+        )
+    # If no auth header in spec, inject wrong content type instead
+    return inject_wrong_content_type(request, headers, spec, rng)
+
+
+# =========================================================================
+# 13. wrong_status_code (for response validation / chained scenarios)
+# =========================================================================
+
+def inject_wrong_status_code(
+    request: Dict[str, Any],
+    headers: Dict[str, str],
+    spec: Dict[str, Any],
+    rng: random_module.Random,
+) -> InjectorResult:
+    """Record that the wrong HTTP status code would be returned.
+
+    Simulates a server returning an unexpected status code.
+    The ground truth stores the wrong code and the expected code.
+    """
+    correct_status = 200 if spec["http_method"] == "GET" else 201
+    wrong_codes = [
+        (301, "Moved Permanently - resource redirected"),
+        (302, "Found - temporary redirect to different endpoint"),
+        (400, "Bad Request - but request is actually valid"),
+        (403, "Forbidden - incorrect permissions applied"),
+        (404, "Not Found - wrong endpoint routing"),
+        (429, "Too Many Requests - rate limit misconfigured"),
+        (500, "Internal Server Error - server-side issue"),
+        (502, "Bad Gateway - upstream service down"),
+        (503, "Service Unavailable - maintenance mode"),
+    ]
+    wrong_status, description = rng.choice(wrong_codes)
+    gt = _ground_truth("wrong_status_code", ["status_code"], request, headers)
+    gt["wrong_status"] = wrong_status
+    gt["correct_status"] = correct_status
+    gt["description"] = description
+    return request, headers, gt
+
+
+# =========================================================================
+# 14. redirect_loop
+# =========================================================================
+
+def inject_redirect_loop(
+    request: Dict[str, Any],
+    headers: Dict[str, str],
+    spec: Dict[str, Any],
+    rng: random_module.Random,
+) -> InjectorResult:
+    """Simulate a redirect chain issue.
+
+    The agent must identify that the endpoint redirects and provide
+    the correct target endpoint.
+    """
+    redirect_scenarios = [
+        {
+            "from": spec["endpoint"],
+            "to": spec["endpoint"].rstrip("/") + "/v2",
+            "reason": "API version upgrade - v1 redirects to v2",
+        },
+        {
+            "from": spec["endpoint"],
+            "to": spec["endpoint"].replace("/api/", "/api/v2/"),
+            "reason": "Base path migration",
+        },
+        {
+            "from": spec["endpoint"],
+            "to": spec["endpoint"] + "?format=json",
+            "reason": "Content negotiation redirect",
+        },
+    ]
+    scenario = rng.choice(redirect_scenarios)
+    gt = _ground_truth("redirect_loop", ["endpoint"], request, headers)
+    gt["redirect_from"] = scenario["from"]
+    gt["redirect_to"] = scenario["to"]
+    gt["reason"] = scenario["reason"]
+    return request, headers, gt
+
+
+# =========================================================================
+# 15. rate_limit_headers
+# =========================================================================
+
+def inject_rate_limit_headers(
+    request: Dict[str, Any],
+    headers: Dict[str, str],
+    spec: Dict[str, Any],
+    rng: random_module.Random,
+) -> InjectorResult:
+    """Inject missing or wrong rate limit headers.
+
+    Real APIs require headers like X-RateLimit-Limit, Retry-After.
+    The agent must identify the rate limiting issue and provide correct headers.
+    """
+    broken_headers = copy.deepcopy(headers)
+    # Add rate limit headers that indicate the client is being throttled
+    broken_headers["X-RateLimit-Remaining"] = "0"
+    broken_headers["X-RateLimit-Reset"] = "1712000000"
+    broken_headers["Retry-After"] = "60"
+
+    gt = _ground_truth(
+        "rate_limit_headers",
+        ["X-RateLimit-Remaining", "Retry-After"],
+        request, headers,
+    )
+    gt["issue"] = "Client is rate-limited, must wait or reduce request frequency"
+    return request, broken_headers, gt
+
+
 # =========================================================================
 # Registry and helpers
 # =========================================================================
 
+# Header-only error types (used by the headers task)
+HEADER_ERROR_TYPES = [
+    "missing_auth_header",
+    "wrong_content_type",
+    "expired_auth_token",
+]
+
 ERROR_TYPES = [
     "missing_required_field",
     "wrong_field_type",
@@ -339,6 +507,11 @@ ERROR_TYPES = [
     "malformed_json_value",
     "invalid_enum_value",
     "datetime_format_error",
+    "wrong_content_type",
+    "expired_auth_token",
+    "wrong_status_code",
+    "redirect_loop",
+    "rate_limit_headers",
 ]
 
 INJECTOR_MAP = {
@@ -352,6 +525,11 @@ INJECTOR_MAP = {
     "malformed_json_value": inject_malformed_json_value,
     "invalid_enum_value": inject_invalid_enum_value,
     "datetime_format_error": inject_datetime_format_error,
+    "wrong_content_type": inject_wrong_content_type,
+    "expired_auth_token": inject_expired_token,
+    "wrong_status_code": inject_wrong_status_code,
+    "redirect_loop": inject_redirect_loop,
+    "rate_limit_headers": inject_rate_limit_headers,
 }
 
 
@@ -367,6 +545,89 @@ def inject_error(
     return injector(request, headers, spec, rng)
 
 
+# Chain patterns for realistic multi-step debugging scenarios
+CHAIN_PATTERNS = [
+    # Pattern 1: Auth gate -> body errors
+    # Real-world: API returns 401 first, body validation only runs after auth passes
+    {
+        "name": "auth_gate",
+        "gate_types": ["missing_auth_header", "expired_auth_token"],
+        "body_pool": None,  # uses all body types
+    },
+    # Pattern 2: Content-type gate -> type mismatches
+    # Real-world: Wrong Content-Type causes parser to misinterpret the body
+    {
+        "name": "content_type_gate",
+        "gate_types": ["wrong_content_type"],
+        "body_pool": ["wrong_field_type", "malformed_json_value", "invalid_enum_value"],
+    },
+    # Pattern 3: Method + endpoint chain
+    # Real-world: Wrong method returns 405, then wrong fields for the correct method
+    {
+        "name": "method_chain",
+        "gate_types": ["wrong_http_method"],
+        "body_pool": ["missing_required_field", "extra_unknown_field", "null_value_in_required"],
+    },
+    # Pattern 4: Rate limit + auth
+    # Real-world: Rate limited, and when retrying the token has expired
+    {
+        "name": "rate_limit_chain",
+        "gate_types": ["rate_limit_headers"],
+        "body_pool": ["expired_auth_token", "missing_required_field"],
+    },
+    # Pattern 5: Redirect + body errors
+    # Real-world: Endpoint moved, client follows redirect but sends wrong body format
+    {
+        "name": "redirect_chain",
+        "gate_types": ["redirect_loop"],
+        "body_pool": ["wrong_field_type", "datetime_format_error", "invalid_email_format"],
+    },
+]
+
+
+def inject_chained_errors(
+    request: Dict[str, Any],
+    headers: Dict[str, str],
+    spec: Dict[str, Any],
+    rng: random_module.Random,
+    count: int = 2,
+) -> Tuple[Dict[str, Any], Dict[str, str], List[GroundTruth]]:
+    """Inject errors in a realistic dependency chain.
+
+    Picks a chain pattern, injects the gate error first, then body errors.
+    Ground truths are ordered: gate errors first, body errors second.
+    This ordering lets the environment progressively reveal errors.
+    """
+    broken_req = copy.deepcopy(request)
+    broken_hdrs = copy.deepcopy(headers)
+    chain: List[GroundTruth] = []
+
+    # Pick a random chain pattern
+    pattern = rng.choice(CHAIN_PATTERNS)
+
+    # Inject the gate error
+    gate_type = rng.choice(pattern["gate_types"])
+    injector = INJECTOR_MAP[gate_type]
+    broken_req, broken_hdrs, gt = injector(broken_req, broken_hdrs, spec, rng)
+    chain.append(gt)
+
+    # Inject body errors from the pattern's pool (or all body types)
+    body_pool = pattern["body_pool"]
+    if body_pool is None:
+        body_pool = [t for t in ERROR_TYPES if t not in HEADER_ERROR_TYPES
+                     and t not in ("wrong_status_code", "redirect_loop", "rate_limit_headers")]
+
+    body_count = max(1, count - 1)
+    available = [t for t in body_pool if t in INJECTOR_MAP]
+    chosen = rng.sample(available, min(body_count, len(available)))
+    for err_type in chosen:
+        injector = INJECTOR_MAP[err_type]
+        broken_req, broken_hdrs, gt = injector(broken_req, broken_hdrs, spec, rng)
+        chain.append(gt)
+
+    return broken_req, broken_hdrs, chain
+
+
 def inject_multiple_errors(
     request: Dict[str, Any],
     headers: Dict[str, str],

server/response_specs.py

@@ -0,0 +1,268 @@
+"""
+Response templates for the response validation task.
+
+Each template defines what a correct API response looks like for a given
+request type. The environment generates a broken response by injecting
+issues (wrong status code, missing fields, wrong types, extra fields).
+
+Response issue types:
+- wrong_status_code: Response has incorrect HTTP status code
+- missing_response_field: Required field missing from response body
+- wrong_response_type: Field present but wrong data type
+- extra_response_field: Unexpected field in response (data leak risk)
+- inconsistent_error_format: Error response doesn't follow spec format
+"""
+
+import copy
+import random
+from typing import Any, Dict, List, Tuple
+
+# Response issue types the agent must identify
+RESPONSE_ISSUE_TYPES = [
+    "wrong_status_code",
+    "missing_response_field",
+    "wrong_response_type",
+    "extra_response_field",
+    "inconsistent_error_format",
+]
+
+
+# Maps API operation type to expected success response
+RESPONSE_TEMPLATES = [
+    {
+        "name": "Create Resource",
+        "success_status": 201,
+        "success_body": {
+            "id": "res_abc123",
+            "status": "created",
+            "created_at": "2025-01-15T10:30:00Z",
+        },
+        "required_response_fields": ["id", "status", "created_at"],
+        "field_types": {"id": "string", "status": "string", "created_at": "string"},
+        "error_status": 400,
+        "error_body": {
+            "error": {"code": "VALIDATION_ERROR", "message": "Invalid input", "details": []},
+        },
+    },
+    {
+        "name": "List Resources",
+        "success_status": 200,
+        "success_body": {
+            "data": [{"id": "res_1", "name": "Item 1"}, {"id": "res_2", "name": "Item 2"}],
+            "total": 2,
+            "page": 1,
+            "per_page": 20,
+        },
+        "required_response_fields": ["data", "total", "page", "per_page"],
+        "field_types": {"data": "array", "total": "integer", "page": "integer", "per_page": "integer"},
+        "error_status": 401,
+        "error_body": {
+            "error": {"code": "UNAUTHORIZED", "message": "Invalid API key", "details": []},
+        },
+    },
+    {
+        "name": "Update Resource",
+        "success_status": 200,
+        "success_body": {
+            "id": "res_abc123",
+            "status": "updated",
+            "updated_at": "2025-01-15T12:00:00Z",
+        },
+        "required_response_fields": ["id", "status", "updated_at"],
+        "field_types": {"id": "string", "status": "string", "updated_at": "string"},
+        "error_status": 404,
+        "error_body": {
+            "error": {"code": "NOT_FOUND", "message": "Resource not found", "details": []},
+        },
+    },
+    {
+        "name": "Delete Resource",
+        "success_status": 204,
+        "success_body": {},
+        "required_response_fields": [],
+        "field_types": {},
+        "error_status": 403,
+        "error_body": {
+            "error": {"code": "FORBIDDEN", "message": "Insufficient permissions", "details": []},
+        },
+    },
+    {
+        "name": "Batch Operation",
+        "success_status": 200,
+        "success_body": {
+            "processed": 5,
+            "failed": 0,
+            "results": [
+                {"id": "item_1", "status": "success"},
+                {"id": "item_2", "status": "success"},
+            ],
+        },
+        "required_response_fields": ["processed", "failed", "results"],
+        "field_types": {"processed": "integer", "failed": "integer", "results": "array"},
+        "error_status": 422,
+        "error_body": {
+            "error": {"code": "UNPROCESSABLE", "message": "Batch validation failed", "details": []},
+        },
+    },
+    {
+        "name": "Authentication",
+        "success_status": 200,
+        "success_body": {
+            "access_token": "eyJhbGciOiJIUzI1NiJ9.token",
+            "token_type": "Bearer",
+            "expires_in": 3600,
+        },
+        "required_response_fields": ["access_token", "token_type", "expires_in"],
+        "field_types": {"access_token": "string", "token_type": "string", "expires_in": "integer"},
+        "error_status": 401,
+        "error_body": {
+            "error": {"code": "INVALID_CREDENTIALS", "message": "Bad credentials", "details": []},
+        },
+    },
+    {
+        "name": "File Upload",
+        "success_status": 201,
+        "success_body": {
+            "file_id": "file_xyz789",
+            "filename": "report.pdf",
+            "size_bytes": 1048576,
+            "url": "https://cdn.example.com/files/file_xyz789",
+        },
+        "required_response_fields": ["file_id", "filename", "size_bytes", "url"],
+        "field_types": {"file_id": "string", "filename": "string", "size_bytes": "integer", "url": "string"},
+        "error_status": 413,
+        "error_body": {
+            "error": {"code": "PAYLOAD_TOO_LARGE", "message": "File exceeds 10MB limit", "details": []},
+        },
+    },
+    {
+        "name": "Search Query",
+        "success_status": 200,
+        "success_body": {
+            "query": "test search",
+            "results": [{"id": "doc_1", "score": 0.95, "title": "Test Document"}],
+            "total_results": 1,
+            "search_time_ms": 42,
+        },
+        "required_response_fields": ["query", "results", "total_results", "search_time_ms"],
+        "field_types": {"query": "string", "results": "array", "total_results": "integer", "search_time_ms": "integer"},
+        "error_status": 400,
+        "error_body": {
+            "error": {"code": "INVALID_QUERY", "message": "Query syntax error", "details": []},
+        },
+    },
+]
+
+
+def get_random_response_template(rng: random.Random) -> Dict[str, Any]:
+    """Pick a random response template."""
+    return copy.deepcopy(rng.choice(RESPONSE_TEMPLATES))
+
+
+def inject_response_issues(
+    template: Dict[str, Any],
+    rng: random.Random,
+    issue_count: int = 1,
+) -> Tuple[Dict[str, Any], int, List[Dict[str, str]]]:
+    """Inject issues into a response and return (broken_body, broken_status, ground_truths).
+
+    Each ground truth has: issue_type, description, affected_field (if applicable).
+    """
+    # Decide if we break a success response or an error response
+    use_success = rng.random() < 0.6
+    if use_success:
+        body = copy.deepcopy(template["success_body"])
+        status = template["success_status"]
+    else:
+        body = copy.deepcopy(template["error_body"])
+        status = template["error_status"]
+
+    ground_truths: List[Dict[str, str]] = []
+    available_issues = list(RESPONSE_ISSUE_TYPES)
+    rng.shuffle(available_issues)
+
+    injected = 0
+    for issue_type in available_issues:
+        if injected >= issue_count:
+            break
+
+        if issue_type == "wrong_status_code":
+            wrong_codes = [200, 201, 204, 301, 400, 401, 403, 404, 422, 429, 500, 502, 503]
+            wrong_codes = [c for c in wrong_codes if c != status]
+            old_status = status
+            status = rng.choice(wrong_codes)
+            ground_truths.append({
+                "issue_type": "wrong_status_code",
+                "description": f"Expected status {old_status}, got {status}",
+                "affected_field": "status_code",
+                "correct_value": str(old_status),
+            })
+            injected += 1
+
+        elif issue_type == "missing_response_field" and template["required_response_fields"]:
+            fields = list(template["required_response_fields"])
+            rng.shuffle(fields)
+            field = fields[0]
+            if field in body:
+                del body[field]
+                ground_truths.append({
+                    "issue_type": "missing_response_field",
+                    "description": f"Required field '{field}' missing from response",
+                    "affected_field": field,
+                })
+                injected += 1
+
+        elif issue_type == "wrong_response_type" and template["field_types"]:
+            typed_fields = [f for f in template["field_types"] if f in body]
+            if typed_fields:
+                field = rng.choice(typed_fields)
+                original_type = template["field_types"][field]
+                # Replace with wrong type
+                if original_type == "string":
+                    body[field] = 12345
+                elif original_type == "integer":
+                    body[field] = "not_a_number"
+                elif original_type == "array":
+                    body[field] = "should_be_array"
+                else:
+                    body[field] = [1, 2, 3]
+                ground_truths.append({
+                    "issue_type": "wrong_response_type",
+                    "description": f"Field '{field}' should be {original_type}",
+                    "affected_field": field,
+                })
+                injected += 1
+
+        elif issue_type == "extra_response_field":
+            leak_fields = {
+                "internal_id": "int_9f8a2b",
+                "debug_trace": "stack trace at line 42",
+                "db_query": "SELECT * FROM users WHERE id=123",
+                "server_ip": "10.0.0.42",
+                "session_token": "sess_leaked_abc",
+            }
+            field, value = rng.choice(list(leak_fields.items()))
+            body[field] = value
+            ground_truths.append({
+                "issue_type": "extra_response_field",
+                "description": f"Unexpected field '{field}' in response (potential data leak)",
+                "affected_field": field,
+            })
+            injected += 1
+
+        elif issue_type == "inconsistent_error_format" and not use_success:
+            # Break the error format -- flatten it or use wrong keys
+            variants = [
+                {"msg": body.get("error", {}).get("message", "error"), "err_code": "UNKNOWN"},
+                {"error_message": "Something went wrong", "status": "error"},
+                {"errors": [{"msg": "bad request"}]},
+            ]
+            body = rng.choice(variants)
+            ground_truths.append({
+                "issue_type": "inconsistent_error_format",
+                "description": "Error response doesn't follow standard format: {error: {code, message, details}}",
+                "affected_field": "error",
+            })
+            injected += 1
+
+    return body, status, ground_truths

tests/test_environment.py

@@ -182,6 +182,88 @@ class TestEasyGrader:
         assert "CORRECT" in obs.feedback
 
 
+# ---------------------------------------------------------------------------
+# TestClassifyGrader
+# ---------------------------------------------------------------------------
+
+class TestClassifyGrader:
+    """Tests for the classify task: multi-error classification."""
+
+    def test_all_types_all_fields_scores_high(self):
+        env = make_env("classify", seed=42)
+        gt_types = [gt["error_type"] for gt in env.ground_truths]
+        gt_fields = []
+        for gt in env.ground_truths:
+            gt_fields.extend(gt.get("affected_fields", []))
+        obs = env.step(APIDebugAction(
+            error_types=gt_types,
+            affected_fields=gt_fields,
+        ))
+        assert obs.reward >= 0.9
+
+    def test_partial_types_gives_partial_score(self):
+        env = make_env("classify", seed=42)
+        first_type = env.ground_truths[0]["error_type"]
+        obs = env.step(APIDebugAction(error_types=[first_type]))
+        # Got 1 of 2-3 types correct, no fields -> partial score
+        assert 0.1 < obs.reward < 0.7
+
+    def test_empty_action_scores_near_0(self):
+        env = make_env("classify", seed=42)
+        obs = env.step(APIDebugAction())
+        assert obs.reward <= 0.01
+
+    def test_single_error_type_field_accepted(self):
+        """Accepts error_type (singular) as fallback for classify."""
+        env = make_env("classify", seed=42)
+        first_type = env.ground_truths[0]["error_type"]
+        first_fields = env.ground_truths[0].get("affected_fields", [])
+        obs = env.step(APIDebugAction(
+            error_type=first_type,
+            affected_fields=first_fields,
+        ))
+        assert obs.reward > 0.1
+
+    def test_wrong_types_score_low(self):
+        env = make_env("classify", seed=42)
+        obs = env.step(APIDebugAction(
+            error_types=["nonexistent_error"],
+            affected_fields=["fake_field"],
+        ))
+        assert obs.reward <= 0.01
+
+    def test_extra_types_reduces_jaccard(self):
+        env = make_env("classify", seed=42)
+        gt_types = [gt["error_type"] for gt in env.ground_truths]
+        obs = env.step(APIDebugAction(
+            error_types=gt_types + ["extra_fake_error"],
+        ))
+        # Extra type reduces Jaccard, but correct ones still score
+        assert obs.reward > 0.1
+
+    def test_max_steps_is_4(self):
+        env = make_env("classify", seed=42)
+        assert env.max_steps == 4
+
+    def test_multiple_ground_truths(self):
+        env = make_env("classify", seed=42)
+        assert len(env.ground_truths) >= 2
+
+    def test_feedback_contains_type_info(self):
+        env = make_env("classify", seed=42)
+        gt_types = [gt["error_type"] for gt in env.ground_truths]
+        obs = env.step(APIDebugAction(error_types=gt_types))
+        assert "error_types" in obs.feedback
+
+    def test_reward_in_valid_range(self):
+        env = make_env("classify", seed=42)
+        obs = env.step(APIDebugAction(
+            error_types=["missing_required_field"],
+            affected_fields=["email"],
+        ))
+        assert 0.001 <= obs.reward <= 0.999
+
+
 # ---------------------------------------------------------------------------
 # TestMediumGrader
 # ---------------------------------------------------------------------------
@@ -271,6 +353,164 @@ class TestMediumGrader:
         assert 0.0 <= obs.reward <= 1.0
 
 
+# ---------------------------------------------------------------------------
+# TestHeadersGrader
+# ---------------------------------------------------------------------------
+
+class TestHeadersGrader:
+    """Tests for the headers task: header-focused debugging."""
+
+    def test_correct_headers_and_type_scores_high(self):
+        env = make_env("headers", seed=42)
+        gt = env.ground_truths[0]
+        obs = env.step(APIDebugAction(
+            error_type=gt["error_type"],
+            fixed_headers=gt["valid_headers"],
+        ))
+        assert obs.reward >= 0.9
+
+    def test_correct_headers_no_type_scores_partial(self):
+        env = make_env("headers", seed=42)
+        gt = env.ground_truths[0]
+        obs = env.step(APIDebugAction(
+            fixed_headers=gt["valid_headers"],
+        ))
+        # 0.7 for headers, 0 for type = ~0.7
+        assert 0.5 < obs.reward < 0.9
+
+    def test_correct_type_no_headers_scores_low(self):
+        env = make_env("headers", seed=42)
+        gt = env.ground_truths[0]
+        obs = env.step(APIDebugAction(
+            error_type=gt["error_type"],
+        ))
+        # 0.3 for type, 0 for headers
+        assert obs.reward < 0.5
+
+    def test_empty_action_scores_near_0(self):
+        env = make_env("headers", seed=42)
+        obs = env.step(APIDebugAction())
+        assert obs.reward <= 0.01
+
+    def test_max_steps_is_4(self):
+        env = make_env("headers", seed=42)
+        assert env.max_steps == 4
+
+    def test_header_error_type_is_header_related(self):
+        env = make_env("headers", seed=42)
+        header_types = {"missing_auth_header", "wrong_content_type", "expired_auth_token"}
+        for gt in env.ground_truths:
+            assert gt["error_type"] in header_types
+
+    def test_feedback_contains_header_info(self):
+        env = make_env("headers", seed=42)
+        gt = env.ground_truths[0]
+        obs = env.step(APIDebugAction(fixed_headers=gt["valid_headers"]))
+        assert "Header" in obs.feedback or "error_type" in obs.feedback
+
+    def test_wrong_headers_score_low(self):
+        env = make_env("headers", seed=42)
+        obs = env.step(APIDebugAction(
+            fixed_headers={"X-Wrong": "value"},
+        ))
+        assert obs.reward < 0.5
+
+    def test_reward_in_valid_range(self):
+        env = make_env("headers", seed=42)
+        obs = env.step(APIDebugAction(
+            fixed_headers={"Authorization": "Bearer test"},
+        ))
+        assert 0.001 <= obs.reward <= 0.999
+
+    def test_single_ground_truth(self):
+        env = make_env("headers", seed=42)
+        assert len(env.ground_truths) == 1
+
+
+# ---------------------------------------------------------------------------
+# TestResponseGrader
+# ---------------------------------------------------------------------------
+
+class TestResponseGrader:
+    """Tests for the response validation task."""
+
+    def test_correct_issues_scores_high(self):
+        env = make_env("response", seed=42)
+        gt_issues = [gt["issue_type"] for gt in env.ground_truths]
+        gt_fields = [gt.get("affected_field", "") for gt in env.ground_truths if gt.get("affected_field")]
+        # Find correct status if applicable
+        expected_status = None
+        for gt in env.ground_truths:
+            if gt["issue_type"] == "wrong_status_code":
+                expected_status = int(gt.get("correct_value", 0))
+        obs = env.step(APIDebugAction(
+            response_issues=gt_issues,
+            affected_fields=gt_fields,
+            expected_status_code=expected_status,
+        ))
+        assert obs.reward >= 0.7
+
+    def test_partial_issues_scores_partial(self):
+        env = make_env("response", seed=10)
+        # Only provide one issue type even if there are more
+        gt = env.ground_truths[0]
+        obs = env.step(APIDebugAction(
+            response_issues=[gt["issue_type"]],
+        ))
+        assert 0.001 <= obs.reward <= 0.999
+
+    def test_empty_action_scores_near_0(self):
+        env = make_env("response", seed=42)
+        obs = env.step(APIDebugAction())
+        assert obs.reward <= 0.01
+
+    def test_max_steps_is_4(self):
+        env = make_env("response", seed=42)
+        assert env.max_steps == 4
+
+    def test_observation_has_response_body(self):
+        env = APIDebugEnvironment()
+        obs = env.reset(task="response", seed=42)
+        assert obs.response_body != ""
+        assert obs.response_status_code > 0
+
+    def test_observation_has_correct_task(self):
+        env = APIDebugEnvironment()
+        obs = env.reset(task="response", seed=42)
+        assert obs.task == "response"
+
+    def test_ground_truth_has_issue_type(self):
+        env = make_env("response", seed=42)
+        valid_types = {
+            "wrong_status_code", "missing_response_field",
+            "wrong_response_type", "extra_response_field",
+            "inconsistent_error_format",
+        }
+        for gt in env.ground_truths:
+            assert gt["issue_type"] in valid_types
+
+    def test_wrong_issues_score_low(self):
+        env = make_env("response", seed=42)
+        obs = env.step(APIDebugAction(
+            response_issues=["nonexistent_issue"],
+        ))
+        assert obs.reward < 0.3
+
+    def test_reward_in_valid_range(self):
+        env = make_env("response", seed=42)
+        obs = env.step(APIDebugAction(
+            response_issues=["wrong_status_code"],
+            expected_status_code=200,
+        ))
+        assert 0.001 <= obs.reward <= 0.999
+
+    def test_response_body_is_valid_json(self):
+        env = APIDebugEnvironment()
+        obs = env.reset(task="response", seed=42)
+        parsed = json.loads(obs.response_body)
+        assert isinstance(parsed, dict)
+
+
 # ---------------------------------------------------------------------------
 # TestHardGrader
 # ---------------------------------------------------------------------------

training/README.md

@@ -0,0 +1,71 @@
+# Training with GRPO on API Debug Environment
+
+Trains a small LLM using **GRPO** (Group Relative Policy Optimization)
+on the live API Debug Environment with **curriculum learning**.
+
+## What is GRPO?
+
+For each prompt, GRPO:
+1. Generates multiple completions (debug attempts)
+2. Scores each with the environment's grader (reward signal)
+3. Updates the model to prefer higher-scoring responses
+
+Over thousands of episodes, the LLM learns to debug API requests
+purely from reward signals -- no labelled data needed.
+
+## Curriculum Learning
+
+The training auto-promotes through difficulty levels:
+
+| Level | Task | Threshold | Max Turns | Skill |
+|-------|------|-----------|-----------|-------|
+| 1 | easy | 0.7 avg reward | 3 | Identify single error type + fields |
+| 2 | classify | 0.6 avg reward | 4 | Identify ALL error types + fields |
+| 3 | medium | 0.6 avg reward | 5 | Fix the broken request body |
+| 4 | headers | 0.5 avg reward | 4 | Fix header-level errors |
+| 5 | response | 0.5 avg reward | 4 | Validate API response issues |
+| 6 | hard | -- | 7 | Fix mixed errors + explain reasoning |
+
+Promotion happens when the rolling average reward (window=10) exceeds
+the threshold for the current level.
+
+## Architecture
+```
+Dataset prompt ("Debug this broken API request.")
+     |
+GRPOTrainer calls rollout_func()
+     |
+rollout_func() connects to live HF Space via WebSocket
+     |
+env.reset(task=current_task) -> broken API request
+     |
+LLM generates JSON response -> env.step(action) -> reward
+     |  (repeat up to max_turns)
+Returns: prompt_ids, completion_ids, logprobs, env_reward
+     |
+reward_from_env() extracts env_reward
+     |
+GRPO updates model weights
+     |
+maybe_promote() checks if agent should advance to next task
+```
+
+## Run on Google Colab (free T4 GPU)
+```python
+# Cell 1 -- Install
+!pip install trl>=0.26.0 transformers torch datasets openenv-core openai
+
+# Cell 2 -- Clone repo
+!git clone https://github.com/Avi-chauhan/api-debug-env.git
+%cd api-debug-env
+
+# Cell 3 -- Train
+!python training/train.py
+```
+
+## Requirements
+
+- GPU: T4 or better (free Colab works)
+- RAM: 8GB+
+- The live HF Space must be running:
+  https://huggingface.co/spaces/avichauhan/api-debug-env

training/requirements.txt

@@ -0,0 +1,6 @@
+trl>=0.26.0
+transformers
+torch
+datasets
+openenv-core
+openai

training/train.py

@@ -0,0 +1,309 @@
+import os
+os.environ["PYTORCH_ALLOC_CONF"] = "expandable_segments:True"
+
+"""
+GRPO Training on API Debug Environment
+=======================================
+Trains a small LLM (Qwen 0.5B) to debug malformed API requests using
+reward signals from the live HuggingFace Space environment.
+
+Supports curriculum learning: starts on easy task, promotes to classify
+and medium as the agent improves.
+
+Run on Colab (free T4 GPU):
+    pip install -r training/requirements.txt
+    python training/train.py
+"""
+
+import json
+import re
+import sys
+
+import torch
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from datasets import Dataset
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from trl import GRPOConfig, GRPOTrainer
+from trl.experimental.openenv import generate_rollout_completions
+
+from client import APIDebugEnv
+from models import APIDebugAction
+
+# -- GPU check ----------------------------------------------------------------
+print(f"GPU available   : {torch.cuda.is_available()}")
+print(f"GPU name        : {torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'None (CPU)'}")
+
+has_gpu       = torch.cuda.is_available()
+supports_bf16 = has_gpu and torch.cuda.is_bf16_supported()
+
+# -- Config -------------------------------------------------------------------
+MODEL_ID    = "Qwen/Qwen2.5-0.5B-Instruct"
+ENV_URL     = "https://avichauhan-api-debug-env.hf.space"
+MAX_TURNS   = 3   # easy task: 3 steps max
+NUM_SAMPLES = 64
+
+# -- Curriculum state ---------------------------------------------------------
+# Tracks which task the agent is currently training on.
+# Promotes when rolling average reward exceeds threshold.
+CURRICULUM = {
+    "easy":     {"next": "classify", "threshold": 0.7, "max_turns": 3},
+    "classify": {"next": "medium",   "threshold": 0.6, "max_turns": 4},
+    "medium":   {"next": "headers",  "threshold": 0.6, "max_turns": 5},
+    "headers":  {"next": "response", "threshold": 0.5, "max_turns": 4},
+    "response": {"next": "hard",     "threshold": 0.5, "max_turns": 4},
+    "hard":     {"next": None,       "threshold": None, "max_turns": 7},
+}
+current_task = "easy"
+recent_rewards: list[float] = []
+WINDOW_SIZE = 10
+
+SYSTEM_PROMPT = """You are an API debugging expert. You receive a broken API request and its specification.
+Your job: identify the error type and the affected fields.
+
+Respond with ONLY a JSON object in this format:
+{"error_type": "<type>", "affected_fields": ["field1", "field2"]}
+
+Valid error types:
+missing_required_field, wrong_field_type, invalid_email_format,
+missing_auth_header, extra_unknown_field, null_value_in_required,
+wrong_http_method, malformed_json_value, invalid_enum_value,
+datetime_format_error, wrong_content_type, expired_auth_token"""
+
+CLASSIFY_PROMPT = """You are an API debugging expert. This request has MULTIPLE errors.
+Identify ALL error types and ALL affected fields.
+
+Respond with ONLY a JSON object:
+{"error_types": ["type1", "type2"], "affected_fields": ["field1", "field2"]}"""
+
+MEDIUM_PROMPT = """You are an API debugging expert. Fix the broken request to match the API spec.
+
+Respond with ONLY a JSON object:
+{"fixed_request": {"field": "value"}, "fixed_headers": {"Header": "value"}}"""
+
+HEADERS_PROMPT = """You are an API debugging expert. This request has ONLY header-level errors.
+Identify the error type and fix the headers to match the API spec.
+
+Respond with ONLY a JSON object:
+{"error_type": "<type>", "fixed_headers": {"Header-Name": "correct_value"}}
+
+Common header error types: wrong_content_type, expired_auth_token, missing_auth_header"""
+
+RESPONSE_PROMPT = """You are an API response validation expert. You receive an API request, its spec, and the server response.
+Identify issues in the response: wrong status codes, missing fields, wrong types, extra fields, inconsistent error format.
+
+Respond with ONLY a JSON object:
+{"response_issues": ["issue_type1"], "affected_fields": ["field1"], "expected_status_code": 200}
+
+Valid issue types: wrong_status_code, missing_response_field, wrong_response_type, extra_response_field, inconsistent_error_format"""
+
+HARD_PROMPT = """You are an API debugging expert. This request has MULTIPLE errors across headers and body.
+Some errors are chained -- fixing one may reveal others. Fix everything and explain your reasoning.
+
+Respond with ONLY a JSON object:
+{"fixed_request": {"field": "value"}, "fixed_headers": {"Header": "value"}, "explanation": "why each fix was needed"}"""
+
+TASK_PROMPTS = {
+    "easy": SYSTEM_PROMPT,
+    "classify": CLASSIFY_PROMPT,
+    "medium": MEDIUM_PROMPT,
+    "headers": HEADERS_PROMPT,
+    "response": RESPONSE_PROMPT,
+    "hard": HARD_PROMPT,
+}
+
+# -- Environment client -------------------------------------------------------
+print(f"Connecting to environment: {ENV_URL}")
+env_client = APIDebugEnv(base_url=ENV_URL)
+
+
+# -- JSON parser (reused from inference.py) -----------------------------------
+def parse_llm_response(text: str) -> dict:
+    if not text:
+        return {}
+    try:
+        return json.loads(text)
+    except json.JSONDecodeError:
+        pass
+    code_block = re.search(r"```(?:json)?\s*\n?(.*?)\n?\s*```", text, re.DOTALL)
+    if code_block:
+        try:
+            return json.loads(code_block.group(1))
+        except json.JSONDecodeError:
+            pass
+    brace_match = re.search(r"\{[^{}]*\}", text, re.DOTALL)
+    if brace_match:
+        try:
+            return json.loads(brace_match.group(0))
+        except json.JSONDecodeError:
+            pass
+    return {}
+
+
+def build_action(data: dict) -> APIDebugAction:
+    fixed_req = data.get("fixed_request")
+    if isinstance(fixed_req, dict):
+        fixed_req = json.dumps(fixed_req)
+    return APIDebugAction(
+        error_type=data.get("error_type"),
+        error_types=data.get("error_types"),
+        affected_fields=data.get("affected_fields"),
+        fixed_request=fixed_req,
+        fixed_headers=data.get("fixed_headers"),
+        response_issues=data.get("response_issues"),
+        expected_status_code=data.get("expected_status_code"),
+    )
+
+
+# -- Curriculum learning ------------------------------------------------------
+def maybe_promote():
+    """Check if agent should be promoted to next difficulty."""
+    global current_task
+    config = CURRICULUM[current_task]
+    if config["next"] is None or config["threshold"] is None:
+        return
+    if len(recent_rewards) < WINDOW_SIZE:
+        return
+    avg = sum(recent_rewards[-WINDOW_SIZE:]) / WINDOW_SIZE
+    if avg >= config["threshold"]:
+        old_task = current_task
+        current_task = config["next"]
+        recent_rewards.clear()
+        print(f"[CURRICULUM] Promoted: {old_task} -> {current_task} (avg_reward={avg:.3f})")
+
+
+# -- Rollout function ---------------------------------------------------------
+def rollout_func(prompts: list[str], trainer: GRPOTrainer) -> dict:
+    tokenizer = trainer.processing_class
+    all_prompt_ids     = []
+    all_completion_ids = []
+    all_logprobs       = []
+    all_rewards        = []
+
+    task = current_task
+    max_turns = CURRICULUM[task]["max_turns"]
+    system_prompt = TASK_PROMPTS[task]
+
+    for base_prompt in prompts:
+        with env_client.sync() as env:
+            obs = env.reset(task=task)
+            episode_reward     = 0.0
+            episode_prompt_ids = []
+            episode_comp_ids   = []
+            episode_logprobs   = []
+
+            for turn in range(max_turns):
+                messages = [
+                    {"role": "system", "content": system_prompt},
+                    {"role": "user", "content": (
+                        f"{base_prompt}\n\n"
+                        f"API: {obs.observation.http_method} {obs.observation.endpoint} "
+                        f"({obs.observation.api_name})\n"
+                        f"Error count: {obs.observation.error_count}\n"
+                        f"Step {turn + 1}/{max_turns}\n\n"
+                        f"Broken request:\n{obs.observation.broken_request}\n\n"
+                        f"Headers: {json.dumps(obs.observation.broken_headers)}\n\n"
+                        f"API Spec:\n{obs.observation.api_spec}\n"
+                        + (f"\nFeedback:\n{obs.observation.feedback}" if obs.observation.feedback else "")
+                    )},
+                ]
+                prompt_text = tokenizer.apply_chat_template(
+                    messages,
+                    add_generation_prompt=True,
+                    tokenize=False,
+                    enable_thinking=False,
+                )
+
+                outputs = generate_rollout_completions(trainer, [prompt_text])[0]
+                completion_text = tokenizer.decode(
+                    outputs["completion_ids"], skip_special_tokens=True
+                ).strip()
+
+                episode_prompt_ids.extend(outputs["prompt_ids"])
+                episode_comp_ids.extend(outputs["completion_ids"])
+                episode_logprobs.extend(outputs["logprobs"])
+
+                # Parse LLM output into action
+                parsed = parse_llm_response(completion_text)
+                action = build_action(parsed)
+                obs = env.step(action)
+                episode_reward = float(obs.reward or 0.0)
+
+                if obs.done:
+                    break
+
+            all_prompt_ids.append(episode_prompt_ids)
+            all_completion_ids.append(episode_comp_ids)
+            all_logprobs.append(episode_logprobs)
+            all_rewards.append(episode_reward)
+
+            # Track for curriculum
+            recent_rewards.append(episode_reward)
+
+    # Check if agent should be promoted
+    maybe_promote()
+
+    return {
+        "prompt_ids":     all_prompt_ids,
+        "completion_ids": all_completion_ids,
+        "logprobs":       all_logprobs,
+        "env_reward":     all_rewards,
+    }
+
+
+# -- Reward function ----------------------------------------------------------
+def reward_from_env(completions, **kwargs):
+    env_rewards = kwargs.get("env_reward", [])
+    return [float(r) for r in env_rewards] if env_rewards else [0.0] * len(completions)
+
+
+# -- Dataset ------------------------------------------------------------------
+dataset = Dataset.from_dict({
+    "prompt": ["Debug this broken API request."] * NUM_SAMPLES
+})
+
+# -- Trainer ------------------------------------------------------------------
+tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+model = AutoModelForCausalLM.from_pretrained(MODEL_ID, attn_implementation="eager")
+
+grpo_args = GRPOConfig(
+    use_vllm=True,
+    vllm_mode="colocate",
+    num_train_epochs=1,
+    num_generations=2,
+    max_completion_length=128,
+    per_device_train_batch_size=1,
+    gradient_accumulation_steps=16,
+    learning_rate=5e-6,
+    output_dir="./outputs/api-debug-grpo",
+    logging_steps=1,
+    report_to="none",
+    bf16=supports_bf16,
+    fp16=has_gpu and not supports_bf16,
+    no_cuda=not has_gpu,
+    gradient_checkpointing=True,
+    vllm_gpu_memory_utilization=0.3,
+    dataloader_pin_memory=False,
+)
+
+trainer = GRPOTrainer(
+    model=model,
+    processing_class=tokenizer,
+    reward_funcs=reward_from_env,
+    train_dataset=dataset,
+    rollout_func=rollout_func,
+    args=grpo_args,
+)
+
+if __name__ == "__main__":
+    print("Starting GRPO training on API Debug Environment...")
+    print(f"Model      : {MODEL_ID}")
+    print(f"Environment: {ENV_URL}")
+    print(f"Episodes   : {NUM_SAMPLES}")
+    print(f"Task       : {current_task} (with curriculum learning)")
+    print(f"bf16       : {supports_bf16}")
+    print(f"fp16       : {has_gpu and not supports_bf16}")
+    trainer.train()
+    print(f"Training complete! Final task: {current_task}")
+    print("Model saved to ./outputs/api-debug-grpo")