Upload folder using huggingface_hub

Dockerfile

@@ -1,81 +1,32 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
-
-# Multi-stage build using openenv-base
-# This Dockerfile is flexible and works for both:
-# - In-repo environments (with local OpenEnv sources)
-# - Standalone environments (with openenv from PyPI/Git)
-# The build script (openenv build) handles context detection and sets appropriate build args.
-
-ARG BASE_IMAGE=ghcr.io/meta-pytorch/openenv-base:latest
-FROM ${BASE_IMAGE} AS builder
-
-WORKDIR /app
-
-# Ensure git is available (required for installing dependencies from VCS)
-RUN apt-get update && \
-    apt-get install -y --no-install-recommends git && \
-    rm -rf /var/lib/apt/lists/*
-
-# Build argument to control whether we're building standalone or in-repo
-ARG BUILD_MODE=in-repo
-ARG ENV_NAME=python_env
-
-# Copy environment code (always at root of build context)
-COPY . /app/env
-
-# For in-repo builds, openenv is already vendored in the build context
-# For standalone builds, openenv will be installed via pyproject.toml
-WORKDIR /app/env
-
-# Ensure uv is available (for local builds where base image lacks it)
-RUN if ! command -v uv >/dev/null 2>&1; then \
-        curl -LsSf https://astral.sh/uv/install.sh | sh && \
-        mv /root/.local/bin/uv /usr/local/bin/uv && \
-        mv /root/.local/bin/uvx /usr/local/bin/uvx; \
-    fi
-    
-# Install dependencies using uv sync
-# If uv.lock exists, use it; otherwise resolve on the fly
-RUN --mount=type=cache,target=/root/.cache/uv \
-    if [ -f uv.lock ]; then \
-        uv sync --frozen --no-install-project --no-editable; \
-    else \
-        uv sync --no-install-project --no-editable; \
-    fi
-
-RUN --mount=type=cache,target=/root/.cache/uv \
-    if [ -f uv.lock ]; then \
-        uv sync --frozen --no-editable; \
-    else \
-        uv sync --no-editable; \
-    fi
-
-# Final runtime stage
-FROM ${BASE_IMAGE}
+FROM python:3.11-slim
 
 WORKDIR /app
 
-# Copy the virtual environment from builder
-COPY --from=builder /app/env/.venv /app/.venv
+# Install system dependencies
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    gcc \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
 
-# Copy the environment code
-COPY --from=builder /app/env /app/env
+# Copy source code
+COPY . /app
 
-# Set PATH to use the virtual environment
-ENV PATH="/app/.venv/bin:$PATH"
+# Install Python dependencies
+RUN pip install --no-cache-dir -r requirements.txt
 
-# Set PYTHONPATH so imports work correctly
-ENV PYTHONPATH="/app/env:$PYTHONPATH"
+# Set environment variables
+ENV PYTHONUNBUFFERED=1
+ENV HOST=0.0.0.0
+ENV PORT=8000
+ENV WORKERS=1
+ENV MAX_CONCURRENT_ENVS=16
 
 # Health check
-HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
-    CMD curl -f http://localhost:8000/health || exit 1
+HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
+    CMD curl -f http://localhost:${PORT}/health || exit 1
 
-# Run the FastAPI server
-# The module path is constructed to work with the /app/env structure
+# Run FastAPI app
+EXPOSE ${PORT}
 ENV ENABLE_WEB_INTERFACE=true
-CMD ["sh", "-c", "cd /app/env && uvicorn server.app:app --host 0.0.0.0 --port 8000"]
+CMD ["python", "-m", "server.app"]

Project.md

@@ -0,0 +1,1111 @@
+python inference.py --model gpt-3.5-turbo --base-url "http://localhost:8000/v1"
+python inference.py --model gemini-2.0-flash --base-url "https://generativelanguage.googleapis.com/openai/"
+python inference.py --model deepseek-chat --base-url "https://api.deepseek.com"# Python Env Project Guide
+
+This document explains how to work with the `python_env` project end to end:
+
+1. What the environment is trying to do
+2. How the current code is structured
+3. How each route works
+4. How to test each route manually
+5. How to use the inference script
+6. How to prepare data so an RL or agent-training setup can learn more effectively
+7. How the project maps to the hackathon functional requirements
+
+The goal is practical: after reading this file, you should be able to start the server, hit every route, understand what each response means, run the baseline, and know what data to collect next.
+
+## 1. Project Goal
+
+This environment simulates a real software engineering workflow: Python code review.
+
+An agent is given Python code and must:
+
+- detect correctness bugs
+- detect security risks
+- detect maintainability problems
+- detect obvious performance issues
+- optionally suggest improved code
+
+This is a valid real-world environment because code review is an actual human task used in engineering teams every day.
+
+## 2. High-Level Architecture
+
+The project has four main parts:
+
+- `models.py`
+  Defines the typed Pydantic models for actions, observations, evaluations, config, health, and direct-review payloads.
+
+- `server/code_review_environment.py`
+  Implements the environment logic: `reset()`, `step()`, reward shaping, task progression, hints, history, and grading integration.
+
+- `server/task_bank.py`, `server/grading.py`, `server/static_review.py`
+  These files define the benchmark tasks, deterministic graders, and direct static review rules.
+
+- `server/app.py`
+  Exposes both:
+  - OpenEnv-compatible endpoints such as `/reset`, `/step`, `/state`, `/schema`, `/ws`
+  - custom REST endpoints such as `/health`, `/tasks`, `/review`, `/config`, `/history`
+
+- `inference.py`
+  Runs an OpenAI-compatible model against the environment and writes a reproducible report.
+
+## 3. File-by-File Understanding
+
+### `models.py`
+
+Important models:
+
+- `ReviewFinding`
+  One code-review issue found by the agent.
+  Fields:
+  - `title`
+  - `line`
+  - `category`
+  - `severity`
+  - `rationale`
+  - `recommendation`
+  - `rule_id`
+
+- `PythonReviewAction`
+  What the agent sends to the environment.
+  Fields:
+  - `operation`
+  - `findings`
+  - `patched_code`
+  - `note`
+
+- `PythonReviewObservation`
+  What the environment returns back.
+  Fields:
+  - `task`
+  - `instructions`
+  - `feedback`
+  - `submitted_findings`
+  - `hints_used`
+  - `attempts_remaining`
+  - `evaluation`
+  - `score`
+  - `review_time_ms`
+  - inherited OpenEnv fields such as `reward`, `done`, `metadata`
+
+- `TaskEvaluation`
+  Deterministic grading output.
+  Fields:
+  - `matched_reference_ids`
+  - `matched_findings`
+  - `total_findings`
+  - `false_positives`
+  - `duplicate_findings`
+  - `weighted_recall`
+  - `patch_score`
+  - `score`
+  - `passed`
+
+### `server/task_bank.py`
+
+Contains the benchmark tasks.
+
+Current tasks:
+
+1. `py-review-easy`
+   Detect unsafe `eval` and division-by-zero risk.
+
+2. `py-review-medium`
+   Detect mutable default list, quadratic membership check, and bare `except`.
+
+3. `py-review-hard`
+   Detect `shell=True` command injection, stale cache bug, and shared output file risk.
+
+Each task contains:
+
+- code to review
+- hints
+- reference findings
+- pass threshold
+
+### `server/grading.py`
+
+This is the benchmark grader.
+
+It compares submitted findings to hidden reference findings and computes:
+
+- weighted recall
+- penalties for false positives
+- penalties for duplicates
+- optional patch quality score
+- final score in `0.0` to `1.0`
+
+This makes the task deterministic and reproducible, which is important for hackathon judging.
+
+### `server/static_review.py`
+
+This powers the `/review` endpoint for arbitrary code snippets.
+
+It uses AST inspection to detect:
+
+- `eval` / `exec`
+- mutable default arguments
+- `shell=True`
+- bare `except`
+- list-membership-inside-loop performance smell
+- syntax errors
+- `print()` used in application logic
+
+This is not the task grader. It is the direct-review helper.
+
+### Reward System
+
+The reward system is **dynamic and multi-component**, designed to provide meaningful feedback at every step of the agent's learning process.
+
+#### Reward Architecture
+
+The system computes rewards using **6 independent components**:
+
+1. **Progress Reward** (max +0.25)
+   - Awarded when the agent improves the score from one step to the next
+   - Formula: `min(PROGRESS_SCALE * score_delta, 0.25)`
+   - Encourages continuous improvement
+
+2. **Syntax Reward** (max +0.35)
+   - One-time bonus awarded for fixing syntax errors (first time compiling)
+   - Applied once per episode when code transitions from uncompilable to compilable
+   - Acknowledges the critical first step of making code valid
+
+3. **Test Reward** (max +0.20)
+   - Based on improvement in test pass rate
+   - Computed as: `min(TEST_PASS_REWARD_SCALE * test_improvement_fraction, 0.20)`
+   - Rewards incremental progress on passing more tests
+
+4. **Quality Reward** (max +0.15)
+   - Based on AST-detected code quality metrics
+   - Rewards improvements in code structure, readability, and best practices
+   - Uses deterministic grader feedback
+
+5. **Stagnation Penalty** (−0.10)
+   - Applied when the agent takes action but code doesn't change
+   - Encourages the agent to edit the code rather than analyze repeatedly
+   - Configurable via `STAGNATION_PENALTY` constant
+
+6. **Regression Penalty** (scale −0.20)
+   - Applied when score decreases from previous step
+   - Formula: `REGRESSION_PENALTY_SCALE * abs(score_delta)`
+   - Discourages actions that make code worse
+
+#### Reward Constants
+
+Defined at the top of `server/env.py`:
+
+```python
+SYNTAX_FIX_BONUS = 0.35          # One-time syntax reward
+TEST_PASS_REWARD_SCALE = 0.30    # Per test improvement
+QUALITY_BONUS_SCALE = 0.15       # Code quality improvement
+PROGRESS_SCALE = 0.25             # Score improvement
+COMPLETION_BONUS = 0.50           # Full correctness bonus
+INVALID_ACTION_PENALTY = 0.15     # For unsupported actions
+STAGNATION_PENALTY = 0.10         # For unchanged code
+REGRESSION_PENALTY_SCALE = 0.20   # For score decline
+TIMEOUT_PENALTY = 0.15            # For execution timeout
+```
+
+#### Final Reward Computation
+
+The final reward is:
+
+```
+total = progress + syntax + test + quality - stagnation - regression
+final_reward = clamp(total, -1.0, +1.0)
+```
+
+The result is always between −1.0 and +1.0, providing bounded, interpretable feedback.
+
+#### RewardDetails: Transparent Feedback
+
+Every reward is returned as a `RewardDetails` object with these fields:
+
+- `value`: The scalar reward for this step
+- `syntax_reward`: Contribution from syntax fixes
+- `test_reward`: Contribution from test improvements
+- `quality_bonus`: Contribution from code quality
+- `progress_delta`: Contribution from score improvement
+- `stagnation_penalty`: Penalty for unchanged code
+- `regression_penalty`: Penalty for score decline
+- `prev_score` / `curr_score`: Score before and after the action
+- `code_changed`: Whether the action modified the code
+- `reason`: Human-readable explanation of the reward
+
+This transparency is crucial for:
+- Debugging agent behavior
+- Understanding what drives reward
+- Tuning the constants
+- Training supervised models on reward components
+
+#### Why This Design Helps Agents Learn
+
+1. **Non-Constant**: Different actions produce different rewards, enabling meaningful gradient signals
+2. **Progressive**: Early bonuses (syntax) are high; later improvements are smaller, promoting efficiency
+3. **Transparent**: Detailed component breakdown helps agents understand what matters
+4. **Bounded**: Clamping to [−1, 1] prevents reward hacking and explosion
+5. **Balanced**: Positive and negative signals teach precision and recall together
+
+### `server/code_review_environment.py`
+
+This is the environment core.
+
+Main methods:
+
+- `reset()`
+  Rotates to the next task, resets episode state, and returns the initial observation.
+
+- `step(action)`
+  Accepts a `PythonReviewAction`, grades it, shapes reward, updates history, and returns the new observation.
+
+- `direct_review(code, context)`
+  Calls the static reviewer for arbitrary code.
+
+- `list_tasks()`
+  Returns public descriptors for all tasks.
+
+- `grade_task_submission(task_id, findings, patched_code)`
+  Grades a proposed submission against the deterministic rubric without stepping through an episode.
+
+### `server/app.py`
+
+This file wires everything to FastAPI and OpenEnv.
+
+Important note:
+
+- OpenEnv endpoints are managed through `create_app(PythonEnvironment, PythonReviewAction, PythonReviewObservation)`
+- custom routes such as `/health`, `/tasks`, `/review`, `/history`, `/config` use a singleton `python_env`
+
+That means:
+
+- `/reset` and `/step` are served by OpenEnv session handling
+- `/review`, `/tasks`, `/config`, `/history` are served by the singleton helper instance
+
+This is fine for startup and manual testing, but if you want one fully unified state model later, you should refactor custom routes to read from the same managed environment/session layer.
+
+## 4. Route-by-Route Guide
+
+### OpenEnv Routes
+
+These are important for validation and agents.
+
+#### `POST /reset`
+
+Purpose:
+- starts a new episode
+- rotates to the next benchmark task
+- returns an initial observation
+
+Use this when:
+- you want to start evaluating an agent on a task
+
+#### `POST /step`
+
+Purpose:
+- submit agent actions
+- get reward, observation, and done flag
+
+Use this when:
+- manually simulating agent steps
+- testing reward shaping and grading
+
+#### `GET /state`
+
+Purpose:
+- returns current OpenEnv session state, typically `episode_id` and `step_count`
+
+Use this when:
+- debugging session behavior
+
+#### `GET /schema`
+
+Purpose:
+- shows the action/observation schema expected by OpenEnv
+
+Use this when:
+- debugging payload formats
+- verifying OpenEnv compatibility
+
+#### `WS /ws`
+
+Purpose:
+- persistent lower-latency session transport for clients
+
+Use this when:
+- building actual agent loops with the `EnvClient`
+
+### Custom REST Routes
+
+#### `GET /health`
+
+Purpose:
+- quick health check for Docker and Hugging Face Spaces
+
+Use this when:
+- checking whether the server is alive
+- validating deployment health
+
+#### `GET /tasks`
+
+Purpose:
+- returns the three benchmark task descriptors
+
+Use this when:
+- reviewing available tasks
+- building curriculum/eval metadata
+
+#### `GET /tasks/{task_id}`
+
+Purpose:
+- returns one task descriptor
+
+Use this when:
+- inspecting a task before submitting findings
+
+#### `POST /tasks/{task_id}/grade`
+
+Purpose:
+- grade a proposed set of findings against the deterministic task rubric
+
+Use this when:
+- validating benchmark grading directly
+- building offline evaluation sets
+
+#### `POST /review`
+
+Purpose:
+- run direct static review on arbitrary Python code
+
+Use this when:
+- testing the static analyzer
+- building training examples
+- verifying that common issues are caught
+
+#### `GET /history`
+
+Purpose:
+- returns the singleton environment history
+
+Use this when:
+- checking what the custom singleton environment has processed
+
+Note:
+- this history is not the same as OpenEnv session history from `/step`
+
+#### `DELETE /history`
+
+Purpose:
+- clears the singleton history
+
+Use this when:
+- resetting the custom review log before a test run
+
+#### `GET /config`
+
+Purpose:
+- inspect config values such as penalties and task order
+
+#### `PUT /config`
+
+Purpose:
+- update the environment config
+
+Use this when:
+- testing different reward penalties or task order
+
+## 5. Manual Testing: Step by Step
+
+Start the server:
+
+```powershell
+uvicorn server.app:app --reload --host 0.0.0.0 --port 8000
+```
+
+Open the docs:
+
+```text
+http://127.0.0.1:8000/docs
+```
+
+That is the easiest manual route explorer.
+
+### Test 1: Health
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/health" -Method Get
+```
+
+Expected:
+- `status` should be `ok`
+- `task_count` should be `3`
+
+### Test 2: List Tasks
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/tasks" -Method Get
+```
+
+Expected:
+- three tasks
+- each task has `task_id`, `difficulty`, `title`, `objective`, `code`
+
+### Test 3: Get One Task
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/tasks/py-review-easy" -Method Get
+```
+
+### Test 4: Direct Static Review
+
+```powershell
+$body = @{
+  code = @"
+def load_settings(config_text):
+    return eval(config_text)
+"@
+} | ConvertTo-Json
+
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/review" `
+  -Method Post `
+  -Body $body `
+  -ContentType "application/json"
+```
+
+Expected:
+- at least one issue
+- one issue should have `rule_id = "avoid-eval"`
+
+### Test 5: Reset Episode
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/reset" `
+  -Method Post `
+  -Body "{}" `
+  -ContentType "application/json"
+```
+
+Expected:
+- an observation with a `task`
+- `done = false`
+- `reward = 0`
+
+### Test 6: Submit Partial Findings To `/step`
+
+```powershell
+$body = @{
+  operation = "submit_findings"
+  findings = @(
+    @{
+      title = "Avoid eval on untrusted configuration data"
+      line = 2
+      category = "security"
+      severity = "critical"
+      rationale = "eval can execute attacker-controlled code."
+      recommendation = "Use json.loads or ast.literal_eval."
+      rule_id = "avoid-eval"
+    }
+  )
+  patched_code = $null
+  note = "First pass review"
+} | ConvertTo-Json -Depth 5
+
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/step" `
+  -Method Post `
+  -Body $body `
+  -ContentType "application/json"
+```
+
+Expected:
+- positive reward
+- improved `score`
+- feedback mentioning a matched rubric item
+
+### Test 7: Request A Hint
+
+```powershell
+$body = @{
+  operation = "request_hint"
+  findings = @()
+  patched_code = $null
+  note = "Need help"
+} | ConvertTo-Json -Depth 5
+
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/step" `
+  -Method Post `
+  -Body $body `
+  -ContentType "application/json"
+```
+
+Expected:
+- small negative reward
+- feedback containing `Hint 1: ...`
+
+### Test 8: Finalize A Full Submission
+
+```powershell
+$body = @{
+  operation = "finalize"
+  findings = @(
+    @{
+      title = "Avoid eval on untrusted configuration data"
+      line = 2
+      category = "security"
+      severity = "critical"
+      rationale = "eval can execute attacker-controlled code."
+      recommendation = "Use json.loads or ast.literal_eval."
+      rule_id = "avoid-eval"
+    },
+    @{
+      title = "Default count of zero causes a division by zero"
+      line = 5
+      category = "bug"
+      severity = "warning"
+      rationale = "count defaults to zero and division crashes."
+      recommendation = "Validate count before dividing."
+      rule_id = "division-by-zero-default"
+    }
+  )
+  patched_code = $null
+  note = "Final review"
+} | ConvertTo-Json -Depth 6
+
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/step" `
+  -Method Post `
+  -Body $body `
+  -ContentType "application/json"
+```
+
+Expected:
+- `done = true`
+- `evaluation.passed = true`
+- `score` near or above task threshold
+
+### Test 9: Inspect State
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/state" -Method Get
+```
+
+### Test 10: Inspect Schemas
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/schema" -Method Get
+```
+
+### Test 11: Grade A Task Without Running An Episode
+
+```powershell
+$body = @{
+  operation = "submit_findings"
+  findings = @(
+    @{
+      title = "shell=True with interpolated input allows command injection"
+      line = 10
+      category = "security"
+      severity = "critical"
+      rationale = "The command string includes user input and runs via shell."
+      recommendation = "Pass args as a list and keep shell=False."
+      rule_id = "shell-true-command-injection"
+    }
+  )
+  patched_code = $null
+  note = "Offline grader test"
+} | ConvertTo-Json -Depth 6
+
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/tasks/py-review-hard/grade" `
+  -Method Post `
+  -Body $body `
+  -ContentType "application/json"
+```
+
+### Test 12: Config Read And Update
+
+Read:
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/config" -Method Get
+```
+
+Update:
+
+```powershell
+$body = @{
+  task_order = @("py-review-easy", "py-review-medium", "py-review-hard")
+  max_steps_per_task = 4
+  hint_penalty = 0.05
+  false_positive_penalty = 0.08
+  duplicate_penalty = 0.03
+  patch_bonus_multiplier = 0.2
+  max_history_entries = 50
+} | ConvertTo-Json
+
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/config" `
+  -Method Put `
+  -Body $body `
+  -ContentType "application/json"
+```
+
+### Test 13: History
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/history" -Method Get
+```
+
+Clear:
+
+```powershell
+Invoke-RestMethod -Uri "http://127.0.0.1:8000/history" -Method Delete
+```
+
+## 6. How To Test Using The Inference Script
+
+The inference script is for model-vs-environment evaluation.
+
+### Required Variables
+
+```powershell
+$env:API_BASE_URL="https://api.openai.com/v1"
+$env:MODEL_NAME="gpt-4.1-mini"
+$env:OPENAI_API_KEY="your_key_here"
+```
+
+If you want it to hit your local server instead of launching Docker:
+
+```powershell
+$env:ENV_BASE_URL="http://127.0.0.1:8000"
+```
+
+Optional:
+
+```powershell
+$env:MAX_TASKS="3"
+$env:MAX_STEPS="3"
+$env:INFERENCE_REPORT_PATH="inference_results.json"
+```
+
+Run:
+
+```powershell
+python inference.py
+```
+
+What it does:
+
+1. connects to the environment
+2. resets through up to 3 tasks
+3. sends task code and feedback to the model
+4. expects strict JSON findings back
+5. submits them through `step()`
+6. logs score and reward per step
+7. writes a final report JSON file
+
+### How To Interpret The Output
+
+Focus on:
+
+- `mean_score`
+  Overall average benchmark score
+
+- per-task `score`
+  How well the model solved each task
+
+- `passed`
+  Whether score met that task’s threshold
+
+- step logs
+  Show whether the model is improving over trajectory or getting stuck
+
+If the model keeps returning empty findings:
+
+- improve the system prompt
+- reduce task ambiguity
+- add examples of desired findings
+- ensure the model endpoint supports the chosen format well
+
+## 7. How To Build Better Training Data
+
+If you want an RL environment to actually learn, the biggest bottleneck is data quality.
+
+You need more than just three final benchmark tasks. You need trajectories, partial attempts, and failure examples.
+
+### Data Types You Should Collect
+
+#### A. Gold Task Rubrics
+
+For each task, store:
+
+- code snippet
+- hidden reference findings
+- severity
+- category
+- expected line numbers
+- good recommendations
+
+This is already partially represented by `server/task_bank.py`.
+
+#### B. Positive Demonstrations
+
+Create solved examples where the review is high quality.
+
+Each example should include:
+
+- task code
+- one or more strong findings
+- strong rationales
+- strong recommendations
+- optional patch
+- final score
+
+This helps supervised warm-start and behavior cloning.
+
+#### C. Partial Trajectories
+
+This is important for RL.
+
+Store intermediate attempts like:
+
+- first attempt finds one issue
+- second attempt adds another issue
+- third attempt finalizes
+
+This is what teaches agents to improve over time, not just emit one final perfect answer.
+
+#### D. Negative Examples
+
+You should also store:
+
+- false positives
+- irrelevant complaints
+- duplicate findings
+- hallucinated issues
+- weak recommendations
+
+Why:
+- the reward function penalizes these
+- the model must learn precision, not just recall
+
+#### E. Hint Usage Examples
+
+Store trajectories where:
+
+- the agent requests a hint
+- then improves its findings
+
+This teaches policy behavior around when hints are worth the penalty.
+
+#### F. Patch Examples
+
+For tasks where patch quality matters, store:
+
+- original code
+- weak patch
+- good patch
+- patch score
+
+This helps the model learn that code edits should remove actual problems, not just change formatting.
+
+## 8. Recommended Dataset Format
+
+Use JSONL so it is easy to stream and train on.
+
+### Benchmark Task Record
+
+```json
+{
+  "task_id": "py-review-easy",
+  "difficulty": "easy",
+  "code": "def load_settings(config_text):\n    return eval(config_text)",
+  "reference_findings": [
+    {
+      "rule_id": "avoid-eval",
+      "line": 2,
+      "category": "security",
+      "severity": "critical"
+    }
+  ]
+}
+```
+
+### Trajectory Record
+
+```json
+{
+  "task_id": "py-review-medium",
+  "episode_id": "abc123",
+  "steps": [
+    {
+      "observation_feedback": "Review the Python snippet.",
+      "action": {
+        "operation": "submit_findings",
+        "findings": [
+          {
+            "title": "Mutable default argument leaks state",
+            "line": 1,
+            "category": "bug",
+            "severity": "warning"
+          }
+        ]
+      },
+      "reward": 0.35,
+      "score": 0.35
+    },
+    {
+      "observation_feedback": "Matched 1 new rubric item(s): mutable-default-list",
+      "action": {
+        "operation": "finalize",
+        "findings": [
+          {
+            "title": "Mutable default argument leaks state",
+            "line": 1,
+            "category": "bug",
+            "severity": "warning"
+          },
+          {
+            "title": "Bare except hides failures",
+            "line": 12,
+            "category": "maintainability",
+            "severity": "warning"
+          }
+        ]
+      },
+      "reward": 0.27,
+      "score": 0.62
+    }
+  ]
+}
+```
+
+## 9. How To Make RL Learn Better
+
+### A. Add More Tasks
+
+Three tasks are enough for the minimum requirement, but not enough for strong training.
+
+You should expand with:
+
+- file I/O bugs
+- API misuse
+- SQL injection
+- unsafe deserialization
+- concurrency issues
+- caching mistakes
+- resource leaks
+- logic edge cases
+
+Target:
+
+- 50 to 200 deterministic tasks
+- grouped by difficulty and domain
+
+### B. Add More Partial Reward Signals
+
+Current reward is already better than binary success/fail, but you can improve it.
+
+Possible additions:
+
+- small bonus when the first critical issue is found early
+- higher reward for critical issues than style issues
+- bonus when rationale quality is high
+- bonus when recommendation mentions a correct mitigation pattern
+- penalty if line numbers are missing when they should be known
+
+### C. Improve Context In Observation
+
+Right now the observation already gives:
+
+- task metadata
+- previous feedback
+- submitted findings
+- attempts remaining
+
+You can improve learning further by including:
+
+- a short list of matched findings so far
+- a short list of remaining categories not yet covered
+- normalized review rubric hints without leaking answers
+- last action summary
+
+This helps the agent reason about what it already did and what is still missing.
+
+### D. Separate Training Tasks From Benchmark Tasks
+
+Important:
+
+- training tasks should be large and varied
+- benchmark tasks should stay hidden and fixed
+
+Do not train directly on the same exact benchmark set you plan to judge on.
+
+### E. Add Preference Data
+
+You can train preference models on:
+
+- strong vs weak findings
+- precise vs vague recommendations
+- useful vs noisy patches
+
+This is valuable for ranking quality beyond exact rubric matches.
+
+## 10. Functional Requirements Mapping
+
+Here is how your environment should be judged against the stated requirements.
+
+### Requirement: Real-World Task Simulation
+
+Status:
+- satisfied in direction
+
+Why:
+- code review is a genuine engineering task
+
+How to improve further:
+- expand beyond tiny snippets into multi-function modules
+- include operational and maintainability review, not just security lints
+
+### Requirement: OpenEnv Spec Compliance
+
+Status:
+- mostly implemented in code
+
+Implemented pieces:
+- typed action model
+- typed observation model
+- `reset()`
+- `step()`
+- `state`
+- `openenv.yaml`
+- FastAPI/OpenEnv routes
+
+What you still need to verify:
+- `openenv validate`
+- schema compatibility under your installed OpenEnv version
+
+### Requirement: Minimum 3 Tasks With Agent Graders
+
+Status:
+- implemented
+
+You have:
+- easy
+- medium
+- hard
+- deterministic grader returning `0.0` to `1.0`
+
+### Requirement: Meaningful Reward Function
+
+Status:
+- implemented
+
+Current reward signals:
+- new rubric matches
+- false positive penalties
+- duplicate penalties
+- hint penalties
+- patch bonus
+- finalize pass bonus
+
+### Requirement: Baseline Inference Script
+
+Status:
+- implemented
+
+Current `inference.py`:
+- uses OpenAI client
+- reads env vars
+- runs tasks
+- writes report
+
+What to verify:
+- actual runtime under 20 minutes
+- reproducible output with your chosen model endpoint
+
+### Requirement: HF Spaces + Docker
+
+Status:
+- code is prepared
+
+You still need to verify:
+
+- `docker build -f server/Dockerfile .`
+- local container startup
+- `openenv push`
+- `/health` returns 200 on the deployed Space
+
+## 11. Recommended Manual Validation Checklist
+
+Before submission, run these in order:
+
+1. Start server locally
+2. Hit `/health`
+3. Hit `/docs`
+4. Test `/tasks`
+5. Test `/review` with unsafe examples
+6. Test `/reset`
+7. Test `/step` with partial findings
+8. Test `/step` with finalize
+9. Test `/tasks/{task_id}/grade`
+10. Run `pytest`
+11. Run `openenv validate`
+12. Run `python inference.py`
+13. Build Docker image
+14. Deploy to Hugging Face Space
+15. Re-test `/health` and `/reset` on the live Space
+
+## 12. Suggested Immediate Next Steps
+
+If you want the environment to become stronger quickly, do this next:
+
+1. Add 10 to 20 more benchmark-style tasks in `server/task_bank.py`
+2. Save solved and failed trajectories as JSONL files under a new `dataset/` directory
+3. Refactor custom route state so `/history` and OpenEnv `/step` share one coherent session story
+4. Run `openenv validate`
+5. Run `inference.py` against your local server and inspect the report
+
+## 13. Quick Commands Summary
+
+Start server:
+
+```powershell
+uvicorn server.app:app --reload --host 0.0.0.0 --port 8000
+```
+
+Open docs:
+
+```text
+http://127.0.0.1:8000/docs
+```
+
+Run example tests:
+
+```powershell
+python -m pytest tests -q
+```
+
+Run inference locally:
+
+```powershell
+$env:API_BASE_URL="https://api.openai.com/v1"
+$env:MODEL_NAME="gpt-4.1-mini"
+$env:OPENAI_API_KEY="your_key"
+$env:ENV_BASE_URL="http://127.0.0.1:8000"
+python inference.py
+```
+
+Validate OpenEnv:
+
+```powershell
+openenv validate
+```
+
+Build Docker:
+
+```powershell
+docker build -t python_env-env:latest -f server/Dockerfile .
+```
+
+Deploy:
+
+```powershell
+openenv push
+```

README.md

@@ -1,266 +1,272 @@
 ---
-title: Python Env Environment Server
-emoji: 🎶
-colorFrom: purple
-colorTo: red
+title: Python Code Review Environment Server
 sdk: docker
-pinned: false
 app_port: 8000
 base_path: /web
+pinned: false
 tags:
   - openenv
+  - code-review
 ---
 
-# Python Env Environment
-
-A simple test environment that echoes back messages. Perfect for testing the env APIs as well as demonstrating environment usage patterns.
-
-## Quick Start
-
-The simplest way to use the Python Env environment is through the `PythonEnv` class:
-
-```python
-from python_env import PythonAction, PythonEnv
-
-try:
-    # Create environment from Docker image
-    python_envenv = PythonEnv.from_docker_image("python_env-env:latest")
-
-    # Reset
-    result = python_envenv.reset()
-    print(f"Reset: {result.observation.echoed_message}")
-
-    # Send multiple messages
-    messages = ["Hello, World!", "Testing echo", "Final message"]
-
-    for msg in messages:
-        result = python_envenv.step(PythonAction(message=msg))
-        print(f"Sent: '{msg}'")
-        print(f"  → Echoed: '{result.observation.echoed_message}'")
-        print(f"  → Length: {result.observation.message_length}")
-        print(f"  → Reward: {result.reward}")
-
-finally:
-    # Always clean up
-    python_envenv.close()
-```
-
-That's it! The `PythonEnv.from_docker_image()` method handles:
-- Starting the Docker container
-- Waiting for the server to be ready
-- Connecting to the environment
-- Container cleanup when you call `close()`
-
-## Building the Docker Image
-
-Before using the environment, you need to build the Docker image:
-
-```bash
-# From project root
-docker build -t python_env-env:latest -f server/Dockerfile .
-```
-
-## Deploying to Hugging Face Spaces
-
-You can easily deploy your OpenEnv environment to Hugging Face Spaces using the `openenv push` command:
-
-```bash
-# From the environment directory (where openenv.yaml is located)
-openenv push
-
-# Or specify options
-openenv push --namespace my-org --private
-```
-
-The `openenv push` command will:
-1. Validate that the directory is an OpenEnv environment (checks for `openenv.yaml`)
-2. Prepare a custom build for Hugging Face Docker space (enables web interface)
-3. Upload to Hugging Face (ensuring you're logged in)
-
-### Prerequisites
-
-- Authenticate with Hugging Face: The command will prompt for login if not already authenticated
-
-### Options
-
-- `--directory`, `-d`: Directory containing the OpenEnv environment (defaults to current directory)
-- `--repo-id`, `-r`: Repository ID in format 'username/repo-name' (defaults to 'username/env-name' from openenv.yaml)
-- `--base-image`, `-b`: Base Docker image to use (overrides Dockerfile FROM)
-- `--private`: Deploy the space as private (default: public)
-
-### Examples
-
-```bash
-# Push to your personal namespace (defaults to username/env-name from openenv.yaml)
-openenv push
-
-# Push to a specific repository
-openenv push --repo-id my-org/my-env
-
-# Push with a custom base image
-openenv push --base-image ghcr.io/meta-pytorch/openenv-base:latest
-
-# Push as a private space
-openenv push --private
-
-# Combine options
-openenv push --repo-id my-org/my-env --base-image custom-base:latest --private
-```
-
-After deployment, your space will be available at:
-`https://huggingface.co/spaces/<repo-id>`
-
-The deployed space includes:
-- **Web Interface** at `/web` - Interactive UI for exploring the environment
-- **API Documentation** at `/docs` - Full OpenAPI/Swagger interface
-- **Health Check** at `/health` - Container health monitoring
-- **WebSocket** at `/ws` - Persistent session endpoint for low-latency interactions
-
-## Environment Details
-
-### Action
-**PythonAction**: Contains a single field
-- `message` (str) - The message to echo back
-
-### Observation
-**PythonObservation**: Contains the echo response and metadata
-- `echoed_message` (str) - The message echoed back
-- `message_length` (int) - Length of the message
-- `reward` (float) - Reward based on message length (length × 0.1)
-- `done` (bool) - Always False for echo environment
-- `metadata` (dict) - Additional info like step count
-
-### Reward
-The reward is calculated as: `message_length × 0.1`
-- "Hi" → reward: 0.2
-- "Hello, World!" → reward: 1.3
-- Empty message → reward: 0.0
-
-## Advanced Usage
-
-### Connecting to an Existing Server
-
-If you already have a Python Env environment server running, you can connect directly:
-
-```python
-from python_env import PythonEnv
-
-# Connect to existing server
-python_envenv = PythonEnv(base_url="<ENV_HTTP_URL_HERE>")
-
-# Use as normal
-result = python_envenv.reset()
-result = python_envenv.step(PythonAction(message="Hello!"))
-```
-
-Note: When connecting to an existing server, `python_envenv.close()` will NOT stop the server.
-
-### Using the Context Manager
-
-The client supports context manager usage for automatic connection management:
-
-```python
-from python_env import PythonAction, PythonEnv
-
-# Connect with context manager (auto-connects and closes)
-with PythonEnv(base_url="http://localhost:8000") as env:
-    result = env.reset()
-    print(f"Reset: {result.observation.echoed_message}")
-    # Multiple steps with low latency
-    for msg in ["Hello", "World", "!"]:
-        result = env.step(PythonAction(message=msg))
-        print(f"Echoed: {result.observation.echoed_message}")
-```
-
-The client uses WebSocket connections for:
-- **Lower latency**: No HTTP connection overhead per request
-- **Persistent session**: Server maintains your environment state
-- **Efficient for episodes**: Better for many sequential steps
-
-### Concurrent WebSocket Sessions
-
-The server supports multiple concurrent WebSocket connections. To enable this,
-modify `server/app.py` to use factory mode:
-
-```python
-# In server/app.py - use factory mode for concurrent sessions
-app = create_app(
-    PythonEnvironment,  # Pass class, not instance
-    PythonAction,
-    PythonObservation,
-    max_concurrent_envs=4,  # Allow 4 concurrent sessions
-)
-```
-
-Then multiple clients can connect simultaneously:
-
-```python
-from python_env import PythonAction, PythonEnv
-from concurrent.futures import ThreadPoolExecutor
-
-def run_episode(client_id: int):
-    with PythonEnv(base_url="http://localhost:8000") as env:
-        result = env.reset()
-        for i in range(10):
-            result = env.step(PythonAction(message=f"Client {client_id}, step {i}"))
-        return client_id, result.observation.message_length
-
-# Run 4 episodes concurrently
-with ThreadPoolExecutor(max_workers=4) as executor:
-    results = list(executor.map(run_episode, range(4)))
-```
-
-## Development & Testing
-
-### Direct Environment Testing
-
-Test the environment logic directly without starting the HTTP server:
-
-```bash
-# From the server directory
-python3 server/python_env_environment.py
-```
-
-This verifies that:
-- Environment resets correctly
-- Step executes actions properly
-- State tracking works
-- Rewards are calculated correctly
-
-### Running Locally
-
-Run the server locally for development:
-
-```bash
-uvicorn server.app:app --reload
-```
-
-## Project Structure
-
-```
-python_env/
-├── .dockerignore         # Docker build exclusions
-├── __init__.py            # Module exports
-├── README.md              # This file
-├── openenv.yaml           # OpenEnv manifest
-├── pyproject.toml         # Project metadata and dependencies
-├── uv.lock                # Locked dependencies (generated)
-├── client.py              # PythonEnv client
-├── models.py              # Action and Observation models
-└── server/
-    ├── __init__.py        # Server module exports
-    ├── python_env_environment.py  # Core environment logic
-    ├── app.py             # FastAPI application (HTTP + WebSocket endpoints)
-    └── Dockerfile         # Container image definition
-```
----------------------------------------
-
-cd F:\python_env
-  # Edit your environment implementation in server/python_env_environment.py
-  # Edit your models in models.py
-  # Install dependencies: uv sync
-
-  # To integrate into OpenEnv repo:
-  # 1. Copy this directory to <repo_root>/envs/python_env_env
-  # 2. Build from repo root: docker build -t python_env_env:latest -f envs/python_env_env/server/Dockerfile .
-  # 3. Run your image: docker run -p 8000:8000 python_env_env:latest
+# Python Code Review Environment
+
+A production-grade OpenEnv environment for Python code review, repair, and optimization tasks. This environment simulates real-world developer workflows where an AI agent reviews, fixes, and improves Python code.
+
+## Overview
+
+**`python_code_review_env`** is a deterministic benchmark environment featuring:
+
+- ✅ **3 real-world tasks** with increasing difficulty (Syntax, Bug Fix, Optimization)
+- ✅ **Deterministic graders** using AST analysis, pytest execution, and performance benchmarking
+- ✅ **OpenAI-compatible API** supporting free/open models (Gemini, DeepSeek, Together, OpenRouter)
+- ✅ **Production-ready Docker** deployment for Hugging Face Spaces
+- ✅ **Structured Observations & Actions** following OpenEnv spec
+- ✅ **Rich reward shaping** with bonuses for syntax fixes, test passes, and optimization
+
+## Tasks
+
+### 1. 🟢 Easy: Syntax Fixing
+
+**Task ID**: `syntax-fix-easy`
+
+Fix broken Python code with syntax errors.
+
+- **Difficulty**: Easy
+- **Goal**: Repair syntax errors to make code compile
+- **Starter Code**: Function with missing closing parenthesis
+- **Grading**: Compilation check + code similarity to reference
+- **Score Range**: 0.0–1.0
+
+### 2. 🟡 Medium: Bug Fixing
+
+**Task ID**: `bug-fix-medium`
+
+Fix logic bugs with visible and hidden test cases.
+
+- **Difficulty**: Medium  
+- **Goal**: Repair a logic error in invoice calculation
+- **Starter Code**: Function that returns wrong total (returns subtotal instead of discounted)
+- **Grading**: Test pass fraction (visible & hidden)
+- **Score Range**: 0.0–1.0
+
+### 3. 🔴 Hard: Optimization & Refactoring
+
+**Task ID**: `optimization-hard`
+
+Optimize inefficient code while maintaining correctness.
+
+- **Difficulty**: Hard
+- **Goal**: Convert O(n²) duplicate removal to O(n) with set
+- **Starter Code**: Slow nested-loop implementation
+- **Grading**: 50% correctness + 30% speedup + 15% code quality + 5% style
+- **Score Range**: 0.0–1.0
+- **Bonus**: Runtime benchmarking against reference implementation
+
+## Quick Start
+
+### Run Locally
+
+```bash
+cd python-code-review-env
+pip install -r server/requirements.txt
+python -m server.app
+```
+
+Visit http://localhost:8000/docs for interactive API
+
+### Run with Docker
+
+```bash
+docker build -f server/Dockerfile -t python_code_review_env:latest .
+docker run -p 8000:8000 python_code_review_env:latest
+```
+
+### Run Inference
+
+```bash
+python inference.py --model "gpt-3.5-turbo" --base-url "http://localhost:8000/v1"
+```
+
+## OpenEnv Specification
+
+### Observation
+
+```json
+{
+  "task_id": "syntax-fix-easy",
+  "difficulty": "easy",
+  "task_description": "Fix syntax errors...",
+  "current_code": "def normalize_username(raw_name: str) -> str:\n    cleaned = raw_name.strip().lower(\n    ...",
+  "errors": "invalid syntax ( line 2, column 40 )",
+  "test_results": "Not run yet.",
+  "visible_tests": ["normalize_username('  Alice Smith  ') == 'alice_smith'"],
+  "history": [],
+  "attempts_remaining": 8,
+  "score": 0.0,
+  "reward": {
+    "value": 0.0,
+    "reason": "Episode reset."
+  }
+}
+```
+
+### Action
+
+```json
+{
+  "action_type": "edit_code",
+  "code": "def normalize_username(raw_name: str) -> str:\n    cleaned = raw_name.strip().lower()\n    if not cleaned:\n        return \"anonymous\"\n    return cleaned.replace(\" \", \"_\")"
+}
+```
+
+### Reward Details
+
+- **+0.2**: Syntax fixed (one-time per episode)
+- **+0.15**: Passing additional test (cumulative per test)
+- **+0.1**: Code quality improvement  
+- **+0.5**: Full correctness (100% hidden tests, one-time)
+- **-0.1**: Invalid action
+
+## Architecture
+
+```
+python_code_review_env/
+├── models.py          # Pydantic models (Observation, Action, Reward)
+├── server/
+│   ├── app.py         # FastAPI server  
+│   ├── env.py         # OpenEnv environment
+│   ├── Dockerfile     # Docker config
+│   └── requirements.txt
+├── graders/
+│   ├── common.py      # Shared utilities
+│   ├── syntax.py      # Syntax/bug graders
+│   ├── optimization.py# Optimization grader
+│   └── pytest_runner.py
+├── tasks/
+│   ├── task_bank.py   # 3 deterministic tasks
+│   └── __init__.py
+├── inference.py       # Baseline evaluation script
+├── openenv.yaml       # OpenEnv spec
+├── pyproject.toml     # Project metadata
+└── README.md
+```
+
+## FastAPI Endpoints
+
+- `GET /health` – Health check
+- `GET /tasks` – List all tasks
+- `GET /tasks/{task_id}` – Get task details
+- `POST /tasks/{task_id}/grade` – Grade code offline
+- Standard OpenEnv endpoints (`/reset`, `/step`, `/state`)
+
+## Deterministic Graders
+
+### Syntax Fix
+```
+if code compiles:
+  score = 1.0
+else:
+  score = 0.15 + 0.55 * similarity_to_reference
+```
+
+### Bug Fix  
+```
+score = test_pass_fraction (0.0 to 1.0)
+```
+
+### Optimization
+```
+score = (
+  0.5 * test_fraction +
+  0.3 * speedup_score +
+  0.15 * code_quality +
+  0.05 * pep8_style
+)
+```
+
+## Examples
+
+### Using Python
+
+```python
+from server.env import PythonCodeReviewEnvironment
+from models import PythonCodeReviewAction
+
+env = PythonCodeReviewEnvironment()
+obs = env.reset(task_id="syntax-fix-easy")
+
+action = PythonCodeReviewAction(
+    action_type="edit_code",
+    code="""def normalize_username(raw_name: str) -> str:
+    cleaned = raw_name.strip().lower()
+    if not cleaned:
+        return "anonymous"
+    return cleaned.replace(" ", "_")
+"""
+)
+
+obs = env.step(action)
+print(f"Score: {obs.score}")
+print(f"Reward: {obs.reward.value:+.3f}")
+```
+
+### Using cURL
+
+```bash
+# Check health
+curl http://localhost:8000/health
+
+# List tasks
+curl http://localhost:8000/tasks
+
+# Grade code
+curl -X POST http://localhost:8000/tasks/syntax-fix-easy/grade \
+  -H "Content-Type: application/json" \
+  -d '{"action_type": "edit_code", "code": "..."}'
+```
+
+## Deployment
+
+### Hugging Face Spaces
+
+1. Create Space > Docker
+2. Upload files + `server/Dockerfile`
+3. Space auto-deploys on CPU
+4. Monitor `/health` endpoint
+
+### Local Docker
+
+```bash
+docker build -f server/Dockerfile -t python_code_review_env .
+docker run -p 8000:8000 \
+  -e MAX_CONCURRENT_ENVS=16 \
+  python_code_review_env
+```
+
+## Performance
+
+- Startup: < 5s
+- Reset: < 100ms
+- Step: 50ms–3s (depends on action)
+- Inference (3 tasks): < 20 minutes
+- CPU: Works on 2 vCPU, 8GB RAM
+
+## Validation Checklist
+
+- ✅ 3 deterministic tasks
+- ✅ Deterministic graders (AST, pytest, benchmarks)
+- ✅ `/health` → 200
+- ✅ Scores vary per task (not constant)
+- ✅ Docker builds successfully
+- ✅ OpenEnv spec compliant
+- ✅ Reward shaping working
+- ✅ All tests deterministic and reproducible
+
+## License
+
+MIT
+
+---
+
+**Built for production. Deterministic. Deployable. Extensible.**

REWARD_SYSTEM_GUIDE.md

@@ -0,0 +1,206 @@
+# Reward System Implementation Guide
+
+This document shows how the reward system is implemented in code and how to use it.
+
+## Module Documentation
+
+The reward system architecture is documented at the module level:
+
+```python
+import server.env
+print(server.env.__doc__)
+```
+
+Output shows all 6 reward components and the final computation formula.
+
+## Reward Constants
+
+All reward constants are defined in `server/env.py` (lines 57-87):
+
+```python
+# Component 1: Score improvement reward
+PROGRESS_SCALE = 0.25
+
+# Component 2: Syntax/compilation fix reward
+SYNTAX_FIX_BONUS = 0.35
+
+# Component 3: Test improvement reward
+TEST_PASS_REWARD_SCALE = 0.30
+
+# Component 4: Code quality reward
+QUALITY_BONUS_SCALE = 0.15
+
+# Component 5: Stagnation penalty
+STAGNATION_PENALTY = 0.10
+
+# Component 6: Regression penalty
+REGRESSION_PENALTY_SCALE = 0.20
+
+# One-time completion bonus
+COMPLETION_BONUS = 0.50
+
+# Invalid/error penalties
+INVALID_ACTION_PENALTY = 0.15
+TIMEOUT_PENALTY = 0.15
+```
+
+To tune the reward system, edit these constants and re-test.
+
+## RewardDetails Model Documentation
+
+Located in `models.py` (lines 26-80):
+
+```python
+from models import RewardDetails
+print(RewardDetails.__doc__)
+```
+
+Shows all 15 fields and their meanings:
+- `value`: Final scalar reward [-1.0, +1.0]
+- `progress_delta`: Score improvement component
+- `syntax_reward`: Syntax fix bonus
+- `test_reward`: Test improvement bonus
+- `quality_bonus`: Code quality improvement
+- `stagnation_penalty`: Unchanged code penalty
+- `regression_penalty`: Score decline penalty
+- `reason`: Human-readable explanation
+- `prev_score`, `curr_score`: Score before/after
+- `code_changed`: Whether code was modified
+
+## Core Computation Method
+
+The main reward computation is in `_compute_reward_components()` (server/env.py, lines 507-703):
+
+```python
+def _compute_reward_components(
+    self,
+    curr_score: float,
+    prev_score: float,
+    curr_grade: TaskGrade,
+    code_changed: bool,
+    prev_grade_score: float = 0.0,
+) -> dict:
+    """Compute all six reward components and return combined result."""
+```
+
+### What It Does
+
+1. **Initializes** empty component dict
+2. **Computes each component**:
+   - Progress: Score improvement scaled by PROGRESS_SCALE
+   - Syntax: One-time bonus if first compile
+   - Test: Test pass rate improvement scaled by TEST_PASS_REWARD_SCALE
+   - Quality: Code quality improvement scaled by QUALITY_BONUS_SCALE
+   - Stagnation: Penalty if code unchanged
+   - Regression: Penalty if score decreased
+3. **Combines**: Sums positives, subtracts negatives
+4. **Clamps**: Bounds result to [-1.0, +1.0]
+
+### Key Design Decisions
+
+- **Monotonic tracking**: Best test rate and quality in episode are tracked
+- **One-time bonuses**: Syntax reward awarded once per episode
+- **Scale capping**: Each component has a maximum (e.g., progress max +0.25)
+- **Timeout handling**: Special penalty instead of score-based
+- **Clamping**: Final reward bounded for numerical stability
+
+## Debug Logging
+
+When `verbose=True`, the environment prints detailed debug output via `_log_debug_step()`:
+
+```python
+env = PythonCodeReviewEnvironment(verbose=True)
+obs = env.reset()
+obs = env.step(action)
+```
+
+Output format:
+```
+Step  1 | Score: 0.698 | Delta: +0.698 | Reward: +0.4239 | Changed: False
+         | Progress=+0.174 | Quality=+0.149 | Stagnation=+0.100
+         | Reason: Syntax error detected: '(' was never closed
+```
+
+Shows:
+- Step number
+- Current score and delta from previous
+- Final reward value
+- Whether code changed
+- Non-zero components only
+- Human-readable reason
+
+## Example: Full Episode with Rewards
+
+```python
+from server.env import PythonCodeReviewEnvironment
+from models import PythonCodeReviewAction
+
+env = PythonCodeReviewEnvironment(verbose=True)
+obs = env.reset(task_id='syntax-fix-easy')
+
+# Step 1: Analyze (no code change)
+action = PythonCodeReviewAction(action_type='analyze_code')
+obs = env.step(action)
+print(f"Reward 1: {obs.reward_details.value:.4f}")
+
+# Step 2: Edit with fix
+code = 'x = 1; y = 2; print(x + y)'
+action = PythonCodeReviewAction(action_type='edit_code', code=code)
+obs = env.step(action)
+print(f"Reward 2: {obs.reward_details.value:.4f}")
+
+# Step 3: Submit
+action = PythonCodeReviewAction(action_type='submit_solution')
+obs = env.step(action)
+print(f"Final Reward: {obs.reward_details.value:.4f}")
+```
+
+## Interpreting Rewards
+
+### Positive Rewards (+0 to +1.0)
+- **+0.5 - +1.0**: Major progress (syntax fix, many tests passing)
+- **+0.2 - +0.5**: Good progress (score improvement, test gains)
+- **+0.0 - +0.2**: Small progress (quality improvement, minor gains)
+
+### Negative Rewards (−1.0 to −0)
+- **−0.1 - 0**: Stagnation (analyzed without changing code)
+- **−0.2 - −0.1**: Slight regression (small score drop)
+- **−0.5 - −0.2**: Major regression (significant score drop)
+- **−1.0 - −0.5**: Invalid action or timeout
+
+## Tuning the Reward System
+
+### For Faster Early Learning
+↑ Increase `SYNTAX_FIX_BONUS` and `COMPLETION_BONUS`
+
+### To Encourage Editing Over Analysis
+↑ Increase `STAGNATION_PENALTY`
+
+### To Reward Test Improvements More
+↑ Increase `TEST_PASS_REWARD_SCALE`
+
+### To Penalize Mistakes More
+↑ Increase `REGRESSION_PENALTY_SCALE`
+
+### To Balance All Components
+Adjust the Scale constants (all in range 0.15-0.35 for stability)
+
+## Accessing Documentation Programmatically
+
+```python
+from server.env import PythonCodeReviewEnvironment
+from models import RewardDetails
+import server.env
+
+# Module-level architecture
+print(server.env.__doc__)
+
+# RewardDetails fields
+print(RewardDetails.__doc__)
+
+# One method
+env = PythonCodeReviewEnvironment()
+help(env._compute_reward_components)
+```
+
+All major functions and classes have comprehensive docstrings.

__init__.py

@@ -1,16 +1,40 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
+"""Public package API for the Python code review OpenEnv benchmark."""
 
-"""Python Env Environment."""
-
-from .client import PythonEnv
-from .models import PythonAction, PythonObservation
+try:
+    from .client import CodeReviewEnv, MyEnv, PythonEnv
+    from .models import (
+        HealthResponse,
+        HistoryEntry,
+        PythonCodeReviewAction,
+        PythonCodeReviewObservation,
+        PythonCodeReviewState,
+        RewardDetails,
+        TaskDescriptor,
+        TaskGrade,
+    )
+except ImportError:  # pragma: no cover
+    from client import CodeReviewEnv, MyEnv, PythonEnv
+    from models import (
+        HealthResponse,
+        HistoryEntry,
+        PythonCodeReviewAction,
+        PythonCodeReviewObservation,
+        PythonCodeReviewState,
+        RewardDetails,
+        TaskDescriptor,
+        TaskGrade,
+    )
 
 __all__ = [
-    "PythonAction",
-    "PythonObservation",
     "PythonEnv",
+    "CodeReviewEnv",
+    "MyEnv",
+    "PythonCodeReviewAction",
+    "PythonCodeReviewObservation",
+    "PythonCodeReviewState",
+    HealthResponse,
+    HistoryEntry,
+    RewardDetails,
+    TaskDescriptor,
+    TaskGrade,
 ]

client.py

@@ -1,46 +1,75 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
+"""Client for the Python code review environment."""
+
+from __future__ import annotations
 
-"""Python Env Environment Client."""
+from typing import Dict
 
-from typing import Any, Dict
+from compat import install_openenv_fastmcp_compat
+
+install_openenv_fastmcp_compat()
 
 from openenv.core import EnvClient
 from openenv.core.client_types import StepResult
-from openenv.core.env_server.types import State
-
-try:
-    from .models import PythonAction, PythonObservation
-except ImportError:
-    from models import PythonAction, PythonObservation  # type: ignore
-
-
-class PythonEnv(EnvClient[PythonAction, PythonObservation, State]):
-    """Typed client for the Python code-review environment."""
-
-    def _step_payload(self, action: PythonAction) -> Dict[str, Any]:
-        """Convert a validated action model to the JSON payload expected by the server."""
-
-        return action.model_dump(exclude_none=True)
-
-    def _parse_result(self, payload: Dict[str, Any]) -> StepResult[PythonObservation]:
-        """Parse a server response into a typed step result."""
-
-        obs_data = dict(payload.get("observation", {}))
-        obs_data.setdefault("done", payload.get("done", False))
-        obs_data.setdefault("reward", payload.get("reward"))
-        observation = PythonObservation.model_validate(obs_data)
-
-        return StepResult(
-            observation=observation,
-            reward=payload.get("reward"),
-            done=payload.get("done", False),
-        )
-
-    def _parse_state(self, payload: Dict[str, Any]) -> State:
-        """Parse the server state payload into the shared state model."""
-
-        return State.model_validate(payload)
+
+from models import (
+    HistoryEntry,
+    PythonCodeReviewAction,
+    PythonCodeReviewObservation,
+    PythonCodeReviewState,
+    RewardDetails,
+)
+
+
+class PythonEnv(
+    EnvClient[PythonCodeReviewAction, PythonCodeReviewObservation, PythonCodeReviewState]
+):
+    """OpenEnv HTTP client for the Python code review benchmark."""
+
+    def _step_payload(self, action: PythonCodeReviewAction) -> Dict:
+        return action.model_dump(exclude_none=True)
+
+    def _parse_result(self, payload: Dict) -> StepResult[PythonCodeReviewObservation]:
+        obs = payload.get("observation", {})
+        observation = PythonCodeReviewObservation(
+            task_id=obs["task_id"],
+            title=obs["title"],
+            difficulty=obs["difficulty"],
+            task_kind=obs["task_kind"],
+            task_description=obs["task_description"],
+            current_code=obs.get("current_code", ""),
+            errors=obs.get("errors", ""),
+            test_results=obs.get("test_results", ""),
+            history=[HistoryEntry(**entry) for entry in obs.get("history", [])],
+            attempts_remaining=obs.get("attempts_remaining", 0),
+            last_action_status=obs.get("last_action_status", ""),
+            score=obs.get("score", 0.0),
+            reward_details=RewardDetails(**obs.get("reward_details", {})),
+            done=payload.get("done", obs.get("done", False)),
+            reward=payload.get("reward", obs.get("reward")),
+            metadata=obs.get("metadata", {}),
+        )
+        return StepResult(
+            observation=observation,
+            reward=payload.get("reward", obs.get("reward")),
+            done=payload.get("done", obs.get("done", False)),
+        )
+
+    def _parse_state(self, payload: Dict) -> PythonCodeReviewState:
+        return PythonCodeReviewState(
+            episode_id=payload.get("episode_id"),
+            step_count=payload.get("step_count", 0),
+            task_id=payload.get("task_id"),
+            difficulty=payload.get("difficulty"),
+            task_kind=payload.get("task_kind"),
+            attempts_remaining=payload.get("attempts_remaining", 0),
+            current_code=payload.get("current_code", ""),
+            errors=payload.get("errors", ""),
+            test_results=payload.get("test_results", ""),
+            history=[HistoryEntry(**entry) for entry in payload.get("history", [])],
+            score=payload.get("score", 0.0),
+            done=payload.get("done", False),
+        )
+
+
+CodeReviewEnv = PythonEnv
+MyEnv = PythonEnv

compat.py

@@ -0,0 +1,92 @@
+"""Compatibility helpers for OpenEnv and FastMCP runtime drift."""
+
+from __future__ import annotations
+
+import sys
+import types
+from typing import Any, Optional
+
+
+def install_openenv_fastmcp_compat() -> None:
+    """Patch FastMCP API differences so older OpenEnv builds keep importing."""
+    try:
+        import fastmcp  # type: ignore
+    except Exception:
+        return
+
+    try:
+        if not hasattr(fastmcp, "Client"):
+            class CompatClient:
+                """Minimal async MCP client used for legacy OpenEnv imports."""
+
+                def __init__(self, *args: Any, **kwargs: Any) -> None:
+                    self.args = args
+                    self.kwargs = kwargs
+
+                async def __aenter__(self) -> "CompatClient":
+                    return self
+
+                async def __aexit__(self, exc_type: Any, exc: Any, tb: Any) -> bool:
+                    return False
+
+                async def list_tools(self) -> list[Any]:
+                    return []
+
+                async def call_tool(self, tool_name: str, arguments: dict[str, Any]) -> Any:
+                    raise RuntimeError(
+                        f"MCP client compatibility mode cannot call tool: {tool_name}"
+                    )
+
+            fastmcp.Client = CompatClient  # type: ignore[attr-defined]
+    except Exception:
+        pass
+
+    try:
+        client_pkg = sys.modules.get("fastmcp.client")
+        if client_pkg is None:
+            client_pkg = types.ModuleType("fastmcp.client")
+            sys.modules["fastmcp.client"] = client_pkg
+
+        client_mod = sys.modules.get("fastmcp.client.client")
+        if client_mod is None:
+            client_mod = types.ModuleType("fastmcp.client.client")
+            sys.modules["fastmcp.client.client"] = client_mod
+
+        if not hasattr(client_mod, "CallToolResult"):
+            class CallToolResult:
+                """Compatibility container for legacy OpenEnv response handling."""
+
+                def __init__(
+                    self,
+                    content: Any = None,
+                    structured_content: Any = None,
+                    meta: Any = None,
+                    data: Any = None,
+                    is_error: bool = False,
+                ) -> None:
+                    self.content = content
+                    self.structured_content = structured_content
+                    self.meta = meta
+                    self.data = data
+                    self.is_error = is_error
+
+            client_mod.CallToolResult = CallToolResult
+
+        client_pkg.client = client_mod  # type: ignore[attr-defined]
+    except Exception:
+        pass
+
+
+install_openenv_fastmcp_compat()
+
+
+try:
+    from openenv.core.env_server.http_server import create_app as openenv_create_app
+    from openenv.core.env_server.interfaces import Environment
+    from openenv.core.env_server.types import Action, Observation, State
+except Exception as exc:  # pragma: no cover
+    raise RuntimeError(f"OpenEnv runtime import failed after compatibility patch: {exc}") from exc
+
+
+create_app = openenv_create_app
+

examples/__init__.py

@@ -0,0 +1 @@
+"""Example snippets for the Python review environment."""

examples/python_review_examples.py

@@ -0,0 +1,58 @@
+"""Example Python snippets for exercising the review environment."""
+
+EXAMPLE_SNIPPETS = {
+    "unsafe_eval": "\n".join(
+        [
+            "def load_settings(config_text):",
+            "    return eval(config_text)",
+        ]
+    ),
+    "mutable_default": "\n".join(
+        [
+            "def append_name(name, names=[]):",
+            "    names.append(name)",
+            "    return names",
+        ]
+    ),
+    "bare_except": "\n".join(
+        [
+            "def publish_report(report):",
+            "    try:",
+            '        return report[\"summary\"]',
+            "    except:",
+            "        return None",
+        ]
+    ),
+    "shell_injection": "\n".join(
+        [
+            "import subprocess",
+            "",
+            "def run_script(script_path, user_input):",
+            '    cmd = f\"python {script_path} {user_input}\"',
+            "    return subprocess.check_output(cmd, shell=True, text=True)",
+        ]
+    ),
+    "syntax_error": "\n".join(
+        [
+            "def broken_function(",
+            "    return 42",
+        ]
+    ),
+    "clean_function": "\n".join(
+        [
+            "def normalize_name(name: str) -> str:",
+            "    cleaned = name.strip().lower()",
+            "    return cleaned.replace(\"  \", \" \")",
+        ]
+    ),
+}
+
+
+EXPECTED_RULE_IDS = {
+    "unsafe_eval": {"avoid-eval"},
+    "mutable_default": {"mutable-default-list"},
+    "bare_except": {"bare-except"},
+    "shell_injection": {"shell-true-command-injection"},
+    "syntax_error": {"syntax-error"},
+    "clean_function": set(),
+}

graders/__init__.py

@@ -0,0 +1,16 @@
+"""Deterministic graders for the Python code review environment."""
+
+from .common import clamp_score
+from .optimization import grade_optimization_task
+from .pytest_runner import PytestExecution, run_pytest_suite
+from .syntax import grade_bug_fix_task, grade_syntax_task, grade_task
+
+__all__ = [
+    "PytestExecution",
+    "clamp_score",
+    "grade_bug_fix_task",
+    "grade_optimization_task",
+    "grade_syntax_task",
+    "grade_task",
+    "run_pytest_suite",
+]

graders/common.py

@@ -0,0 +1,82 @@
+"""Shared deterministic scoring helpers."""
+
+from __future__ import annotations
+
+import ast
+import difflib
+import traceback
+from typing import Tuple
+
+
+def clamp_score(value: float) -> float:
+    """Clamp any scalar score into the required 0..1 interval."""
+
+    return max(0.0, min(1.0, round(value, 6)))
+
+
+def syntax_error_message(code: str) -> str:
+    """Return a concise syntax error string or an empty string."""
+
+    try:
+        ast.parse(code)
+    except SyntaxError as exc:
+        return f"{exc.msg} (line {exc.lineno}, column {exc.offset})"
+    except Exception:  # pragma: no cover
+        return traceback.format_exc(limit=1).strip()
+    return ""
+
+
+def compiles(code: str) -> bool:
+    """Return whether the code parses and compiles."""
+
+    try:
+        compile(code, "<candidate>", "exec")
+    except Exception:
+        return False
+    return True
+
+
+def normalized_diff_score(code: str, reference_code: str) -> float:
+    """Score textual similarity to the reference solution."""
+
+    ratio = difflib.SequenceMatcher(
+        a="".join(code.split()),
+        b="".join(reference_code.split()),
+    ).ratio()
+    return clamp_score(ratio)
+
+
+def style_score(code: str, max_line_length: int = 88) -> float:
+    """Simple deterministic PEP8-inspired style score."""
+
+    lines = code.splitlines() or [""]
+    line_length_ok = sum(1 for line in lines if len(line) <= max_line_length) / len(lines)
+    tab_ok = 1.0 if all("\t" not in line for line in lines) else 0.0
+    trailing_ws_ok = 1.0 if all(line == line.rstrip() for line in lines) else 0.0
+    return clamp_score((line_length_ok * 0.6) + (tab_ok * 0.2) + (trailing_ws_ok * 0.2))
+
+
+def nested_loop_depth(tree: ast.AST) -> int:
+    """Return the maximum nested loop depth in the AST."""
+
+    best = 0
+
+    def walk(node: ast.AST, depth: int) -> None:
+        nonlocal best
+        if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
+            depth += 1
+            best = max(best, depth)
+        for child in ast.iter_child_nodes(node):
+            walk(child, depth)
+
+    walk(tree, 0)
+    return best
+
+
+def compile_tree(code: str) -> Tuple[ast.AST | None, str]:
+    """Return AST tree and optional parse error."""
+
+    try:
+        return ast.parse(code), ""
+    except SyntaxError as exc:
+        return None, f"{exc.msg} (line {exc.lineno}, column {exc.offset})"

graders/optimization.py

@@ -0,0 +1,167 @@
+"""Deterministic grading for optimization and refactor tasks."""
+
+from __future__ import annotations
+
+import json
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+from graders.common import clamp_score, compile_tree, nested_loop_depth, style_score
+from graders.pytest_runner import run_pytest_suite
+from models import TaskGrade
+from tasks.task_bank import TaskSpec
+
+
+def _benchmark_script(task: TaskSpec) -> str:
+    return f"""import json
+import time
+from candidate import {task.benchmark_entrypoint}
+
+{task.benchmark_builder}
+
+events = build_benchmark_events()
+start = time.perf_counter()
+for _ in range({task.benchmark_repeats}):
+    result = {task.benchmark_entrypoint}(events)
+elapsed = time.perf_counter() - start
+Path = __import__("pathlib").Path
+Path("benchmark.json").write_text(json.dumps({{"elapsed": elapsed, "rows": len(result)}}), encoding="utf-8")
+"""
+
+
+def benchmark_runtime(candidate_code: str, task: TaskSpec) -> tuple[float, bool, str]:
+    """Benchmark runtime deterministically against the starter implementation."""
+
+    assert task.benchmark_entrypoint is not None
+    try:
+        with tempfile.TemporaryDirectory(prefix="python-code-review-bench-") as temp_dir:
+            temp_path = Path(temp_dir)
+            (temp_path / "candidate.py").write_text(candidate_code, encoding="utf-8")
+            (temp_path / "starter.py").write_text(task.starter_code, encoding="utf-8")
+            (temp_path / "candidate_runner.py").write_text(_benchmark_script(task), encoding="utf-8")
+
+            starter_script = _benchmark_script(task).replace("from candidate import", "from starter import")
+            (temp_path / "starter_runner.py").write_text(starter_script, encoding="utf-8")
+
+            try:
+                starter_run = subprocess.run(
+                    [sys.executable, "starter_runner.py"],
+                    cwd=temp_path,
+                    capture_output=True,
+                    text=True,
+                    timeout=task.benchmark_timeout_s,
+                    check=False,
+                )
+                starter_payload = json.loads((temp_path / "benchmark.json").read_text(encoding="utf-8"))
+
+                candidate_run = subprocess.run(
+                    [sys.executable, "candidate_runner.py"],
+                    cwd=temp_path,
+                    capture_output=True,
+                    text=True,
+                    timeout=task.benchmark_timeout_s,
+                    check=False,
+                )
+                candidate_payload = json.loads((temp_path / "benchmark.json").read_text(encoding="utf-8"))
+            except subprocess.TimeoutExpired as exc:
+                output = (exc.stdout or "") + (exc.stderr or "")
+                return 0.0, True, (output or "benchmark timed out").strip()
+            except Exception as exc:  # pragma: no cover
+                return 0.0, False, str(exc)
+
+            starter_elapsed = max(float(starter_payload["elapsed"]), 1e-9)
+            candidate_elapsed = max(float(candidate_payload["elapsed"]), 1e-9)
+            speedup = starter_elapsed / candidate_elapsed
+            runtime_score = clamp_score(min((speedup - 1.0) / 3.0, 1.0))
+            output = "\n".join(
+                part
+                for part in [
+                    starter_run.stdout.strip(),
+                    starter_run.stderr.strip(),
+                    candidate_run.stdout.strip(),
+                    candidate_run.stderr.strip(),
+                    f"starter={starter_elapsed:.6f}s candidate={candidate_elapsed:.6f}s speedup={speedup:.2f}x",
+                ]
+                if part
+            )
+            return runtime_score, False, output
+    except Exception as exc:  # pragma: no cover
+        return 0.0, False, str(exc)
+
+
+def ast_quality_score(code: str, task: TaskSpec) -> float:
+    """Score maintainability and algorithmic structure."""
+
+    tree, parse_error = compile_tree(code)
+    if tree is None:
+        return 0.0
+
+    import ast
+
+    function_node = next(
+        (node for node in tree.body if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef))),
+        None,
+    )
+    docstring_points = 0.2 if function_node and ast.get_docstring(function_node, clean=False) else 0.0
+    nested_points = 0.4 if nested_loop_depth(tree) <= 1 else 0.0
+    marker_points = 0.0
+    for marker in task.expected_quality_markers:
+        if marker in code:
+            marker_points += 0.2
+    return clamp_score(docstring_points + nested_points + marker_points)
+
+
+def grade_optimization_task(candidate_code: str, task: TaskSpec) -> TaskGrade:
+    """Grade optimization tasks using correctness, runtime, AST quality, and style."""
+
+    execution = run_pytest_suite(
+        candidate_code,
+        [*task.visible_tests, *task.hidden_tests],
+        timeout_s=task.benchmark_timeout_s,
+    )
+    test_fraction = execution.passed / execution.total if execution.total else 0.0
+
+    if execution.timed_out:
+        return TaskGrade(
+            score=0.0,
+            tests_passed=execution.passed,
+            tests_total=execution.total,
+            timed_out=True,
+            details={"tests": execution.output},
+        )
+
+    runtime_score, timed_out, benchmark_output = benchmark_runtime(candidate_code, task)
+    if timed_out:
+        return TaskGrade(
+            score=0.0,
+            tests_passed=execution.passed,
+            tests_total=execution.total,
+            timed_out=True,
+            details={"tests": execution.output, "benchmark": benchmark_output},
+        )
+
+    quality_score = ast_quality_score(candidate_code, task)
+    pep8_score = style_score(candidate_code, task.style_max_line_length)
+    score = clamp_score(
+        (0.5 * test_fraction)
+        + (0.3 * runtime_score)
+        + (0.15 * quality_score)
+        + (0.05 * pep8_score)
+    )
+    return TaskGrade(
+        score=score,
+        syntax_score=1.0,
+        tests_passed=execution.passed,
+        tests_total=execution.total,
+        quality_score=quality_score,
+        runtime_score=runtime_score,
+        details={
+            "tests": execution.output,
+            "benchmark": benchmark_output,
+            "test_fraction": round(test_fraction, 4),
+            "runtime_score": round(runtime_score, 4),
+            "style_score": round(pep8_score, 4),
+        },
+    )

graders/pytest_runner.py

@@ -0,0 +1,149 @@
+"""Helpers for deterministic pytest execution in temp sandboxes."""
+
+from __future__ import annotations
+
+import json
+import subprocess
+import sys
+import tempfile
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Iterable
+
+
+@dataclass(frozen=True)
+class PytestExecution:
+    """Exact pytest execution summary."""
+
+    passed: int
+    failed: int
+    total: int
+    timed_out: bool
+    output: str
+
+
+def _test_module_source(tests: Iterable[str]) -> str:
+    """Build a valid pytest module from expression-style or full test snippets."""
+    blocks: list[str] = ["from candidate import *  # noqa: F401,F403"]
+    for index, test in enumerate(tests, start=1):
+        snippet = str(test).strip()
+        if not snippet:
+            continue
+        if snippet.startswith("def test_"):
+            blocks.append(snippet)
+            continue
+        blocks.append(
+            "\n".join(
+                [
+                    f"def test_case_{index:03d}():",
+                    f"    assert {snippet}",
+                ]
+            )
+        )
+    return "\n\n".join(blocks) or "def test_placeholder():\n    assert True\n"
+
+
+def _runner_script() -> str:
+    return """import json
+import pathlib
+import pytest
+
+
+class Collector:
+    def __init__(self) -> None:
+        self.passed = 0
+        self.failed = 0
+
+    def pytest_runtest_logreport(self, report):
+        if report.when != "call":
+            return
+        if report.passed:
+            self.passed += 1
+        elif report.failed:
+            self.failed += 1
+
+
+collector = Collector()
+exit_code = pytest.main(["-q", "test_candidate.py"], plugins=[collector])
+payload = {
+    "passed": collector.passed,
+    "failed": collector.failed,
+    "exit_code": int(exit_code),
+}
+pathlib.Path("pytest_results.json").write_text(json.dumps(payload), encoding="utf-8")
+"""
+
+
+def run_pytest_suite(candidate_code: str, tests: Iterable[str], timeout_s: float = 3.0) -> PytestExecution:
+    """Run a pytest suite against candidate.py and return structured results."""
+
+    test_cases = list(tests)
+    try:
+        with tempfile.TemporaryDirectory(prefix="python-code-review-") as temp_dir:
+            temp_path = Path(temp_dir)
+            (temp_path / "candidate.py").write_text(candidate_code, encoding="utf-8")
+            (temp_path / "test_candidate.py").write_text(_test_module_source(test_cases), encoding="utf-8")
+            (temp_path / "runner.py").write_text(_runner_script(), encoding="utf-8")
+
+            try:
+                completed = subprocess.run(
+                    [sys.executable, "runner.py"],
+                    cwd=temp_path,
+                    capture_output=True,
+                    text=True,
+                    timeout=timeout_s,
+                    check=False,
+                )
+            except subprocess.TimeoutExpired as exc:
+                output = (exc.stdout or "") + (exc.stderr or "")
+                return PytestExecution(
+                    passed=0,
+                    failed=max(len(test_cases), 1),
+                    total=max(len(test_cases), 1),
+                    timed_out=True,
+                    output=(output or "pytest timed out").strip(),
+                )
+
+            result_path = temp_path / "pytest_results.json"
+            if not result_path.exists():
+                output = (completed.stdout or "") + (completed.stderr or "")
+                total = max(len(test_cases), 1)
+                return PytestExecution(
+                    passed=0,
+                    failed=total,
+                    total=total,
+                    timed_out=False,
+                    output=output.strip(),
+                )
+
+            try:
+                payload = json.loads(result_path.read_text(encoding="utf-8"))
+            except Exception as exc:
+                output = ((completed.stdout or "") + (completed.stderr or "")).strip()
+                return PytestExecution(
+                    passed=0,
+                    failed=max(len(test_cases), 1),
+                    total=max(len(test_cases), 1),
+                    timed_out=False,
+                    output=(output or str(exc)).strip(),
+                )
+
+            passed = int(payload.get("passed", 0))
+            failed = int(payload.get("failed", 0))
+            total = max(passed + failed, len(test_cases))
+            output = ((completed.stdout or "") + (completed.stderr or "")).strip()
+            return PytestExecution(
+                passed=passed,
+                failed=failed,
+                total=total,
+                timed_out=False,
+                output=output,
+            )
+    except Exception as exc:
+        return PytestExecution(
+            passed=0,
+            failed=max(len(test_cases), 1),
+            total=max(len(test_cases), 1),
+            timed_out=False,
+            output=str(exc),
+        )

graders/syntax.py

@@ -0,0 +1,78 @@
+"""Task graders for syntax and bug-fix tasks."""
+
+from __future__ import annotations
+
+from graders.common import clamp_score, compiles, normalized_diff_score, style_score, syntax_error_message
+from graders.optimization import grade_optimization_task
+from graders.pytest_runner import run_pytest_suite
+from models import TaskGrade
+from tasks.task_bank import TaskSpec
+
+
+def grade_syntax_task(candidate_code: str, task: TaskSpec) -> TaskGrade:
+    """Grade syntax repair tasks with partial credit for progress toward the reference."""
+
+    error = syntax_error_message(candidate_code)
+    diff_score = normalized_diff_score(candidate_code, task.reference_code)
+    style_base = style_score(candidate_code, task.style_max_line_length)
+    
+    if not error:
+        return TaskGrade(
+            score=1.0,
+            syntax_score=1.0,
+            quality_score=style_base,
+            details={"compile_error": ""},
+        )
+
+    partial = clamp_score(0.15 + (0.55 * diff_score))
+    return TaskGrade(
+        score=partial,
+        syntax_score=0.0,
+        quality_score=diff_score * style_base,
+        details={"compile_error": error},
+    )
+
+
+def grade_bug_fix_task(candidate_code: str, task: TaskSpec, include_hidden: bool = True) -> TaskGrade:
+    """Grade logic bug tasks with pytest pass fraction."""
+
+    if not compiles(candidate_code):
+        error = syntax_error_message(candidate_code)
+        return TaskGrade(score=0.0, syntax_score=0.0, details={"compile_error": error})
+
+    tests = list(task.visible_tests)
+    if include_hidden:
+        tests.extend(task.hidden_tests)
+    
+    execution = run_pytest_suite(candidate_code, tests, timeout_s=3.0)
+    if execution.timed_out:
+        return TaskGrade(
+            score=0.0,
+            syntax_score=1.0,
+            tests_passed=execution.passed,
+            tests_total=execution.total,
+            timed_out=True,
+            details={"compile_error": "", "tests": execution.output},
+        )
+
+    pass_fraction = execution.passed / execution.total if execution.total else 0.0
+    quality = style_score(candidate_code, task.style_max_line_length)
+    
+    return TaskGrade(
+        score=clamp_score(pass_fraction),
+        syntax_score=1.0,
+        tests_passed=execution.passed,
+        tests_total=execution.total,
+        quality_score=quality,
+        details={"compile_error": "", "tests": execution.output},
+    )
+
+
+def grade_task(candidate_code: str, task: TaskSpec, include_hidden: bool = True) -> TaskGrade:
+    """Dispatch to the correct deterministic grader for one task."""
+
+    if task.task_kind == "syntax_fix":
+        return grade_syntax_task(candidate_code, task)
+    if task.task_kind == "bug_fix":
+        return grade_bug_fix_task(candidate_code, task, include_hidden=include_hidden)
+    return grade_optimization_task(candidate_code, task)

inference.py

@@ -1,314 +1,462 @@
-"""Baseline inference script for the Python code-review environment.
-
-This script is meant to be submission-friendly:
-
-- configuration comes from environment variables
-- model calls use the OpenAI client as required
-- malformed model output is handled gracefully
-- a JSON report is written for reproducibility
-"""
-
-from __future__ import annotations
-
-import json
-import os
-import re
-from pathlib import Path
-from typing import Any, Dict, List, Optional
-
-from openai import OpenAI
-
-from client import PythonEnv
-from models import PythonReviewAction, ReviewFinding
-
-
-# Read all runtime configuration from environment variables so the script can
-# be reused unchanged across local runs, CI, and HF Spaces validation.
-API_BASE_URL = os.environ["API_BASE_URL"]
-MODEL_NAME = os.environ["MODEL_NAME"]
-API_KEY = os.getenv("HF_TOKEN") or os.getenv("OPENAI_API_KEY")
-ENV_BASE_URL = os.getenv("ENV_BASE_URL")
-DOCKER_IMAGE = os.getenv("PYTHON_ENV_IMAGE", "python_env-env:latest")
-MAX_STEPS = int(os.getenv("MAX_STEPS", "3"))
-MAX_TASKS = int(os.getenv("MAX_TASKS", "3"))
-REPORT_PATH = Path(os.getenv("INFERENCE_REPORT_PATH", "inference_results.json"))
-TEMPERATURE = float(os.getenv("TEMPERATURE", "0"))
-MAX_TOKENS = int(os.getenv("MAX_TOKENS", "900"))
-
-SYSTEM_PROMPT = """You are a precise Python code reviewer.
-Return strict JSON using this schema:
-{
-  "findings": [
-    {
-      "title": "short title",
-      "line": 1,
-      "category": "bug|security|style|performance|maintainability",
-      "severity": "critical|warning|info",
-      "rationale": "why it matters",
-      "recommendation": "how to fix it",
-      "rule_id": "optional-stable-id"
-    }
-  ],
-  "patched_code": null
-}
-
-Rules:
-- Output JSON only. No markdown fences.
-- Only report issues supported by the visible code.
-- Prefer high precision over quantity.
-- Include line numbers when possible.
-"""
-
-
-def _build_prompt(observation, step: int, history: List[str]) -> str:
-    """Build the task prompt sent to the model for one step."""
-
-    history_text = "\n".join(history[-4:]) if history else "No previous attempts."
-    return (
-        f"Task ID: {observation.task.task_id}\n"
-        f"Difficulty: {observation.task.difficulty}\n"
-        f"Objective: {observation.task.objective}\n"
-        f"Step: {step}\n"
-        f"Attempts remaining: {observation.attempts_remaining}\n"
-        f"Current score: {observation.score:.2f}\n"
-        f"Latest feedback: {observation.feedback or 'None'}\n"
-        f"Attempt history:\n{history_text}\n\n"
-        "Code to review:\n"
-        "```python\n"
-        f"{observation.task.code}\n"
-        "```"
-    )
-
-
-def _extract_text_content(message_content: Any) -> str:
-    """Normalize OpenAI response content into one text string."""
-
-    if isinstance(message_content, str):
-        return message_content
-    if isinstance(message_content, list):
-        parts: List[str] = []
-        for item in message_content:
-            if isinstance(item, dict):
-                text = item.get("text")
-                if isinstance(text, str):
-                    parts.append(text)
-        return "\n".join(parts)
-    return ""
-
-
-def _extract_json_blob(content: str) -> str:
-    """Extract a JSON object from plain or fenced model output."""
-
-    fenced_match = re.search(r"```(?:json)?\s*(\{.*\})\s*```", content, re.DOTALL)
-    if fenced_match:
-        return fenced_match.group(1)
-
-    start = content.find("{")
-    end = content.rfind("}")
-    if start != -1 and end != -1 and end > start:
-        return content[start : end + 1]
-    return content
-
-
-def _parse_response(content: str) -> Dict[str, Any]:
-    """Parse the model response into a normalized payload dict."""
-
-    raw = _extract_json_blob(content)
-    try:
-        data = json.loads(raw)
-    except json.JSONDecodeError:
-        return {"findings": [], "patched_code": None, "_parse_error": raw}
-
-    findings = data.get("findings", [])
-    if not isinstance(findings, list):
-        findings = []
-    patched_code = data.get("patched_code")
-    if patched_code is not None and not isinstance(patched_code, str):
-        patched_code = None
-    return {"findings": findings, "patched_code": patched_code}
-
-
-def _completion(client: OpenAI, prompt: str) -> Dict[str, Any]:
-    """Send one completion request to the configured model endpoint."""
-
-    response = client.chat.completions.create(
-        model=MODEL_NAME,
-        temperature=TEMPERATURE,
-        max_tokens=MAX_TOKENS,
-        messages=[
-            {"role": "system", "content": SYSTEM_PROMPT},
-            {"role": "user", "content": prompt},
-        ],
-    )
-    content = _extract_text_content(response.choices[0].message.content) or "{}"
-    return _parse_response(content)
-
-
-def _normalize_findings(payload: Dict[str, Any]) -> List[ReviewFinding]:
-    """Convert raw dict findings into validated `ReviewFinding` objects."""
-
-    findings: List[ReviewFinding] = []
-    for item in payload.get("findings", []):
-        if not isinstance(item, dict):
-            continue
-        try:
-            findings.append(ReviewFinding(**item))
-        except Exception:
-            continue
-    return findings
-
-
-def _build_fallback_action(observation, note: str) -> PythonReviewAction:
-    """Create a safe fallback action when model output is unusable."""
-
-    return PythonReviewAction(
-        operation="finalize" if observation.attempts_remaining <= 1 else "request_hint",
-        note=note,
-    )
-
-
-def _to_action(
-    payload: Dict[str, Any],
-    observation,
-    finalize: bool,
-) -> PythonReviewAction:
-    """Convert a parsed model payload into a valid environment action."""
-
-    findings = _normalize_findings(payload)
-    if not findings and not payload.get("patched_code"):
-        note = "Model returned no valid findings."
-        if payload.get("_parse_error"):
-            note = f"{note} Raw response could not be parsed as JSON."
-        return _build_fallback_action(observation, note)
-
-    return PythonReviewAction(
-        operation="finalize" if finalize else "submit_findings",
-        findings=findings,
-        patched_code=payload.get("patched_code"),
-    )
-
-
-def _make_env() -> PythonEnv:
-    """Connect to a live environment or launch the Docker image."""
-
-    if ENV_BASE_URL:
-        return PythonEnv(base_url=ENV_BASE_URL)
-    return PythonEnv.from_docker_image(DOCKER_IMAGE)
-
-
-def _task_result_dict(observation, step_logs: List[Dict[str, Any]]) -> Dict[str, Any]:
-    """Build the report payload for one completed task run."""
-
-    evaluation = observation.evaluation
-    return {
-        "task_id": observation.task.task_id,
-        "difficulty": observation.task.difficulty,
-        "title": observation.task.title,
-        "score": observation.score,
-        "passed": evaluation.passed,
-        "matched_findings": evaluation.matched_findings,
-        "total_findings": evaluation.total_findings,
-        "false_positives": evaluation.false_positives,
-        "duplicate_findings": evaluation.duplicate_findings,
-        "weighted_recall": evaluation.weighted_recall,
-        "patch_score": evaluation.patch_score,
-        "steps": step_logs,
-    }
-
-
-def main() -> None:
-    """Run the configured model against the benchmark task set."""
-
-    if not API_KEY:
-        raise RuntimeError("Set HF_TOKEN or OPENAI_API_KEY before running inference.py")
-
-    client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
-    env = _make_env()
-    episode_results: List[Dict[str, Any]] = []
-
-    try:
-        for index in range(MAX_TASKS):
-            result = env.reset()
-            observation = result.observation
-            history: List[str] = []
-            step_logs: List[Dict[str, Any]] = []
-
-            print(
-                f"Task {index + 1}: {observation.task.task_id} "
-                f"({observation.task.difficulty})"
-            )
-
-            for step in range(1, MAX_STEPS + 1):
-                prompt = _build_prompt(observation, step, history)
-                try:
-                    # Model-call failures are captured in the report rather than
-                    # crashing the full benchmark run.
-                    payload = _completion(client, prompt)
-                except Exception as exc:
-                    payload = {"findings": [], "patched_code": None, "_error": str(exc)}
-
-                action = _to_action(
-                    payload=payload,
-                    observation=observation,
-                    finalize=step == MAX_STEPS or observation.attempts_remaining <= 1,
-                )
-
-                result = env.step(action)
-                observation = result.observation
-
-                step_log = {
-                    "step": step,
-                    "operation": action.operation,
-                    "submitted_findings": len(action.findings),
-                    "reward": result.reward or 0.0,
-                    "score": observation.score,
-                    "done": result.done,
-                    "feedback": observation.feedback,
-                }
-                if payload.get("_error"):
-                    step_log["model_error"] = payload["_error"]
-                if payload.get("_parse_error"):
-                    step_log["parse_error"] = True
-                step_logs.append(step_log)
-
-                # The history string is fed back into later prompts so the
-                # model can see what it already tried.
-                history.append(
-                    f"step={step} op={action.operation} findings={len(action.findings)} "
-                    f"score={observation.score:.2f} feedback={observation.feedback}"
-                )
-
-                print(
-                    f"  step={step} op={action.operation} findings={len(action.findings)} "
-                    f"score={observation.score:.2f} reward={(result.reward or 0.0):.2f} "
-                    f"done={result.done}"
-                )
-
-                if result.done:
-                    break
-
-            episode_results.append(_task_result_dict(observation, step_logs))
-    finally:
-        env.close()
-
-    mean_score = (
-        sum(item["score"] for item in episode_results) / len(episode_results)
-        if episode_results
-        else 0.0
-    )
-    summary = {
-        "model_name": MODEL_NAME,
-        "api_base_url": API_BASE_URL,
-        "task_count": len(episode_results),
-        "mean_score": mean_score,
-        "results": episode_results,
-    }
-
-    # Persist the report so scores can be compared across runs and models.
-    REPORT_PATH.write_text(json.dumps(summary, indent=2), encoding="utf-8")
-    print(json.dumps(summary, indent=2))
-    print(f"\nSaved report to {REPORT_PATH}")
-
-
-if __name__ == "__main__":
-    main()
+#!/usr/bin/env python3
+"""Fail-safe inference entrypoint for the Python code review environment."""
+
+from __future__ import annotations
+
+import io
+import json
+import os
+import subprocess
+import sys
+import time
+from collections.abc import Iterable
+from contextlib import redirect_stderr, redirect_stdout
+from typing import Any, Dict, Optional
+
+from compat import install_openenv_fastmcp_compat
+
+try:
+    from openai import OpenAI
+except Exception:
+    OpenAI = None  # type: ignore[assignment]
+
+
+install_openenv_fastmcp_compat()
+
+try:
+    from server.env import PythonCodeReviewEnvironment
+except Exception:
+    PythonCodeReviewEnvironment = None  # type: ignore[assignment]
+
+try:
+    from models import PythonCodeReviewAction
+except Exception:
+    PythonCodeReviewAction = None  # type: ignore[assignment]
+
+try:
+    from tasks import task_ids
+except Exception:
+    task_ids = None  # type: ignore[assignment]
+
+
+ALLOWED_ACTIONS = {
+    "analyze_code",
+    "edit_code",
+    "run_tests",
+    "submit_solution",
+}
+DEFAULT_MODEL_NAME = "mock-model"
+DEFAULT_ACTION = {"action_type": "analyze_code", "code": None, "fallback_reason": "mock_response"}
+API_TIMEOUT_SECONDS = 3.0
+API_RETRIES = 1
+API_RETRY_DELAY_SECONDS = 0.2
+MAX_STEPS = 2
+
+
+def safe_env(name: str, default: str = "") -> str:
+    """Read an allowed environment variable and return a safe string default."""
+    try:
+        value = os.getenv(name)
+        if value is None:
+            return default
+        return str(value)
+    except Exception:
+        return default
+
+
+def clamp(value: float, low: float = 0.0, high: float = 1.0) -> float:
+    """Clamp a numeric value to a bounded range."""
+    try:
+        return max(low, min(high, float(value)))
+    except Exception:
+        return low
+
+
+def safe_float(value: Any, default: float = 0.0) -> float:
+    """Convert a value to float without raising."""
+    try:
+        return float(value)
+    except Exception:
+        return default
+
+
+def safe_text(value: Any, default: str = "") -> str:
+    """Convert any value into a bounded, printable string."""
+    try:
+        text = str(value)
+    except Exception:
+        return default
+    text = " ".join(text.split())
+    return text[:160] if text else default
+
+
+def safe_getattr(obj: Any, name: str, default: Any = None) -> Any:
+    """Fetch an attribute from an object without raising."""
+    try:
+        return getattr(obj, name, default)
+    except Exception:
+        return default
+
+
+def parse_json_response(raw_text: str) -> Dict[str, Any]:
+    """Parse model output into a safe action payload with deterministic fallback."""
+    try:
+        text = raw_text or ""
+        start = text.find("{")
+        end = text.rfind("}") + 1
+        if start >= 0 and end > start:
+            payload = json.loads(text[start:end])
+            if isinstance(payload, dict):
+                action_type = payload.get("action_type", DEFAULT_ACTION["action_type"])
+                code = payload.get("code")
+                if action_type not in ALLOWED_ACTIONS:
+                    action_type = DEFAULT_ACTION["action_type"]
+                if action_type != "edit_code":
+                    code = None
+                return {
+                    "action_type": action_type,
+                    "code": code,
+                    "fallback_reason": "",
+                }
+    except Exception:
+        pass
+    return dict(DEFAULT_ACTION)
+
+
+def build_prompt(observation: Any) -> str:
+    """Build a short prompt from the current observation with safe defaults."""
+    try:
+        task_description = safe_text(safe_getattr(observation, "task_description", ""), "No task description.")
+        current_code = safe_text(safe_getattr(observation, "current_code", ""), "")
+        errors = safe_text(safe_getattr(observation, "errors", ""), "")
+        tests = safe_text(safe_getattr(observation, "test_results", ""), "")
+        score = clamp(safe_getattr(observation, "score", 0.0))
+        visible_tests = safe_getattr(observation, "visible_tests", [])
+        if not isinstance(visible_tests, Iterable) or isinstance(visible_tests, (str, bytes)):
+            visible_tests = []
+        visible_lines = []
+        for item in list(visible_tests)[:4]:
+            visible_lines.append(f"- {safe_text(item, 'unknown test')}")
+        visible_block = "\n".join(visible_lines) if visible_lines else "- none"
+        return (
+            "Return exactly one JSON object with keys action_type and optional code.\n"
+            "Allowed action_type values: analyze_code, edit_code, run_tests, submit_solution.\n"
+            f"Task: {task_description}\n"
+            f"Score: {score:.3f}\n"
+            f"Errors: {errors or 'none'}\n"
+            f"Tests: {tests or 'not available'}\n"
+            f"Visible tests:\n{visible_block}\n"
+            f"Code:\n{current_code}\n"
+        )
+    except Exception:
+        return (
+            "Return exactly one JSON object with keys action_type and optional code. "
+            "Use action_type analyze_code."
+        )
+
+
+def create_client() -> Optional[Any]:
+    """Create an OpenAI-compatible client using only the allowed environment variables."""
+    if OpenAI is None:
+        return None
+    base_url = safe_env("API_BASE_URL", "")
+    if not base_url:
+        return None
+    try:
+        if safe_env("HF_TOKEN", ""):
+            os.environ["OPENAI_API_KEY"] = safe_env("HF_TOKEN", "")
+    except Exception:
+        pass
+    try:
+        client = OpenAI(base_url=os.getenv("API_BASE_URL"))
+        return client
+    except Exception:
+        return None
+
+
+def run_llm(client: Optional[Any], model: str, prompt: str) -> Dict[str, Any]:
+    """Call the LLM with timeout and retry, then fall back to a mock action."""
+    if client is None:
+        fallback = dict(DEFAULT_ACTION)
+        fallback["fallback_reason"] = "client_unavailable"
+        return fallback
+
+    last_reason = "llm_unavailable"
+    for attempt in range(API_RETRIES + 1):
+        try:
+            with redirect_stdout(io.StringIO()), redirect_stderr(io.StringIO()):
+                response = client.with_options(timeout=API_TIMEOUT_SECONDS).chat.completions.create(
+                    model=model,
+                    messages=[{"role": "user", "content": prompt}],
+                    temperature=0,
+                    max_tokens=300,
+                )
+            message = safe_getattr(response.choices[0].message, "content", "")
+            parsed = parse_json_response(message)
+            if parsed.get("fallback_reason"):
+                parsed["fallback_reason"] = "parse_failed"
+            return parsed
+        except Exception as exc:
+            last_reason = safe_text(exc, "llm_error").lower().replace(" ", "_")
+            if attempt < API_RETRIES:
+                try:
+                    time.sleep(API_RETRY_DELAY_SECONDS * (attempt + 1))
+                except Exception:
+                    pass
+
+    fallback = dict(DEFAULT_ACTION)
+    fallback["fallback_reason"] = last_reason[:48] or "llm_retry_exhausted"
+    return fallback
+
+
+def probe_docker(image_name: str) -> Dict[str, Any]:
+    """Safely validate Docker connectivity when a local image name is provided."""
+    if not image_name:
+        return {"checked": False, "available": False, "reason": "docker_skip"}
+    try:
+        with redirect_stdout(io.StringIO()), redirect_stderr(io.StringIO()):
+            result = subprocess.run(
+                ["docker", "image", "inspect", image_name],
+                capture_output=True,
+                text=True,
+                timeout=3,
+                check=False,
+            )
+        if result.returncode == 0:
+            return {"checked": True, "available": True, "reason": "docker_ok"}
+        return {"checked": True, "available": False, "reason": "docker_unreachable"}
+    except Exception as exc:
+        return {"checked": True, "available": False, "reason": safe_text(exc, "docker_error").lower().replace(" ", "_")}
+
+
+def fallback_step_result(reason: str, docker_status: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
+    """Return a deterministic dummy step result when environment execution fails."""
+    docker_reason = safe_text((docker_status or {}).get("reason", "docker_skip"), "docker_skip")
+    short_reason = safe_text(reason, "env_fallback").lower().replace(" ", "_")
+    return {
+        "status": "ok",
+        "fallback": True,
+        "reason": short_reason[:64],
+        "reward": 0.0,
+        "improvement": 0.0,
+        "score": 0.0,
+        "done": True,
+        "docker": docker_reason[:32],
+    }
+
+
+def safe_task_list() -> list[str]:
+    """Load task identifiers without raising."""
+    try:
+        if callable(task_ids):
+            loaded = list(task_ids())
+            if loaded:
+                return [safe_text(item, "fallback-task") for item in loaded]
+    except Exception:
+        pass
+    return ["fallback-task"]
+
+
+def make_action(action_payload: Dict[str, Any]) -> Any:
+    """Build a validated environment action or a safe placeholder."""
+    action_type = action_payload.get("action_type", DEFAULT_ACTION["action_type"])
+    if action_type not in ALLOWED_ACTIONS:
+        action_type = DEFAULT_ACTION["action_type"]
+    code = action_payload.get("code")
+    if action_type != "edit_code":
+        code = None
+    if PythonCodeReviewAction is None:
+        return {"action_type": action_type, "code": code}
+    try:
+        return PythonCodeReviewAction(action_type=action_type, code=code)
+    except Exception:
+        try:
+            return PythonCodeReviewAction(action_type=DEFAULT_ACTION["action_type"], code=None)
+        except Exception:
+            return {"action_type": DEFAULT_ACTION["action_type"], "code": None}
+
+
+def compute_reward(
+    previous_score: float,
+    current_score: float,
+    step_reward: float,
+    used_fallback: bool,
+    done: bool,
+) -> Dict[str, float]:
+    """Compute a deterministic dynamic reward and improvement metric."""
+    prev_value = clamp(previous_score)
+    curr_value = clamp(current_score)
+    improvement = round(curr_value - prev_value, 4)
+    bounded_step_reward = max(-1.0, min(1.0, safe_float(step_reward, 0.0)))
+    reward_value = (
+        0.55 * curr_value
+        + 0.30 * max(improvement, 0.0)
+        + 0.10 * max(bounded_step_reward, 0.0)
+        + (0.05 if done and curr_value >= 0.99 else 0.0)
+        - (0.05 if used_fallback else 0.0)
+    )
+    return {
+        "reward": round(clamp(reward_value), 4),
+        "improvement": improvement,
+    }
+
+
+def safe_step(env: Any, action: Any) -> Any:
+    """Execute one environment step without allowing stdout leaks or exceptions."""
+    try:
+        with redirect_stdout(io.StringIO()), redirect_stderr(io.StringIO()):
+            return env.step(action)
+    except Exception:
+        return None
+
+
+def safe_reset(env: Any, task_id: str) -> Any:
+    """Reset the environment safely for a task."""
+    try:
+        with redirect_stdout(io.StringIO()), redirect_stderr(io.StringIO()):
+            return env.reset(task_id=task_id)
+    except Exception:
+        return None
+
+
+def run_env(client: Optional[Any], model: str) -> Dict[str, Any]:
+    """Run the environment loop safely and return a structured result payload."""
+    docker_status = probe_docker(safe_env("LOCAL_IMAGE_NAME", ""))
+    if PythonCodeReviewEnvironment is None:
+        return fallback_step_result("env_import_failed", docker_status)
+
+    try:
+        with redirect_stdout(io.StringIO()), redirect_stderr(io.StringIO()):
+            env = PythonCodeReviewEnvironment(verbose=False)
+    except Exception as exc:
+        return fallback_step_result(f"env_init_failed_{safe_text(exc, 'unknown')}", docker_status)
+
+    tasks = safe_task_list()
+    task_id = tasks[0] if tasks else "fallback-task"
+    observation = safe_reset(env, task_id)
+    if observation is None:
+        return fallback_step_result("env_reset_failed", docker_status)
+
+    previous_score = clamp(safe_getattr(observation, "score", 0.0))
+    total_step_reward = 0.0
+    used_fallback = False
+    final_status = "ok"
+    final_reason = "completed"
+    final_observation = observation
+
+    for step_index in range(MAX_STEPS):
+        prompt = build_prompt(final_observation)
+        action_payload = run_llm(client, model, prompt)
+        used_fallback = used_fallback or bool(action_payload.get("fallback_reason"))
+        action = make_action(action_payload)
+        next_observation = safe_step(env, action)
+        if next_observation is None:
+            final_status = "ok"
+            final_reason = "env_step_fallback"
+            used_fallback = True
+            break
+
+        final_observation = next_observation
+        total_step_reward += safe_float(safe_getattr(final_observation, "reward", 0.0), 0.0)
+        done = bool(safe_getattr(final_observation, "done", False))
+        score = clamp(safe_getattr(final_observation, "score", 0.0))
+        if safe_getattr(final_observation, "last_action_status", ""):
+            final_reason = safe_text(safe_getattr(final_observation, "last_action_status", ""), "step_completed")
+        elif action_payload.get("fallback_reason"):
+            final_reason = safe_text(action_payload.get("fallback_reason"), "llm_fallback")
+        else:
+            final_reason = f"step_{step_index + 1}_completed"
+        if done:
+            break
+
+        if step_index == 0:
+            submit_action = make_action({"action_type": "submit_solution", "code": None})
+            submitted_observation = safe_step(env, submit_action)
+            if submitted_observation is None:
+                final_reason = "submit_fallback"
+                used_fallback = True
+                break
+            final_observation = submitted_observation
+            total_step_reward += safe_float(safe_getattr(final_observation, "reward", 0.0), 0.0)
+            if safe_getattr(final_observation, "last_action_status", ""):
+                final_reason = safe_text(safe_getattr(final_observation, "last_action_status", ""), "submit_completed")
+            break
+
+    current_score = clamp(safe_getattr(final_observation, "score", previous_score))
+    done = bool(safe_getattr(final_observation, "done", True))
+    metrics = compute_reward(
+        previous_score=previous_score,
+        current_score=current_score,
+        step_reward=total_step_reward,
+        used_fallback=used_fallback,
+        done=done,
+    )
+    return {
+        "status": final_status,
+        "fallback": used_fallback,
+        "reason": safe_text(final_reason, "completed").lower().replace(" ", "_")[:64],
+        "reward": metrics["reward"],
+        "improvement": metrics["improvement"],
+        "score": round(current_score, 4),
+        "done": done,
+        "docker": safe_text(docker_status.get("reason", "docker_skip"), "docker_skip")[:32],
+    }
+
+
+def format_step_message(result: Dict[str, Any]) -> str:
+    """Format the only allowed STEP line for stdout."""
+    try:
+        fallback = bool(result.get("fallback", False))
+        reason = safe_text(result.get("reason", "completed"), "completed").lower().replace(" ", "_")
+        if fallback:
+            reward = safe_float(result.get("reward", 0.0), 0.0)
+            improvement = safe_float(result.get("improvement", 0.0), 0.0)
+            score = safe_float(result.get("score", 0.0), 0.0)
+            status = safe_text(result.get("status", "ok"), "ok").lower().replace(" ", "_")
+            return (
+                f"error handled: {reason} reward={reward:.4f} status={status} "
+                f"fallback=true improvement={improvement:.4f} score={score:.4f}"
+            )
+        reward = safe_float(result.get("reward", 0.0), 0.0)
+        improvement = safe_float(result.get("improvement", 0.0), 0.0)
+        score = safe_float(result.get("score", 0.0), 0.0)
+        status = safe_text(result.get("status", "ok"), "ok").lower().replace(" ", "_")
+        return (
+            f"reward={reward:.4f} status={status} "
+            f"fallback=false improvement={improvement:.4f} score={score:.4f}"
+        )
+    except Exception:
+        return "error handled: formatting_failed"
+
+
+def main() -> int:
+    """Run the inference workflow and always terminate successfully."""
+    step_message = "error handled: initialization_failed"
+    try:
+        model_name = safe_env("MODEL_NAME", DEFAULT_MODEL_NAME) or DEFAULT_MODEL_NAME
+        client = create_client()
+        result = run_env(client, model_name)
+        step_message = format_step_message(result)
+    except BaseException as exc:
+        step_message = f"error handled: {safe_text(exc, 'unexpected_failure').lower().replace(' ', '_')[:64]}"
+    finally:
+        try:
+            print("START")
+            print(f"STEP: {step_message}")
+            print("END")
+        except Exception:
+            pass
+    return 0
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except BaseException:
+        try:
+            print("START")
+            print("STEP: error handled: fatal_guard")
+            print("END")
+        except Exception:
+            pass
+    sys.exit(0)

models.py

@@ -1,217 +1,185 @@
-"""Typed models for the Python code-review environment.
+"""Typed models for Python code review and repair environment."""
 
-This module is the shared contract between:
+from __future__ import annotations
 
-- the OpenEnv server implementation
-- the REST API layer
-- the benchmark grader
-- the inference script
-- the tests
-
-Keeping these models centralized makes the environment easier to validate,
-serialize, and evolve without each module inventing its own payload shape.
-"""
-
-from typing import List, Literal, Optional
+from typing import Any, Dict, List, Literal, Optional
 
 from pydantic import BaseModel, Field
-from openenv.core.env_server.types import Action, Observation
 
+from compat import Action, Observation, State
 
-# Difficulty buckets are intentionally small and fixed so tasks can be
-# grouped for curriculum learning and reporting without extra normalization.
-Difficulty = Literal["easy", "medium", "hard"]
 
-# Severity is separate from category because one category such as "security"
-# can still vary in importance across tasks.
+Difficulty = Literal["easy", "medium", "hard"]
+TaskKind = Literal["syntax_fix", "bug_fix", "optimization"]
+ActionType = Literal["analyze_code", "edit_code", "run_tests", "submit_solution"]
+Category = Literal["bug", "security", "performance", "maintainability", "style", "testing"]
 Severity = Literal["critical", "warning", "info"]
+
+
+class HistoryEntry(BaseModel):
+    """Record of one action taken during an episode."""
+
+    step: int = Field(..., ge=0)
+    action_type: ActionType
+    status: str = Field(..., description="Outcome message")
+    reward: float = Field(...)
+
+
+class RewardDetails(BaseModel):
+    """Detailed reward breakdown for transparent agent feedback.
+    
+    The reward system is dynamic and multi-component, with 6 independent sources:
+    
+    1. Progress Reward (max +0.25)
+       - Awarded for score improvement from previous step
+       - Formula: min(PROGRESS_SCALE * score_delta, 0.25)
+       - Encourages continuous improvement
+    
+    2. Syntax Reward (max +0.35)
+       - One-time bonus for fixing syntax errors (first compile)
+       - Applied when code transitions from uncompilable to compilable
+       - Acknowledges the critical first step of valid code
+    
+    3. Test Reward (max +0.20)
+       - Based on improvement in test pass rate
+       - Formula: min(TEST_PASS_REWARD_SCALE * test_improvement, 0.20)
+       - Rewards incremental test progress
+    
+    4. Quality Reward (max +0.15)
+       - Based on AST-detected code quality metrics
+       - Rewards improvements in structure, readability, best practices
+       - Uses deterministic grader feedback
+    
+    5. Stagnation Penalty (−0.10)
+       - Applied when agent acts but code doesn't change
+       - Encourages editing rather than repeated analysis
+       - Configurable via STAGNATION_PENALTY constant
+    
+    6. Regression Penalty (scale −0.20)
+       - Applied when score decreases from previous step
+       - Formula: REGRESSION_PENALTY_SCALE * abs(score_delta)
+       - Discourages actions that make code worse
+    
+    Final Reward: clamp(progress + syntax + test + quality - stagnation - regression, -1.0, +1.0)
+    
+    The result is always bounded in [-1.0, +1.0], providing interpretable feedback for learning.
+    """
+
+    value: float = Field(..., description="Net scalar reward for this step (bounded in [-1.0, +1.0])")
+    syntax_reward: float = Field(default=0.0, description="Bonus for fixing syntax errors (max +0.35)")
+    test_reward: float = Field(default=0.0, description="Reward from test improvements (max +0.20)")
+    quality_bonus: float = Field(default=0.0, description="Bonus for code quality improvements (max +0.15)")
+    correctness_bonus: float = Field(default=0.0, description="Bonus for full correctness (max +0.50)")
+    progress_delta: float = Field(default=0.0, description="Reward from score improvement (max +0.25)")
+    stagnation_penalty: float = Field(default=0.0, description="Penalty for unchanged code (−0.10)")
+    regression_penalty: float = Field(default=0.0, description="Penalty for score decline (scale −0.20)")
+    invalid_action_penalty: float = Field(default=0.0, description="Penalty for invalid actions (−0.15)")
+    timeout_penalty: float = Field(default=0.0, description="Penalty for execution timeout (−0.15)")
+    reason: str = Field(..., description="Human-readable explanation of the reward")
+    
+    # Debug information for transparency
+    prev_score: float = Field(default=0.0, description="Score before this step")
+    curr_score: float = Field(default=0.0, description="Score after this step")
+    code_changed: bool = Field(default=False, description="Whether the action modified the code")
+
+
+class PythonCodeReviewAction(Action):
+    """Action space for code review environment."""
+
+    action_type: ActionType = Field(..., description="Type of action to perform")
+    code: Optional[str] = Field(default=None, description="New code for edit_code actions")
+
+
+class PythonCodeReviewObservation(Observation):
+    """Observation returned by reset() and step()."""
+
+    task_id: str = Field(..., description="Current task identifier")
+    title: str = Field(default="", description="Human-readable task title")
+    difficulty: Difficulty = Field(..., description="Task difficulty level")
+    task_kind: Optional[TaskKind] = Field(default=None, description="Task type")
+    task_description: str = Field(..., description="Detailed task description")
+    current_code: str = Field(..., description="Current code state")
+    errors: str = Field(..., description="Syntax/compilation errors, if any")
+    test_results: str = Field(..., description="Results from test execution")
+    visible_tests: List[str] = Field(default_factory=list, description="Public test cases")
+    history: List[HistoryEntry] = Field(default_factory=list, description="Action history")
+    attempts_remaining: int = Field(..., ge=0, description="Actions left in episode")
+    last_action_status: str = Field(default="", description="Outcome message from the last action")
+    score: float = Field(..., ge=0.0, le=1.0, description="Current episode score")
+    reward_details: RewardDetails = Field(
+        default_factory=lambda: RewardDetails(value=0.0, reason="Reset"),
+        description="Detailed reward breakdown for the last action",
+    )
+
+
+class PythonCodeReviewState(State):
+    """Exposed environment state."""
+
+    episode_id: str = Field(..., description="Unique episode identifier")
+    step_count: int = Field(default=0, ge=0)
+    task_id: Optional[str] = Field(default=None)
+    difficulty: Optional[Difficulty] = Field(default=None)
+    task_kind: Optional[TaskKind] = Field(default=None)
+    attempts_remaining: int = Field(default=0, ge=0)
+    current_code: str = Field(default="")
+    errors: str = Field(default="")
+    test_results: str = Field(default="")
+    history: List[HistoryEntry] = Field(default_factory=list)
+    score: float = Field(default=0.0, ge=0.0, le=1.0)
+    done: bool = Field(default=False)
+
+
+class TaskDescriptor(BaseModel):
+    """Public task metadata."""
 
-# Categories help both humans and agents understand what type of issue was found.
-Category = Literal["bug", "security", "style", "performance", "maintainability"]
+    task_id: str = Field(..., description="Stable task identifier")
+    title: str = Field(..., description="Human-readable title")
+    difficulty: Difficulty = Field(..., description="Difficulty level")
+    task_kind: Optional[TaskKind] = Field(default=None, description="Type of task")
+    task_description: str = Field(default="", description="Full task description")
+    starter_code: str = Field(default="", description="Initial broken code")
+    visible_tests: List[str] = Field(default_factory=list, description="Public test cases")
+    goal: str = Field(default="", description="Optional goal summary for review-style tasks")
+    repo_summary: str = Field(default="", description="Optional repository context")
+    changed_files: List[str] = Field(default_factory=list, description="Changed files for review-style tasks")
+    available_files: List[str] = Field(default_factory=list, description="Browsable files for review-style tasks")
+    max_steps: int = Field(..., ge=1, description="Maximum steps allowed")
+
+
+class TaskSummary(BaseModel):
+    """Lightweight task metadata for list endpoints."""
 
-# Operations define the small action space an agent can use during an episode.
-Operation = Literal["submit_findings", "request_hint", "finalize"]
+    task_id: str = Field(..., description="Stable task identifier")
+    difficulty: Difficulty = Field(..., description="Difficulty level")
+    title: str = Field(..., description="Human-readable title")
+    goal: str = Field(default="", description="Optional task goal")
 
 
 class ReviewFinding(BaseModel):
-    """A structured review finding.
-
-    Each finding is designed to be machine-gradable while still resembling the
-    sort of issue summary a human reviewer would write in a real code review.
-    """
-
-    title: str = Field(..., description="Short title for the finding")
-    line: Optional[int] = Field(default=None, description="1-based source line number")
-    category: Category = Field(default="bug", description="Issue category")
-    severity: Severity = Field(default="warning", description="Issue severity")
-    rationale: str = Field(
-        default="",
-        description="Why the issue matters and how it affects behaviour or safety",
-    )
-    recommendation: Optional[str] = Field(
-        default=None, description="Concrete fix recommendation"
-    )
-    rule_id: Optional[str] = Field(
-        default=None,
-        description="Stable internal rule identifier when known",
-    )
+    """Structured code review finding used by auxiliary review utilities."""
 
+    title: str = Field(..., description="Short human-readable finding title")
+    file_path: str = Field(default="", description="Optional file path")
+    line: Optional[int] = Field(default=None, ge=1, description="Optional 1-based line number")
+    category: Category = Field(default="bug", description="Finding category")
+    severity: Severity = Field(default="warning", description="Finding severity")
+    rationale: str = Field(default="", description="Why this matters")
+    recommendation: str = Field(default="", description="Suggested remediation")
+    rule_id: str = Field(default="", description="Stable detector or rubric identifier")
 
-class TaskDescriptor(BaseModel):
-    """Public task metadata shown to the agent.
+    @property
+    def explanation(self) -> str:
+        """Backward-compatible alias used by older grading helpers."""
+        return self.rationale
 
-    This is intentionally the "visible" task information. Hidden grading
-    details stay inside the server task bank so the benchmark remains useful.
-    """
-
-    task_id: str = Field(..., description="Stable task identifier")
-    difficulty: Difficulty = Field(..., description="Task difficulty bucket")
-    title: str = Field(..., description="Short task title")
-    objective: str = Field(..., description="What the reviewer should accomplish")
-    code: str = Field(..., description="Python code to review")
-    max_steps: int = Field(..., ge=1, description="Maximum actions allowed")
-    success_threshold: float = Field(
-        ..., ge=0.0, le=1.0, description="Minimum score considered a pass"
-    )
-
-
-class TaskEvaluation(BaseModel):
-    """Deterministic grader output.
-
-    This model is returned in observations and offline grading routes so that
-    both online interaction and offline evaluation use exactly the same metrics.
-    """
-
-    matched_reference_ids: List[str] = Field(default_factory=list)
-    matched_findings: int = Field(default=0, ge=0)
-    total_findings: int = Field(default=0, ge=0)
-    false_positives: int = Field(default=0, ge=0)
-    duplicate_findings: int = Field(default=0, ge=0)
-    weighted_recall: float = Field(default=0.0, ge=0.0, le=1.0)
-    patch_score: float = Field(default=0.0, ge=0.0, le=1.0)
-    score: float = Field(default=0.0, ge=0.0, le=1.0)
-    passed: bool = Field(default=False)
-
-
-class PythonReviewAction(Action):
-    """Action submitted by an agent during an episode.
-
-    The action space is kept intentionally small:
-
-    - `submit_findings` for intermediate progress
-    - `request_hint` when the agent needs guidance at a small penalty
-    - `finalize` when the agent wants the episode to end
-    """
-
-    operation: Operation = Field(
-        default="submit_findings",
-        description="How to interact with the environment on this step",
-    )
-    findings: List[ReviewFinding] = Field(
-        default_factory=list,
-        description="Structured findings being submitted for grading",
-    )
-    patched_code: Optional[str] = Field(
-        default=None,
-        description="Optional improved version of the code under review",
-    )
-    note: Optional[str] = Field(
-        default=None,
-        description="Optional free-form reviewer note for logging or context",
-    )
-
-
-class PythonEnvConfig(BaseModel):
-    """Environment-level configuration knobs.
-
-    These values are useful for experimentation because they let you adjust
-    reward shaping and curriculum ordering without changing the grader logic.
-    """
-
-    task_order: List[str] = Field(
-        default_factory=lambda: ["py-review-easy", "py-review-medium", "py-review-hard"],
-        description="Deterministic task order used across resets",
-    )
-    max_steps_per_task: int = Field(default=4, ge=1, le=10)
-    hint_penalty: float = Field(default=0.05, ge=0.0, le=1.0)
-    false_positive_penalty: float = Field(default=0.08, ge=0.0, le=1.0)
-    duplicate_penalty: float = Field(default=0.03, ge=0.0, le=1.0)
-    patch_bonus_multiplier: float = Field(default=0.2, ge=0.0, le=1.0)
-    max_history_entries: int = Field(default=50, ge=1, le=500)
-
-
-class PythonReviewObservation(Observation):
-    """Observation returned by `reset()` and `step()`.
-
-    The observation combines:
-
-    - visible task context
-    - immediate feedback on the previous action
-    - cumulative evaluation state
-    - OpenEnv-standard reward/done/metadata fields
-    """
-
-    task: TaskDescriptor = Field(..., description="Current task details")
-    instructions: str = Field(
-        default="Inspect the code and submit structured findings.",
-        description="Episode instructions shown to the agent",
-    )
-    feedback: str = Field(default="", description="Feedback for the last action")
-    submitted_findings: List[ReviewFinding] = Field(
-        default_factory=list,
-        description="All findings submitted so far in this episode",
-    )
-    hints_used: int = Field(default=0, ge=0)
-    attempts_remaining: int = Field(default=0, ge=0)
-    evaluation: TaskEvaluation = Field(default_factory=TaskEvaluation)
-    score: float = Field(
-        default=0.0,
-        ge=0.0,
-        le=1.0,
-        description="Current task score after this step",
-    )
-    review_time_ms: float = Field(default=0.0, ge=0.0)
-
-
-class EpisodeRecord(BaseModel):
-    """Stored summary of a completed or in-progress episode.
-
-    This model is used by the custom history routes and is intentionally
-    compact enough to archive for later analysis or dataset creation.
-    """
-
-    episode_id: str
-    task_id: str
-    difficulty: Difficulty
-    title: str
-    final_score: float = Field(ge=0.0, le=1.0)
-    passed: bool = Field(default=False)
-    steps_taken: int = Field(default=0, ge=0)
-    hints_used: int = Field(default=0, ge=0)
-    matched_findings: int = Field(default=0, ge=0)
-    total_findings: int = Field(default=0, ge=0)
-    false_positives: int = Field(default=0, ge=0)
-    duplicate_findings: int = Field(default=0, ge=0)
-    status: Literal["active", "completed"] = Field(default="completed")
-    created_at: str
-    updated_at: str
-
-
-class DirectReviewRequest(BaseModel):
-    """Request model for ad-hoc review outside the benchmark tasks."""
-
-    code: str = Field(..., description="Python source code to inspect")
-    context: Optional[str] = Field(
-        default=None, description="Optional explanation of the code's purpose"
-    )
+    @property
+    def suggested_fix(self) -> str:
+        """Backward-compatible alias used by older grading helpers."""
+        return self.recommendation
 
 
 class DirectReviewResponse(BaseModel):
-    """Static review result for arbitrary Python code.
-
-    This route is useful for manual testing and dataset generation because it
-    lets you review arbitrary snippets without entering the benchmark loop.
-    """
+    """Response payload for deterministic direct-review utilities."""
 
     issues: List[ReviewFinding] = Field(default_factory=list)
     summary: str = Field(default="")
@@ -219,30 +187,26 @@ class DirectReviewResponse(BaseModel):
     improved_code: Optional[str] = Field(default=None)
 
 
-class DeleteResponse(BaseModel):
-    """Small acknowledgement payload for DELETE routes."""
-
-    detail: str
-
+class TaskGrade(BaseModel):
+    """Grading result for task submission."""
 
-class HealthResponse(BaseModel):
-    """Health payload used by Docker and Spaces checks.
-
-    This payload stays intentionally simple because health checks are often
-    consumed by infrastructure rather than by human users.
-    """
-
-    status: Literal["ok"] = "ok"
-    environment: str = "python_env"
-    task_count: int = Field(default=0, ge=0)
-    active_task_id: Optional[str] = None
-    active_episode_id: Optional[str] = None
-
-
-# Backward-compatible aliases keep older imports working while the project
-# standardizes on the `Python*` naming convention.
-PythonAction = PythonReviewAction
-PythonObservation = PythonReviewObservation
-CodeReviewAction = PythonReviewAction
-CodeReviewObservation = PythonReviewObservation
-CodeReviewConfig = PythonEnvConfig
+    score: float = Field(..., ge=0.0, le=1.0, description="Overall score")
+    syntax_score: float = Field(default=0.0, ge=0.0, le=1.0)
+    tests_passed: int = Field(default=0, ge=0)
+    tests_total: int = Field(default=0, ge=0)
+    quality_score: float = Field(default=0.0, ge=0.0, le=1.0)
+    runtime_score: float = Field(default=0.0, ge=0.0, le=1.0)
+    timed_out: bool = Field(default=False)
+    matched_issue_ids: List[str] = Field(default_factory=list)
+    false_positives: int = Field(default=0, ge=0)
+    duplicate_findings: int = Field(default=0, ge=0)
+    matched_weight: float = Field(default=0.0, ge=0.0, le=1.0)
+    details: Dict[str, Any] = Field(default_factory=dict)
+
+
+class HealthResponse(BaseModel):
+    """Health check response."""
+
+    status: Literal["ok"] = "ok"
+    environment: str = "python_code_review_env"
+    task_count: int = Field(default=0, ge=0)

openenv.yaml

@@ -1,7 +1,20 @@
-spec_version: 1
-name: python_env
-type: space
-runtime: fastapi
-app: server.app:app
-port: 8000
-
+spec_version: 1
+name: python_code_review_env
+type: space
+runtime: fastapi
+app: server.app:app
+port: 8000
+
+metadata:
+  description: "Production-grade Python code review and repair benchmark for OpenEnv"
+  domain: code-review
+  task_count: 3
+  task_ids:
+    - syntax-fix-easy
+    - bug-fix-medium
+    - optimization-hard
+  difficulty_levels:
+    - easy
+    - medium
+    - hard
+

openenv_python_env.egg-info/PKG-INFO

@@ -1,10 +1,13 @@
 Metadata-Version: 2.4
 Name: openenv-python_env
-Version: 0.1.0
-Summary: Python Env environment for OpenEnv
+Version: 0.2.0
+Summary: Deterministic Python code review and repair benchmark environment for OpenEnv
 Requires-Python: >=3.10
 Requires-Dist: openenv-core[core]>=0.2.2
-Requires-Dist: pydantic>=2.12.5
+Requires-Dist: fastapi>=0.115.0
+Requires-Dist: uvicorn>=0.30.0
+Requires-Dist: openai>=1.40.0
+Requires-Dist: pytest>=8.0.0
 Provides-Extra: dev
 Requires-Dist: pytest>=8.0.0; extra == "dev"
 Requires-Dist: pytest-cov>=4.0.0; extra == "dev"

openenv_python_env.egg-info/SOURCES.txt

@@ -1,11 +1,8 @@
 README.md
-__init__.py
-client.py
-inference.py
-models.py
 pyproject.toml
 ./__init__.py
 ./client.py
+./compat.py
 ./inference.py
 ./models.py
 openenv_python_env.egg-info/PKG-INFO
@@ -16,4 +13,15 @@ openenv_python_env.egg-info/requires.txt
 openenv_python_env.egg-info/top_level.txt
 server/__init__.py
 server/app.py
-server/python_env_environment.py
+server/code_review_env_environment.py
+server/code_review_environment.py
+server/env.py
+server/env_safe.py
+server/grading.py
+server/python_env_environment.py
+server/static_review.py
+server/task_bank.py
+tests/test_api.py
+tests/test_environment.py
+tests/test_examples.py
+tests/test_reward_dynamics.py

openenv_python_env.egg-info/requires.txt

@@ -1,5 +1,8 @@
 openenv-core[core]>=0.2.2
-pydantic>=2.12.5
+fastapi>=0.115.0
+uvicorn>=0.30.0
+openai>=1.40.0
+pytest>=8.0.0
 
 [dev]
 pytest>=8.0.0

pyproject.toml

@@ -1,46 +1,33 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
-
-[build-system]
-requires = ["setuptools>=45", "wheel"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "openenv-python_env"
-version = "0.1.0"
-description = "Python Env environment for OpenEnv"
-requires-python = ">=3.10"
-dependencies = [
-    # Core OpenEnv runtime (provides FastAPI server + HTTP client types)
-    # install from github
-    # "openenv-core[core] @ git+https://github.com/meta-pytorch/OpenEnv.git",
-    "openenv-core[core]>=0.2.2",
-    # Environment-specific dependencies
-    # Add all dependencies needed for your environment here
-    # Examples:
-    # "numpy>=1.19.0",
-    # "torch>=2.0.0",
-    # "gymnasium>=0.29.0",
-    # "openspiel>=1.0.0",
-    # "smolagents>=1.22.0,<2",
-    "pydantic>=2.12.5",
-]
-
-[project.optional-dependencies]
-dev = [
-    "pytest>=8.0.0",
-    "pytest-cov>=4.0.0",
-]
-
-[project.scripts]
-# Server entry point - enables running via: uv run --project . server
-# or: python -m python_env.server.app
-server = "python_env.server.app:main"
-
-[tool.setuptools]
-include-package-data = true
-packages = ["python_env", "python_env.server"]
-package-dir = { "python_env" = ".", "python_env.server" = "server" }
+[build-system]
+requires = ["setuptools>=45", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "openenv-python_env"
+version = "0.2.0"
+description = "Deterministic Python code review and repair benchmark environment for OpenEnv"
+requires-python = ">=3.10"
+dependencies = [
+    "openenv-core[core]>=0.2.2",
+    "fastapi>=0.115.0",
+    "uvicorn>=0.30.0",
+    "openai>=1.40.0",
+    "pytest>=8.0.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0.0",
+    "pytest-cov>=4.0.0",
+]
+
+[project.scripts]
+server = "python_env.server.app:main"
+
+[tool.setuptools]
+include-package-data = true
+packages = ["python_env", "python_env.server"]
+package-dir = { "python_env" = ".", "python_env.server" = "server" }
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

pytest-cache-files-1f62ra1g/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

pytest-cache-files-1f62ra1g/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

pytest-cache-files-i2cpw3zw/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

pytest-cache-files-i2cpw3zw/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

pytest-cache-files-le0qcl0z/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

pytest-cache-files-le0qcl0z/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

pytest-cache-files-qm8xzmpt/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

pytest-cache-files-qm8xzmpt/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

pytest-cache-files-qun9v98v/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

pytest-cache-files-qun9v98v/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

pytest-cache-files-srp2otxc/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

pytest-cache-files-srp2otxc/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

pytest-cache-files-u6t7g29i/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

pytest-cache-files-u6t7g29i/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

pytest-cache-files-x1yzwik9/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

pytest-cache-files-x1yzwik9/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

server/__init__.py

@@ -1,11 +1,5 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
-
-"""Python Env environment server components."""
-
-from .python_env_environment import PythonEnvironment
-
-__all__ = ["PythonEnvironment"]
+"""Server exports for the Python code review environment."""
+
+from .code_review_environment import CodeReviewEnvironment, PythonCodeReviewEnvironment, PythonEnvironment
+
+__all__ = ["PythonEnvironment", "PythonCodeReviewEnvironment", "CodeReviewEnvironment"]

server/app.py

@@ -1,84 +1,117 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
-
-"""
-FastAPI application for the Python Env Environment.
-
-This module creates an HTTP server that exposes the PythonEnvironment
-over HTTP and WebSocket endpoints, compatible with EnvClient.
-
-Endpoints:
-    - POST /reset: Reset the environment
-    - POST /step: Execute an action
-    - GET /state: Get current environment state
-    - GET /schema: Get action/observation schemas
-    - WS /ws: WebSocket endpoint for persistent sessions
-
-Usage:
-    # Development (with auto-reload):
-    uvicorn server.app:app --reload --host 0.0.0.0 --port 8000
-
-    # Production:
-    uvicorn server.app:app --host 0.0.0.0 --port 8000 --workers 4
-
-    # Or run directly:
-    python -m server.app
-"""
-
+"""FastAPI application for the Python code review environment."""
+
+from __future__ import annotations
+
+import os
+
+from fastapi import APIRouter, HTTPException
+from fastapi.responses import RedirectResponse
+
+from compat import create_app
+
+from models import (
+    HealthResponse,
+    PythonCodeReviewAction,
+    PythonCodeReviewObservation,
+    PythonCodeReviewState,
+    TaskDescriptor,
+    TaskGrade,
+)
+from server.env import PythonCodeReviewEnvironment
+
+
 try:
-    from openenv.core.env_server.http_server import create_app
-except Exception as e:  # pragma: no cover
-    raise ImportError(
-        "openenv is required for the web interface. Install dependencies with '\n    uv sync\n'"
-    ) from e
-
-try:
-    from ..models import PythonAction, PythonObservation
-    from .python_env_environment import PythonEnvironment
-except ImportError:
-    from models import PythonAction, PythonObservation
-    from server.python_env_environment import PythonEnvironment
-
+    MAX_CONCURRENT_ENVS = max(int(os.getenv("MAX_CONCURRENT_ENVS", "16")), 1)
+except Exception:
+    MAX_CONCURRENT_ENVS = 16
 
-# Create the app with web interface and README integration
+python_env = PythonCodeReviewEnvironment(verbose=False)
 app = create_app(
-    PythonEnvironment,
-    PythonAction,
-    PythonObservation,
-    env_name="python_env",
-    max_concurrent_envs=1,  # increase this number to allow more concurrent WebSocket sessions
-)
-
-
-def main(host: str = "0.0.0.0", port: int = 8000):
-    """
-    Entry point for direct execution via uv run or python -m.
-
-    This function enables running the server without Docker:
-        uv run --project . server
-        uv run --project . server --port 8001
-        python -m python_env.server.app
-
-    Args:
-        host: Host address to bind to (default: "0.0.0.0")
-        port: Port number to listen on (default: 8000)
-
-    For production deployments, consider using uvicorn directly with
-    multiple workers:
-        uvicorn python_env.server.app:app --workers 4
-    """
-    import uvicorn
-
-    uvicorn.run(app, host=host, port=port)
-
-
-if __name__ == "__main__":
-    import argparse
-
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--port", type=int, default=8000)
-    args = parser.parse_args()
-    main(port=args.port)
+    PythonCodeReviewEnvironment,
+    PythonCodeReviewAction,
+    PythonCodeReviewObservation,
+    max_concurrent_envs=MAX_CONCURRENT_ENVS,
+)
+router = APIRouter(tags=["python-code-review"])
+
+
+@router.get("/", include_in_schema=False)
+def root() -> RedirectResponse:
+    """Redirect root to API documentation."""
+    return RedirectResponse(url="/docs")
+
+
+@router.get("/health", response_model=HealthResponse)
+def health() -> HealthResponse:
+    """Health check endpoint for deployment monitoring."""
+    return python_env.health()
+
+
+@router.get("/tasks", response_model=list)
+def list_tasks() -> list:
+    """List all available deterministic tasks."""
+    return python_env.list_task_summaries()
+
+
+@router.get("/tasks/{task_id}", response_model=object)
+def get_task(task_id: str) -> object:
+    """Get a specific task by ID."""
+    try:
+        return python_env.get_task(task_id)
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail=str(exc)) from exc
+
+
+@router.post("/tasks/{task_id}/grade", response_model=TaskGrade)
+def grade_task(task_id: str, payload: PythonCodeReviewAction) -> TaskGrade:
+    """Grade code submission for a task without running an episode."""
+    if payload.action_type != "edit_code" or not payload.code:
+        raise HTTPException(
+            status_code=400, 
+            detail="Requires action_type='edit_code' with code parameter."
+        )
+    try:
+        return python_env.grade_task_submission(task_id=task_id, code=payload.code)
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail=str(exc)) from exc
+
+
+@router.post("/state", response_model=PythonCodeReviewState)
+def get_state_post() -> RedirectResponse:
+    """Redirect POST /state to GET for compatibility."""
+    return RedirectResponse(url="/state", status_code=303)
+
+
+app.include_router(router)
+
+
+def _prioritize_route(path: str, methods: set[str]) -> None:
+    """Move a matching custom route ahead of default OpenEnv routes."""
+    try:
+        for index in range(len(app.router.routes) - 1, -1, -1):
+            route = app.router.routes[index]
+            route_path = getattr(route, "path", None)
+            route_methods = set(getattr(route, "methods", set()) or set())
+            if route_path == path and methods.issubset(route_methods):
+                app.router.routes.insert(0, app.router.routes.pop(index))
+                break
+    except Exception:
+        pass
+
+
+_prioritize_route("/health", {"GET"})
+
+
+def main(host: str = "0.0.0.0", port: int = 8000) -> None:
+    """Run the FastAPI application with uvicorn."""
+    import uvicorn
+    uvicorn.run(
+        app,
+        host=os.getenv("HOST", host),
+        port=int(os.getenv("PORT", str(port))),
+    )
+
+
+if __name__ == "__main__":
+    main()
+

server/code_review_env_environment.py

@@ -0,0 +1,9 @@
+"""Compatibility shim for older imports."""
+
+try:
+    from server.code_review_environment import CodeReviewEnvironment
+except ModuleNotFoundError:  # pragma: no cover
+    from .code_review_environment import CodeReviewEnvironment
+
+
+__all__ = ["CodeReviewEnvironment"]

server/code_review_environment.py

@@ -0,0 +1,5 @@
+"""Compatibility wrapper for older imports."""
+
+from .env import CodeReviewEnvironment, PythonCodeReviewEnvironment, PythonEnvironment
+
+__all__ = ["CodeReviewEnvironment", "PythonCodeReviewEnvironment", "PythonEnvironment"]

server/env.py

@@ -0,0 +1 @@
+from .env_safe import *  # noqa: F401,F403

server/env_safe.py

@@ -0,0 +1,492 @@
+"""Safe OpenEnv environment for deterministic Python code repair tasks."""
+
+from __future__ import annotations
+
+from typing import Any, Optional
+from uuid import uuid4
+
+from compat import Environment
+from graders import grade_task
+from models import (
+    HealthResponse,
+    HistoryEntry,
+    PythonCodeReviewAction,
+    PythonCodeReviewObservation,
+    PythonCodeReviewState,
+    RewardDetails,
+    TaskGrade,
+)
+from tasks import TaskSpec, get_task as load_task, list_task_summaries, task_ids
+
+
+INVALID_ACTION_PENALTY = 0.10
+NO_PROGRESS_PENALTY = 0.08
+REPEATED_ACTION_PENALTY = 0.05
+BASE_STEP_PENALTY = 0.02
+ANALYZE_STEP_PENALTY = 0.01
+SUBMIT_COMPLETION_BONUS = 0.30
+TIMEOUT_PENALTY = 0.12
+VALID_ACTIONS = {"analyze_code", "edit_code", "run_tests", "submit_solution"}
+
+
+def _clamp(value: float, low: float = 0.0, high: float = 1.0) -> float:
+    """Clamp a scalar to a bounded numeric interval."""
+    try:
+        return max(low, min(high, float(value)))
+    except Exception:
+        return low
+
+
+def _safe_text(value: Any, default: str = "") -> str:
+    """Convert values into short stable strings."""
+    try:
+        text = str(value)
+    except Exception:
+        return default
+    text = " ".join(text.split())
+    return text[:240] if text else default
+
+
+class PythonCodeReviewEnvironment(
+    Environment[PythonCodeReviewAction, PythonCodeReviewObservation, PythonCodeReviewState]
+):
+    """Deterministic, bounded, evaluator-safe environment for code repair tasks."""
+
+    SUPPORTS_CONCURRENT_SESSIONS = True
+
+    def __init__(self, verbose: bool = False) -> None:
+        super().__init__()
+        self._verbose = bool(verbose)
+        self._task_order = self._safe_task_order()
+        self._task_cursor = -1
+        self._task: Optional[TaskSpec] = None
+        self._state = PythonCodeReviewState(episode_id=str(uuid4()))
+        self._done = False
+        self._last_status = "Call reset() to start."
+        self._last_reward = RewardDetails(value=0.0, reason="Environment initialized.")
+        self._metrics = self._blank_metrics()
+        self._last_action_type = ""
+
+    def reset(
+        self,
+        seed: Optional[int] = None,
+        episode_id: Optional[str] = None,
+        task_id: Optional[str] = None,
+        **_: object,
+    ) -> PythonCodeReviewObservation:
+        """Reset the environment for a deterministic task and return an observation."""
+        del seed
+        try:
+            self._reset_rubric()
+        except Exception:
+            pass
+
+        task = self._select_task(task_id)
+        self._task = task
+        self._done = False
+        self._metrics = self._blank_metrics()
+        self._last_action_type = ""
+        self._last_status = "Inspect the code, run checks, edit the code, then submit."
+        self._last_reward = RewardDetails(
+            value=0.0,
+            reason="Episode reset.",
+            prev_score=0.0,
+            curr_score=0.0,
+        )
+        self._state = PythonCodeReviewState(
+            episode_id=episode_id or str(uuid4()),
+            step_count=0,
+            task_id=task.task_id,
+            difficulty=task.difficulty,
+            task_kind=task.task_kind,
+            attempts_remaining=max(int(task.max_steps), 1),
+            current_code=task.starter_code,
+            errors="",
+            test_results="No checks run yet.",
+            history=[],
+            score=0.0,
+            done=False,
+        )
+        return self._build_observation()
+
+    def step(
+        self,
+        action: PythonCodeReviewAction,
+        timeout_s: Optional[float] = None,
+        **_: object,
+    ) -> PythonCodeReviewObservation:
+        """Execute one safe environment step and always return a valid observation."""
+        del timeout_s
+        try:
+            if self._task is None:
+                return self.reset()
+
+            if self._done:
+                self._last_status = "Episode already completed. Call reset() to continue."
+                self._last_reward = RewardDetails(
+                    value=-INVALID_ACTION_PENALTY,
+                    invalid_action_penalty=INVALID_ACTION_PENALTY,
+                    reason="Episode already completed.",
+                    prev_score=self._metrics["score"],
+                    curr_score=self._metrics["score"],
+                    code_changed=False,
+                )
+                return self._build_observation()
+
+            self._state.step_count += 1
+            action_type = _safe_text(getattr(action, "action_type", "analyze_code"), "analyze_code")
+            code = getattr(action, "code", None)
+
+            if action_type == "analyze_code":
+                self._handle_scored_action(action_type=action_type, candidate_code=self._state.current_code, include_hidden=False)
+            elif action_type == "run_tests":
+                self._handle_scored_action(action_type=action_type, candidate_code=self._state.current_code, include_hidden=False)
+            elif action_type == "edit_code":
+                self._handle_edit(code)
+            elif action_type == "submit_solution":
+                self._handle_scored_action(action_type=action_type, candidate_code=self._state.current_code, include_hidden=True)
+                self._done = True
+            else:
+                self._apply_invalid_action(f"Unsupported action_type '{action_type}'.")
+
+            self._state.attempts_remaining = max(self._task.max_steps - self._state.step_count, 0)
+            if self._state.attempts_remaining == 0 and not self._done:
+                self._auto_submit()
+
+            self._state.done = self._done
+            return self._build_observation()
+        except Exception as exc:
+            self._apply_invalid_action(f"Step failure handled: {_safe_text(exc, 'unknown_error')}")
+            self._state.done = self._done
+            return self._build_observation()
+
+    @property
+    def state(self) -> PythonCodeReviewState:
+        """Return a deep copy of the current environment state."""
+        try:
+            return self._state.model_copy(deep=True)
+        except Exception:
+            return PythonCodeReviewState(episode_id=str(uuid4()))
+
+    def list_task_summaries(self) -> list[object]:
+        """Return public task summaries."""
+        try:
+            return list_task_summaries()
+        except Exception:
+            return []
+
+    def get_task(self, task_id: str) -> object:
+        """Return a single public task descriptor."""
+        return self._select_task(task_id).to_descriptor()
+
+    def health(self) -> HealthResponse:
+        """Return a simple health response."""
+        return HealthResponse(task_count=len(self._task_order))
+
+    def grade_task_submission(self, task_id: str, code: str) -> TaskGrade:
+        """Grade a task submission outside an episode without raising."""
+        try:
+            task = self._select_task(task_id)
+            return self._safe_grade(task=task, candidate_code=code, include_hidden=True)
+        except Exception as exc:
+            return TaskGrade(score=0.0, details={"error": _safe_text(exc, "grading_failed")})
+
+    def run_tests(self, code: str, include_hidden: bool = False) -> tuple[float, dict[str, int], TaskGrade]:
+        """Run deterministic grading and return score plus test summary."""
+        task = self._task or self._select_task(None)
+        grade = self._safe_grade(task=task, candidate_code=code, include_hidden=include_hidden)
+        return (
+            _clamp(grade.score),
+            {"passed": int(grade.tests_passed), "total": int(grade.tests_total)},
+            grade,
+        )
+
+    def apply_action(self, action: PythonCodeReviewAction) -> str:
+        """Return the candidate code implied by the action."""
+        if getattr(action, "action_type", "") == "edit_code":
+            code = getattr(action, "code", None)
+            return str(code) if code is not None else self._state.current_code
+        return self._state.current_code
+
+    def compute_reward(
+        self,
+        action_type: str,
+        previous_metrics: dict[str, float],
+        current_metrics: dict[str, float],
+        grade: TaskGrade,
+        code_changed: bool,
+        invalid_action: bool = False,
+    ) -> RewardDetails:
+        """Compute a bounded dynamic reward with progress and efficiency shaping."""
+        prev_score = _clamp(previous_metrics.get("score", 0.0))
+        curr_score = _clamp(current_metrics.get("score", 0.0))
+        score_delta = curr_score - prev_score
+        test_delta = current_metrics.get("test_fraction", 0.0) - previous_metrics.get("test_fraction", 0.0)
+        syntax_delta = current_metrics.get("syntax_score", 0.0) - previous_metrics.get("syntax_score", 0.0)
+        quality_delta = current_metrics.get("quality_score", 0.0) - previous_metrics.get("quality_score", 0.0)
+
+        step_penalty = BASE_STEP_PENALTY + (ANALYZE_STEP_PENALTY if action_type == "analyze_code" else 0.0)
+        repeated_penalty = REPEATED_ACTION_PENALTY if action_type == self._last_action_type else 0.0
+        no_progress = (
+            score_delta <= 1e-9
+            and test_delta <= 1e-9
+            and syntax_delta <= 1e-9
+            and quality_delta <= 1e-9
+            and not code_changed
+        )
+        stagnation_penalty = NO_PROGRESS_PENALTY if no_progress and not invalid_action else 0.0
+        regression_penalty = max(-score_delta, 0.0) * 0.6 + repeated_penalty + step_penalty
+        invalid_penalty = INVALID_ACTION_PENALTY if invalid_action else 0.0
+        timeout_penalty = TIMEOUT_PENALTY if bool(grade.timed_out) else 0.0
+
+        progress_reward = max(score_delta, 0.0) * 0.7
+        syntax_reward = max(syntax_delta, 0.0) * 0.5
+        test_reward = max(test_delta, 0.0) * 1.0
+        quality_bonus = max(quality_delta, 0.0) * 0.2
+        correctness_bonus = SUBMIT_COMPLETION_BONUS if action_type == "submit_solution" and curr_score >= 0.999 else 0.0
+
+        reward_value = (
+            progress_reward
+            + syntax_reward
+            + test_reward
+            + quality_bonus
+            + correctness_bonus
+            - stagnation_penalty
+            - regression_penalty
+            - invalid_penalty
+            - timeout_penalty
+        )
+        reward_value = max(-1.0, min(1.0, round(reward_value, 6)))
+        return RewardDetails(
+            value=reward_value,
+            syntax_reward=round(syntax_reward, 6),
+            test_reward=round(test_reward, 6),
+            quality_bonus=round(quality_bonus, 6),
+            correctness_bonus=round(correctness_bonus, 6),
+            progress_delta=round(progress_reward, 6),
+            stagnation_penalty=round(stagnation_penalty, 6),
+            regression_penalty=round(regression_penalty, 6),
+            invalid_action_penalty=round(invalid_penalty, 6),
+            timeout_penalty=round(timeout_penalty, 6),
+            reason=f"{action_type} reward computed safely",
+            prev_score=round(prev_score, 6),
+            curr_score=round(curr_score, 6),
+            code_changed=bool(code_changed),
+        )
+
+    def _safe_task_order(self) -> list[str]:
+        """Load deterministic task ids with a hard fallback."""
+        try:
+            loaded = list(task_ids())
+            if loaded:
+                return [str(task_id) for task_id in loaded]
+        except Exception:
+            pass
+        return ["syntax-fix-easy", "bug-fix-medium", "optimization-hard"]
+
+    def _blank_metrics(self) -> dict[str, float]:
+        """Return an empty metric snapshot."""
+        return {
+            "score": 0.0,
+            "test_fraction": 0.0,
+            "syntax_score": 0.0,
+            "quality_score": 0.0,
+        }
+
+    def _select_task(self, task_id: Optional[str]) -> TaskSpec:
+        """Select the requested task or advance deterministically."""
+        try:
+            if task_id:
+                task = load_task(task_id)
+                if task.task_id in self._task_order:
+                    self._task_cursor = self._task_order.index(task.task_id)
+                return task
+        except Exception:
+            pass
+
+        try:
+            self._task_cursor = (self._task_cursor + 1) % len(self._task_order)
+            return load_task(self._task_order[self._task_cursor])
+        except Exception:
+            return load_task("syntax-fix-easy")
+
+    def _safe_grade(self, task: TaskSpec, candidate_code: str, include_hidden: bool) -> TaskGrade:
+        """Run grading without allowing exceptions to escape."""
+        try:
+            return grade_task(candidate_code, task, include_hidden=include_hidden)
+        except Exception as exc:
+            return TaskGrade(
+                score=0.0,
+                syntax_score=0.0,
+                tests_passed=0,
+                tests_total=max(len(task.visible_tests), 1),
+                details={"compile_error": "", "error": _safe_text(exc, "grading_failed")},
+            )
+
+    def _metrics_from_grade(self, grade: TaskGrade) -> dict[str, float]:
+        """Derive normalized reward metrics from a grading result."""
+        tests_total = max(int(grade.tests_total), 0)
+        tests_passed = max(int(grade.tests_passed), 0)
+        test_fraction = (tests_passed / tests_total) if tests_total else _clamp(grade.syntax_score)
+        return {
+            "score": _clamp(grade.score),
+            "test_fraction": _clamp(test_fraction),
+            "syntax_score": _clamp(grade.syntax_score),
+            "quality_score": _clamp(grade.quality_score),
+        }
+
+    def _format_test_results(self, grade: TaskGrade, include_hidden: bool) -> str:
+        """Format test execution results for the observation."""
+        compile_error = _safe_text(grade.details.get("compile_error", ""), "")
+        scope = "all checks" if include_hidden else "visible checks"
+        if compile_error:
+            return f"{scope}: compile error: {compile_error}"
+        if grade.timed_out:
+            return f"{scope}: execution timed out"
+        if self._task and self._task.task_kind == "syntax_fix":
+            return "visible checks: code compiles successfully"
+        return f"{scope}: {int(grade.tests_passed)}/{int(grade.tests_total)} passing"
+
+    def _build_status(self, action_type: str, grade: TaskGrade) -> str:
+        """Build a human-readable status message."""
+        if action_type == "submit_solution":
+            return f"Solution submitted. Final score: {_clamp(grade.score):.3f}"
+        if action_type == "edit_code":
+            if grade.details.get("compile_error"):
+                return "Code updated, but syntax issues remain."
+            return "Code updated and evaluated."
+        if action_type == "run_tests":
+            return "Test run completed."
+        if action_type == "analyze_code":
+            return "Analysis completed."
+        return "Action handled safely."
+
+    def _apply_grade_to_state(self, grade: TaskGrade, include_hidden: bool) -> None:
+        """Update environment state from the latest grading result."""
+        compile_error = _safe_text(grade.details.get("compile_error", ""), "")
+        self._state.score = _clamp(grade.score)
+        self._state.errors = compile_error
+        self._state.test_results = self._format_test_results(grade, include_hidden=include_hidden)
+
+    def _handle_scored_action(self, action_type: str, candidate_code: str, include_hidden: bool) -> None:
+        """Grade code, update state, and compute reward for a valid action."""
+        task = self._task or self._select_task(None)
+        previous_metrics = dict(self._metrics)
+        prior_code = self._state.current_code
+        code_changed = candidate_code.strip() != prior_code.strip()
+        if action_type == "edit_code":
+            self._state.current_code = candidate_code
+        grade = self._safe_grade(task=task, candidate_code=self._state.current_code, include_hidden=include_hidden)
+        current_metrics = self._metrics_from_grade(grade)
+        self._apply_grade_to_state(grade, include_hidden=include_hidden)
+        self._last_reward = self.compute_reward(
+            action_type=action_type,
+            previous_metrics=previous_metrics,
+            current_metrics=current_metrics,
+            grade=grade,
+            code_changed=code_changed,
+            invalid_action=False,
+        )
+        self._last_status = self._build_status(action_type, grade)
+        self._metrics = current_metrics
+        self._last_action_type = action_type
+        self._append_history(action_type, self._last_status, self._last_reward.value)
+
+    def _handle_edit(self, code: Optional[str]) -> None:
+        """Validate edit input and evaluate the new candidate code."""
+        safe_code = (code or "").strip()
+        if not safe_code:
+            self._apply_invalid_action("edit_code requires code parameter.")
+            return
+        self._handle_scored_action(action_type="edit_code", candidate_code=safe_code, include_hidden=False)
+
+    def _apply_invalid_action(self, reason: str) -> None:
+        """Record an invalid action without crashing the episode."""
+        previous_metrics = dict(self._metrics)
+        grade = TaskGrade(score=previous_metrics["score"], syntax_score=previous_metrics["syntax_score"])
+        self._last_reward = self.compute_reward(
+            action_type="invalid",
+            previous_metrics=previous_metrics,
+            current_metrics=previous_metrics,
+            grade=grade,
+            code_changed=False,
+            invalid_action=True,
+        )
+        self._last_status = reason
+        self._append_history("analyze_code", reason, self._last_reward.value)
+
+    def _auto_submit(self) -> None:
+        """Finalize the episode when attempts are exhausted."""
+        task = self._task or self._select_task(None)
+        grade = self._safe_grade(task=task, candidate_code=self._state.current_code, include_hidden=True)
+        self._apply_grade_to_state(grade, include_hidden=True)
+        self._done = True
+        self._state.done = True
+        self._last_status = f"Auto-submitted. Final score: {_clamp(grade.score):.3f}"
+
+    def _append_history(self, action_type: str, status: str, reward: float) -> None:
+        """Append one action record to the episode history."""
+        try:
+            stable_action = action_type if action_type in VALID_ACTIONS else "analyze_code"
+            self._state.history.append(
+                HistoryEntry(
+                    step=max(int(self._state.step_count), 0),
+                    action_type=stable_action,
+                    status=_safe_text(status, "handled"),
+                    reward=float(reward),
+                )
+            )
+        except Exception:
+            pass
+
+    def _build_observation(self) -> PythonCodeReviewObservation:
+        """Build a valid observation from current state."""
+        task = self._task
+        try:
+            return PythonCodeReviewObservation(
+                task_id=self._state.task_id or "",
+                title=task.title if task else "",
+                difficulty=self._state.difficulty or "easy",
+                task_kind=self._state.task_kind,
+                task_description=task.task_description if task else "",
+                current_code=self._state.current_code,
+                errors=self._state.errors,
+                test_results=self._state.test_results,
+                visible_tests=list(task.visible_tests) if task else [],
+                history=list(self._state.history),
+                attempts_remaining=max(int(self._state.attempts_remaining), 0),
+                last_action_status=self._last_status,
+                score=_clamp(self._state.score),
+                reward_details=self._last_reward,
+                reward=self._last_reward.value,
+                done=bool(self._state.done),
+                metadata={
+                    "prev_score": self._last_reward.prev_score,
+                    "curr_score": self._last_reward.curr_score,
+                },
+            )
+        except Exception as exc:
+            return PythonCodeReviewObservation(
+                task_id=self._state.task_id or "",
+                title="",
+                difficulty="easy",
+                task_kind=None,
+                task_description="",
+                current_code=getattr(self._state, "current_code", ""),
+                errors=_safe_text(exc, "observation_build_failed"),
+                test_results="visible checks: unavailable",
+                visible_tests=[],
+                history=[],
+                attempts_remaining=0,
+                last_action_status="Observation fallback returned safely.",
+                score=0.0,
+                reward_details=RewardDetails(value=0.0, reason="Observation fallback."),
+                reward=0.0,
+                done=bool(getattr(self._state, "done", False)),
+                metadata={},
+            )
+
+
+PythonEnvironment = PythonCodeReviewEnvironment
+CodeReviewEnvironment = PythonCodeReviewEnvironment
+

server/grading.py

@@ -0,0 +1,147 @@
+"""Deterministic grading helpers for PR-review tasks."""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from typing import Iterable, List, Optional, Sequence, Set
+
+try:
+    from models import ReviewFinding, TaskGrade
+    from server.task_bank import RubricIssue, TaskSpec
+except ModuleNotFoundError:  # pragma: no cover
+    from ..models import ReviewFinding, TaskGrade
+    from .task_bank import RubricIssue, TaskSpec
+
+
+FALSE_POSITIVE_PENALTY = 0.10
+DUPLICATE_PENALTY = 0.05
+
+
+@dataclass(frozen=True)
+class FindingMatch:
+    """Result of matching one finding against the rubric."""
+
+    issue_id: Optional[str]
+    duplicate: bool = False
+
+
+def finding_fingerprint(finding: ReviewFinding) -> str:
+    """Build a deterministic fingerprint for duplicate detection."""
+
+    text = " ".join(
+        [
+            finding.file_path,
+            str(finding.line or 0),
+            finding.category,
+            finding.severity,
+            finding.title,
+            finding.explanation,
+            finding.suggested_fix,
+        ]
+    )
+    return "|".join(sorted(tokens(text)))
+
+
+def match_finding(
+    finding: ReviewFinding,
+    task: TaskSpec,
+    matched_issue_ids: Set[str],
+    seen_fingerprints: Set[str],
+) -> FindingMatch:
+    """Match one finding against the remaining rubric issues."""
+
+    fingerprint = finding_fingerprint(finding)
+    if fingerprint in seen_fingerprints:
+        return FindingMatch(issue_id=None, duplicate=True)
+
+    for issue in task.rubric_issues:
+        if issue.issue_id in matched_issue_ids:
+            continue
+        if finding_matches_issue(finding, issue):
+            return FindingMatch(issue_id=issue.issue_id)
+    return FindingMatch(issue_id=None)
+
+
+def finding_matches_issue(finding: ReviewFinding, issue: RubricIssue) -> bool:
+    """Return True when a finding deterministically matches a rubric issue."""
+
+    if finding.file_path != issue.file_path:
+        return False
+    if finding.category != issue.category:
+        return False
+    if finding.severity != issue.severity:
+        return False
+    if finding.line is None or abs(finding.line - issue.line) > 2:
+        return False
+
+    finding_tokens = tokens(
+        " ".join([finding.title, finding.explanation, finding.suggested_fix])
+    )
+    keyword_hits = sum(1 for keyword in issue.keywords if keyword in finding_tokens)
+    return keyword_hits >= issue.min_keyword_hits
+
+
+def score_task(
+    task: TaskSpec,
+    matched_issue_ids: Iterable[str],
+    false_positives: int = 0,
+    duplicate_findings: int = 0,
+) -> TaskGrade:
+    """Score a task from cumulative episode state."""
+
+    matched_set = set(matched_issue_ids)
+    matched_weight = sum(
+        issue.weight for issue in task.rubric_issues if issue.issue_id in matched_set
+    )
+    raw_score = matched_weight
+    raw_score -= false_positives * FALSE_POSITIVE_PENALTY
+    raw_score -= duplicate_findings * DUPLICATE_PENALTY
+    score = max(0.0, min(1.0, round(raw_score, 6)))
+    return TaskGrade(
+        score=score,
+        matched_issue_ids=sorted(matched_set),
+        false_positives=false_positives,
+        duplicate_findings=duplicate_findings,
+        matched_weight=min(1.0, round(matched_weight, 6)),
+    )
+
+
+def grade_findings(task: TaskSpec, findings: Sequence[ReviewFinding]) -> TaskGrade:
+    """Offline-grade a batch of findings for one task."""
+
+    matched_issue_ids: Set[str] = set()
+    seen_fingerprints: Set[str] = set()
+    false_positives = 0
+    duplicate_findings = 0
+
+    for finding in findings:
+        result = match_finding(
+            finding=finding,
+            task=task,
+            matched_issue_ids=matched_issue_ids,
+            seen_fingerprints=seen_fingerprints,
+        )
+        fingerprint = finding_fingerprint(finding)
+        if result.duplicate:
+            duplicate_findings += 1
+            continue
+        seen_fingerprints.add(fingerprint)
+        if result.issue_id is None:
+            false_positives += 1
+            continue
+        matched_issue_ids.add(result.issue_id)
+
+    return score_task(
+        task=task,
+        matched_issue_ids=matched_issue_ids,
+        false_positives=false_positives,
+        duplicate_findings=duplicate_findings,
+    )
+
+
+def tokens(text: str) -> Set[str]:
+    """Normalize free text into deterministic comparison tokens."""
+
+    return set(re.findall(r"[a-z0-9_]+", text.lower()))
+

server/python_env_environment.py

@@ -1,421 +1,9 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
-
-"""Python code-review environment implementation."""
-
-from __future__ import annotations
-
-from dataclasses import dataclass
-from datetime import UTC, datetime
-from typing import Dict, Iterable, List, Optional
-from uuid import uuid4
-
-from openenv.core.env_server.interfaces import Environment
-from openenv.core.env_server.types import State
-
-try:
-    from ..models import (
-        PythonAction,
-        PythonEnvConfig,
-        PythonObservation,
-        ReviewFinding,
-        TaskDescriptor,
-        TaskEvaluation,
-    )
-except ImportError:
-    from models import (  # type: ignore
-        PythonAction,
-        PythonEnvConfig,
-        PythonObservation,
-        ReviewFinding,
-        TaskDescriptor,
-        TaskEvaluation,
-    )
-
-
-@dataclass(frozen=True)
-class ReferenceFinding:
-    """Hidden finding metadata used for deterministic grading."""
-
-    rule_id: str
-    title: str
-    line: int
-    category: str
-    severity: str
-    rationale: str
-    recommendation: str
-    weight: float
-
-
-@dataclass(frozen=True)
-class ReviewTask:
-    """A visible task plus its hidden grading references."""
-
-    descriptor: TaskDescriptor
-    references: tuple[ReferenceFinding, ...]
-    hint: str
-    patched_code: Optional[str] = None
-
-
-TASK_BANK: Dict[str, ReviewTask] = {
-    "py-review-easy": ReviewTask(
-        descriptor=TaskDescriptor(
-            task_id="py-review-easy",
-            difficulty="easy",
-            title="Mutable default argument",
-            objective="Find the correctness issue and explain a safe fix.",
-            code=(
-                "def add_tag(tag, tags=[]):\n"
-                "    tags.append(tag)\n"
-                "    return tags\n"
-            ),
-            max_steps=4,
-            success_threshold=0.7,
-        ),
-        references=(
-            ReferenceFinding(
-                rule_id="mutable-default",
-                title="Mutable default list is shared across calls",
-                line=1,
-                category="bug",
-                severity="warning",
-                rationale="The list persists between calls and leaks state.",
-                recommendation="Use None as the default and create a new list inside the function.",
-                weight=1.0,
-            ),
-        ),
-        hint="Look for state that survives between separate function calls.",
-        patched_code=(
-            "def add_tag(tag, tags=None):\n"
-            "    if tags is None:\n"
-            "        tags = []\n"
-            "    tags.append(tag)\n"
-            "    return tags\n"
-        ),
-    ),
-    "py-review-medium": ReviewTask(
-        descriptor=TaskDescriptor(
-            task_id="py-review-medium",
-            difficulty="medium",
-            title="Unsafe shell invocation",
-            objective="Review the snippet for security-sensitive behavior.",
-            code=(
-                "import os\n\n"
-                "def run_backup(path):\n"
-                "    os.system(f\"tar -czf backup.tgz {path}\")\n"
-            ),
-            max_steps=4,
-            success_threshold=0.72,
-        ),
-        references=(
-            ReferenceFinding(
-                rule_id="shell-injection",
-                title="User input is interpolated into a shell command",
-                line=4,
-                category="security",
-                severity="critical",
-                rationale="An attacker can inject shell metacharacters through the path argument.",
-                recommendation="Use subprocess with an argument list instead of os.system.",
-                weight=1.0,
-            ),
-        ),
-        hint="Check how external commands are invoked and whether user input is escaped.",
-        patched_code=(
-            "import subprocess\n\n"
-            "def run_backup(path):\n"
-            "    subprocess.run([\"tar\", \"-czf\", \"backup.tgz\", path], check=True)\n"
-        ),
-    ),
-    "py-review-hard": ReviewTask(
-        descriptor=TaskDescriptor(
-            task_id="py-review-hard",
-            difficulty="hard",
-            title="Retry helper hides failures",
-            objective="Identify correctness and maintainability issues in the retry logic.",
-            code=(
-                "import time\n\n"
-                "def fetch_with_retry(client, url, retries=3):\n"
-                "    last_error = None\n"
-                "    for _ in range(retries):\n"
-                "        try:\n"
-                "            return client.get(url, timeout=1)\n"
-                "        except Exception as exc:\n"
-                "            last_error = exc\n"
-                "            time.sleep(0.1)\n"
-                "    return None\n"
-            ),
-            max_steps=4,
-            success_threshold=0.74,
-        ),
-        references=(
-            ReferenceFinding(
-                rule_id="swallowed-error",
-                title="Function swallows the final exception and returns None",
-                line=10,
-                category="bug",
-                severity="warning",
-                rationale="Callers cannot distinguish a failed request from a valid None result.",
-                recommendation="Re-raise the last exception after retries are exhausted.",
-                weight=0.65,
-            ),
-            ReferenceFinding(
-                rule_id="broad-except",
-                title="Broad exception handler catches unexpected failures",
-                line=7,
-                category="maintainability",
-                severity="info",
-                rationale="Catching Exception masks programming errors and interrupts.",
-                recommendation="Catch only the client or network exceptions you expect to retry.",
-                weight=0.35,
-            ),
-        ),
-        hint="Consider what happens to the final error after the retry loop finishes.",
-        patched_code=(
-            "import time\n\n"
-            "def fetch_with_retry(client, url, retries=3):\n"
-            "    last_error = None\n"
-            "    for _ in range(retries):\n"
-            "        try:\n"
-            "            return client.get(url, timeout=1)\n"
-            "        except client.retryable_exceptions as exc:\n"
-            "            last_error = exc\n"
-            "            time.sleep(0.1)\n"
-            "    if last_error is not None:\n"
-            "        raise last_error\n"
-        ),
-    ),
-}
-
-
-def _utc_now() -> str:
-    return datetime.now(UTC).isoformat()
-
-
-def _normalize_text(value: Optional[str]) -> str:
-    return " ".join((value or "").strip().lower().split())
-
-
-def _normalize_code(value: Optional[str]) -> str:
-    return "\n".join(line.rstrip() for line in (value or "").strip().splitlines())
-
-
-class PythonEnvironment(Environment[PythonAction, PythonObservation, State]):
-    """Deterministic benchmark environment for Python code review tasks."""
-
-    SUPPORTS_CONCURRENT_SESSIONS: bool = True
-
-    def __init__(self, config: Optional[PythonEnvConfig] = None):
-        super().__init__()
-        self._config = config or PythonEnvConfig()
-        self._state = State(episode_id=str(uuid4()), step_count=0)
-        self._task_cursor = -1
-        self._current_task: Optional[ReviewTask] = None
-        self._submitted_findings: List[ReviewFinding] = []
-        self._hints_used = 0
-        self._created_at = _utc_now()
-
-    def reset(
-        self,
-        seed: Optional[int] = None,
-        episode_id: Optional[str] = None,
-        **kwargs,
-    ) -> PythonObservation:
-        """Start the next configured review task."""
-
-        del seed, kwargs
-        self._task_cursor = (self._task_cursor + 1) % len(self._config.task_order)
-        task_id = self._config.task_order[self._task_cursor]
-        self._current_task = TASK_BANK.get(task_id, TASK_BANK["py-review-easy"])
-        self._state = State(
-            episode_id=episode_id or str(uuid4()),
-            step_count=0,
-        )
-        self._submitted_findings = []
-        self._hints_used = 0
-        self._created_at = _utc_now()
-        return self._build_observation(
-            feedback="New review task loaded. Submit findings or request a hint.",
-            reward=0.0,
-            done=False,
-        )
-
-    def step(
-        self,
-        action: PythonAction,
-        timeout_s: Optional[float] = None,
-        **kwargs,
-    ) -> PythonObservation:
-        """Process one review action and return updated feedback."""
-
-        del timeout_s, kwargs
-        if self._current_task is None:
-            return self.reset()
-
-        self._state.step_count += 1
-        operation = action.operation
-        feedback = ""
-        reward = 0.0
-        done = False
-
-        if operation == "request_hint":
-            self._hints_used += 1
-            feedback = self._current_task.hint
-            evaluation = self._evaluate(self._submitted_findings, action.patched_code)
-            reward = evaluation.score
-        else:
-            if action.findings:
-                self._submitted_findings.extend(action.findings)
-            evaluation = self._evaluate(self._submitted_findings, action.patched_code)
-            reward = evaluation.score
-            if operation == "finalize":
-                done = True
-                feedback = (
-                    "Review finalized. "
-                    f"Matched {evaluation.matched_findings}/{evaluation.total_findings} "
-                    "reference findings."
-                )
-            else:
-                feedback = (
-                    f"Progress saved. Matched {evaluation.matched_findings}/"
-                    f"{evaluation.total_findings} findings with score {evaluation.score:.2f}."
-                )
-
-        if self._state.step_count >= self._max_steps():
-            done = True
-            if operation != "finalize":
-                feedback = (
-                    f"{feedback} Maximum steps reached."
-                    if feedback
-                    else "Maximum steps reached."
-                )
-
-        return self._build_observation(
-            feedback=feedback,
-            reward=reward,
-            done=done,
-            patched_code=action.patched_code,
-        )
-
-    def _build_observation(
-        self,
-        *,
-        feedback: str,
-        reward: float,
-        done: bool,
-        patched_code: Optional[str] = None,
-    ) -> PythonObservation:
-        assert self._current_task is not None
-        evaluation = self._evaluate(self._submitted_findings, patched_code)
-        attempts_remaining = max(
-            self._max_steps() - self._state.step_count,
-            0,
-        )
-        return PythonObservation(
-            task=self._current_task.descriptor,
-            feedback=feedback,
-            submitted_findings=list(self._submitted_findings),
-            hints_used=self._hints_used,
-            attempts_remaining=attempts_remaining,
-            evaluation=evaluation,
-            score=evaluation.score,
-            review_time_ms=float(self._state.step_count * 125),
-            done=done,
-            reward=reward,
-            metadata={
-                "episode_id": self._state.episode_id,
-                "created_at": self._created_at,
-                "updated_at": _utc_now(),
-            },
-        )
-
-    def _evaluate(
-        self,
-        findings: Iterable[ReviewFinding],
-        patched_code: Optional[str],
-    ) -> TaskEvaluation:
-        assert self._current_task is not None
-
-        references = self._current_task.references
-        matched_reference_ids: List[str] = []
-        matched_weight = 0.0
-        false_positives = 0
-        duplicate_findings = 0
-
-        seen_ids = set()
-        for finding in findings:
-            ref_id = self._match_reference(finding, references)
-            if ref_id is None:
-                false_positives += 1
-                continue
-            if ref_id in seen_ids:
-                duplicate_findings += 1
-                continue
-            seen_ids.add(ref_id)
-            matched_reference_ids.append(ref_id)
-            matched_weight += next(ref.weight for ref in references if ref.rule_id == ref_id)
-
-        total_weight = sum(ref.weight for ref in references) or 1.0
-        weighted_recall = min(matched_weight / total_weight, 1.0)
-
-        patch_score = 0.0
-        if self._current_task.patched_code and patched_code:
-            patch_score = float(
-                _normalize_code(patched_code) == _normalize_code(self._current_task.patched_code)
-            )
-
-        raw_score = (
-            weighted_recall
-            + (self._config.patch_bonus_multiplier * patch_score)
-            - (self._config.false_positive_penalty * false_positives)
-            - (self._config.duplicate_penalty * duplicate_findings)
-            - (self._config.hint_penalty * self._hints_used)
-        )
-        score = max(0.0, min(raw_score, 1.0))
-
-        return TaskEvaluation(
-            matched_reference_ids=matched_reference_ids,
-            matched_findings=len(matched_reference_ids),
-            total_findings=len(references),
-            false_positives=false_positives,
-            duplicate_findings=duplicate_findings,
-            weighted_recall=weighted_recall,
-            patch_score=patch_score,
-            score=score,
-            passed=score >= self._current_task.descriptor.success_threshold,
-        )
-
-    def _match_reference(
-        self,
-        finding: ReviewFinding,
-        references: Iterable[ReferenceFinding],
-    ) -> Optional[str]:
-        finding_rule = _normalize_text(finding.rule_id)
-        finding_title = _normalize_text(finding.title)
-        for reference in references:
-            if finding_rule and finding_rule == _normalize_text(reference.rule_id):
-                return reference.rule_id
-            line_matches = finding.line is not None and finding.line == reference.line
-            category_matches = finding.category == reference.category
-            title_matches = finding_title and (
-                finding_title in _normalize_text(reference.title)
-                or _normalize_text(reference.title) in finding_title
-            )
-            if line_matches and (category_matches or title_matches):
-                return reference.rule_id
-        return None
-
-    def _max_steps(self) -> int:
-        assert self._current_task is not None
-        return min(
-            self._current_task.descriptor.max_steps,
-            self._config.max_steps_per_task,
-        )
-
-    @property
-    def state(self) -> State:
-        """Return the current environment state."""
-
-        return self._state
+"""Compatibility shim for older imports."""
+
+try:
+    from server.code_review_environment import PythonEnvironment
+except ModuleNotFoundError:  # pragma: no cover
+    from .code_review_environment import PythonEnvironment
+
+
+__all__ = ["PythonEnvironment"]

server/requirements.txt

@@ -1,6 +1,6 @@
-openenv[core]>=0.2.0
-fastapi>=0.115.0
-uvicorn>=0.24.0
-
-
-
+openenv-core[core]>=0.2.2
+fastapi>=0.115.0
+uvicorn[standard]>=0.30.0
+openai>=1.40.0
+pytest>=8.0.0
+pydantic>=2.0.0

server/static_review.py

@@ -0,0 +1,273 @@
+"""Deterministic static-review helpers for arbitrary Python code.
+
+Unlike the benchmark grader, this module does not compare against hidden rubric
+items. Instead, it performs direct AST-based review on arbitrary snippets so it
+can be used for manual testing, examples, and future dataset generation.
+"""
+
+from __future__ import annotations
+
+import ast
+from typing import List, Optional
+
+try:
+    from models import DirectReviewResponse, ReviewFinding
+except ModuleNotFoundError:  # pragma: no cover
+    from ..models import DirectReviewResponse, ReviewFinding
+
+
+class _StaticAnalyzer(ast.NodeVisitor):
+    """AST visitor that emits structured review findings.
+
+    The visitor intentionally focuses on a small set of high-signal patterns so
+    the direct-review endpoint stays predictable and easy to understand.
+    """
+
+    def __init__(self) -> None:
+        self.issues: List[ReviewFinding] = []
+
+    def visit_FunctionDef(self, node: ast.FunctionDef) -> None:  # noqa: N802
+        """Flag mutable default arguments in function definitions."""
+
+        for default in list(node.args.defaults):
+            if isinstance(default, (ast.List, ast.Dict, ast.Set)):
+                self.issues.append(
+                    ReviewFinding(
+                        title="Mutable default argument",
+                        line=getattr(default, "lineno", node.lineno),
+                        category="bug",
+                        severity="warning",
+                        rationale=(
+                            "Mutable defaults persist across calls and can leak state "
+                            "between unrelated requests."
+                        ),
+                        recommendation="Use None as the default and create the object inside the function.",
+                        rule_id="mutable-default-list",
+                    )
+                )
+        self.generic_visit(node)
+
+    def visit_Call(self, node: ast.Call) -> None:  # noqa: N802
+        """Inspect function calls for obviously unsafe or noisy patterns."""
+
+        func_name = self._call_name(node)
+        if func_name in {"eval", "exec"}:
+            self.issues.append(
+                ReviewFinding(
+                    title=f"Avoid {func_name} on untrusted input",
+                    line=node.lineno,
+                    category="security",
+                    severity="critical",
+                    rationale=(
+                        f"{func_name} executes arbitrary code and is unsafe on "
+                        "user-controlled input."
+                    ),
+                    recommendation="Use a safe parser or a whitelist-based evaluator.",
+                    rule_id="avoid-eval" if func_name == "eval" else "avoid-exec",
+                )
+            )
+        if func_name.endswith("check_output") or func_name.endswith("run"):
+            for keyword in node.keywords:
+                # `shell=True` is only a problem when the command comes from a
+                # shell-parsed string, but this heuristic is high value for
+                # review and intentionally conservative.
+                if keyword.arg == "shell" and isinstance(keyword.value, ast.Constant) and keyword.value.value is True:
+                    self.issues.append(
+                        ReviewFinding(
+                            title="shell=True with dynamic input",
+                            line=node.lineno,
+                            category="security",
+                            severity="critical",
+                            rationale=(
+                                "shell=True executes through the shell and can allow "
+                                "command injection when the command string is interpolated."
+                            ),
+                            recommendation="Pass a list of arguments and keep shell=False.",
+                            rule_id="shell-true-command-injection",
+                        )
+                    )
+        if func_name == "print":
+            self.issues.append(
+                ReviewFinding(
+                    title="Print statement in application logic",
+                    line=node.lineno,
+                    category="style",
+                    severity="info",
+                    rationale="Production services should prefer structured logging over print statements.",
+                    recommendation="Use the logging module or return the value to the caller.",
+                    rule_id="print-statement",
+                )
+            )
+        self.generic_visit(node)
+
+    def visit_ExceptHandler(self, node: ast.ExceptHandler) -> None:  # noqa: N802
+        """Flag bare exception handlers that hide failures."""
+
+        if node.type is None:
+            self.issues.append(
+                ReviewFinding(
+                    title="Bare except",
+                    line=node.lineno,
+                    category="maintainability",
+                    severity="warning",
+                    rationale="Bare except catches KeyboardInterrupt and other system-level exceptions.",
+                    recommendation="Catch a specific exception and record the failure.",
+                    rule_id="bare-except",
+                )
+            )
+        self.generic_visit(node)
+
+    def visit_For(self, node: ast.For) -> None:  # noqa: N802
+        """Look for list-membership checks nested in loops."""
+
+        for child in ast.walk(node):
+            if isinstance(child, ast.Compare) and any(
+                isinstance(operator, (ast.In, ast.NotIn)) for operator in child.ops
+            ):
+                if isinstance(child.comparators[0], ast.Name):
+                    self.issues.append(
+                        ReviewFinding(
+                            title="Potential quadratic membership check inside loop",
+                            line=child.lineno,
+                            category="performance",
+                            severity="warning",
+                            rationale=(
+                                "Repeated membership checks against a list inside a loop "
+                                "can degrade to quadratic runtime."
+                            ),
+                            recommendation="Use a set or dict for O(1) membership checks.",
+                            rule_id="quadratic-membership-check",
+                        )
+                    )
+                    break
+        self.generic_visit(node)
+
+    @staticmethod
+    def _call_name(node: ast.Call) -> str:
+        """Extract a dotted function name such as `subprocess.run`."""
+
+        func = node.func
+        if isinstance(func, ast.Name):
+            return func.id
+        if isinstance(func, ast.Attribute):
+            prefix = _StaticAnalyzer._attribute_prefix(func.value)
+            return f"{prefix}.{func.attr}" if prefix else func.attr
+        return ""
+
+    @staticmethod
+    def _attribute_prefix(node: ast.AST) -> str:
+        """Reconstruct the left-hand side of an attribute chain."""
+
+        if isinstance(node, ast.Name):
+            return node.id
+        if isinstance(node, ast.Attribute):
+            prefix = _StaticAnalyzer._attribute_prefix(node.value)
+            return f"{prefix}.{node.attr}" if prefix else node.attr
+        return ""
+
+
+def analyze_python_code(code: str) -> List[ReviewFinding]:
+    """Analyze arbitrary Python code and return structured findings."""
+
+    if not code.strip():
+        return [
+            ReviewFinding(
+                title="No code provided",
+                category="bug",
+                severity="warning",
+                rationale="The reviewer cannot inspect an empty submission.",
+                recommendation="Provide Python source code.",
+                rule_id="empty-input",
+            )
+        ]
+
+    # Syntax errors are turned into findings rather than exceptions so API
+    # consumers always get a valid response shape.
+    try:
+        tree = ast.parse(code)
+    except SyntaxError as exc:
+        return [
+            ReviewFinding(
+                title="Syntax error",
+                line=exc.lineno,
+                category="bug",
+                severity="critical",
+                rationale=exc.msg,
+                recommendation="Fix the syntax error before running static review.",
+                rule_id="syntax-error",
+            )
+        ]
+
+    analyzer = _StaticAnalyzer()
+    analyzer.visit(tree)
+    return _deduplicate(analyzer.issues)
+
+
+def build_direct_review_response(
+    code: str, context: Optional[str] = None
+) -> DirectReviewResponse:
+    """Build the public direct-review response for the `/review` route."""
+
+    issues = analyze_python_code(code)
+    weighted_penalty = 0.0
+    # The direct-review score is intentionally simple: more severe issues lower
+    # the score more aggressively.
+    for issue in issues:
+        if issue.severity == "critical":
+            weighted_penalty += 0.3
+        elif issue.severity == "warning":
+            weighted_penalty += 0.15
+        else:
+            weighted_penalty += 0.05
+
+    score = max(0.0, min(1.0, 1.0 - weighted_penalty))
+    summary = _build_summary(issues, context)
+    improved_code = _suggest_improved_code(code, issues)
+    return DirectReviewResponse(
+        issues=issues,
+        summary=summary,
+        score=score,
+        improved_code=improved_code,
+    )
+
+
+def _build_summary(issues: List[ReviewFinding], context: Optional[str]) -> str:
+    """Create a concise human-readable summary for the direct-review response."""
+
+    if not issues:
+        base = "No obvious issues were detected by the deterministic reviewer."
+    else:
+        critical = sum(1 for issue in issues if issue.severity == "critical")
+        warnings = sum(1 for issue in issues if issue.severity == "warning")
+        infos = sum(1 for issue in issues if issue.severity == "info")
+        base = (
+            f"Detected {len(issues)} issue(s): {critical} critical, "
+            f"{warnings} warning, {infos} info."
+        )
+    if context:
+        return f"{base} Context: {context}"
+    return base
+
+
+def _suggest_improved_code(code: str, issues: List[ReviewFinding]) -> Optional[str]:
+    """Append high-level fix directions to the submitted code."""
+
+    if not issues:
+        return None
+    suggestions = [issue.recommendation for issue in issues if issue.recommendation]
+    comment = " | ".join(dict.fromkeys(suggestions))
+    return f"{code.rstrip()}\n\n# Suggested review directions: {comment}"
+
+
+def _deduplicate(findings: List[ReviewFinding]) -> List[ReviewFinding]:
+    """Drop duplicate findings that refer to the same rule and line."""
+
+    seen = set()
+    unique: List[ReviewFinding] = []
+    for finding in findings:
+        key = (finding.rule_id, finding.line, finding.category)
+        if key in seen:
+            continue
+        seen.add(key)
+        unique.append(finding)
+    return unique

server/task_bank.py

@@ -0,0 +1,340 @@
+"""Static PR-review tasks and hidden grading rubrics."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Dict, Iterable, List, Sequence
+
+try:
+    from models import Category, Difficulty, Severity, TaskDescriptor, TaskSummary
+except ModuleNotFoundError:  # pragma: no cover
+    from ..models import Category, Difficulty, Severity, TaskDescriptor, TaskSummary
+
+
+@dataclass(frozen=True)
+class RubricIssue:
+    """One hidden issue that can be matched by the deterministic grader."""
+
+    issue_id: str
+    file_path: str
+    line: int
+    category: Category
+    severity: Severity
+    keywords: Sequence[str]
+    min_keyword_hits: int
+    weight: float
+
+
+@dataclass(frozen=True)
+class TaskSpec:
+    """Complete task definition, including hidden rubric metadata."""
+
+    task_id: str
+    difficulty: Difficulty
+    title: str
+    goal: str
+    repo_summary: str
+    visible_diff: str
+    file_contents: Dict[str, str]
+    changed_files: Sequence[str]
+    rubric_issues: Sequence[RubricIssue]
+    max_steps: int
+
+    @property
+    def available_files(self) -> List[str]:
+        return list(self.file_contents.keys())
+
+    def to_descriptor(self) -> TaskDescriptor:
+        return TaskDescriptor(
+            task_id=self.task_id,
+            difficulty=self.difficulty,
+            title=self.title,
+            goal=self.goal,
+            repo_summary=self.repo_summary,
+            changed_files=list(self.changed_files),
+            available_files=self.available_files,
+            max_steps=self.max_steps,
+        )
+
+    def to_summary(self) -> TaskSummary:
+        return TaskSummary(
+            task_id=self.task_id,
+            difficulty=self.difficulty,
+            title=self.title,
+            goal=self.goal,
+        )
+
+
+TASKS: List[TaskSpec] = [
+    TaskSpec(
+        task_id="py-pr-review-easy",
+        difficulty="easy",
+        title="Retry Delay Regression",
+        goal=(
+            "Review the pull request and identify the real bug introduced in the retry "
+            "delay helper before it ships."
+        ),
+        repo_summary=(
+            "This service computes retry delays for background notification delivery. "
+            "The change is intended to relax validation for legacy callers."
+        ),
+        visible_diff="\n".join(
+            [
+                "diff --git a/src/notifications/retry.py b/src/notifications/retry.py",
+                "@@",
+                "-    if base_delay <= 0:",
+                "+    if base_delay < 0:",
+                "         return 0.0",
+            ]
+        ),
+        file_contents={
+            "src/notifications/retry.py": "\n".join(
+                [
+                    "from __future__ import annotations",
+                    "",
+                    "def calculate_retry_delay(attempt: int, base_delay: float = 2.0) -> float:",
+                    '    """Return the retry delay in seconds."""',
+                    "    if attempt < 0:",
+                    '        raise ValueError(\"attempt must be >= 0\")',
+                    "    if base_delay < 0:",
+                    "        return 0.0",
+                    "    return attempt / base_delay",
+                ]
+            )
+        },
+        changed_files=("src/notifications/retry.py",),
+        rubric_issues=(
+            RubricIssue(
+                issue_id="zero-base-delay-divides",
+                file_path="src/notifications/retry.py",
+                line=7,
+                category="bug",
+                severity="warning",
+                keywords=("zero", "division", "base_delay"),
+                min_keyword_hits=2,
+                weight=1.0,
+            ),
+        ),
+        max_steps=4,
+    ),
+    TaskSpec(
+        task_id="py-pr-review-medium",
+        difficulty="medium",
+        title="Coupon Billing Rollout",
+        goal=(
+            "Review the billing change and identify both the production regression and "
+            "the missing coverage that would have caught it."
+        ),
+        repo_summary=(
+            "The billing service is adding coupon support for one-off invoices. The PR "
+            "touches both the service code and its unit tests."
+        ),
+        visible_diff="\n".join(
+            [
+                "diff --git a/app/billing/invoice_service.py b/app/billing/invoice_service.py",
+                "@@",
+                " def charge_invoice(order: dict, gateway: Gateway) -> str:",
+                "-    return gateway.charge(order[\"customer_id\"], order[\"amount_cents\"])",
+                "+    total = order[\"amount_cents\"]",
+                "+    coupon = order.get(\"coupon_code\")",
+                "+    if coupon:",
+                "+        discount = gateway.lookup_discount(coupon)",
+                "+        total = max(total - discount, 0)",
+                "+    return gateway.charge(order[\"customer_id\"], order[\"amount_cents\"])",
+                "",
+                "diff --git a/tests/test_invoice_service.py b/tests/test_invoice_service.py",
+                "@@",
+                " class FakeGateway:",
+                "+    def lookup_discount(self, coupon: str) -> int:",
+                "+        return 250",
+            ]
+        ),
+        file_contents={
+            "app/billing/invoice_service.py": "\n".join(
+                [
+                    "from gateway import Gateway",
+                    "",
+                    "def charge_invoice(order: dict, gateway: Gateway) -> str:",
+                    '    total = order["amount_cents"]',
+                    '    coupon = order.get("coupon_code")',
+                    "    if coupon:",
+                    "        discount = gateway.lookup_discount(coupon)",
+                    "        total = max(total - discount, 0)",
+                    '    return gateway.charge(order["customer_id"], order["amount_cents"])',
+                ]
+            ),
+            "tests/test_invoice_service.py": "\n".join(
+                [
+                    "from app.billing.invoice_service import charge_invoice",
+                    "",
+                    "class FakeGateway:",
+                    "    def lookup_discount(self, coupon: str) -> int:",
+                    "        return 250",
+                    "",
+                    "    def charge(self, customer_id: str, amount_cents: int) -> str:",
+                    "        self.last_charge = (customer_id, amount_cents)",
+                    '        return "charge_123"',
+                    "",
+                    "def test_charge_invoice_without_coupon():",
+                    "    gateway = FakeGateway()",
+                    '    charge_invoice({"customer_id": "cus_1", "amount_cents": 1000}, gateway)',
+                    '    assert gateway.last_charge == ("cus_1", 1000)',
+                ]
+            ),
+        },
+        changed_files=("app/billing/invoice_service.py", "tests/test_invoice_service.py"),
+        rubric_issues=(
+            RubricIssue(
+                issue_id="discount-total-unused",
+                file_path="app/billing/invoice_service.py",
+                line=8,
+                category="bug",
+                severity="warning",
+                keywords=("discount", "total", "charge", "amount"),
+                min_keyword_hits=2,
+                weight=0.6,
+            ),
+            RubricIssue(
+                issue_id="missing-coupon-test",
+                file_path="tests/test_invoice_service.py",
+                line=11,
+                category="testing",
+                severity="warning",
+                keywords=("missing", "test", "coupon", "discount"),
+                min_keyword_hits=2,
+                weight=0.4,
+            ),
+        ),
+        max_steps=5,
+    ),
+    TaskSpec(
+        task_id="py-pr-review-hard",
+        difficulty="hard",
+        title="Async Job Runner Deduplication",
+        goal=(
+            "Review the async job-runner PR and find the subtle concurrency issues "
+            "without inventing extra problems."
+        ),
+        repo_summary=(
+            "A shared webhook backfill service is deduplicating in-flight work with an "
+            "async task cache and writing the latest result for operators to inspect."
+        ),
+        visible_diff="\n".join(
+            [
+                "diff --git a/app/jobs/runner.py b/app/jobs/runner.py",
+                "@@",
+                " async def run_job(job_id: str, payload: dict, worker) -> str:",
+                "     if job_id in ACTIVE_RUNS:",
+                "         return await ACTIVE_RUNS[job_id]",
+                "+    lock = asyncio.Lock()",
+                "+    async with lock:",
+                "+        task = asyncio.create_task(worker.run(payload))",
+                "+        ACTIVE_RUNS[job_id] = task",
+                "     try:",
+                "         result = await task",
+                "     finally:",
+                "         ACTIVE_RUNS.pop(job_id, None)",
+                "+    Path(\"latest-result.json\").write_text(result)",
+                "     return result",
+            ]
+        ),
+        file_contents={
+            "app/jobs/runner.py": "\n".join(
+                [
+                    "import asyncio",
+                    "from pathlib import Path",
+                    "",
+                    "ACTIVE_RUNS: dict[str, asyncio.Task[str]] = {}",
+                    "",
+                    "async def run_job(job_id: str, payload: dict, worker) -> str:",
+                    "    if job_id in ACTIVE_RUNS:",
+                    "        return await ACTIVE_RUNS[job_id]",
+                    "",
+                    "    lock = asyncio.Lock()",
+                    "    async with lock:",
+                    "        task = asyncio.create_task(worker.run(payload))",
+                    "        ACTIVE_RUNS[job_id] = task",
+                    "    try:",
+                    "        result = await task",
+                    "    finally:",
+                    "        ACTIVE_RUNS.pop(job_id, None)",
+                    "",
+                    '    Path("latest-result.json").write_text(result)',
+                    "    return result",
+                ]
+            ),
+            "tests/test_runner.py": "\n".join(
+                [
+                    "import pytest",
+                    "",
+                    "from app.jobs.runner import run_job",
+                    "",
+                    "class FakeWorker:",
+                    "    async def run(self, payload: dict) -> str:",
+                    '        return payload["job_id"]',
+                    "",
+                    "@pytest.mark.asyncio",
+                    "async def test_run_job_returns_worker_result():",
+                    "    worker = FakeWorker()",
+                    '    result = await run_job("job-1", {"job_id": "job-1"}, worker)',
+                    '    assert result == "job-1"',
+                ]
+            ),
+        },
+        changed_files=("app/jobs/runner.py", "tests/test_runner.py"),
+        rubric_issues=(
+            RubricIssue(
+                issue_id="per-call-lock-race",
+                file_path="app/jobs/runner.py",
+                line=9,
+                category="bug",
+                severity="warning",
+                keywords=("lock", "race", "concurrent", "duplicate"),
+                min_keyword_hits=2,
+                weight=0.55,
+            ),
+            RubricIssue(
+                issue_id="shared-output-file-race",
+                file_path="app/jobs/runner.py",
+                line=18,
+                category="maintainability",
+                severity="warning",
+                keywords=("latest", "result", "file", "concurrent", "overwrite"),
+                min_keyword_hits=2,
+                weight=0.45,
+            ),
+        ),
+        max_steps=6,
+    ),
+]
+
+
+TASKS_BY_ID: Dict[str, TaskSpec] = {task.task_id: task for task in TASKS}
+
+
+def list_task_descriptors() -> List[TaskDescriptor]:
+    """Return public descriptors for all tasks."""
+
+    return [task.to_descriptor() for task in TASKS]
+
+
+def list_task_summaries() -> List[TaskSummary]:
+    """Return task summaries for lightweight route responses."""
+
+    return [task.to_summary() for task in TASKS]
+
+
+def get_task(task_id: str) -> TaskSpec:
+    """Return a task by id."""
+
+    try:
+        return TASKS_BY_ID[task_id]
+    except KeyError as exc:  # pragma: no cover
+        raise ValueError(f"Unknown task_id: {task_id}") from exc
+
+
+def task_ids() -> Iterable[str]:
+    """Return task ids in benchmark order."""
+
+    return [task.task_id for task in TASKS]
+

summary/01_introduction_quickstart.md

@@ -0,0 +1,66 @@
+# 01. Introduction & Quick Start
+
+Source:
+- https://meta-pytorch.org/OpenEnv/auto_getting_started/plot_01_introduction_quickstart.html
+
+## Main idea
+
+OpenEnv is a standardized framework for building, sharing, and using RL environments as typed, containerized services.
+
+The official docs frame it as:
+
+- Gym-style interaction
+- Docker-based isolation
+- typed contracts
+- HTTP/WebSocket access
+- easy sharing through Hugging Face
+
+## Core loop
+
+The RL interaction model is still the normal loop:
+
+1. reset environment
+2. observe state
+3. choose action
+4. call step
+5. receive reward + next observation
+6. repeat until done
+
+The difference is that OpenEnv wraps this loop in a typed client/server system.
+
+## Why OpenEnv instead of only Gym
+
+The docs emphasize these advantages:
+
+- type safety
+- environment isolation through containers
+- better reproducibility
+- easier sharing and deployment
+- language-agnostic communication
+- cleaner debugging
+
+The key contrast is:
+
+- old style: raw arrays and same-process execution
+- OpenEnv style: typed objects and isolated environment runtime
+
+## Important mental model
+
+OpenEnv treats environments more like services than in-process libraries.
+
+That means:
+
+- your environment logic can run separately from the agent code
+- failures in the environment do not automatically crash the training loop
+- deployment and usage are closer to how production systems work
+
+## What this means for `python_env`
+
+Your repo should keep these properties intact:
+
+- typed `Action`, `Observation`, and evaluation models
+- a clean environment class with `reset()`, `step()`, and `state`
+- a client that hides transport details
+- a deployable container
+
+For hackathon purposes, this page is the justification for why your project is not just a script. It is a reusable environment artifact.

summary/02_using_environments.md

@@ -0,0 +1,98 @@
+# 02. Using Environments
+
+Source:
+- https://meta-pytorch.org/OpenEnv/auto_getting_started/plot_02_using_environments.html
+
+## Main idea
+
+This page is about how users consume an existing OpenEnv environment.
+
+The docs highlight three connection methods:
+
+1. from Hugging Face Hub
+2. from Docker image
+3. from direct base URL
+
+## Connection methods
+
+### 1. From Hugging Face Hub
+
+The easiest route for end users.
+
+Typical flow:
+
+- pull the image from the HF registry
+- start the container locally
+- connect to it
+- clean it up on close
+
+The docs show the pattern conceptually as:
+
+```python
+MyEnv.from_hub("owner/env-name")
+```
+
+## 2. From Docker image
+
+Useful when:
+
+- you already built the image locally
+- you want reproducible local runs
+- you do not want to depend on a live remote Space
+
+Typical pattern:
+
+```python
+MyEnv.from_docker_image("my-env:latest")
+```
+
+## 3. Direct URL connection
+
+Useful when:
+
+- the server is already running
+- you want to connect to localhost or a deployed Space
+
+Typical pattern:
+
+```python
+MyEnv(base_url="http://localhost:8000")
+```
+
+## WebSocket model
+
+The docs emphasize that OpenEnv uses WebSocket-backed sessions for persistent environment interaction.
+
+Why this matters:
+
+- lower overhead than stateless HTTP on every step
+- cleaner session management
+- better fit for multi-step RL loops
+
+## Environment loop
+
+The intended use pattern is:
+
+1. connect
+2. reset
+3. repeatedly call `step(action)`
+4. inspect `reward`, `done`, and `observation`
+5. close cleanly
+
+## What this means for `python_env`
+
+Your environment should be easy to consume in all three modes:
+
+- local URL
+- local Docker image
+- HF Space
+
+That means the most important user-facing checks are:
+
+- `reset()` works
+- `step()` works
+- the client can parse the observation correctly
+- Docker image starts cleanly
+- deployed Space responds on `/health`, `/docs`, and session routes
+
+For hackathon validation, this page is basically the “user experience” standard you need to match.