README.md

metadata

title: SQL Debugger Environment
emoji: 🔧
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
app_port: 8000
license: mit
short_description: SQL Query Debugger - OpenEnv RL Environment
tags:
  - openenv

SQL Debugger — OpenEnv Environment

An RL environment where an AI agent receives broken SQL queries and attempts to fix them, scored with partial credit.

Environment Description

The SQL Debugger simulates a real-world task: debugging broken SQL queries. Given a buggy query, database schema, and expected output, the agent must produce a corrected query. This models a genuine developer workflow — reading error messages, understanding schema, and iteratively fixing queries.

Action Space

@dataclass
class SQLAction(Action):
    corrected_query: str  # The agent's fixed SQL query

Observation Space

@dataclass
class SQLObservation(Observation):
    broken_query: str           # The buggy SQL to fix
    schema: list                # CREATE TABLE statements
    description: str            # Hint about what's wrong
    expected_output: list       # What the correct query should return
    actual_output: list | None  # What the agent's query returned
    score: float                # Partial credit score (0.0 - 1.0)
    error_message: str | None   # Error if the agent's SQL crashed
    done: bool                  # Whether the episode is over
    attempts_left: int          # Remaining attempts

Tasks

9 tasks across 3 difficulty levels:

Difficulty	Count	Bug Type	Example
Easy	3	Syntax errors	Typos: SELCT, FORME, ORDRE
Medium	3	Logic errors	Wrong JOIN type, wrong WHERE filter, wrong GROUP BY
Hard	3	Subtle multi-table bugs	Wrong threshold, missing DISTINCT, global vs per-category average

Reward Function

Partial credit scoring with 5 levels:

Score	Criteria
0.0	Query doesn't parse
0.2	Parses (EXPLAIN works)
0.4	Runs without error
0.6	Correct number of columns
0.8	Correct number of rows
1.0	Exact match with expected output

Setup

# Install dependencies
pip install -r requirements.txt

# Run the environment server locally
uvicorn server.app:app --host 0.0.0.0 --port 8000

# Run inference (requires LLM API access)
export HF_TOKEN=your_token_here
python inference.py

Docker

docker build -t sql-debugger .
docker run -p 8000:8000 sql-debugger

Baseline Scores

Baseline agent (Qwen2.5-72B-Instruct) with 3 attempts per task. Scores vary per run due to LLM non-determinism.