add files

Dockerfile

@@ -0,0 +1,12 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+COPY requirements-server.txt .
+RUN pip install --no-cache-dir -r requirements-server.txt
+
+COPY models.py tasks.py rewards.py environment.py main.py openenv.yaml ./
+
+EXPOSE 8000
+
+CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

README.md

@@ -0,0 +1,163 @@
+# IndicScriptureQA — OpenEnv Environment
+
+**Semantic structure and factual grounding evaluation for low-resource Indic languages.**
+
+Most LLM benchmarks for Hindi, Sanskrit, and other Indic languages test surface-level factual recall — did the model get the right answer? This environment goes further. It evaluates whether an agent can produce answers that are not only **factually correct** but also **semantically well-formed**: logically ordered, terminologically precise, structurally complete, and coherently written — the qualities that separate a genuinely useful answer from one that merely contains the right words in the wrong shape.
+
+The domain is Indic scriptural knowledge (Vedas, Upanishads, Ramayana, Mahabharata, Bhagavad Gita, Puranas), chosen because it stresses every axis at once: factual precision matters (misattributing a verse to the wrong text is a hallucination), but so does structural literacy — knowing that an explanation of Rta should distinguish its natural-law and moral-law dimensions, that the Samudra Manthan narrative has a specific dramatic arc, or that "nishkama karma" is the correct term, not the English gloss "selfless action."
+
+## The problem with low-resource language evaluation
+
+LLMs fail on low-resource languages in ways that pure accuracy metrics miss:
+
+**Terminology collapse.** Models substitute English glosses for domain-specific terms — writing "cosmic order" instead of "Rta", "meditation" instead of "dhyana", "duty" instead of "svadharma." This strips cultural and semantic precision even when the underlying fact is technically correct.
+
+**Structural incoherence.** Answers about complex topics arrive as bags of loosely related facts instead of logically sequenced arguments. An explanation of the six Darshanas that jumbles founders with commentators, or a Dashavatara account that breaks chronological ordering, fails structurally even if every individual claim is true.
+
+**Completeness gaps.** Models cover one dimension of a multi-faceted concept and call it done — describing dharma only as "duty" without addressing its subtlety (sukshma), context-dependence, or the rajadharma/apaddharma/moksha-dharma triad that the Mahabharata actually teaches.
+
+**Misconception propagation.** Some errors are so common in training data that models reproduce them confidently — Shankaracharya "founding" Vedanta (he was a commentator, not the founder), or Indra "maintaining" Rta (that's Varuna). These need active detection and penalisation, not just factual comparison.
+
+This environment provides a structured RL benchmark for training and evaluating agents that address **all four failure modes simultaneously**.
+
+---
+
+## How it works
+
+An agent receives a question and a pre-generated answer that may be flawed along any combination of axes — factual errors, poor structure, missing terminology, wrong ordering, incomplete coverage. The agent interacts with the environment through a fixed action space to iteratively improve the answer before finalising it.
+
+## Action Space
+
+| Action        | Payload                                   | Effect                                                |
+|---------------|-------------------------------------------|-------------------------------------------------------|
+| `RETRIEVE`    | Optional query string                     | Surfaces the next available source passage             |
+| `EDIT`        | New answer text                           | Rewrites to fix factual errors and improve content     |
+| `RESTRUCTURE` | Reorganised answer text                   | Reorganises flow, ordering, and terminology without changing facts |
+| `CITE`        | Citation string (e.g. `"Bhagavad Gita 2.47"`) | Attaches a citation                               |
+| `ACCEPT`      | —                                         | Accepts answer as final (terminal)                     |
+| `REJECT`      | —                                         | Rejects the answer entirely (terminal)                 |
+
+The distinction between `EDIT` and `RESTRUCTURE` is deliberate. EDIT changes what the answer says. RESTRUCTURE changes how it says it — reordering paragraphs, inserting transitions, swapping an English gloss for the correct Sanskrit term, expanding a single sentence into the three conceptual aspects the topic requires. The grader scores them differently: RESTRUCTURE is penalised if it destroys factual content, and EDIT is measured on both factual and structural improvement.
+
+## Observation Space
+
+| Field                | Type         | Description                                                 |
+|----------------------|--------------|-------------------------------------------------------------|
+| `question`           | `str`        | The question being answered                                  |
+| `current_answer`     | `str`        | Current (possibly flawed) answer                             |
+| `retrieved_passages` | `list[str]`  | Source passages retrieved so far                              |
+| `current_citations`  | `list[str]`  | Citations attached so far                                     |
+| `steps_remaining`    | `int`        | Steps left in the episode                                     |
+| `task_name`          | `str`        | Active task identifier                                        |
+| `feedback`           | `str?`       | Feedback from the last action (includes structural breakdown) |
+| `structural_hints`   | `list[str]`  | Non-spoiler hints about expected answer structure             |
+
+`structural_hints` are the agent's window into what the grader expects structurally — things like "Use the Sanskrit term for selfless action", "Cover scriptural, ritual, AND mathematical dimensions", or "Follow narrative arc: setup → churning → crisis → treasures → resolution." They don't reveal the answer but guide the agent toward well-formed output.
+
+## Tasks
+
+| Task                  | Difficulty | Max Steps | Focus |
+|-----------------------|------------|-----------|-------|
+| `verify-factual`      | Easy       | 5         | Can the agent distinguish a correct answer from a wrong one, accounting for both factual accuracy and structural adequacy? |
+| `correct-and-cite`    | Medium     | 8         | Given a partially correct answer with missing citations and poor structure, can the agent retrieve sources, fix gaps, add terminology, and reorganise? |
+| `fix-hallucination`   | Hard       | 12        | Can the agent detect subtle hallucinations woven into plausible text while simultaneously fixing structural problems: wrong concept ordering, banned misconception terms, incomplete aspect coverage? |
+
+Each task has 5 scenarios covering the Vedas, Upanishads, Ramayana, Mahabharata, Bhagavad Gita, and Puranas. Every scenario carries both factual ground truth and a `StructuralMeta` specification defining required terms, required sections, expected ordering, and banned misconception markers.
+
+## Reward Structure
+
+The final score blends **factual quality** and **structural quality** into **[0.0, 1.0]**.
+
+### Terminal reward (on ACCEPT)
+
+| Component           | Max   | What it measures                                     |
+|---------------------|-------|------------------------------------------------------|
+| Factual similarity  | 0.90  | Token-F1 between final answer and ground truth       |
+| Citation recall     | 0.30  | Fraction of expected citations matched               |
+| Structural quality  | 0.70  | Composite of 4 axes (see below)                      |
+| Efficiency bonus    | 0.20  | Reward for finishing in fewer steps                   |
+
+### Structural quality composite (0.70 max)
+
+| Axis             | Weight | What it catches                                                |
+|------------------|--------|----------------------------------------------------------------|
+| **Terminology**  | 0.30   | Are the correct Sanskrit/domain terms present? Are banned misconception markers absent? |
+| **Completeness** | 0.25   | Does the answer cover all required conceptual aspects of the topic? |
+| **Ordering**     | 0.25   | Do concepts appear in the expected logical/narrative sequence?  |
+| **Coherence**    | 0.20   | Transition quality, sentence-structure uniformity, multi-sentence flow |
+
+All four axes are computed without ML dependencies — token matching, keyword heuristics, positional analysis, and discourse marker detection — so the environment runs on minimal hardware (2 vCPU, 8 GB RAM).
+
+### Per-step shaping
+
+| Action       | Good outcome          | Bad outcome                    |
+|--------------|-----------------------|--------------------------------|
+| `RETRIEVE`   | +0.05 (useful)        | −0.15 (redundant, >3 times)    |
+| `EDIT`       | +0.20 + quality delta | −0.20 (degradation)            |
+| `RESTRUCTURE`| +0.25 + struct delta  | −0.25 (destroyed facts)        |
+| `CITE`       | +0.15 (correct)       | −0.05 (wrong)                  |
+
+Step-level rewards blend factual and structural deltas (60/40 for EDIT, structure-dominant for RESTRUCTURE), giving the agent continuous signal throughout the episode rather than only at termination.
+
+---
+
+## Setup
+
+### Server (Docker)
+
+```bash
+docker build -t indic-scripture-qa .
+docker run -p 8000:8000 indic-scripture-qa
+```
+
+Verify: `curl -X POST http://localhost:8000/reset -H 'Content-Type: application/json' -d '{}'`
+
+### Inference
+
+```bash
+pip install -r requirements.txt
+
+export API_BASE_URL="https://router.huggingface.co/v1"
+export MODEL_NAME="Qwen/Qwen2.5-72B-Instruct"
+export HF_TOKEN="your-token"
+export PING_URL="http://localhost:8000"
+
+python inference.py
+```
+
+### Validate
+
+```bash
+pip install openenv-core
+openenv validate
+```
+
+## Baseline Scores
+
+| Task                 | Score  |
+|----------------------|--------|
+| `verify-factual`     | ~0.40  |
+| `correct-and-cite`   | ~0.30  |
+| `fix-hallucination`  | ~0.22  |
+| **Average**          | ~0.31  |
+
+*(Qwen2.5-72B-Instruct, temperature=0.4, scenario 0, structural eval enabled)*
+
+## Project Structure
+
+```
+├── openenv.yaml            # OpenEnv metadata
+├── Dockerfile              # Server container
+├── main.py                 # FastAPI server (reset/step/state)
+├── environment.py          # Core env logic
+├── models.py               # Typed Pydantic models + StructuralMeta
+├── tasks.py                # Task definitions, scenarios, structural metadata
+├── rewards.py              # Factual + structural reward computation
+├── inference.py            # Baseline inference script
+├── requirements.txt        # Client deps
+└── requirements-server.txt # Server deps
+```
+
+## License
+
+MIT

environment.py

@@ -0,0 +1,158 @@
+"""
+Core environment logic for IndicScriptureQA.
+
+Implements the OpenEnv interface:
+  reset(task_name, scenario_index) → StepResult
+  step(action)                     → StepResult
+  state()                          → EnvState
+"""
+
+from __future__ import annotations
+
+import random
+from typing import Optional
+
+from models import Action, ActionType, EnvState, Observation, StepResult, StructuralMeta
+from rewards import normalize_score, step_reward, terminal_reward
+from tasks import TASKS, Scenario, TaskConfig
+
+
+class IndicScriptureQAEnv:
+    """Stateful environment — one instance per episode."""
+
+    def __init__(self) -> None:
+        self._state: Optional[EnvState] = None
+
+    # ── reset ─────────────────────────────────────────────────────────────
+
+    def reset(
+        self,
+        task_name: str = "verify-factual",
+        scenario_index: Optional[int] = None,
+    ) -> StepResult:
+        if task_name not in TASKS:
+            raise ValueError(f"Unknown task {task_name!r}. Choose from {list(TASKS)}")
+
+        cfg: TaskConfig = TASKS[task_name]
+        if scenario_index is not None:
+            idx = scenario_index % len(cfg.scenarios)
+        else:
+            idx = random.randint(0, len(cfg.scenarios) - 1)
+
+        sc: Scenario = cfg.scenarios[idx]
+
+        self._state = EnvState(
+            question=sc.question,
+            current_answer=sc.given_answer,
+            original_answer=sc.given_answer,
+            ground_truth_answer=sc.ground_truth_answer,
+            ground_truth_citations=list(sc.ground_truth_citations),
+            available_passages=list(sc.available_passages),
+            answer_is_correct=sc.answer_is_correct,
+            factual_is_correct=sc.factual_is_correct,
+            structural_meta=sc.structural_meta,
+            structural_hints=list(sc.structural_hints),
+            task_name=task_name,
+            max_steps=cfg.max_steps,
+            steps_remaining=cfg.max_steps,
+            step_count=0,
+            done=False,
+            cumulative_reward=0.0,
+            rewards=[],
+            retrieval_count=0,
+            edit_count=0,
+            restructure_count=0,
+            feedback="Episode started. Examine the answer for factual accuracy AND semantic structure.",
+        )
+        return StepResult(observation=self._state.to_observation(), reward=0.0, done=False)
+
+    # ── step ──────────────────────────────────────────────────────────────
+
+    def step(self, action: Action) -> StepResult:
+        s = self._state
+        if s is None:
+            raise RuntimeError("Call reset() before step().")
+        if s.done:
+            raise RuntimeError("Episode already finished. Call reset().")
+
+        s.step_count += 1
+        s.steps_remaining -= 1
+        act = action.action_type
+        payload = (action.payload or "").strip()
+
+        reward = 0.0
+        feedback = ""
+        done = False
+
+        # ── action dispatch ───────────────────────────────────────────────
+        if act == ActionType.RETRIEVE:
+            s.retrieval_count += 1
+            if s.available_passages:
+                idx = (s.retrieval_count - 1) % len(s.available_passages)
+                passage = s.available_passages[idx]
+                if passage not in s.retrieved_passages:
+                    s.retrieved_passages.append(passage)
+            reward, feedback = step_reward(s, act, payload)
+
+        elif act == ActionType.EDIT:
+            s.edit_count += 1
+            reward, feedback = step_reward(s, act, payload)
+            if payload:
+                s.current_answer = payload
+
+        elif act == ActionType.RESTRUCTURE:
+            s.restructure_count += 1
+            reward, feedback = step_reward(s, act, payload)
+            if payload:
+                s.current_answer = payload
+
+        elif act == ActionType.CITE:
+            if payload and payload not in s.current_citations:
+                s.current_citations.append(payload)
+            reward, feedback = step_reward(s, act, payload)
+
+        elif act == ActionType.ACCEPT:
+            t_reward, feedback = terminal_reward(s, act)
+            reward = t_reward
+            done = True
+
+        elif act == ActionType.REJECT:
+            t_reward, feedback = terminal_reward(s, act)
+            reward = t_reward
+            done = True
+
+        else:
+            reward = -0.10
+            feedback = f"Unknown action type: {act}"
+
+        # ── check step limit ──────────────────────────────────────────────
+        if not done and s.steps_remaining <= 0:
+            t_reward, t_fb = terminal_reward(s, ActionType.ACCEPT)
+            reward += t_reward - 0.20
+            feedback += f" | Forced termination (step limit). {t_fb}"
+            done = True
+
+        # ── bookkeeping ──────────────────────────────────────────────────
+        s.rewards.append(reward)
+        s.cumulative_reward += reward
+        s.done = done
+        s.feedback = feedback
+
+        info = {}
+        if done:
+            info["score"] = normalize_score(s.cumulative_reward)
+            info["cumulative_reward"] = s.cumulative_reward
+
+        return StepResult(
+            observation=s.to_observation(),
+            reward=reward,
+            done=done,
+            info=info,
+        )
+
+    # ── state ─────────────────────────────────────────────────────────────
+
+    def state(self) -> EnvState:
+        if self._state is None:
+            raise RuntimeError("Call reset() first.")
+        return self._state.model_copy(deep=True)

inference.py

@@ -0,0 +1,242 @@
+"""
+Baseline inference script for IndicScriptureQA.
+
+Runs an LLM agent against all 3 tasks via the OpenEnv HTTP API.
+Emits structured [START]/[STEP]/[END] logs per the OpenEnv spec.
+
+The agent evaluates BOTH factual accuracy AND semantic structure:
+  - factual: hallucination detection, correction
+  - structural: coherence, completeness, terminology, logical ordering
+
+Environment variables:
+  API_BASE_URL   LLM endpoint         (default: HF router)
+  MODEL_NAME     Model identifier      (default: Qwen2.5-72B-Instruct)
+  HF_TOKEN       API key
+  PING_URL       Environment server    (default: http://localhost:8000)
+"""
+
+import json
+import os
+import textwrap
+from typing import Dict, List, Optional
+
+import requests
+from openai import OpenAI
+
+# ── Config ────────────────────────────────────────────────────────────────────
+
+API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")
+API_BASE_URL = os.getenv("API_BASE_URL", "https://router.huggingface.co/v1")
+MODEL_NAME = os.getenv("MODEL_NAME", "Qwen/Qwen2.5-72B-Instruct")
+ENV_URL = os.getenv("PING_URL", "http://localhost:8000")
+BENCHMARK = "indic_scripture_qa"
+TEMPERATURE = 0.4
+MAX_TOKENS = 600
+
+TASKS = [
+    {"name": "verify-factual",    "max_steps": 5},
+    {"name": "correct-and-cite",  "max_steps": 8},
+    {"name": "fix-hallucination", "max_steps": 12},
+]
+
+SYSTEM_PROMPT = textwrap.dedent("""\
+You are an expert agent that both CORRECTS hallucinations and IMPROVES the
+semantic structure of answers about Indic scriptures (Vedas, Upanishads,
+Ramayana, Mahabharata, Bhagavad Gita, Puranas).
+
+Each turn you receive an observation with:
+  - question, current_answer, retrieved_passages, current_citations,
+    steps_remaining, feedback, structural_hints
+
+You must reply with EXACTLY ONE JSON object (no markdown, no explanation):
+{
+  "action_type": "RETRIEVE" | "EDIT" | "RESTRUCTURE" | "CITE" | "ACCEPT" | "REJECT",
+  "payload": "<string or null>"
+}
+
+Actions:
+  RETRIEVE    — fetch source passages to verify facts
+  EDIT        — rewrite the answer to fix factual errors AND improve content
+  RESTRUCTURE — reorganise the answer's flow, ordering, and coherence WITHOUT
+                changing facts (use when facts are right but structure is poor)
+  CITE        — add a scripture citation (e.g. "Bhagavad Gita 2.47")
+  ACCEPT      — finalise when answer is both accurate and well-structured
+  REJECT      — only if the answer is fundamentally unsalvageable
+
+Strategy:
+  1. RETRIEVE first (1–2 times) to get authoritative source passages.
+  2. Check facts against retrieved passages. EDIT to fix any errors.
+  3. Read structural_hints. If the answer's flow, terminology, or completeness
+     is poor, use RESTRUCTURE to reorganise it.
+  4. CITE relevant scripture references.
+  5. ACCEPT when the answer is factually accurate, well-structured, uses
+     correct Sanskrit terminology, and covers all required aspects.
+  6. Be efficient — fewer steps score higher.
+
+Evaluation axes (the grader checks ALL of these):
+  - Factual similarity to ground truth
+  - Citation accuracy
+  - Terminology precision (correct Sanskrit/domain terms, no misconception markers)
+  - Completeness (all required conceptual aspects covered)
+  - Logical ordering (concepts in proper sequence)
+  - Coherence (smooth transitions, balanced sentence structure)
+""")
+
+
+# ── Logging helpers ───────────────────────────────────────────────────────────
+
+def log_start(task: str, env: str, model: str) -> None:
+    print(f"[START] task={task} env={env} model={model}", flush=True)
+
+
+def log_step(step: int, action: str, reward: float, done: bool, error: Optional[str]) -> None:
+    print(
+        f"[STEP] step={step} action={action} reward={reward:.2f} "
+        f"done={str(done).lower()} error={error or 'null'}",
+        flush=True,
+    )
+
+
+def log_end(success: bool, steps: int, score: float, rewards: List[float]) -> None:
+    rewards_str = ",".join(f"{r:.2f}" for r in rewards)
+    print(
+        f"[END] success={str(success).lower()} steps={steps} "
+        f"score={score:.2f} rewards={rewards_str}",
+        flush=True,
+    )
+
+
+# ── Env HTTP helpers ──────────────────────────────────────────────────────────
+
+def env_reset(task_name: str, scenario_index: int = 0) -> Dict:
+    resp = requests.post(
+        f"{ENV_URL}/reset",
+        json={"task_name": task_name, "scenario_index": scenario_index},
+        timeout=30,
+    )
+    resp.raise_for_status()
+    return resp.json()
+
+
+def env_step(action_type: str, payload: Optional[str] = None) -> Dict:
+    resp = requests.post(
+        f"{ENV_URL}/step",
+        json={"action_type": action_type, "payload": payload},
+        timeout=30,
+    )
+    resp.raise_for_status()
+    return resp.json()
+
+
+# ── Agent ─────────────────────────────────────────────────────────────────────
+
+def build_user_prompt(obs: Dict, step: int) -> str:
+    return json.dumps({
+        "step": step,
+        "question": obs["question"],
+        "current_answer": obs["current_answer"],
+        "retrieved_passages": obs["retrieved_passages"],
+        "current_citations": obs["current_citations"],
+        "steps_remaining": obs["steps_remaining"],
+        "feedback": obs.get("feedback"),
+        "structural_hints": obs.get("structural_hints", []),
+    }, indent=2)
+
+
+def get_agent_action(client: OpenAI, obs: Dict, step: int) -> Dict:
+    """Ask the LLM for the next action."""
+    user_prompt = build_user_prompt(obs, step)
+    try:
+        completion = client.chat.completions.create(
+            model=MODEL_NAME,
+            messages=[
+                {"role": "system", "content": SYSTEM_PROMPT},
+                {"role": "user", "content": user_prompt},
+            ],
+            temperature=TEMPERATURE,
+            max_tokens=MAX_TOKENS,
+            stream=False,
+        )
+        raw = (completion.choices[0].message.content or "").strip()
+        if raw.startswith("```"):
+            raw = raw.split("\n", 1)[-1].rsplit("```", 1)[0].strip()
+        return json.loads(raw)
+    except Exception as exc:
+        print(f"[DEBUG] LLM parse error: {exc}", flush=True)
+        if step <= 2:
+            return {"action_type": "RETRIEVE", "payload": None}
+        return {"action_type": "ACCEPT", "payload": None}
+
+
+# ── Main loop ─────────────────────────────────────────────────────────────────
+
+def run_task(client: OpenAI, task_name: str, max_steps: int, scenario_index: int = 0) -> float:
+    """Run one episode. Returns score in [0, 1]."""
+    log_start(task=task_name, env=BENCHMARK, model=MODEL_NAME)
+
+    rewards: List[float] = []
+    steps_taken = 0
+    score = 0.0
+    success = False
+
+    try:
+        result = env_reset(task_name, scenario_index)
+        obs = result["observation"]
+
+        for step in range(1, max_steps + 1):
+            if result.get("done", False):
+                break
+
+            agent_action = get_agent_action(client, obs, step)
+            action_type = agent_action.get("action_type", "ACCEPT")
+            payload = agent_action.get("payload")
+
+            result = env_step(action_type, payload)
+            obs = result["observation"]
+            reward = result.get("reward", 0.0)
+            done = result.get("done", False)
+
+            rewards.append(reward)
+            steps_taken = step
+
+            action_str = f"{action_type}({payload!r})" if payload else action_type
+            log_step(step=step, action=action_str, reward=reward, done=done, error=None)
+
+            if done:
+                score = result.get("info", {}).get("score", 0.0)
+                break
+
+        success = score >= 0.10
+
+    except Exception as exc:
+        print(f"[DEBUG] Episode error: {exc}", flush=True)
+
+    finally:
+        log_end(success=success, steps=steps_taken, score=score, rewards=rewards)
+
+    return score
+
+
+def main() -> None:
+    client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
+
+    all_scores: Dict[str, float] = {}
+
+    for task in TASKS:
+        task_name = task["name"]
+        max_steps = task["max_steps"]
+        score = run_task(client, task_name, max_steps, scenario_index=0)
+        all_scores[task_name] = score
+        print(flush=True)
+
+    print("=" * 60, flush=True)
+    print("BASELINE RESULTS", flush=True)
+    for name, sc in all_scores.items():
+        print(f"  {name:25s}  score={sc:.3f}", flush=True)
+    avg = sum(all_scores.values()) / len(all_scores) if all_scores else 0.0
+    print(f"  {'AVERAGE':25s}  score={avg:.3f}", flush=True)
+    print("=" * 60, flush=True)
+
+
+if __name__ == "__main__":
+    main()

main.py

@@ -0,0 +1,109 @@
+"""
+FastAPI server for IndicScriptureQA — OpenEnv compatible.
+
+Endpoints:
+  POST /reset   — start a new episode
+  POST /step    — take an action
+  GET  /state   — get current environment state
+  GET  /health  — liveness check
+  GET  /tasks   — list available tasks
+"""
+
+from __future__ import annotations
+
+from typing import Optional
+
+from fastapi import FastAPI, HTTPException
+from pydantic import BaseModel
+
+from environment import IndicScriptureQAEnv
+from models import Action, ActionType
+from tasks import TASKS
+
+app = FastAPI(
+    title="IndicScriptureQA",
+    description=(
+        "OpenEnv environment for evaluating agents on Indic scripture "
+        "hallucination correction AND semantic structure quality."
+    ),
+    version="1.1.0",
+)
+
+_env = IndicScriptureQAEnv()
+
+
+# ── Request / Response schemas ────────────────────────────────────────────────
+
+class ResetRequest(BaseModel):
+    task_name: str = "verify-factual"
+    scenario_index: Optional[int] = None
+
+
+class StepRequest(BaseModel):
+    action_type: str     # RETRIEVE | EDIT | RESTRUCTURE | CITE | ACCEPT | REJECT
+    payload: Optional[str] = None
+
+
+class TaskInfo(BaseModel):
+    name: str
+    description: str
+    max_steps: int
+    num_scenarios: int
+
+
+# ── Endpoints ─────────────────────────────────────────────────────────────────
+
+@app.post("/reset")
+def reset(body: ResetRequest = ResetRequest()):
+    try:
+        result = _env.reset(
+            task_name=body.task_name,
+            scenario_index=body.scenario_index,
+        )
+        return result.model_dump()
+    except ValueError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
+@app.post("/step")
+def step(body: StepRequest):
+    try:
+        action_type = ActionType(body.action_type.upper())
+    except ValueError:
+        raise HTTPException(
+            status_code=400,
+            detail=f"Invalid action_type {body.action_type!r}. Must be one of: {[a.value for a in ActionType]}",
+        )
+    try:
+        action = Action(action_type=action_type, payload=body.payload)
+        result = _env.step(action)
+        return result.model_dump()
+    except RuntimeError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
+@app.get("/state")
+def state():
+    try:
+        s = _env.state()
+        return s.model_dump()
+    except RuntimeError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
+@app.get("/health")
+def health():
+    return {"status": "ok"}
+
+
+@app.get("/tasks")
+def list_tasks():
+    return [
+        TaskInfo(
+            name=cfg.name,
+            description=cfg.description,
+            max_steps=cfg.max_steps,
+            num_scenarios=len(cfg.scenarios),
+        ).model_dump()
+        for cfg in TASKS.values()
+    ]

models.py

@@ -0,0 +1,119 @@
+"""Typed Pydantic models for the IndicScriptureQA environment."""
+
+from __future__ import annotations
+
+from enum import Enum
+from typing import Dict, List, Optional
+
+from pydantic import BaseModel, Field
+
+
+# ── Action Space ──────────────────────────────────────────────────────────────
+
+class ActionType(str, Enum):
+    RETRIEVE     = "RETRIEVE"      # Retrieve source passages (payload = optional query)
+    EDIT         = "EDIT"          # Replace current answer   (payload = new answer text)
+    RESTRUCTURE  = "RESTRUCTURE"   # Reorganise answer flow   (payload = restructured text)
+    CITE         = "CITE"          # Attach a citation        (payload = e.g. "Bhagavad Gita 2.47")
+    ACCEPT       = "ACCEPT"        # Accept answer as final    (terminal)
+    REJECT       = "REJECT"        # Reject answer entirely    (terminal)
+
+
+class Action(BaseModel):
+    action_type: ActionType
+    payload: Optional[str] = None
+
+
+# ── Structural metadata (hidden from agent, used by grader) ──────────────────
+
+class StructuralMeta(BaseModel):
+    """Describes the *expected* semantic structure of a correct answer."""
+    required_terms: List[str] = Field(
+        default_factory=list,
+        description="Sanskrit / domain terms the answer must contain.",
+    )
+    required_sections: List[str] = Field(
+        default_factory=list,
+        description="Conceptual aspects the answer should cover (order-independent).",
+    )
+    expected_order: List[str] = Field(
+        default_factory=list,
+        description="Concepts that should appear in this logical sequence.",
+    )
+    banned_terms: List[str] = Field(
+        default_factory=list,
+        description="Terms that indicate a common misconception if present.",
+    )
+
+
+# ── Observation Space ─────────────────────────────────────────────────────────
+
+class Observation(BaseModel):
+    question: str
+    current_answer: str
+    retrieved_passages: List[str] = Field(default_factory=list)
+    current_citations: List[str] = Field(default_factory=list)
+    steps_remaining: int
+    task_name: str
+    feedback: Optional[str] = None
+    # structural hints exposed to the agent (non-spoiler)
+    structural_hints: List[str] = Field(
+        default_factory=list,
+        description="High-level hints about expected answer structure.",
+    )
+
+
+# ── Step Result ───────────────────────────────────────────────────────────────
+
+class StepResult(BaseModel):
+    observation: Observation
+    reward: float = 0.0
+    done: bool = False
+    info: dict = Field(default_factory=dict)
+
+
+# ── Internal State (superset of observation + grading internals) ──────────────
+
+class EnvState(BaseModel):
+    # observable
+    question: str
+    current_answer: str
+    retrieved_passages: List[str] = Field(default_factory=list)
+    current_citations: List[str] = Field(default_factory=list)
+    steps_remaining: int = 0
+    task_name: str = ""
+    feedback: Optional[str] = None
+    structural_hints: List[str] = Field(default_factory=list)
+
+    # hidden / grading — factual
+    original_answer: str = ""
+    ground_truth_answer: str = ""
+    ground_truth_citations: List[str] = Field(default_factory=list)
+    available_passages: List[str] = Field(default_factory=list)
+    answer_is_correct: bool = False        # overall: facts AND structure both OK
+    factual_is_correct: bool = False       # facts alone are OK (structure may be bad)
+
+    # hidden / grading — structural
+    structural_meta: StructuralMeta = Field(default_factory=StructuralMeta)
+
+    # episode bookkeeping
+    step_count: int = 0
+    max_steps: int = 8
+    done: bool = False
+    cumulative_reward: float = 0.0
+    rewards: List[float] = Field(default_factory=list)
+    retrieval_count: int = 0
+    edit_count: int = 0
+    restructure_count: int = 0
+
+    def to_observation(self) -> Observation:
+        return Observation(
+            question=self.question,
+            current_answer=self.current_answer,
+            retrieved_passages=list(self.retrieved_passages),
+            current_citations=list(self.current_citations),
+            steps_remaining=self.steps_remaining,
+            task_name=self.task_name,
+            feedback=self.feedback,
+            structural_hints=list(self.structural_hints),
+        )

openenv.yaml

@@ -0,0 +1,109 @@
+name: indic_scripture_qa
+version: "1.1.0"
+description: >
+  An RL environment for evaluating language agents on Indic scripture
+  hallucination detection, correction, AND semantic structure quality.
+  Agents receive a question and a potentially flawed answer — which may
+  contain factual hallucinations, structural incoherence, missing
+  terminology, or poor logical ordering — and must retrieve source passages,
+  edit, restructure, cite, and decide whether to accept or reject.
+
+author: kishlay-notabot
+license: MIT
+
+env:
+  module: main:app
+  port: 8000
+  health_endpoint: /health
+
+action_space:
+  type: object
+  fields:
+    action_type:
+      type: string
+      enum: [RETRIEVE, EDIT, RESTRUCTURE, CITE, ACCEPT, REJECT]
+      description: >
+        RETRIEVE fetches source passages. EDIT fixes factual content.
+        RESTRUCTURE reorganises flow/coherence without changing facts.
+        CITE adds a scripture reference. ACCEPT/REJECT are terminal.
+    payload:
+      type: string
+      nullable: true
+      description: >
+        Content for the action — query for RETRIEVE, new answer text for
+        EDIT or RESTRUCTURE, citation string for CITE, unused for ACCEPT/REJECT.
+
+observation_space:
+  type: object
+  fields:
+    question:
+      type: string
+    current_answer:
+      type: string
+    retrieved_passages:
+      type: array
+      items: { type: string }
+    current_citations:
+      type: array
+      items: { type: string }
+    steps_remaining:
+      type: integer
+    task_name:
+      type: string
+    feedback:
+      type: string
+      nullable: true
+    structural_hints:
+      type: array
+      items: { type: string }
+      description: >
+        Non-spoiler hints about expected answer structure (e.g. required
+        terminology, conceptual ordering, completeness expectations).
+
+tasks:
+  - name: verify-factual
+    description: >
+      Easy — Verify whether a given answer is factually correct and
+      structurally sound. Accept good answers, reject or fix bad ones.
+    max_steps: 5
+    num_scenarios: 5
+
+  - name: correct-and-cite
+    description: >
+      Medium — Improve a partially correct answer by fixing factual gaps,
+      restructuring for coherence, adding proper terminology, and citing
+      scripture sources.
+    max_steps: 8
+    num_scenarios: 5
+
+  - name: fix-hallucination
+    description: >
+      Hard — Detect subtle hallucinations, fix factual errors, correct
+      misused Sanskrit terms, reorder logical flow, and ensure complete
+      coverage of required conceptual aspects.
+    max_steps: 12
+    num_scenarios: 5
+
+reward:
+  range: [0.0, 1.0]
+  components:
+    factual:
+      weight: 0.43
+      description: >
+        Token-F1 similarity to ground truth answer (×0.90) plus citation
+        recall (×0.30).
+    structural:
+      weight: 0.33
+      description: >
+        Composite of four axes: terminology precision (0.30), completeness
+        (0.25), logical ordering (0.25), and coherence (0.20). Includes
+        banned-term penalties for common misconceptions.
+    efficiency:
+      weight: 0.10
+      description: Bonus for using fewer steps.
+    step_shaping:
+      weight: 0.14
+      description: >
+        Per-step signals: +0.05 for useful retrieval, +0.20–0.50 for
+        quality edits/restructures, +0.15 for correct citations, with
+        penalties for redundancy and degradation.

requirements-server.txt

@@ -0,0 +1,3 @@
+fastapi>=0.110.0
+uvicorn[standard]>=0.27.0
+pydantic>=2.0.0

requirements.txt

@@ -0,0 +1,2 @@
+openai>=1.0.0
+requests>=2.31.0

rewards.py

@@ -0,0 +1,396 @@
+"""
+Reward computation for IndicScriptureQA.
+
+Two evaluation axes, weighted into a single scalar:
+  A. Factual quality  — token-F1 similarity to ground truth, citation recall
+  B. Structural quality — coherence, completeness, terminology, ordering
+
+All scoring is zero-dependency (no ML models) so the env runs on 2 vCPU / 8 GB.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import List, Tuple
+
+from models import ActionType, EnvState, StructuralMeta
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# A. FACTUAL SCORING
+# ═══════════════════════════════════════════════════════════════════════════════
+
+def _tokenize(text: str) -> List[str]:
+    """Lowercase split on non-alphanumeric (keeps Devanagari chars)."""
+    return [t for t in re.split(r"[^a-zA-Z0-9\u0900-\u097F]+", text.lower()) if t]
+
+
+def token_f1(candidate: str, reference: str) -> float:
+    """Token-level F1 between candidate and reference. Returns 0–1."""
+    cand_toks = _tokenize(candidate)
+    ref_toks = _tokenize(reference)
+    if not cand_toks or not ref_toks:
+        return 0.0
+    cand_set = set(cand_toks)
+    ref_set = set(ref_toks)
+    common = cand_set & ref_set
+    if not common:
+        return 0.0
+    precision = len(common) / len(cand_set)
+    recall = len(common) / len(ref_set)
+    return 2 * precision * recall / (precision + recall)
+
+
+def _normalize_citation(c: str) -> str:
+    return re.sub(r"\s+", " ", c.strip().lower())
+
+
+def citation_recall(predicted: List[str], ground_truth: List[str]) -> float:
+    """Fraction of ground-truth citations matched (fuzzy substring)."""
+    if not ground_truth:
+        return 1.0
+    gt_norms = [_normalize_citation(g) for g in ground_truth]
+    pred_norms = [_normalize_citation(p) for p in predicted]
+    matched = 0
+    for gt in gt_norms:
+        for pred in pred_norms:
+            if gt in pred or pred in gt:
+                matched += 1
+                break
+    return matched / len(gt_norms)
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# B. STRUCTURAL SCORING
+# ═══════════════════════════════════════════════════════════════════════════════
+
+# ── B1. Terminology precision ────────────────────────────────────────────────
+
+def terminology_score(answer: str, meta: StructuralMeta) -> float:
+    """
+    Checks:
+      +  required_terms present   → recall over required_terms
+      -  banned_terms present     → hard penalty per banned term found
+    Returns float in [-1.0, 1.0].
+    """
+    answer_lower = answer.lower()
+
+    # required term recall
+    if meta.required_terms:
+        hits = sum(1 for t in meta.required_terms if t.lower() in answer_lower)
+        term_recall = hits / len(meta.required_terms)
+    else:
+        term_recall = 1.0
+
+    # banned term penalty
+    ban_penalty = 0.0
+    if meta.banned_terms:
+        for bt in meta.banned_terms:
+            if bt.lower() in answer_lower:
+                ban_penalty += 0.25
+        ban_penalty = min(ban_penalty, 1.0)
+
+    return term_recall - ban_penalty
+
+
+# ── B2. Completeness (section coverage) ──────────────────────────────────────
+
+def completeness_score(answer: str, meta: StructuralMeta) -> float:
+    """
+    Heuristic: for each required_section, check whether characteristic
+    keywords from that section label appear in the answer.
+    Returns 0–1 (fraction of sections covered).
+    """
+    if not meta.required_sections:
+        return 1.0
+    answer_lower = answer.lower()
+    covered = 0
+    for section in meta.required_sections:
+        # use the keywords from the section label itself
+        section_keywords = _tokenize(section)
+        # count a section as covered if ≥ half its keywords appear
+        if section_keywords:
+            hits = sum(1 for kw in section_keywords if kw in answer_lower)
+            if hits / len(section_keywords) >= 0.5:
+                covered += 1
+    return covered / len(meta.required_sections)
+
+
+# ── B3. Logical ordering (sequence adherence) ────────────────────────────────
+
+def ordering_score(answer: str, meta: StructuralMeta) -> float:
+    """
+    Checks whether concepts in expected_order appear in the correct sequence
+    in the answer. Uses first-occurrence position of each concept's keywords.
+    Returns 0–1.
+    """
+    if len(meta.expected_order) < 2:
+        return 1.0
+
+    answer_lower = answer.lower()
+    positions: List[int] = []
+
+    for concept in meta.expected_order:
+        keywords = _tokenize(concept)
+        # find earliest position of any keyword
+        earliest = len(answer_lower) + 1
+        for kw in keywords:
+            idx = answer_lower.find(kw)
+            if idx != -1 and idx < earliest:
+                earliest = idx
+        positions.append(earliest)
+
+    # count correctly ordered adjacent pairs
+    correct_pairs = sum(
+        1 for i in range(len(positions) - 1) if positions[i] <= positions[i + 1]
+    )
+    return correct_pairs / (len(positions) - 1)
+
+
+# ── B4. Coherence (transition quality + sentence structure) ──────────────────
+
+_TRANSITION_MARKERS = {
+    "therefore", "however", "moreover", "furthermore", "thus", "consequently",
+    "specifically", "in contrast", "for example", "similarly", "additionally",
+    "because", "since", "although", "while", "first", "second", "third",
+    "finally", "in particular", "notably", "according to", "this means",
+    "as a result", "in other words",
+}
+
+
+def coherence_score(answer: str) -> float:
+    """
+    Lightweight coherence proxy:
+      - Sentence count (more than 1 sentence expected)
+      - Transition markers (discourse connectives)
+      - Sentence-length variance (very uneven → lower coherence)
+    Returns 0–1.
+    """
+    sentences = [s.strip() for s in re.split(r"[.!?]+", answer) if s.strip()]
+    if len(sentences) <= 1:
+        return 0.3  # single sentence is structurally weak for these tasks
+
+    # transition marker density
+    answer_lower = answer.lower()
+    marker_count = sum(1 for m in _TRANSITION_MARKERS if m in answer_lower)
+    marker_density = min(marker_count / max(len(sentences) - 1, 1), 1.0)
+
+    # sentence length variance (normalised). Very uneven → incoherent.
+    lengths = [len(s.split()) for s in sentences]
+    mean_len = sum(lengths) / len(lengths)
+    if mean_len == 0:
+        return 0.2
+    variance = sum((l - mean_len) ** 2 for l in lengths) / len(lengths)
+    cv = (variance ** 0.5) / mean_len  # coefficient of variation
+    uniformity = max(0.0, 1.0 - cv)  # lower CV → more uniform → higher score
+
+    # blend: 50 % markers, 30 % uniformity, 20 % baseline for multi-sentence
+    return 0.5 * marker_density + 0.3 * uniformity + 0.2
+
+
+# ── Composite structural score ───────────────────────────────────────────────
+
+def structural_quality(answer: str, meta: StructuralMeta) -> Tuple[float, dict]:
+    """
+    Weighted composite of all structural axes.
+    Returns (score_0_to_1, breakdown_dict).
+    """
+    ts = terminology_score(answer, meta)
+    cs = completeness_score(answer, meta)
+    os_ = ordering_score(answer, meta)
+    coh = coherence_score(answer)
+
+    # weights
+    composite = (
+        0.30 * max(ts, 0.0)   # terminology  (clamp negatives to 0 for composite)
+      + 0.25 * cs              # completeness
+      + 0.25 * os_             # ordering
+      + 0.20 * coh             # coherence
+    )
+
+    # apply banned-term penalty on top
+    if ts < 0:
+        composite += 0.15 * ts  # propagate penalty
+
+    composite = max(0.0, min(1.0, composite))
+
+    breakdown = {
+        "terminology": round(ts, 3),
+        "completeness": round(cs, 3),
+        "ordering": round(os_, 3),
+        "coherence": round(coh, 3),
+        "composite": round(composite, 3),
+    }
+    return composite, breakdown
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# PER-STEP REWARD
+# ═══════════════════════════════════════════════════════════════════════════════
+
+def step_reward(state: EnvState, action_type: ActionType, payload: str | None) -> Tuple[float, str]:
+    """
+    Compute per-step reward and feedback message.
+    Now accounts for structural improvement on EDIT and RESTRUCTURE.
+    """
+    reward = 0.0
+    feedback = ""
+
+    if action_type == ActionType.RETRIEVE:
+        if state.retrieval_count >= 3:
+            reward = -0.15
+            feedback = "Redundant retrieval — you've already retrieved 3 times."
+        elif state.available_passages:
+            reward = 0.05
+            feedback = "Passages retrieved."
+        else:
+            reward = -0.05
+            feedback = "No passages available for retrieval."
+
+    elif action_type == ActionType.EDIT:
+        if not payload:
+            reward = -0.10
+            feedback = "Empty edit — no content provided."
+        else:
+            # factual delta
+            old_sim = token_f1(state.current_answer, state.ground_truth_answer)
+            new_sim = token_f1(payload, state.ground_truth_answer)
+            fact_delta = new_sim - old_sim
+
+            # structural delta
+            old_struct, _ = structural_quality(state.current_answer, state.structural_meta)
+            new_struct, bk = structural_quality(payload, state.structural_meta)
+            struct_delta = new_struct - old_struct
+
+            combined_delta = 0.6 * fact_delta + 0.4 * struct_delta
+
+            if combined_delta > 0.03:
+                reward = 0.20 + combined_delta
+                feedback = f"Edit improved answer (fact Δ{fact_delta:+.2f}, struct Δ{struct_delta:+.2f})."
+            elif combined_delta < -0.03:
+                reward = -0.20
+                feedback = f"Edit degraded answer (fact Δ{fact_delta:+.2f}, struct Δ{struct_delta:+.2f})."
+            else:
+                reward = -0.05
+                feedback = "Edit had negligible effect."
+
+    elif action_type == ActionType.RESTRUCTURE:
+        if not payload:
+            reward = -0.10
+            feedback = "Empty restructure — no content provided."
+        else:
+            # restructure should preserve facts but improve structure
+            old_sim = token_f1(state.current_answer, state.ground_truth_answer)
+            new_sim = token_f1(payload, state.ground_truth_answer)
+            fact_delta = new_sim - old_sim
+
+            old_struct, _ = structural_quality(state.current_answer, state.structural_meta)
+            new_struct, bk = structural_quality(payload, state.structural_meta)
+            struct_delta = new_struct - old_struct
+
+            if fact_delta < -0.10:
+                # restructure destroyed factual content
+                reward = -0.25
+                feedback = f"Restructure lost factual content (fact Δ{fact_delta:+.2f}). Use EDIT if changing facts."
+            elif struct_delta > 0.05:
+                reward = 0.25 + struct_delta
+                feedback = (
+                    f"Restructure improved structure (Δ{struct_delta:+.2f}). "
+                    f"Breakdown: term={bk['terminology']:.2f} comp={bk['completeness']:.2f} "
+                    f"order={bk['ordering']:.2f} coh={bk['coherence']:.2f}"
+                )
+            elif struct_delta < -0.03:
+                reward = -0.15
+                feedback = f"Restructure degraded structure (Δ{struct_delta:+.2f})."
+            else:
+                reward = -0.05
+                feedback = "Restructure had negligible structural effect."
+
+    elif action_type == ActionType.CITE:
+        if not payload:
+            reward = -0.05
+            feedback = "Empty citation."
+        else:
+            cr = citation_recall([payload], state.ground_truth_citations)
+            if cr > 0:
+                reward = 0.15
+                feedback = "Correct citation added."
+            else:
+                reward = -0.05
+                feedback = "Citation does not match expected sources."
+
+    elif action_type in (ActionType.ACCEPT, ActionType.REJECT):
+        pass  # terminal rewards handled separately
+
+    return reward, feedback
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# TERMINAL REWARD
+# ═══════════════════════════════════════════════════════════════════════════════
+
+def terminal_reward(state: EnvState, action_type: ActionType) -> Tuple[float, str]:
+    """
+    Terminal reward blends factual quality AND structural quality.
+    """
+    if action_type == ActionType.REJECT:
+        if not state.answer_is_correct:
+            return 0.30, "Correctly rejected a flawed answer."
+        else:
+            return -0.50, "Incorrectly rejected a valid answer."
+
+    # ── ACCEPT ────────────────────────────────────────────────────────────
+    # factual component
+    answer_sim = token_f1(state.current_answer, state.ground_truth_answer)
+    cit_score = citation_recall(state.current_citations, state.ground_truth_citations)
+
+    # structural component
+    struct_score, struct_breakdown = structural_quality(
+        state.current_answer, state.structural_meta
+    )
+
+    # efficiency bonus (0–0.2)
+    efficiency = 0.20 * (state.steps_remaining / state.max_steps)
+
+    # weighted terminal reward
+    terminal = (
+        0.90 * answer_sim        # factual similarity    (max 0.90)
+      + 0.30 * cit_score         # citation recall       (max 0.30)
+      + 0.70 * struct_score      # structural quality    (max 0.70)
+      + efficiency               # efficiency bonus      (max 0.20)
+    )
+    # theoretical max ≈ 2.10
+
+    # penalty for accepting a still-bad answer
+    if answer_sim < 0.3 and struct_score < 0.3:
+        terminal -= 0.50
+        quality_label = "poor"
+    elif answer_sim < 0.5:
+        quality_label = "mediocre"
+    else:
+        quality_label = "good"
+
+    feedback = (
+        f"Accepted a {quality_label} answer "
+        f"(fact={answer_sim:.2f}, cite={cit_score:.2f}, struct={struct_score:.2f} "
+        f"[term={struct_breakdown['terminology']:.2f} "
+        f"comp={struct_breakdown['completeness']:.2f} "
+        f"ord={struct_breakdown['ordering']:.2f} "
+        f"coh={struct_breakdown['coherence']:.2f}])"
+    )
+
+    return terminal, feedback
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# SCORE NORMALISATION
+# ═══════════════════════════════════════════════════════════════════════════════
+
+# theoretical max: terminal ~2.10 + step bonuses ~0.5 ≈ 2.6
+MAX_REASONABLE_REWARD = 2.80
+
+
+def normalize_score(cumulative_reward: float) -> float:
+    """Clamp cumulative reward into [0, 1]."""
+    score = cumulative_reward / MAX_REASONABLE_REWARD
+    return max(0.0, min(1.0, score))

tasks.py

@@ -0,0 +1,448 @@
+"""
+Task definitions for IndicScriptureQA.
+
+Each scenario now carries *structural metadata* alongside factual ground truth.
+The grader evaluates BOTH factual accuracy AND semantic structure:
+  - required_terms     → Sanskrit/domain terms the answer must use
+  - required_sections  → conceptual aspects that must be covered
+  - expected_order     → logical ordering of concepts
+  - banned_terms       → misconception markers (penalty if present)
+
+Difficulty controls:
+  easy   → blatant factual error OR correct answer; max 5 steps
+  medium → partial errors + missing citations + poor structure; max 8 steps
+  hard   → subtle hallucinations + jumbled structure + terminology misuse; max 12 steps
+"""
+
+from __future__ import annotations
+
+import dataclasses
+from typing import Dict, List, Optional
+
+from models import StructuralMeta
+
+
+@dataclasses.dataclass
+class Scenario:
+    question: str
+    given_answer: str
+    ground_truth_answer: str
+    ground_truth_citations: List[str]
+    available_passages: List[str]
+    answer_is_correct: bool            # overall (facts + structure)
+    factual_is_correct: bool           # facts alone
+    structural_meta: StructuralMeta
+    structural_hints: List[str]        # non-spoiler hints shown to agent
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# TASK 1 — verify-factual  (Easy)
+# ═══════════════════════════════════════════════════════════════════════════════
+
+VERIFY_FACTUAL_SCENARIOS: List[Scenario] = [
+    Scenario(
+        question="Who killed Ravana in the Ramayana?",
+        given_answer="Lakshmana killed Ravana with a divine arrow during the battle of Lanka.",
+        ground_truth_answer="Rama killed Ravana using the Brahmastra during the battle of Lanka.",
+        ground_truth_citations=["Valmiki Ramayana, Yuddha Kanda, Sarga 108"],
+        available_passages=[
+            "Yuddha Kanda 108: Rama, following Agastya's counsel, invoked the Brahmastra and struck Ravana in the chest, ending his life.",
+            "Yuddha Kanda 101: Lakshmana fought Indrajit and defeated him but did not fight Ravana directly in the final duel.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["Rama", "Ravana", "Brahmastra"],
+            required_sections=["who performed the act", "weapon used", "context of battle"],
+            expected_order=["context of battle", "weapon used", "outcome"],
+            banned_terms=[],
+        ),
+        structural_hints=["Answer should identify the warrior, the weapon, and the battle context."],
+    ),
+    Scenario(
+        question="How many chapters does the Bhagavad Gita contain?",
+        given_answer="The Bhagavad Gita contains 18 chapters, each called an Adhyaya.",
+        ground_truth_answer="The Bhagavad Gita contains 18 chapters (Adhyayas), comprising 700 verses.",
+        ground_truth_citations=["Bhagavad Gita, Chapters 1–18"],
+        available_passages=[
+            "The Bhagavad Gita is a 700-verse scripture that is part of the Mahabharata (Bhishma Parva, chapters 25–42). It consists of 18 chapters.",
+        ],
+        answer_is_correct=True,
+        factual_is_correct=True,
+        structural_meta=StructuralMeta(
+            required_terms=["Adhyaya", "18"],
+            required_sections=["chapter count", "structure note"],
+            expected_order=[],
+            banned_terms=[],
+        ),
+        structural_hints=["Mention chapter count and the term for chapters."],
+    ),
+    Scenario(
+        question="Who narrated the Bhagavad Gita to Arjuna?",
+        given_answer="Vyasa narrated the Bhagavad Gita to Arjuna on the battlefield of Kurukshetra.",
+        ground_truth_answer="Krishna narrated the Bhagavad Gita to Arjuna on the battlefield of Kurukshetra.",
+        ground_truth_citations=["Bhagavad Gita 1.1", "Bhagavad Gita 2.11"],
+        available_passages=[
+            "Bhagavad Gita 2.11: Sri Bhagavan (Krishna) said — You grieve for those who should not be grieved for.",
+            "Vyasa composed the Mahabharata and dictated it to Ganesha but was not the speaker of the Gita.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["Krishna", "Arjuna", "Kurukshetra"],
+            required_sections=["speaker", "listener", "setting"],
+            expected_order=["speaker", "listener", "setting"],
+            banned_terms=[],
+        ),
+        structural_hints=["Identify the speaker, the listener, and the setting."],
+    ),
+    Scenario(
+        question="What is the first word of the Rigveda?",
+        given_answer="The first word of the Rigveda is 'Agnim' (अग्निम्), invoking the fire deity Agni.",
+        ground_truth_answer="The first word of the Rigveda is 'Agnim' (अग्निम्), the accusative form of Agni, beginning the hymn to the fire deity.",
+        ground_truth_citations=["Rigveda 1.1.1"],
+        available_passages=[
+            "Rigveda 1.1.1: Agnim ile purohitam yajnasya devam ritvijam — I praise Agni, the foremost priest, the divine minister of the sacrifice.",
+        ],
+        answer_is_correct=True,
+        factual_is_correct=True,
+        structural_meta=StructuralMeta(
+            required_terms=["Agnim", "Agni", "Rigveda"],
+            required_sections=["the word", "its meaning", "its significance"],
+            expected_order=["the word", "its meaning"],
+            banned_terms=[],
+        ),
+        structural_hints=["State the word, explain its grammatical form, and note its significance."],
+    ),
+    Scenario(
+        question="In the Mahabharata, who was the commander-in-chief of the Kaurava army on the first day?",
+        given_answer="Drona was the first commander-in-chief of the Kaurava army at Kurukshetra.",
+        ground_truth_answer="Bhishma was the first commander-in-chief of the Kaurava army, leading from days 1 through 10.",
+        ground_truth_citations=["Mahabharata, Bhishma Parva"],
+        available_passages=[
+            "Bhishma Parva: Bhishma was appointed supreme commander of the Kaurava forces. He led the army for the first ten days of the war.",
+            "Drona Parva: After Bhishma fell, Drona was appointed the second commander-in-chief.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["Bhishma", "Kaurava"],
+            required_sections=["commander identity", "duration of command"],
+            expected_order=["commander identity", "duration of command"],
+            banned_terms=[],
+        ),
+        structural_hints=["Identify the commander and state how long they held command."],
+    ),
+]
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# TASK 2 — correct-and-cite  (Medium)
+#   Scenarios now include structural problems: missing aspects, wrong ordering,
+#   imprecise terminology, alongside the citation/factual issues.
+# ═══════════════════════════════════════════════════════════════════════════════
+
+CORRECT_AND_CITE_SCENARIOS: List[Scenario] = [
+    Scenario(
+        question="What does Krishna say about Karma Yoga in the Bhagavad Gita?",
+        given_answer="Krishna tells Arjuna to perform his duty without attachment to results. He says action is superior to inaction and that one should work selflessly.",
+        ground_truth_answer="Krishna teaches that one has the right to perform prescribed duties but is not entitled to the fruits of actions. He advocates nishkama karma — selfless action without attachment to outcomes — as the path to liberation.",
+        ground_truth_citations=["Bhagavad Gita 2.47", "Bhagavad Gita 3.19"],
+        available_passages=[
+            "Bhagavad Gita 2.47: Karmanye vadhikaraste ma phaleshu kadachana — You have the right to perform your duty, but you are not entitled to the fruits of your actions.",
+            "Bhagavad Gita 3.19: Therefore, without attachment, always perform the work that has to be done; for by performing work without attachment, one attains the Supreme.",
+            "Bhagavad Gita 3.4: Not by merely abstaining from action can one achieve freedom from reaction, nor by renunciation alone can one attain perfection.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["nishkama karma", "Karma Yoga", "phala"],
+            required_sections=["core teaching", "key verse reference", "philosophical implication"],
+            expected_order=["core teaching", "key verse reference", "philosophical implication"],
+            banned_terms=[],
+        ),
+        structural_hints=[
+            "Use the Sanskrit term for selfless action.",
+            "Structure: teaching → supporting verse → implication for liberation.",
+        ],
+    ),
+    Scenario(
+        question="What are the four Mahavakyas of the Upanishads?",
+        given_answer="The four Mahavakyas are: Prajnanam Brahma, Aham Brahmasmi, Tat Tvam Asi, and Ayam Atma Brahma. They express the identity of the self with Brahman.",
+        ground_truth_answer="The four Mahavakyas are: 'Prajnanam Brahma' (Consciousness is Brahman) from Aitareya Upanishad, 'Aham Brahmasmi' (I am Brahman) from Brihadaranyaka Upanishad, 'Tat Tvam Asi' (Thou art That) from Chandogya Upanishad, and 'Ayam Atma Brahma' (This Self is Brahman) from Mandukya Upanishad.",
+        ground_truth_citations=[
+            "Aitareya Upanishad 3.3",
+            "Brihadaranyaka Upanishad 1.4.10",
+            "Chandogya Upanishad 6.8.7",
+            "Mandukya Upanishad 1.2",
+        ],
+        available_passages=[
+            "Aitareya Upanishad 3.3: Prajnanam Brahma — Consciousness is Brahman. This Mahavakya belongs to the Rigveda.",
+            "Brihadaranyaka Upanishad 1.4.10: Aham Brahmasmi — I am Brahman. This declaration belongs to the Yajurveda.",
+            "Chandogya Upanishad 6.8.7: Tat Tvam Asi — Thou art That. Uddalaka teaches this to Shvetaketu. Belongs to the Samaveda.",
+            "Mandukya Upanishad 1.2: Ayam Atma Brahma — This Self is Brahman. Belongs to the Atharvaveda.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=True,  # the four names are correct; missing source attribution
+        structural_meta=StructuralMeta(
+            required_terms=["Mahavakya", "Brahman", "Atman"],
+            required_sections=["each vakya with translation", "source Upanishad for each", "unifying theme"],
+            expected_order=["vakya list", "unifying theme"],
+            banned_terms=[],
+        ),
+        structural_hints=[
+            "Each Mahavakya should be paired with its source Upanishad.",
+            "Conclude with the unifying Advaita theme.",
+        ],
+    ),
+    Scenario(
+        question="Describe the concept of Dharma as explained in the Mahabharata.",
+        given_answer="Dharma in the Mahabharata is presented as a complex moral code. Bhishma explains dharma extensively while lying on the bed of arrows. The text suggests dharma is subtle and context-dependent.",
+        ground_truth_answer="The Mahabharata presents dharma as subtle (sukshma) and context-dependent. Bhishma's discourse on dharma in the Shanti Parva and Anushasana Parva covers duties of kings, ethics of war, and personal righteousness. The famous dictum states: 'Dharma is that which sustains all beings.'",
+        ground_truth_citations=["Mahabharata, Shanti Parva", "Mahabharata, Anushasana Parva"],
+        available_passages=[
+            "Shanti Parva: Bhishma, lying on the bed of arrows, instructs Yudhishthira on rajadharma (duties of kings), moksha-dharma, and apaddharma (ethics in emergencies).",
+            "Anushasana Parva: Continuation of Bhishma's teachings on dana (charity), dharma, and moral conduct.",
+            "Vana Parva 313.117: Dharmo rakshati rakshitah — Dharma protects those who protect dharma.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=True,
+        structural_meta=StructuralMeta(
+            required_terms=["sukshma", "rajadharma", "apaddharma", "dharma"],
+            required_sections=["definition of dharma", "Bhishma's discourse topics", "key dictum"],
+            expected_order=["definition of dharma", "Bhishma's discourse topics", "key dictum"],
+            banned_terms=[],
+        ),
+        structural_hints=[
+            "Use the Sanskrit term for dharma's subtlety.",
+            "Cover the three categories Bhishma teaches: rajadharma, moksha-dharma, apaddharma.",
+        ],
+    ),
+    Scenario(
+        question="What is the significance of the number 108 in Hindu scriptures?",
+        given_answer="The number 108 is sacred because there are 108 Upanishads in the Muktika canon. Japa malas have 108 beads for chanting.",
+        ground_truth_answer="108 is considered sacred in Hinduism for multiple reasons: the Muktika canon lists 108 Upanishads, japa malas contain 108 beads, there are 108 names (ashtottara) for major deities, and mathematically 1×(2²)×(3³)=108. The distance between the Earth and Sun is approximately 108 times the Sun's diameter.",
+        ground_truth_citations=["Muktika Upanishad 1.30–39"],
+        available_passages=[
+            "Muktika Upanishad: Rama narrates 108 Upanishads to Hanuman. The list includes major (mukhya) and minor Upanishads.",
+            "In Hindu tradition, 108 appears in mala beads, temple steps, and as the count of names in ashtottara-namavali (108 names of deities like Vishnu, Shiva, Lakshmi).",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=True,
+        structural_meta=StructuralMeta(
+            required_terms=["ashtottara", "Muktika", "japa"],
+            required_sections=["scriptural significance", "ritual significance", "mathematical note"],
+            expected_order=["scriptural significance", "ritual significance"],
+            banned_terms=[],
+        ),
+        structural_hints=[
+            "Cover scriptural, ritual, AND mathematical dimensions.",
+            "Use the term 'ashtottara-namavali' for the 108-name tradition.",
+        ],
+    ),
+    Scenario(
+        question="Who composed the Yoga Sutras?",
+        given_answer="Patanjali composed the Yoga Sutras, a foundational text of Raja Yoga, consisting of 196 sutras organized into four padas.",
+        ground_truth_answer="Patanjali composed the Yoga Sutras, a foundational text of Raja Yoga comprising 196 sutras in four padas: Samadhi Pada, Sadhana Pada, Vibhuti Pada, and Kaivalya Pada.",
+        ground_truth_citations=["Yoga Sutras of Patanjali 1.1"],
+        available_passages=[
+            "Yoga Sutras 1.1: Atha yoga-anushasanam — Now, the exposition of Yoga is being made.",
+            "The four padas are: Samadhi Pada (51 sutras), Sadhana Pada (55 sutras), Vibhuti Pada (56 sutras), and Kaivalya Pada (34 sutras).",
+        ],
+        answer_is_correct=True,
+        factual_is_correct=True,
+        structural_meta=StructuralMeta(
+            required_terms=["Patanjali", "Raja Yoga", "pada"],
+            required_sections=["author", "text structure", "pada names"],
+            expected_order=["author", "text structure", "pada names"],
+            banned_terms=[],
+        ),
+        structural_hints=["Name all four padas explicitly."],
+    ),
+]
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# TASK 3 — fix-hallucination  (Hard)
+#   Includes subtle factual errors PLUS structural/terminological problems:
+#   wrong ordering of concepts, misused Sanskrit terms, incomplete coverage.
+# ═══════════════════════════════════════════════════════════════════════════════
+
+FIX_HALLUCINATION_SCENARIOS: List[Scenario] = [
+    Scenario(
+        question="Explain the Dashavatara (ten avatars) of Vishnu as described in the Puranas.",
+        given_answer="The ten avatars of Vishnu are: Matsya (fish), Kurma (tortoise), Varaha (boar), Narasimha (man-lion), Vamana (dwarf), Parashurama, Rama, Balarama, Buddha, and Kalki. Each avatar appeared in a specific yuga to restore cosmic order. Narasimha appeared in Treta Yuga to defeat the demon Hiranyakashipu. Vamana tricked the demon king Ravana by asking for three steps of land.",
+        ground_truth_answer="The ten avatars of Vishnu are: Matsya, Kurma, Varaha, Narasimha, Vamana, Parashurama, Rama, Krishna (or Balarama in some lists), Buddha, and Kalki. Narasimha appeared in Satya Yuga (not Treta) to defeat Hiranyakashipu. Vamana tricked the demon king Mahabali (not Ravana) by asking for three paces of land. The avatars follow an evolutionary sequence reflecting increasing biological complexity.",
+        ground_truth_citations=[
+            "Bhagavata Purana 1.3",
+            "Garuda Purana 1.86",
+            "Bhagavata Purana 7.8 (Narasimha)",
+            "Bhagavata Purana 8.18–22 (Vamana)",
+        ],
+        available_passages=[
+            "Bhagavata Purana 1.3: Lists the avatars of Vishnu including Matsya, Kurma, Varaha, Narasimha, Vamana, Parashurama, Rama, Krishna, Buddha, and Kalki.",
+            "Bhagavata Purana 7.8: Narasimha avatar manifested in Satya Yuga to protect Prahlada and slay the demon Hiranyakashipu.",
+            "Bhagavata Purana 8.18–22: Vamana (dwarf avatar) approached the generous demon king Mahabali during a yajna and requested three paces of land, then expanded to cosmic form.",
+            "Garuda Purana 1.86: Alternate lists sometimes place Balarama as the eighth avatar instead of Krishna.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["Dashavatara", "Satya Yuga", "Mahabali", "Vishnu"],
+            required_sections=["complete avatar list", "yuga assignment for key avatars", "purpose of each avatar", "evolutionary sequence note"],
+            expected_order=["avatar list in order", "individual avatar details", "overarching pattern"],
+            banned_terms=["Treta Yuga for Narasimha", "Ravana for Vamana"],
+        ),
+        structural_hints=[
+            "List avatars in canonical order, then elaborate on key ones.",
+            "Note the evolutionary progression from aquatic to human forms.",
+            "Narasimha is Satya Yuga, not Treta.",
+        ],
+    ),
+    Scenario(
+        question="What is the story of Savitri and Satyavan from the Mahabharata?",
+        given_answer="Savitri was a princess who chose Satyavan as her husband despite the sage Vishwamitra's warning that Satyavan would die within a year. When Yama came to claim Satyavan's soul, Savitri followed Yama for seven days. Impressed by her devotion, Yama granted three boons including Satyavan's life. The story is found in the Ramayana's Aranya Kanda.",
+        ground_truth_answer="Savitri chose Satyavan despite the sage Narada's warning (not Vishwamitra) that Satyavan was fated to die within a year. When Yama came, Savitri followed him and through her wisdom and persistence obtained boons. The story appears in the Mahabharata's Vana Parva (not Ramayana's Aranya Kanda). Savitri's victory came through dialectical skill — she argued Yama into granting life, not through mere devotion alone.",
+        ground_truth_citations=[
+            "Mahabharata, Vana Parva, Chapters 277–283 (Pativrata Mahatmya)",
+        ],
+        available_passages=[
+            "Mahabharata, Vana Parva 277: Sage Narada warned King Asvapati and Savitri that Satyavan, though virtuous, was destined to die exactly one year after their marriage.",
+            "Mahabharata, Vana Parva 281–283: Savitri followed Yama as he carried Satyavan's soul. Through her persistent arguments and wisdom, Yama granted boons: her father-in-law's sight restored, his lost kingdom returned, and finally Satyavan's life.",
+            "The Savitri-Satyavan story is entirely within the Mahabharata (Vana Parva). It does not appear in the Ramayana.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["Narada", "Yama", "Vana Parva", "pativrata"],
+            required_sections=["Savitri's choice", "Narada's warning", "confrontation with Yama", "method of victory", "source text"],
+            expected_order=["Savitri's choice", "Narada's warning", "confrontation with Yama", "method of victory"],
+            banned_terms=["Vishwamitra", "Aranya Kanda", "Ramayana"],
+        ),
+        structural_hints=[
+            "Narrative structure: choice → prophecy → confrontation → resolution.",
+            "Emphasise Savitri's dialectical skill, not just devotion.",
+            "This story is in the Mahabharata, not the Ramayana.",
+        ],
+    ),
+    Scenario(
+        question="Describe the Samudra Manthan (Churning of the Ocean) episode.",
+        given_answer="The devas and asuras churned the ocean of milk using Mount Meru as the churning rod and the serpent Takshaka as the rope. Vishnu took the form of Kurma (tortoise) to support the mountain. Fourteen treasures emerged including Amrita (nectar), Lakshmi, the deadly poison Halahala which was consumed by Vishnu, and the divine horse Uchchaihshravas. Mohini distributed the Amrita exclusively to the devas.",
+        ground_truth_answer="The devas and asuras churned the ocean using Mount Mandara (not Meru) as the rod and the serpent Vasuki (not Takshaka) as the rope. Kurma supported Mandara on his back. Among the fourteen treasures were Amrita, Lakshmi, Halahala, and Uchchaihshravas. The Halahala poison was consumed by Shiva (not Vishnu), who held it in his throat, earning the name Neelakantha. Vishnu as Mohini distributed the Amrita.",
+        ground_truth_citations=[
+            "Bhagavata Purana 8.5–12",
+            "Vishnu Purana 1.9",
+            "Mahabharata, Adi Parva 15–17",
+        ],
+        available_passages=[
+            "Bhagavata Purana 8.7: Mount Mandara was used as the churning rod. The serpent Vasuki served as the rope. Vishnu as Kurma supported the mountain from below.",
+            "Bhagavata Purana 8.7: The Halahala poison emerged first, threatening all creation. Shiva drank the poison at Parvati's urging, holding it in his throat. His throat turned blue — hence the name Neelakantha.",
+            "Vishnu Purana 1.9: Fourteen ratnas (treasures) emerged: Lakshmi, Kaustubha gem, Parijata tree, Varuni, Dhanvantari with Amrita, Chandra (moon), Kamadhenu, Airavata, Uchchaihshravas, and others.",
+            "Bhagavata Purana 8.12: Vishnu assumed the form of Mohini to distribute Amrita among the devas, deceiving the asuras.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["Mandara", "Vasuki", "Halahala", "Neelakantha", "Mohini", "ratna"],
+            required_sections=["setup and participants", "churning mechanism", "crisis (poison)", "treasures", "resolution (Amrita distribution)"],
+            expected_order=["setup and participants", "churning mechanism", "crisis (poison)", "treasures", "resolution (Amrita distribution)"],
+            banned_terms=["Meru as churning rod", "Takshaka as rope", "Vishnu drank poison"],
+        ),
+        structural_hints=[
+            "Follow the narrative arc: setup → churning → crisis → treasures → resolution.",
+            "The mountain is Mandara, the serpent is Vasuki.",
+            "Shiva consumed the poison, not Vishnu.",
+        ],
+    ),
+    Scenario(
+        question="What are the main philosophical schools (Darshanas) of Hinduism?",
+        given_answer="The six orthodox darshanas are: Samkhya (founded by Kapila), Yoga (by Patanjali), Nyaya (by Gotama), Vaisheshika (by Kanada), Mimamsa (by Kumarila Bhatta), and Vedanta (by Shankaracharya). These are called astika schools because they accept the authority of the Vedas. The heterodox schools include Buddhism, Jainism, and the Charvaka school founded by Brihaspati.",
+        ground_truth_answer="The six orthodox darshanas are: Samkhya (Kapila), Yoga (Patanjali), Nyaya (Gotama/Akshapada), Vaisheshika (Kanada), Purva Mimamsa (Jaimini, not Kumarila Bhatta — Kumarila was a later commentator), and Uttara Mimamsa/Vedanta (Badarayana/Vyasa composed the Brahma Sutras; Shankaracharya was a later commentator, not the founder). The heterodox (nastika) schools include Buddhism, Jainism, and Charvaka/Lokayata. Each school uses a distinct epistemological method (pramana).",
+        ground_truth_citations=[
+            "Sarva-Darshana-Sangraha by Madhvacharya",
+            "Brahma Sutras by Badarayana",
+            "Mimamsa Sutras by Jaimini",
+        ],
+        available_passages=[
+            "The six astika darshanas accept Vedic authority. Purva Mimamsa was founded by Jaimini (author of Mimamsa Sutras). Kumarila Bhatta and Prabhakara were later commentators, not founders.",
+            "Vedanta (Uttara Mimamsa) is based on the Brahma Sutras by Badarayana. Adi Shankaracharya (8th century CE) was the most influential commentator but not the founder of the school.",
+            "Sarva-Darshana-Sangraha by Madhvacharya surveys all philosophical schools including both astika and nastika traditions.",
+            "Charvaka (also called Lokayata) is a materialist school. Attribution to Brihaspati is traditional but historically uncertain.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["astika", "nastika", "pramana", "Jaimini", "Badarayana"],
+            required_sections=["astika schools with founders", "distinction between founder and commentator", "nastika schools", "epistemological note"],
+            expected_order=["astika schools with founders", "nastika schools", "epistemological note"],
+            banned_terms=["Kumarila Bhatta founded Mimamsa", "Shankaracharya founded Vedanta"],
+        ),
+        structural_hints=[
+            "Distinguish between original founders and later commentators.",
+            "Mention epistemological methods (pramanas) as a unifying thread.",
+            "Mimamsa founder is Jaimini, Vedanta founder is Badarayana.",
+        ],
+    ),
+    Scenario(
+        question="Describe the concept of Rta in the Rigveda.",
+        given_answer="Rta is the Rigvedic concept of cosmic order and truth that governs the universe. It is maintained by the god Indra, who is called Rtavan (possessor of Rta). The rivers flow, seasons change, and dawn appears because of Rta. Rta is the precursor to the later concept of Dharma. Varuna has no specific connection to Rta.",
+        ground_truth_answer="Rta is the cosmic order governing natural and moral law in the Rigveda. Varuna (not Indra) is the principal guardian of Rta, called Rtasya Gopah (guardian of Rta). While Indra is important in the Rigveda, Rta's custodianship belongs to Varuna and to a lesser extent Mitra. Rta governs natural phenomena and ethical conduct, and is indeed the precursor to Dharma. The concept bridges cosmic regularity with human moral obligation.",
+        ground_truth_citations=[
+            "Rigveda 1.24.8",
+            "Rigveda 7.87 (Varuna hymns)",
+        ],
+        available_passages=[
+            "Rigveda 1.24.8: Varuna is praised as the upholder of Rta, the cosmic moral order.",
+            "Rigveda 7.87: Hymns to Varuna describe him as Rtasya Gopah — the guardian of cosmic truth and order.",
+            "Rta in the Rigveda encompasses both natural law (the regularity of cosmic phenomena) and moral law (truth and righteous conduct). It later evolved into the concept of Dharma.",
+            "Indra is celebrated primarily as a warrior god (Vritrahan) and lord of storms, not as the guardian of Rta.",
+        ],
+        answer_is_correct=False,
+        factual_is_correct=False,
+        structural_meta=StructuralMeta(
+            required_terms=["Rta", "Varuna", "Rtasya Gopah", "Dharma"],
+            required_sections=["definition of Rta", "guardian deity", "natural law aspect", "moral law aspect", "evolution to Dharma"],
+            expected_order=["definition of Rta", "guardian deity", "natural law aspect", "moral law aspect", "evolution to Dharma"],
+            banned_terms=["Indra maintains Rta", "Varuna has no connection"],
+        ),
+        structural_hints=[
+            "Distinguish natural-law and moral-law dimensions of Rta.",
+            "Varuna is the guardian, not Indra.",
+            "End with the Rta → Dharma evolution.",
+        ],
+    ),
+]
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+# Task registry
+# ═══════════════════════════════════════════════════════════════════════════════
+
+@dataclasses.dataclass
+class TaskConfig:
+    name: str
+    description: str
+    max_steps: int
+    scenarios: List[Scenario]
+
+
+TASKS: Dict[str, TaskConfig] = {
+    "verify-factual": TaskConfig(
+        name="verify-factual",
+        description="Verify whether a given answer about Indic scriptures is factually correct and structurally sound. Accept correct answers, reject or edit incorrect ones.",
+        max_steps=5,
+        scenarios=VERIFY_FACTUAL_SCENARIOS,
+    ),
+    "correct-and-cite": TaskConfig(
+        name="correct-and-cite",
+        description="Improve a partially correct answer by fixing factual gaps, adding citations, and restructuring for coherence and proper terminology.",
+        max_steps=8,
+        scenarios=CORRECT_AND_CITE_SCENARIOS,
+    ),
+    "fix-hallucination": TaskConfig(
+        name="fix-hallucination",
+        description="Detect and correct subtle hallucinations, fix semantic structure, eliminate terminology errors, and ensure logical narrative flow.",
+        max_steps=12,
+        scenarios=FIX_HALLUCINATION_SCENARIOS,
+    ),
+}