Upload 12 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+static/ruc-logo.png filter=lfs diff=lfs merge=lfs -text

Dockerfile

@@ -0,0 +1,21 @@
+FROM python:3.9
+
+WORKDIR /code
+
+COPY ./requirements.txt /code/requirements.txt
+
+# Set cache directory permissions to prevent HF errors
+RUN mkdir -p /code/cache && chmod 777 /code/cache
+ENV HF_HOME=/code/cache
+
+RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
+
+COPY . .
+
+# Ensure submissions directory exists and is writable
+RUN mkdir -p /code/submissions && chmod 777 /code/submissions
+
+# Expose port 7860
+EXPOSE 7860
+
+CMD ["gunicorn", "-b", "0.0.0.0:7860", "app:app"]

README.md

@@ -1,10 +1,138 @@
 ---
 title: DISBench Leaderboard
-emoji: 📉
-colorFrom: red
-colorTo: yellow
+emoji: 🏆
+colorFrom: yellow
+colorTo: red
 sdk: docker
-pinned: false
+app_port: 7860
+pinned: true
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# 🏆 DISBench Leaderboard
+
+Welcome to the official leaderboard for **DISBench (DeepImageSearch Benchmark)**!
+
+DISBench is a comprehensive benchmark for evaluating DeepImageSearch methods on photo collections. This leaderboard tracks and compares the performance of various approaches on standardized evaluation metrics.
+
+## 📊 Evaluation Metrics
+
+### Core Metrics
+
+- **Exact Match (EM)**: Percentage of queries where the predicted photo set exactly matches the ground truth
+- **F1 Score**: Harmonic mean of precision and recall at the set level
+
+### Query Types
+
+1. **Intra-Event**: Search within a single event or time period (e.g., "sunset photos from our beach vacation")
+2. **Inter-Event**: Search across multiple events or time periods (e.g., "all birthday party photos from last year")
+
+### Tracks
+
+- **Standard Track**: Uses predefined constraints and standard model configurations
+- **Open Track**: Allows custom models, additional training data, and external resources
+
+All metrics are reported as:
+- Overall (all queries)
+- Intra-event only
+- Inter-event only
+
+## 🚀 How to Submit
+
+### Step-by-Step Guide
+
+1. **Prepare Your Results**
+   - Run your method on the DISBench test set
+   - Format predictions according to the submission schema (see below)
+
+2. **Submit via Web Interface**
+   - Navigate to the **Submit** tab on this Space
+   - Upload your JSON file containing metadata and predictions
+   - Click "Submit"
+
+3. **Automated Processing**
+   - The system validates your submission format
+   - A Pull Request is automatically created
+   - Maintainers review the submission
+
+4. **Leaderboard Update**
+   - Once approved and merged, the Space automatically rebuilds
+   - Your results are evaluated against ground truth
+   - The leaderboard updates with your scores
+
+### Submission Format
+
+```json
+{
+  "meta": {
+    "method_name": "Your Method Name",
+    "organization": "Your Organization",
+    "track": "Standard",
+    "agent_framework": "Your Agent Framework (if applicable)",
+    "backbone_model": "Your Backbone Model",
+    "retriever_model": "Your Retriever Model (if applicable)",
+    "project_url": "https://github.com/your-repo"
+  },
+  "predictions": {
+    "1": ["photo_id_1", "photo_id_2", "photo_id_3"],
+    "2": ["photo_id_4"],
+    "3": ["photo_id_5", "photo_id_6"],
+    ...
+  }
+}
+```
+
+### Field Descriptions
+
+**Meta Fields:**
+- `method_name` (required): Name of your method/system
+- `organization` (optional): Your institution or organization
+- `track` (required): Either "Standard" or "Open"
+- `agent_framework` (optional): Agent framework used (e.g., "ReAct", "AutoGPT")
+- `backbone_model` (required): Core model used (e.g., "GPT-4", "Claude-3")
+- `retriever_model` (optional): Retrieval model used (e.g., "CLIP-ViT-L/14", "BM25")
+- `project_url` (optional): Link to your project/paper
+
+**Predictions:**
+- Keys are query IDs (as strings)
+- Values are arrays of photo IDs (as strings)
+- Photo IDs should match those in the ground truth dataset
+
+## 📋 Leaderboard Rules
+
+### Uniqueness & Deduplication
+
+Each entry is uniquely identified by the combination of:
+- Method name
+- Agent framework
+- Backbone model
+- Retriever model
+- Track
+
+If you submit multiple times with the same configuration, only the **latest submission** will appear on the leaderboard.
+
+### Ranking
+
+Entries are ranked by **Overall EM Score** in descending order. The leaderboard displays:
+- Overall EM & F1
+- Intra-event EM & F1
+- Inter-event EM & F1
+
+### Separate Tracks
+
+Standard and Open track submissions are ranked separately to ensure fair comparison.
+
+## 📄 Citation
+
+If you use DISBench in your research, please cite:
+
+```bibtex
+@misc{deng2026deepimagesearchbenchmarkingmultimodalagents,
+  title={DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories}, 
+  author={Chenlong Deng and Mengjie Deng and Junjie Wu and Dun Zeng and Teng Wang and Qingsong Xie and Jiadeng Huang and Shengjie Ma and Changwang Zhang and Zhaoxiang Wang and Jun Wang and Yutao Zhu and Zhicheng Dou},
+  year={2026},
+  eprint={2602.10809},
+  archivePrefix={arXiv},
+  primaryClass={cs.CV},
+  url={https://arxiv.org/abs/2602.10809}
+}
+```

app.py

@@ -0,0 +1,239 @@
+"""
+app.py - DISBench Leaderboard Main Application
+
+Startup Flow:
+  1. Space rebuild (triggered by PR merge) → Docker container starts
+  2. Call evaluate.run_evaluation() to scan new submissions in submissions/
+  3. Calculate EM/F1 scores for new submissions, update leaderboard_data.json
+  4. Commit updated data back to repository (persistence)
+  5. Start Flask Web server
+"""
+
+import os
+import json
+import logging
+from datetime import datetime
+from flask import Flask, render_template, request, redirect, url_for, jsonify
+from huggingface_hub import HfApi, CommitOperationAdd
+
+# Evaluation module
+from evaluate import run_evaluation, commit_leaderboard_to_repo
+
+logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
+logger = logging.getLogger(__name__)
+
+app = Flask(__name__)
+app.secret_key = os.environ.get("SECRET_KEY", "disbench-leaderboard-secret-key")
+
+# --- Configuration ---
+LEADERBOARD_FILE = "leaderboard_data.json"
+SUBMISSIONS_DIR = "submissions"
+os.makedirs(SUBMISSIONS_DIR, exist_ok=True)
+
+# HuggingFace Space configuration
+HF_TOKEN = os.environ.get("HF_TOKEN")
+SPACE_ID = os.environ.get("SPACE_ID", "RUC-NLPIR/DISBench-Leaderboard")
+
+
+# ============================================================
+# Automatic Evaluation on Startup
+# ============================================================
+
+def startup_evaluation():
+    """
+    Automatically run evaluation when the app starts.
+
+    When maintainers merge a PR containing new submission files,
+    HF Space will automatically rebuild and restart, and this function will be called:
+      - Scan all files in submissions/ directory
+      - Re-evaluate all submissions (deduplicate using configuration combinations)
+      - Compare with groundtruth.jsonl to calculate scores
+      - Update leaderboard_data.json
+      - Commit results back to repository for persistence
+    
+    Note:
+      - Every startup re-evaluates all files, making the logic simpler
+      - submissions/ is the single source of truth
+      - Evaluation is fast and won't affect startup speed
+    """
+    logger.info("=" * 60)
+    logger.info("DISBench: Running startup evaluation...")
+    logger.info("=" * 60)
+
+    try:
+        total, _ = run_evaluation()
+
+        if total > 0:
+            logger.info(f"Evaluated all submissions. Committing to repo...")
+            commit_leaderboard_to_repo()
+        else:
+            logger.info("No submissions found.")
+
+        logger.info(f"Leaderboard has {total} unique configurations. Ready to serve.")
+
+    except Exception as e:
+        logger.error(f"Startup evaluation failed: {e}")
+        logger.info("Continuing with existing leaderboard data...")
+
+
+# Execute startup evaluation
+startup_evaluation()
+
+
+# ============================================================
+# Data Loading
+# ============================================================
+
+def load_leaderboard():
+    if os.path.exists(LEADERBOARD_FILE):
+        with open(LEADERBOARD_FILE, 'r', encoding='utf-8') as f:
+            return json.load(f)
+    return []
+
+
+# ============================================================
+# Submission Validation
+# ============================================================
+
+def validate_submission(submission):
+    errors = []
+    if not isinstance(submission, dict):
+        return ["Submission must be a JSON object with 'meta' and 'predictions' fields."]
+
+    meta = submission.get("meta")
+    preds = submission.get("predictions")
+
+    if not meta or not isinstance(meta, dict):
+        errors.append("Missing or invalid 'meta' field.")
+    else:
+        required_meta = ["method_name"]
+        for field in required_meta:
+            if field not in meta:
+                errors.append(f"Missing required field: meta.{field}")
+
+        valid_tracks = ["Standard", "Open"]
+        if meta.get("track") and meta["track"] not in valid_tracks:
+            errors.append(f"meta.track must be one of: {valid_tracks}")
+
+    if not preds or not isinstance(preds, dict):
+        errors.append("Missing or invalid 'predictions' field.")
+
+    return errors
+
+
+# ============================================================
+# PR Creation
+# ============================================================
+
+def create_pr_submission(submission_json, method_name):
+    """Create a PR via HF Hub API, upload submission file to submissions/ directory"""
+    if not HF_TOKEN:
+        raise RuntimeError(
+            "HF_TOKEN not configured. Please set the HF_TOKEN secret in your Space settings."
+        )
+
+    api = HfApi(token=HF_TOKEN)
+
+    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+    safe_name = method_name.replace(" ", "-").replace("/", "_")
+    filename = f"{safe_name}_{timestamp}.json"
+    path_in_repo = f"submissions/{filename}"
+
+    content = json.dumps(submission_json, indent=2, ensure_ascii=False).encode("utf-8")
+
+    commit_info = api.create_commit(
+        repo_id=SPACE_ID,
+        repo_type="space",
+        operations=[
+            CommitOperationAdd(
+                path_in_repo=path_in_repo,
+                path_or_fileobj=content,
+            )
+        ],
+        commit_message=f"[Submission] Add results for {method_name}",
+        commit_description=(
+            f"**Method**: {method_name}\n"
+            f"**Organization**: {submission_json.get('meta', {}).get('organization', 'N/A')}\n"
+            f"**Track**: {submission_json.get('meta', {}).get('track', 'N/A')}\n"
+            f"**Agent**: {submission_json.get('meta', {}).get('agent_framework', 'N/A')}\n"
+            f"**Backbone**: {submission_json.get('meta', {}).get('backbone_model', 'N/A')}\n"
+            f"**Retriever**: {submission_json.get('meta', {}).get('retriever_model', 'N/A')}\n\n"
+            f"Submitted at {datetime.now().strftime('%Y-%m-%d %H:%M:%S UTC')}"
+        ),
+        create_pr=True,
+    )
+
+    return commit_info
+
+
+# ============================================================
+# Routes
+# ============================================================
+
+@app.route('/')
+def index():
+    data = load_leaderboard()
+    return render_template('index.html', data=data)
+
+
+@app.route('/upload', methods=['POST'])
+def upload_file():
+    """Handle submission: validate → create PR → return result"""
+    if 'file' not in request.files:
+        return jsonify({"success": False, "error": "No file uploaded."}), 400
+
+    file = request.files['file']
+    if file.filename == '':
+        return jsonify({"success": False, "error": "No file selected."}), 400
+
+    try:
+        submission = json.load(file)
+    except json.JSONDecodeError as e:
+        return jsonify({"success": False, "error": f"Invalid JSON file: {e}"}), 400
+
+    errors = validate_submission(submission)
+    if errors:
+        return jsonify({"success": False, "error": "Validation failed.", "details": errors}), 400
+
+    method_name = submission["meta"]["method_name"]
+
+    # Local backup
+    safe_name = method_name.replace(" ", "-").replace("/", "_")
+    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+    local_path = os.path.join(SUBMISSIONS_DIR, f"{safe_name}_{timestamp}.json")
+    with open(local_path, 'w', encoding='utf-8') as f:
+        json.dump(submission, f, indent=2, ensure_ascii=False)
+
+    # Create PR
+    try:
+        commit_info = create_pr_submission(submission, method_name)
+        pr_url = getattr(commit_info, 'pr_url', None)
+        return jsonify({
+            "success": True,
+            "message": f"Submission for '{method_name}' has been submitted as a Pull Request!",
+            "pr_url": pr_url or f"https://huggingface.co/spaces/{SPACE_ID}/discussions",
+        })
+    except RuntimeError as e:
+        return jsonify({
+            "success": True,
+            "message": (
+                f"Submission for '{method_name}' saved locally. "
+                f"PR creation skipped: {str(e)}. "
+                f"Maintainers will review it manually."
+            ),
+            "pr_url": None,
+        })
+    except Exception as e:
+        return jsonify({
+            "success": True,
+            "message": (
+                f"Submission for '{method_name}' saved locally, "
+                f"but PR creation failed: {str(e)}. "
+                f"Please contact the maintainers."
+            ),
+            "pr_url": None,
+        })
+
+
+if __name__ == '__main__':
+    app.run(debug=False, host="0.0.0.0", port=7860)

evaluate.py

@@ -0,0 +1,371 @@
+"""
+evaluate.py - DISBench Evaluation Script
+
+Responsibilities:
+  1. Scan all JSON submission files in submissions/ directory
+  2. For each unevaluated submission, compare with groundtruth.jsonl to calculate scores
+  3. Append new results to leaderboard_data.json
+  4. (Optional) Commit updated leaderboard_data.json back to HF repository
+
+Execution:
+  - Automatic: Called when app.py starts (automatically triggered on Space rebuild)
+  - Manual: python evaluate.py
+"""
+
+import os
+import json
+import logging
+from datetime import datetime
+from typing import Dict, List, Set, Tuple, Optional
+
+logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
+logger = logging.getLogger(__name__)
+
+# --- Path Configuration ---
+SUBMISSIONS_DIR = "submissions"
+GROUND_TRUTH_FILE = "groundtruth.jsonl"
+LEADERBOARD_FILE = "leaderboard_data.json"
+
+
+# ============================================================
+# Core Evaluation Logic
+# ============================================================
+
+def compute_em(predicted: Set[str], gold: Set[str]) -> float:
+    """Exact Match: returns 1 if predicted set exactly matches gold set, otherwise 0"""
+    return 1.0 if predicted == gold else 0.0
+
+
+def compute_f1(predicted: Set[str], gold: Set[str]) -> float:
+    """F1 Score: harmonic mean of set-based precision and recall"""
+    if not predicted and not gold:
+        return 1.0
+    if not predicted or not gold:
+        return 0.0
+
+    tp = len(predicted & gold)
+    precision = tp / len(predicted)
+    recall = tp / len(gold)
+
+    if precision + recall == 0:
+        return 0.0
+    return 2 * precision * recall / (precision + recall)
+
+
+def load_ground_truth() -> Dict:
+    """
+    Load ground truth file (JSONL format).
+    
+    Input format (groundtruth.jsonl):
+    Each line is a JSON object:
+    {
+        "query_id": "1",
+        "user_id": "...",
+        "query": "...",
+        "answer": ["photo_id_1", "photo_id_2"],
+        "event_type": "intra-event"  // "intra-event" or "inter-event"
+    }
+    
+    Converted to internal format:
+    {
+        "queries": {
+            "1": {
+                "type": "intra",          // "intra" or "inter"
+                "gold_photos": ["photo_id_1", "photo_id_2"]
+            },
+            ...
+        }
+    }
+    """
+    if not os.path.exists(GROUND_TRUTH_FILE):
+        logger.warning(f"Ground truth file not found: {GROUND_TRUTH_FILE}")
+        return {}
+
+    queries = {}
+    with open(GROUND_TRUTH_FILE, 'r', encoding='utf-8') as f:
+        for line_num, line in enumerate(f, 1):
+            line = line.strip()
+            if not line:
+                continue
+            
+            try:
+                entry = json.loads(line)
+                query_id = entry.get("query_id")
+                answer = entry.get("answer", [])
+                event_type = entry.get("event_type", "intra-event")
+                
+                # Convert event_type: "intra-event" -> "intra", "inter-event" -> "inter"
+                query_type = event_type.replace("-event", "")
+                
+                queries[query_id] = {
+                    "type": query_type,
+                    "gold_photos": answer
+                }
+            except json.JSONDecodeError as e:
+                logger.warning(f"Invalid JSON at line {line_num}: {e}")
+                continue
+            except Exception as e:
+                logger.warning(f"Error processing line {line_num}: {e}")
+                continue
+    
+    return {"queries": queries}
+
+
+def evaluate_predictions(
+    predictions: Dict[str, List[str]],
+    ground_truth: Dict
+) -> Dict[str, float]:
+    """
+    Calculate all metrics for a submission's predictions.
+    
+    Returns:
+    {
+        "overall_em": float,
+        "overall_f1": float,
+        "intra_em": float,
+        "intra_f1": float,
+        "inter_em": float,
+        "inter_f1": float
+    }
+    """
+    queries = ground_truth.get("queries", {})
+
+    if not queries:
+        logger.warning("Ground truth has no queries, returning zeros.")
+        return {
+            "overall_em": 0.0, "overall_f1": 0.0,
+            "intra_em": 0.0, "intra_f1": 0.0,
+            "inter_em": 0.0, "inter_f1": 0.0,
+        }
+
+    # Collect scores by type
+    scores_by_type = {"intra": {"em": [], "f1": []}, "inter": {"em": [], "f1": []}}
+    all_em, all_f1 = [], []
+
+    for query_id, query_info in queries.items():
+        gold_set = set(query_info.get("gold_photos", []))
+        pred_set = set(predictions.get(query_id, []))
+        query_type = query_info.get("type", "intra")  # Default to intra
+
+        em = compute_em(pred_set, gold_set)
+        f1 = compute_f1(pred_set, gold_set)
+
+        all_em.append(em)
+        all_f1.append(f1)
+
+        if query_type in scores_by_type:
+            scores_by_type[query_type]["em"].append(em)
+            scores_by_type[query_type]["f1"].append(f1)
+
+    def safe_mean(lst):
+        return round(sum(lst) / len(lst) * 100, 1) if lst else 0.0
+
+    return {
+        "overall_em": safe_mean(all_em),
+        "overall_f1": safe_mean(all_f1),
+        "intra_em": safe_mean(scores_by_type["intra"]["em"]),
+        "intra_f1": safe_mean(scores_by_type["intra"]["f1"]),
+        "inter_em": safe_mean(scores_by_type["inter"]["em"]),
+        "inter_f1": safe_mean(scores_by_type["inter"]["f1"]),
+    }
+
+
+# ============================================================
+# Submission Management
+# ============================================================
+
+def get_entry_key(entry: Dict) -> Tuple:
+    """
+    Generate unique identifier key for an entry.
+    
+    The same method may have multiple different configurations (different backbone, retriever, etc.),
+    Only when all key configuration fields are the same, they are considered the same submission.
+    
+    Returns: (method, agent, backbone, retriever, track)
+    """
+    return (
+        entry.get("method", ""),
+        entry.get("agent", ""),
+        entry.get("backbone", ""),
+        entry.get("retriever", ""),
+        entry.get("track", "Standard"),
+    )
+
+
+def load_leaderboard() -> list:
+    if os.path.exists(LEADERBOARD_FILE):
+        with open(LEADERBOARD_FILE, 'r', encoding='utf-8') as f:
+            return json.load(f)
+    return []
+
+
+def save_leaderboard(data: list):
+    with open(LEADERBOARD_FILE, 'w', encoding='utf-8') as f:
+        json.dump(data, f, indent=2, ensure_ascii=False)
+
+
+def process_submission(filepath: str, ground_truth: Dict) -> Optional[Dict]:
+    """
+    Process a single submission file, return leaderboard entry (or None if error).
+    """
+    try:
+        with open(filepath, 'r', encoding='utf-8') as f:
+            submission = json.load(f)
+
+        meta = submission.get("meta", {})
+        predictions = submission.get("predictions", {})
+
+        if not meta.get("method_name"):
+            logger.warning(f"Skipping {filepath}: missing method_name")
+            return None
+
+        if not predictions:
+            logger.warning(f"Skipping {filepath}: empty predictions")
+            return None
+
+        # Calculate scores
+        scores = evaluate_predictions(predictions, ground_truth)
+
+        entry = {
+            "method": meta.get("method_name", "Unknown"),
+            "url": meta.get("project_url", "#"),
+            "org": meta.get("organization", "Anonymous"),
+            "agent": meta.get("agent_framework", "Unknown"),
+            "backbone": meta.get("backbone_model", "Unknown"),
+            "retriever": meta.get("retriever_model", "Unknown"),
+            "track": meta.get("track", "Standard"),
+            "date": datetime.now().strftime("%Y-%m-%d"),
+            **scores,
+        }
+
+        logger.info(
+            f"Evaluated '{entry['method']}': "
+            f"Overall EM={scores['overall_em']}, F1={scores['overall_f1']}"
+        )
+        return entry
+
+    except Exception as e:
+        logger.error(f"Error processing {filepath}: {e}")
+        return None
+
+
+def run_evaluation():
+    """
+    Main evaluation pipeline:
+      1. Load ground truth
+      2. Scan all files in submissions/ and re-evaluate
+      3. Deduplicate using configuration combinations (method, agent, backbone, retriever, track)
+      4. If multiple submissions exist for the same configuration, keep the latest (sorted by filename, last file is considered latest)
+      5. Return (number of entries, total entries)
+    
+    Notes:
+      - No evaluated.json is maintained, all files are re-evaluated on each startup
+      - submissions/ is the single source of truth
+      - Benefits: simple logic, no state inconsistency, automatic recalculation when evaluation logic changes
+    """
+    # 1. Load ground truth
+    ground_truth = load_ground_truth()
+    if not ground_truth:
+        logger.info("No ground truth file found. Skipping evaluation.")
+        return 0, 0
+
+    # 2. Scan all submissions and evaluate
+    if not os.path.exists(SUBMISSIONS_DIR):
+        logger.info("No submissions directory found.")
+        return 0, 0
+
+    # Use dictionary to store: key is configuration tuple, value is (entry, filename)
+    # If multiple submissions exist for the same configuration, later evaluated ones will overwrite earlier ones (keep latest)
+    entries_by_config = {}
+
+    for filename in sorted(os.listdir(SUBMISSIONS_DIR)):
+        if not filename.endswith(".json"):
+            continue
+
+        filepath = os.path.join(SUBMISSIONS_DIR, filename)
+        logger.info(f"Processing submission: {filename}")
+
+        entry = process_submission(filepath, ground_truth)
+        if entry is not None:
+            config_key = get_entry_key(entry)
+            
+            # If this configuration already exists, it means there's a duplicate submission, replace old with new
+            if config_key in entries_by_config:
+                old_filename = entries_by_config[config_key][1]
+                logger.info(
+                    f"Config {config_key} already exists (from {old_filename}), "
+                    f"replacing with {filename}"
+                )
+            
+            entries_by_config[config_key] = (entry, filename)
+
+    # 3. Extract all unique entries
+    leaderboard = [entry for entry, _ in entries_by_config.values()]
+
+    # 4. Save results
+    save_leaderboard(leaderboard)
+    logger.info(f"Leaderboard updated: {len(leaderboard)} unique configurations.")
+
+    return len(leaderboard), len(leaderboard)
+
+
+def commit_leaderboard_to_repo():
+    """
+    (Optional) Commit the updated leaderboard_data.json back to HF repository,
+    to persist data (avoid re-evaluation on every restart).
+    
+    Note: We no longer commit evaluated.json, as we re-evaluate from submissions/ on each startup.
+    """
+    hf_token = os.environ.get("HF_TOKEN")
+    space_id = os.environ.get("SPACE_ID")
+
+    if not hf_token or not space_id:
+        logger.info("HF_TOKEN or SPACE_ID not set, skipping repo commit.")
+        return
+
+    try:
+        from huggingface_hub import HfApi, CommitOperationAdd
+
+        api = HfApi(token=hf_token)
+
+        # Only commit leaderboard_data.json
+        if not os.path.exists(LEADERBOARD_FILE):
+            logger.warning(f"Leaderboard file {LEADERBOARD_FILE} not found, skipping commit.")
+            return
+
+        with open(LEADERBOARD_FILE, 'rb') as f:
+            api.create_commit(
+                repo_id=space_id,
+                repo_type="space",
+                operations=[
+                    CommitOperationAdd(
+                        path_in_repo=LEADERBOARD_FILE,
+                        path_or_fileobj=f.read(),
+                    )
+                ],
+                commit_message="[Auto] Update leaderboard scores",
+            )
+            logger.info("Leaderboard committed to repo successfully.")
+
+    except Exception as e:
+        logger.error(f"Failed to commit to repo: {e}")
+
+
+# ============================================================
+# Entry Point
+# ============================================================
+
+if __name__ == "__main__":
+    logger.info("=" * 60)
+    logger.info("DISBench Evaluation Pipeline - Manual Run")
+    logger.info("=" * 60)
+
+    total, _ = run_evaluation()
+
+    if total > 0:
+        logger.info(f"Evaluated all submissions. Committing to repo...")
+        commit_leaderboard_to_repo()
+    else:
+        logger.info("No submissions found.")
+
+    logger.info(f"Leaderboard has {total} unique configurations.")

groundtruth.jsonl

@@ -0,0 +1,122 @@
+{"query_id": "1", "user_id": "10287726@N02", "query": "Find photos from the musical performance identified by the blue and white event logo on site, where only the lead singer appears on stage.", "answer": ["7759256930", "7759407170", "7759295108", "7759433016"], "event_type": "intra-event"}
+{"query_id": "2", "user_id": "10287726@N02", "query": "Find photos taken while watching that match of body contact competition without equipment, which contain the scoreboard showing a contest between two purely Asian countries.", "answer": ["7859417230", "7859414230", "7859412098"], "event_type": "intra-event"}
+{"query_id": "3", "user_id": "10297518@N00", "query": "Find the photo taken on the day when the wall covered with drawings of fresh ingredients was seen, that contains metal chairs.", "answer": ["12678291825"], "event_type": "intra-event"}
+{"query_id": "4", "user_id": "10297518@N00", "query": "All photos taken during the calendar week when a foggy cityscape was photographed in a western North American city at dusk, excluding those containing wine bottles.", "answer": ["12345034875", "12345198653", "12345043105", "12345208993", "12345510174", "12345514604"], "event_type": "intra-event"}
+{"query_id": "5", "user_id": "10299779@N03", "query": "Find all indoor photos taken during the city hall visit where you saw the nude statue commemorating a cultural figure of that country, which contain at least one complete European classical lamp.", "answer": ["9687658675", "9690897644", "9690945632", "9687710449"], "event_type": "intra-event"}
+{"query_id": "6", "user_id": "10299779@N03", "query": "Find all photos of flowers that do not contain purple flowers, taken on the day when the main building of the locally famous ship repair and mechanical engineering company whose name starts with the letter \"K\" was seen.", "answer": ["10413130075", "10413006546", "10413168333", "10412794946", "10413195113", "10412848095", "10412814313"], "event_type": "intra-event"}
+{"query_id": "7", "user_id": "12276997@N06", "query": "I took many photos of a certain animal at Mirador de La Antigua; find similar photos of this animal that I took at Torrejón el Rubio.", "answer": ["4849540315", "4849543217", "4850192364", "4850188322", "4850192226"], "event_type": "inter-event"}
+{"query_id": "8", "user_id": "12276997@N06", "query": "Find photos of cows taken on the day I photographed a group of the animal that I took many photos of at Mirador de La Antigua.", "answer": ["4064083683", "4064082669", "4064820510", "4064824294"], "event_type": "inter-event"}
+{"query_id": "9", "user_id": "12276997@N06", "query": "Find photos providing a closer view of the spire-topped building that appears in the distance in the first photo taken on April 18, 2014.", "answer": ["13956729065", "13933619246"], "event_type": "inter-event"}
+{"query_id": "10", "user_id": "12734746@N00", "query": "Find all photos of C-3PO taken on the day when Micky was not photographed.", "answer": ["5740173261", "5740173785"], "event_type": "intra-event"}
+{"query_id": "11", "user_id": "12734746@N00", "query": "Find all photos of Darth Maul taken on the day when both Ahsoka and Darth Maul were photographed.", "answer": ["5977807515", "5977840951", "5872342937"], "event_type": "intra-event"}
+{"query_id": "12", "user_id": "12734746@N00", "query": "Find all photos of the Lego-built United States Capitol taken on the day during the Legoland event when I photographed many Lego-built U.S. presidents.", "answer": ["4531838972", "4531206035", "4531205283"], "event_type": "intra-event"}
+{"query_id": "13", "user_id": "13101981@N03", "query": "Find photos of the Sydney Harbour Bridge at night taken on a day several months before I watched fireworks at the Potts Point apartment, and on that day I also went to a bar.", "answer": ["5770246050", "5769706101", "5770247096", "5769704353", "5769702669"], "event_type": "intra-event"}
+{"query_id": "14", "user_id": "13101981@N03", "query": "Find all photos of the Sydney Opera House taken on the day I visited the exhibition featuring hanging T-shirts.", "answer": ["6529354413", "6529356351", "6529392021"], "event_type": "intra-event"}
+{"query_id": "15", "user_id": "14580956@N08", "query": "Find side-view photos of the large statue of the animated character that appears as a small statue in the photos where I was behind the TV-shaped photo frame.", "answer": ["11330906063", "11332580254"], "event_type": "inter-event"}
+{"query_id": "16", "user_id": "14580956@N08", "query": "Find all photos of animal sculptures wearing props around their necks that were taken on the day when the orangutan was playing with a red prop.", "answer": ["11654699444", "11654633534"], "event_type": "intra-event"}
+{"query_id": "17", "user_id": "14580956@N08", "query": "Find all photos of tigers that I took in early April of the same year when I photographed a tiger yawning with its mouth wide open.", "answer": ["13748820094", "13748673695", "13748447223", "13749066994"], "event_type": "inter-event"}
+{"query_id": "18", "user_id": "14737255@N00", "query": "Find photos of Richard Müller standing almost completely sideways during the Open Air Festival 2010.", "answer": ["4901394195"], "event_type": "intra-event"}
+{"query_id": "19", "user_id": "14737255@N00", "query": "Find photos of the lead singer of Faithless holding a microphone and singing during the Open Air Festival 2010.", "answer": ["4902747962", "4902161109", "4902160123", "4902744724", "4902157461", "4902156713"], "event_type": "intra-event"}
+{"query_id": "20", "user_id": "14737255@N00", "query": "Find all close-up or medium photos of human statues taken during the 2009 Paris trip before the first photo of a Van Gogh painting was taken.", "answer": ["3906532981", "3906533767"], "event_type": "intra-event"}
+{"query_id": "21", "user_id": "14798958@N08", "query": "Find all photos in which a building with \"1926\" on the lintel appears, even if only partially visible.", "answer": ["12713450225", "12713612803", "12713424065", "12713611993"], "event_type": "intra-event"}
+{"query_id": "22", "user_id": "14798958@N08", "query": "Identify the person who posed with a triangulation station in both December 2013 and February 2014, and retrieve the photo of them with the station from the December 2013 event.", "answer": ["11308410435", "11670823024", "11670470795", "11670806594"], "event_type": "inter-event"}
+{"query_id": "23", "user_id": "14798958@N08", "query": "Find all solo photos of the same person with the triangulation station taken on both December 7 and December 30, 2013.", "answer": ["11670442965", "11308452886", "11308450456"], "event_type": "inter-event"}
+{"query_id": "24", "user_id": "15352839@N00", "query": "Find the photo of that large cruise ship, which was photographed in Chile, later docking in another country.", "answer": ["5500698550"], "event_type": "inter-event"}
+{"query_id": "25", "user_id": "15352839@N00", "query": "Find all photos of street performers taken in the city where the centennial monument was photographed.", "answer": ["5500929668", "5500334405"], "event_type": "intra-event"}
+{"query_id": "26", "user_id": "15352839@N00", "query": "Find all photos of anchor sculptures taken in the city where the centennial monument was photographed.", "answer": ["5500340243", "5500927560", "5500927690"], "event_type": "intra-event"}
+{"query_id": "27", "user_id": "15803691@N00", "query": "Find photos of road signs taken on the way to the Zen Debris concert.", "answer": ["199242541", "199242771", "199242963"], "event_type": "intra-event"}
+{"query_id": "28", "user_id": "15803691@N00", "query": "Find photos where a scarf that appeared in the 2004 DCC RHPS performance appears again at a later event.", "answer": ["281594862", "281593571", "281526899"], "event_type": "inter-event"}
+{"query_id": "29", "user_id": "18383978@N00", "query": "Find the photo of me rock climbing alone on the day after the group photo of three people taken in Scotland.", "answer": ["5385417640"], "event_type": "intra-event"}
+{"query_id": "30", "user_id": "18383978@N00", "query": "Find photos of several men wearing the same shoes that I photographed during my trip to Southern Europe two calender years ago.", "answer": ["12017070886"], "event_type": "inter-event"}
+{"query_id": "31", "user_id": "21435131@N06", "query": "Find photos of the person who took pictures in a church during the trip to France, holding a baby.", "answer": ["6400941291", "6400940763"], "event_type": "intra-event"}
+{"query_id": "32", "user_id": "21895046@N08", "query": "Find solo photos of any child taken in a museum with dinosaur skeletons.", "answer": ["5471527175", "3571341554", "3570529059", "3571332812", "3571336630"], "event_type": "intra-event"}
+{"query_id": "33", "user_id": "21895046@N08", "query": "Find photos of the woman who has worn a heart-shaped necklace holding a newborn baby.", "answer": ["4764933969", "4764970551", "4764930661"], "event_type": "inter-event"}
+{"query_id": "34", "user_id": "22017657@N05", "query": "Find the photos of the man's watch face appeared at the event on Saturday who wears a hat on Sunday.", "answer": ["9106789483", "9106781343", "9106753625", "9108880170"], "event_type": "inter-event"}
+{"query_id": "35", "user_id": "22017657@N05", "query": "The photo of the woman wearing a light blue patterned top that was taken immediately after the photo of the lady in a bright red top at the TV drama fan meeting event.", "answer": ["9422191489"], "event_type": "intra-event"}
+{"query_id": "36", "user_id": "22017657@N05", "query": "Find the solo photos from the fan event that night in which the man who wore a T-shirt with a green logo at the TV series discussion appears.", "answer": ["9429645636", "9426974443"], "event_type": "intra-event"}
+{"query_id": "37", "user_id": "22526649@N03", "query": "Find the group photo of four people taken in front of the yellow wall with a circular pattern that appeared at the Economic Club.", "answer": ["13568401013"], "event_type": "intra-event"}
+{"query_id": "38", "user_id": "22736462@N07", "query": "Check if there is anyone who has both ridden a bike and performed as DJ; if so, find photos of them playing DJ.", "answer": ["4213476097", "4214221364", "4213455031", "4213454895", "4214221174", "4214242858", "4214242796", "4214242702"], "event_type": "inter-event"}
+{"query_id": "39", "user_id": "22736462@N07", "query": "Find photos from a birthday party held in a room with calligraphy or paintings hanging on the wall, where the birthday person is holding a wine glass.", "answer": ["3456340318", "3456340218", "3455521861"], "event_type": "intra-event"}
+{"query_id": "40", "user_id": "22736462@N07", "query": "Find all photos from the hiking trip in the western United States containing the woman who wore multiple necklaces at the birthday party with the large yellow floor lamp.", "answer": ["4000393384", "4000534281", "4001299644", "4001299798", "4000534371"], "event_type": "inter-event"}
+{"query_id": "41", "user_id": "23090753@N06", "query": "Find the logos found elsewhere in the venue (not on the stage) during another recent performance by the female bassist who previously performed in an enclosed shed.", "answer": ["5545472770", "8537595742"], "event_type": "inter-event"}
+{"query_id": "42", "user_id": "23518714@N00", "query": "There have been two occasions, less than 2 years apart, when photos were taken on the same day of the border signs of the same two adjacent states. Please find the border sign photos of the smaller state from those two occasions.", "answer": ["2164102074", "2163302409", "2329450116", "2329512252"], "event_type": "inter-event"}
+{"query_id": "43", "user_id": "23736466@N00", "query": "Find photos where the jersey number of the FMVP on the team playing against the free-throwing shooting guard appears in other sports.", "answer": ["2861882952", "2982419979"], "event_type": "inter-event"}
+{"query_id": "44", "user_id": "24232779@N00", "query": "There is a person wearing a red T-shirt. Find photos of this person wearing the same jacket on two occasions separated by more than 8 months.", "answer": ["13114381394", "6601840711", "6601826625"], "event_type": "inter-event"}
+{"query_id": "45", "user_id": "24413182@N00", "query": "Please help me find the photo that is a landscape shot taken on the road, captured after we photographed the person providing transportation services, but before we arrived at Aleppo Citadel.", "answer": ["4650094397", "4650711100", "4650711710", "4650093789"], "event_type": "intra-event"}
+{"query_id": "46", "user_id": "24413182@N00", "query": "Find photos taken 7 days before the photo of the elderly Maasai woman wearing a layered bead necklace. The target photos feature a vertebrate animal that relies on external heat sources to regulate its body temperature.", "answer": ["10901596755", "10901899493"], "event_type": "intra-event"}
+{"query_id": "47", "user_id": "24468935@N03", "query": "Find photos of a girl taken on the coast where the legend of Odysseus and Princess Nausicaa took place. She has appeared in another photo wearing a blue headscarf and sunglasses.", "answer": ["7769170062", "7769178796", "7769192290", "7769175414", "7769210734", "7769235402", "7769197742", "7769228788", "7769259012", "7769222512", "7769255270", "7769248762"], "event_type": "inter-event"}
+{"query_id": "48", "user_id": "24468935@N03", "query": "Find photos depicting a woman swimming next to the legendary fragment of land that, according to myth, was struck by a trident and separated from the continent. The woman also appears in another photo wearing a blue headscarf and sunglasses.", "answer": ["7767611716", "7767606504", "7767603054"], "event_type": "inter-event"}
+{"query_id": "49", "user_id": "24468935@N03", "query": "Find photos of the person doing rappelling during a rainforest canyon adventure at the edge of a volcanic belt, who previously had selfies with elephants.", "answer": ["5555244846", "5555244956", "5555244744", "5555244914", "5555244802", "5555244688", "5554657497", "5554658813", "5554657587", "5554658759", "5554658551", "5555246166", "5555246112", "5554658467", "5555246296", "5554657545", "5555246004", "5554658427", "5554658587", "5554657643"], "event_type": "inter-event"}
+{"query_id": "50", "user_id": "24736216@N07", "query": "Find the photos I took during the Caribbean cruise that contain animals which closely resemble the animal depicted on one of my souvenir T-shirts.", "answer": ["5726408412", "5726408366", "5725852439"], "event_type": "inter-event"}
+{"query_id": "51", "user_id": "24736216@N07", "query": "Find an interface similar to the one displayed on the LG screen.", "answer": ["12933192564"], "event_type": "inter-event"}
+{"query_id": "52", "user_id": "24819841@N06", "query": "Find the photo of a woman taken at the main civic square of the United Kingdom's second largest city, next to the bronze fountain popularly known as \"The Woman in the Hot Water Bath\", where this woman had previously taken a photo in the country that was the world's second largest by territory at that time.", "answer": ["2887804961"], "event_type": "inter-event"}
+{"query_id": "53", "user_id": "25367139@N00", "query": "Find all photos taken during the January 11 match that include an advertising board of a brand which had at least five of its boards displayed consecutively along the edge of the field during a preseason friendly in the third quarter of 2013.", "answer": ["12009721833", "12010249266", "12010250456", "12009782424", "12009689213", "12009759074"], "event_type": "intra-event"}
+{"query_id": "54", "user_id": "25652622@N00", "query": "Find all group photos of four people, at any location, that include the person who wore a dark blue jacket climbing the largest glacier in the United States in 2009.", "answer": ["4857960798", "4877577777", "4095181696", "6997586514"], "event_type": "inter-event"}
+{"query_id": "55", "user_id": "25652622@N00", "query": "Identify the companion who was holding a red object during my climb of a Washington glacier named after a geologist. Then, locate all photos of this person sitting on large stones.", "answer": ["4613611446", "4613612288"], "event_type": "inter-event"}
+{"query_id": "56", "user_id": "25899413@N04", "query": "Find the man who was closest to the plastic box while resting at the Swiss ski resort at an altitude of 1675 meters. Then  get all photos of him smiling, regardless of location.", "answer": ["4514232684", "4513592917", "3504418456", "3503604671"], "event_type": "inter-event"}
+{"query_id": "57", "user_id": "25899413@N04", "query": "Locate the orange tool that appeared during the trip in Switzerland. Then, find photos from the Pragelato trip where that type of tool was put on the ground.", "answer": ["4416229191"], "event_type": "inter-event"}
+{"query_id": "58", "user_id": "27550543@N02", "query": "Is there a person who has both tried to lift a black tool over their head and has held a steering wheel? If so, try to find all photos of this person holding a wine glass.", "answer": ["3460813802", "3460788528", "3460672156"], "event_type": "inter-event"}
+{"query_id": "59", "user_id": "27550543@N02", "query": "I have photographed a type of vehicle with logo of two red animals. Please find all photos from 2011 that contain exactly two vehicles of this type.", "answer": ["6070729736", "6070183061", "6070728050", "6070183739", "6070179861", "6070739508", "6070181495", "6070193407", "6070180415"], "event_type": "intra-event"}
+{"query_id": "60", "user_id": "27634886@N00", "query": "Find all photos with the sea taken at the beach two days after watching the fireworks show.", "answer": ["6009152901", "6009707544", "6009157655"], "event_type": "intra-event"}
+{"query_id": "61", "user_id": "27634886@N00", "query": "Find all photos of the white-haired grandma at the mini golf course on the day when there was a fireworks show at night.", "answer": ["6009148499", "6009696628"], "event_type": "intra-event"}
+{"query_id": "62", "user_id": "27634886@N00", "query": "Find all photos taken of the person who wore a bathrobe on Christmas Day 2012 during his later trip to Pompei.", "answer": ["9486332346", "9476277061", "9483505769"], "event_type": "inter-event"}
+{"query_id": "63", "user_id": "27637456@N06", "query": "Find photos of the hiker with a backpack that has an orange-red logo on top, taken later when he was setting out towards the stone bridge on the road from Hontanas to Boadilla del Camino.", "answer": ["7036534497", "7036532695"], "event_type": "intra-event"}
+{"query_id": "64", "user_id": "27637456@N06", "query": "Find solo photos of the woman who posed with the fish-carved stone in Arles, taken before she saw the strange face statue during her trip to the ancient Roman seaside port city.", "answer": ["5778612789", "5778617109"], "event_type": "intra-event"}
+{"query_id": "65", "user_id": "27637456@N06", "query": "Find the photos of the hiker interacting with a dog, whose backpack has an orange-red logo in the middle.", "answer": ["8558357036", "6856070036"], "event_type": "intra-event"}
+{"query_id": "66", "user_id": "28157992@N03", "query": "Find photos of the person who posed with the ice hockey statue, specifically the ones where they are wearing a hat.", "answer": ["9896642915", "6123707828"], "event_type": "inter-event"}
+{"query_id": "67", "user_id": "28157992@N03", "query": "Find fireworks photos from my trip, taken on the day I also photographed a city view containing the bridge I crossed at customs a year prior.", "answer": ["9647794441", "9647793803", "9647795907", "9647793275", "9651031044"], "event_type": "inter-event"}
+{"query_id": "68", "user_id": "28495173@N00", "query": "Find the photos from Yeomen where the woman who helped with makeup in both Gondoliers and Utopia Limited is wearing small earrings.", "answer": ["4021141323", "4021900844"], "event_type": "intra-event"}
+{"query_id": "69", "user_id": "28495173@N00", "query": "Find all photos taken at the beginning of the year seven years after 2005 in which I photographed the same cruise ship that I captured at dusk in Otago in 2005.", "answer": ["6657554835", "6666029973", "6657006619", "6666129439", "6657056897", "6658404791", "6657058951", "6657057827", "6657055985"], "event_type": "inter-event"}
+{"query_id": "70", "user_id": "28495173@N00", "query": "Find all photos from 2008 showing ships docked at the warehouse where a grey warship was once docked.", "answer": ["6740675373", "3401390348", "3400575971"], "event_type": "inter-event"}
+{"query_id": "71", "user_id": "30872191@N00", "query": "Find photos of the female athlete who wore different glasses in the first two Brunettes practices in 2013, where she is preparing to catch the ball during the first practice.", "answer": ["9213002082", "9210216039", "9213008110"], "event_type": "inter-event"}
+{"query_id": "72", "user_id": "30872191@N00", "query": "Find the race photos of the woman wearing a headband who participated in the Walk to End Alzheimer's event in 2012 and also received a finisher medal in the Hospital Hill run in the same year.", "answer": ["7339977074", "7154879151"], "event_type": "inter-event"}
+{"query_id": "73", "user_id": "31058815@N00", "query": "Find photos of the bridge crossing the canyon near the giant bicycle sculpture in New Zealand.", "answer": ["7483049266", "7483050402"], "event_type": "intra-event"}
+{"query_id": "74", "user_id": "31058815@N00", "query": "Find group photos of people on the observation deck, where all the people in the photo have taken a helicopter sightseeing tour nearby.", "answer": ["4513080187", "4513722624"], "event_type": "intra-event"}
+{"query_id": "75", "user_id": "31058815@N00", "query": "Find group photos taken on the way to the Koala Conservation Reserve featuring the same tourists who were later photographed at the best-preserved convict settlement in Australia.", "answer": ["5872148402", "5871590137"], "event_type": "inter-event"}
+{"query_id": "76", "user_id": "35032604@N00", "query": "Find walking photos from the 2012 trip during which the beige wide-brimmed sun hat from the 2008 Puerto Rico vacation reappeared.", "answer": ["8367884194", "8367874070"], "event_type": "inter-event"}
+{"query_id": "77", "user_id": "35032604@N00", "query": "Find photos of the purple-blue plush toy received on Christmas morning while it was still in the bag.", "answer": ["2135703854", "2135704246", "2134924297", "2135704784", "2135704966", "2134925041"], "event_type": "intra-event"}
+{"query_id": "78", "user_id": "35032604@N00", "query": "Find Christmas photos of the woman who wore a pink patterned dress and glasses on Thanksgiving, in which she is showing a toothy smile.", "answer": ["2149872721", "2149886507", "2149889275", "2150682448", "2150694380"], "event_type": "inter-event"}
+{"query_id": "79", "user_id": "39979407@N05", "query": "Find photos of the dog that appears in photos for four consecutive years, where it is lying on the ground wearing a leash.", "answer": ["9505410488", "9505377294"], "event_type": "inter-event"}
+{"query_id": "80", "user_id": "39979407@N05", "query": "Find photos where the dog that often goes hiking together is lying on the ground wearing a leash.", "answer": ["9505410488", "9505377294"], "event_type": "inter-event"}
+{"query_id": "81", "user_id": "39979407@N05", "query": "Find photos of pink flowers taken on the two days when many cairns were photographed.", "answer": ["9318157923", "9320981086"], "event_type": "intra-event"}
+{"query_id": "82", "user_id": "40817698@N07", "query": "Find photos of the car, which was photographed on two different days, parked in front of the store.", "answer": ["10912713835"], "event_type": "inter-event"}
+{"query_id": "83", "user_id": "40817698@N07", "query": "Find photos of the car, where I sat inside laughing later, parked in front of a store.", "answer": ["10912713835"], "event_type": "intra-event"}
+{"query_id": "84", "user_id": "40817698@N07", "query": "Find all indoor photos of me taken on the day when I wore two different hats.", "answer": ["11781868496", "11781862686", "11781854676", "11781364664", "11781471144"], "event_type": "intra-event"}
+{"query_id": "85", "user_id": "41610421@N05", "query": "Find all photos of cats on the windowsill taken after the cat tree was assembled.", "answer": ["6925126388", "7071197841", "6925115942", "7301319300", "9286900781"], "event_type": "inter-event"}
+{"query_id": "86", "user_id": "41610421@N05", "query": "Find all photos of the cat that was present during the assembly of the cat tree standing on the table.", "answer": ["5436177059", "5436791744", "5436790026"], "event_type": "intra-event"}
+{"query_id": "87", "user_id": "41610421@N05", "query": "I've drawn a political party's symbol on the beach; find photos of the promotional material of the party placed on the sofa.", "answer": ["4781159336", "4780522565"], "event_type": "inter-event"}
+{"query_id": "88", "user_id": "41838028@N00", "query": "Find the photo where the toy that the dog was biting is later hanging on a backpack.", "answer": ["2274176358"], "event_type": "inter-event"}
+{"query_id": "89", "user_id": "41838028@N00", "query": "Find photos where the toy that the dog was biting is later placed on the bench.", "answer": ["2274156016", "2273358409"], "event_type": "inter-event"}
+{"query_id": "90", "user_id": "41838028@N00", "query": "Find photos where the young boy wore a jacket in 2008 that he also wore in 2007.", "answer": ["2777734502"], "event_type": "inter-event"}
+{"query_id": "91", "user_id": "43145783@N00", "query": "Find photos from the Royal Regiment of Scotland band parade that include leopard print clothing.", "answer": ["416271783", "416272319", "416272242", "416271972", "416271490", "416271886", "416271306"], "event_type": "intra-event"}
+{"query_id": "92", "user_id": "47554402@N00", "query": "Find all photos of that non-plaster statue that were taken twice within half a year.", "answer": ["4766071721", "6101461613", "4305319876"], "event_type": "inter-event"}
+{"query_id": "93", "user_id": "47642109@N04", "query": "Find photos showing the object (which a puppy was playing with outdoors) lying on the ground after having a large portion torn off by another puppy.", "answer": ["6708877621", "6708882313", "6725766223", "6725760161", "6725685701", "6725679177", "6725674339", "6708692717", "6725559125"], "event_type": "intra-event"}
+{"query_id": "94", "user_id": "49475364@N00", "query": "Find the building that has appeared in both real life and non-real form at different locations, and retrieve both its real life version and non-real version photos.", "answer": ["7382741514", "10628350484"], "event_type": "inter-event"}
+{"query_id": "95", "user_id": "49475364@N00", "query": "Find the solo photos of the person who appears in a dining table group photo with two others (where all three cooked in the same kitchen), but who only has individual shots taken in that kitchen.", "answer": ["5279736855", "5522330465"], "event_type": "inter-event"}
+{"query_id": "96", "user_id": "49645113@N07", "query": "Find the photos of dancing with exposed midriff that were taken approximately 30 minutes to 1 hour before a performer holding a torch performed.", "answer": ["9994719874", "9994777326"], "event_type": "intra-event"}
+{"query_id": "97", "user_id": "54368512@N00", "query": "I once photographed an animal sleeping on a tool marked \"4x4\". Now I need to find all photos of animals of the same species, but of the opposite sex, with their eyes open.", "answer": ["8096054958", "8096049037", "8096048161", "8096073106", "8096049471"], "event_type": "inter-event"}
+{"query_id": "98", "user_id": "54368512@N00", "query": "Find photos that contain strictly carnivorous animals, taken within three months after photos of animals about 25 centimeters in body length that like to eat nuts and insects.", "answer": ["8096077441", "8096082466", "8096076611", "8096050475", "8096057334", "8096047771", "8096062198", "8096075411", "8096064490", "8096059707", "8096049037", "8096072144", "8096070276", "8096062065", "8096049989", "8096053895", "8096062876", "8096054958", "8096073106", "8096063428", "8096054475", "8096061340", "8096046937", "8096069206", "8096071444", "8096059069", "8096067368", "8096049471", "8096048161", "8096058114", "8096053005", "8096052363", "8096074701", "8096047347", "8096078287"], "event_type": "intra-event"}
+{"query_id": "99", "user_id": "54368512@N00", "query": "Identify the person who, during a match, wore a black helmet with the same number on top as Iniesta's Barcelona number. Then find all the photos in which this person is not wearing a helmet.", "answer": ["10262806964", "10262887436"], "event_type": "inter-event"}
+{"query_id": "100", "user_id": "55772206@N03", "query": "Find all photos where the person who was wearing flip-flops at the beach appears in a rainforest that has existed on Earth for about 130 million years.", "answer": ["8120951530", "8120946816", "8120936155"], "event_type": "inter-event"}
+{"query_id": "101", "user_id": "55772206@N03", "query": "There's a photo of one people smiling brightly, wearing clothing whose brand logo is a stone. Find all photos from that same trip that contain coconut trees.", "answer": ["7975722484", "7975713847", "7975715557", "7975712984", "8014500983", "7975716537", "7975717703"], "event_type": "intra-event"}
+{"query_id": "102", "user_id": "57269089@N03", "query": "Identify the person who appears most frequently in horseback riding photos. I want to find all photos of this person sitting on the grass.", "answer": ["7660499744", "7660500656", "7660501552"], "event_type": "inter-event"}
+{"query_id": "103", "user_id": "57269089@N03", "query": "Identify the date of the photo showing a person holding a glass item first manufactured in 1753. Then find all photos of seaside sunsets taken during the next calendar month after that date.", "answer": ["5978560626", "5978559064", "5981890977", "5982435276", "5981883283"], "event_type": "inter-event"}
+{"query_id": "104", "user_id": "62606667@N04", "query": "Based on the photo taken during the second trip to Florence showing a building with colonnades reflected on the river, find the river view photos taken from the same side and viewpoint during the first trip.", "answer": ["6168137826", "6167602051"], "event_type": "inter-event"}
+{"query_id": "105", "user_id": "62606667@N04", "query": "Find solo photos of the man who wore a black hat while camping on the Alaska ferry and hiking in Juneau, showing him with a backpack during the trip where he saw wild bears.", "answer": ["6529792617", "6529792013"], "event_type": "inter-event"}
+{"query_id": "106", "user_id": "62606667@N04", "query": "Find solo photos of the man wearing sunglasses at Everest Base Camp who wore the same top on both the Puffing Billy steam train and the Manly Ferry.", "answer": ["8360209009", "8361271630", "8360207761", "8360208375", "8360215489", "8360215211", "8361277548"], "event_type": "inter-event"}
+{"query_id": "107", "user_id": "65367662@N00", "query": "Find the moments right after the woman who performed wearing leopard-print pants at the CalTech Visit fired a handgun at a shooting range in California.", "answer": ["8158465001", "8158462583", "8158491180"], "event_type": "intra-event"}
+{"query_id": "108", "user_id": "65367662@N00", "query": "Find photos taken in 2012 on the beach near where Sheep crab was seen diving, showing the person who wore a blue and white headscarf during the Grant Lakes hike.", "answer": ["6886948892", "6886950944", "6886936510"], "event_type": "inter-event"}
+{"query_id": "109", "user_id": "65367662@N00", "query": "Find underwater solo photos of a diver at Sliver Prince Diving in 2012 whose snorkel color is different from any snorkel color that appeared in the September 2012 diving activity.", "answer": ["7249396480", "7249390648", "7249381698", "7249394202"], "event_type": "inter-event"}
+{"query_id": "110", "user_id": "67298685@N00", "query": "Find the photos from the June 2008 Algonquin canoe trip where the man who interacted with a dog during the August 2008 Algonquin Highland hiking trip is holding a wine glass.", "answer": ["2636424739", "2637249688", "2637247844"], "event_type": "inter-event"}
+{"query_id": "111", "user_id": "67298685@N00", "query": "Find photos from the July 2011 Killarney canoe trip showing the person who participated in cooking during both the June 2008 Algonquin canoe trip and the July 2011 Killarney canoe trip, where this person is sitting in a boat.", "answer": ["5957229745", "5957790476", "5957219347", "5957790338", "5957790178"], "event_type": "intra-event"}
+{"query_id": "112", "user_id": "67298685@N00", "query": "Find photos from the 2012 French River canoe trip showing the same dog from the close-up sleeping shot inside the tent, now seen on a canoe.", "answer": ["8423526781", "8423515763", "8424617216", "8423516443"], "event_type": "intra-event"}
+{"query_id": "113", "user_id": "68712269@N00", "query": "Find the lakeside photos taken during a 2013 hiking trip in which both this trip and the Little Bulger hike in the same year captured the same landmark.", "answer": ["10498268355", "10498271225", "10498273155", "10498265896"], "event_type": "inter-event"}
+{"query_id": "114", "user_id": "69099808@N00", "query": "Find the outdoor group photo of the man who once hugged his knees while watching a game, taken during the calendar week of a milestone 10-year birthday celebration.", "answer": ["4730861417", "4730942747"], "event_type": "intra-event"}
+{"query_id": "115", "user_id": "70408381@N00", "query": "Find all photos with a body of water that I took during the outing with a friend who was wearing a themed T-shirt encouraging females to try an alcoholic beverage.", "answer": ["7626314726", "7626306292", "7626411894", "7626371328", "7626214908", "7626401908", "7626294468", "7626363828", "7626425526"], "event_type": "intra-event"}
+{"query_id": "116", "user_id": "70554294@N00", "query": "Find all photos containing a red warning flag that were taken during the trip when I visited a Mayan ruin.", "answer": ["471404130", "471404248", "471422465", "471424669", "471425395", "471408496", "471429531"], "event_type": "intra-event"}
+{"query_id": "117", "user_id": "70554294@N00", "query": "Find all photos taken at that software-themed gathering held at a location not by the sea, which include two specific men who also appeared together in a photo at another place within the same month.", "answer": ["2258888189", "2258884481"], "event_type": "inter-event"}
+{"query_id": "118", "user_id": "7276266@N05", "query": "Find photos of yellow flowers taken near the memorial site for the heavy coastal defense kinetic weapon that was transported in 1991.", "answer": ["2927208756", "2927211222", "2927203590"], "event_type": "intra-event"}
+{"query_id": "119", "user_id": "7276266@N05", "query": "Find all photos from two trips that include the woman who wore exactly the same outfit (top and bottom) in both trips, where the two trips are more than three months apart.", "answer": ["3743782238", "3743779180", "2859132587"], "event_type": "inter-event"}
+{"query_id": "120", "user_id": "7276266@N05", "query": "Find photos of rodents taken on the day of visiting the area near the artificial lake recreation zone that was developed during the Great Depression in the United States.", "answer": ["1590169209", "1590167229"], "event_type": "intra-event"}
+{"query_id": "121", "user_id": "7664723@N05", "query": "A boy and a girl have worn identical tops (same design). Please find all photos containing them wearing these tops.", "answer": ["6964340856", "7362155924", "7450745238", "7450731212", "7450212688", "7450615604", "7450624464", "7450315166"], "event_type": "inter-event"}
+{"query_id": "122", "user_id": "7664723@N05", "query": "Find the photos containing beer taken during the calendar week when visiting the church whose construction took more than 500 years to complete.", "answer": ["7362595606", "7177235085", "7362587718", "7177019057", "7362611520", "7362616970", "7362543680", "7177024063", "7362252240", "7177399111", "7177243049"], "event_type": "intra-event"}

leaderboard_data.json

@@ -0,0 +1 @@
+[]

requirements.txt

@@ -0,0 +1,5 @@
+flask
+pandas
+huggingface_hub
+python-dotenv
+gunicorn

static/script.js

@@ -0,0 +1,253 @@
+const rawData = window.SERVER_DATA || [];
+
+// --- State Management ---
+let state = {
+    championTrack: "Standard",
+    sortKey: "overall_em",
+    sortDir: "desc",
+    filters: {
+        track: "all",
+        agent: "",
+        backbone: "",
+        retriever: ""       // New retriever filter
+    }
+};
+
+// List of score column keys (used to determine which columns need highlighting)
+const SCORE_KEYS = ["overall_em", "overall_f1", "intra_em", "intra_f1", "inter_em", "inter_f1"];
+
+// --- Main Tab Switching ---
+function switchMainTab(tabName) {
+    document.querySelectorAll('.nav-btn').forEach(btn => btn.classList.remove('active'));
+    const map = {'leaderboard': 0, 'full-metrics': 1, 'submit': 2};
+    document.querySelectorAll('.nav-btn')[map[tabName]].classList.add('active');
+
+    document.querySelectorAll('.view-section').forEach(el => el.style.display = 'none');
+    document.getElementById(`view-${tabName}`).style.display = 'block';
+
+    if (tabName === 'leaderboard') renderChampionTable();
+    if (tabName === 'full-metrics') renderFullTable();
+}
+
+// --- Update active-sort state for all table headers ---
+function updateSortHeaders() {
+    document.querySelectorAll('.sortable').forEach(th => {
+        const key = th.dataset.key;
+        // Remove arrows from text
+        let text = th.textContent.replace(/ ↓/g, '').replace(/ ↑/g, '').trim();
+        
+        if (key === state.sortKey) {
+            th.classList.add('active-sort');
+            text += state.sortDir === 'desc' ? ' ↓' : ' ↑';
+        } else {
+            th.classList.remove('active-sort');
+        }
+        th.textContent = text;
+    });
+}
+
+// --- Logic: Champion Table ---
+function renderChampionTable() {
+    const tbody = document.querySelector('#champion-table tbody');
+    let data = rawData.filter(d => d.track === state.championTrack);
+    
+    // For each method name, keep only the one with the highest overall_em
+    const methodBestMap = new Map();
+    data.forEach(row => {
+        const method = row.method;
+        if (!methodBestMap.has(method) || row.overall_em > methodBestMap.get(method).overall_em) {
+            methodBestMap.set(method, row);
+        }
+    });
+    
+    // Convert to array
+    data = Array.from(methodBestMap.values());
+    data = sortData(data);
+
+    tbody.innerHTML = data.map((row, idx) => {
+        return buildRowHTML(row, idx + 1, false);
+    }).join('');
+
+    updateSortHeaders();
+}
+
+// --- Logic: Full Table ---
+function renderFullTable() {
+    const tbody = document.querySelector('#full-table tbody');
+
+    let data = rawData.filter(d => {
+        const f = state.filters;
+        if (f.track !== 'all' && d.track !== f.track) return false;
+        if (f.agent && !d.agent.toLowerCase().includes(f.agent.toLowerCase())) return false;
+        if (f.backbone && !d.backbone.toLowerCase().includes(f.backbone.toLowerCase())) return false;
+        if (f.retriever && !d.retriever.toLowerCase().includes(f.retriever.toLowerCase())) return false;
+        return true;
+    });
+
+    data = sortData(data);
+
+    if (data.length === 0) {
+        tbody.innerHTML = `<tr><td colspan="12" style="text-align:center; padding:20px; color:#888;">No matching results.</td></tr>`;
+        return;
+    }
+
+    tbody.innerHTML = data.map((row, idx) => {
+        return buildRowHTML(row, idx + 1, true);
+    }).join('');
+
+    updateSortHeaders();
+}
+
+// --- Helper: Sort ---
+function sortData(data) {
+    return data.sort((a, b) => {
+        let valA = a[state.sortKey];
+        let valB = b[state.sortKey];
+        if (typeof valA === 'number') {
+            return state.sortDir === 'desc' ? valB - valA : valA - valB;
+        }
+        return 0;
+    });
+}
+
+// --- Helper: Row HTML Builder ---
+function buildRowHTML(row, rank, showTrack) {
+    const medal = rank === 1 ? '🥇' : rank === 2 ? '🥈' : rank === 3 ? '🥉' : rank;
+
+    const trackTd = showTrack
+        ? `<td><span class="track-tag ${row.track}">${row.track}</span></td>`
+        : '';
+
+    // Generate score cell, add active-col class to current sort column
+    function scoreCell(key, value) {
+        const isActive = (key === state.sortKey) ? ' active-col' : '';
+        return `<td class="score-cell${isActive}">${value.toFixed(1)}</td>`;
+    }
+
+    return `
+    <tr>
+        <td class="rank-col">${medal}</td>
+        <td class="method-col align-left">
+            <a href="${row.url}" target="_blank" class="method-name">${row.method}</a>
+            <div class="org-name">${row.date}</div>
+        </td>
+        ${trackTd}
+        <td>${row.agent}</td>
+        <td>${row.backbone}</td>
+        <td>${row.retriever}</td>
+        ${scoreCell('overall_em', row.overall_em)}
+        ${scoreCell('overall_f1', row.overall_f1)}
+        ${scoreCell('intra_em', row.intra_em)}
+        ${scoreCell('intra_f1', row.intra_f1)}
+        ${scoreCell('inter_em', row.inter_em)}
+        ${scoreCell('inter_f1', row.inter_f1)}
+    </tr>
+    `;
+}
+
+// --- Event Listeners ---
+
+// 1. Champion Sub-Tabs (Standard/Open)
+document.querySelectorAll('.sub-tab-btn').forEach(btn => {
+    btn.addEventListener('click', (e) => {
+        document.querySelectorAll('.sub-tab-btn').forEach(b => b.classList.remove('active'));
+        e.target.classList.add('active');
+        state.championTrack = e.target.dataset.track;
+        renderChampionTable();
+    });
+});
+
+// 2. Sorting
+document.querySelectorAll('.sortable').forEach(th => {
+    th.addEventListener('click', (e) => {
+        const key = e.currentTarget.dataset.key;
+        if (state.sortKey === key) {
+            state.sortDir = state.sortDir === 'desc' ? 'asc' : 'desc';
+        } else {
+            state.sortKey = key;
+            state.sortDir = 'desc';
+        }
+
+        // Re-render the currently visible table
+        if (document.getElementById('view-leaderboard').style.display !== 'none') {
+            renderChampionTable();
+        } else {
+            renderFullTable();
+        }
+    });
+});
+
+// 3. Full Metrics Filters
+document.getElementById('filter-track').addEventListener('change', (e) => {
+    state.filters.track = e.target.value;
+    renderFullTable();
+});
+document.getElementById('filter-agent').addEventListener('input', (e) => {
+    state.filters.agent = e.target.value;
+    renderFullTable();
+});
+document.getElementById('filter-backbone').addEventListener('input', (e) => {
+    state.filters.backbone = e.target.value;
+    renderFullTable();
+});
+document.getElementById('filter-retriever').addEventListener('input', (e) => {
+    state.filters.retriever = e.target.value;
+    renderFullTable();
+});
+
+// 4. Submit Form (AJAX, for displaying PR link feedback)
+document.getElementById('submit-form').addEventListener('submit', async (e) => {
+    e.preventDefault();
+
+    const fileInput = document.getElementById('submit-file');
+    const submitBtn = document.getElementById('submit-btn');
+    const resultDiv = document.getElementById('submit-result');
+
+    if (!fileInput.files.length) return;
+
+    // Disable button, show loading
+    submitBtn.disabled = true;
+    submitBtn.innerHTML = '<i class="fas fa-spinner fa-spin"></i> Submitting...';
+    resultDiv.style.display = 'none';
+
+    const formData = new FormData();
+    formData.append('file', fileInput.files[0]);
+
+    try {
+        const response = await fetch('/upload', {
+            method: 'POST',
+            body: formData,
+        });
+
+        const data = await response.json();
+
+        resultDiv.style.display = 'block';
+
+        if (data.success) {
+            let html = `<strong>✅ ${data.message}</strong>`;
+            if (data.pr_url) {
+                html += `<br><br>🔗 <a href="${data.pr_url}" target="_blank">View your Pull Request →</a>`;
+            }
+            html += `<br><br><small>After maintainers review and merge your PR, scores will be computed and published on the leaderboard.</small>`;
+            resultDiv.className = 'success';
+            resultDiv.innerHTML = html;
+        } else {
+            let html = `<strong>❌ ${data.error || 'Submission failed.'}</strong>`;
+            if (data.details) {
+                html += '<ul>' + data.details.map(d => `<li>${d}</li>`).join('') + '</ul>';
+            }
+            resultDiv.className = 'error';
+            resultDiv.innerHTML = html;
+        }
+    } catch (err) {
+        resultDiv.style.display = 'block';
+        resultDiv.className = 'error';
+        resultDiv.innerHTML = `<strong>❌ Network error:</strong> ${err.message}`;
+    } finally {
+        submitBtn.disabled = false;
+        submitBtn.innerHTML = '<i class="fas fa-paper-plane"></i> Submit & Create PR';
+    }
+});
+
+// --- Init ---
+renderChampionTable();

static/styles.css

@@ -0,0 +1,442 @@
+@import url('https://fonts.googleapis.com/css2?family=Lato:wght@300;400;700&display=swap');
+
+:root {
+  --bg: #f6f1eb;
+  --bg-ink: #141210;
+  --muted: #3d3832;          /* Darkened: original #5a534a was too light */
+  --accent: #cb5a2d;
+  --accent-light: #e8d0c3;
+  --line: #ddd3c9;
+  --card: #fffaf4;
+  --shadow: 0 4px 20px rgba(20, 18, 16, 0.08);
+  --header-bg: #fdfdfd;
+}
+
+body {
+  margin: 0;
+  font-family: "Lato", sans-serif;
+  color: var(--bg-ink);
+  background: radial-gradient(circle at top left, #fdeee0 0%, var(--bg) 50%, #f3efe8 100%);
+  min-height: 100vh;
+}
+
+.page { max-width: 1280px; margin: 0 auto; padding: 0 24px 60px; }
+
+/* ===== Paper Header ===== */
+.paper-header {
+  padding: 48px 0 24px;
+  text-align: center;
+}
+
+/* RUC Logo */
+.institution-logo {
+  margin-bottom: 20px;
+}
+.ruc-logo {
+  height: 64px;
+  width: auto;
+  opacity: 0.9;
+}
+
+.paper-title {
+  font-family: "Lato", sans-serif;
+  font-size: 2.2rem;
+  font-weight: 700;
+  line-height: 1.35;
+  color: var(--bg-ink);
+  margin: 0 auto 16px;
+  max-width: 900px;
+}
+
+.paper-authors {
+  font-size: 0.95rem;
+  color: var(--muted);
+  line-height: 1.7;
+  margin: 0 auto 6px;
+  max-width: 860px;
+}
+.paper-authors sup { font-size: 0.7em; color: var(--accent); margin-right: 1px; }
+
+.paper-affiliations {
+  font-size: 0.88rem;
+  color: #666;              /* Darkened: original #888 was too light */
+  margin: 0 auto 20px;
+}
+.paper-affiliations sup { font-size: 0.7em; color: var(--accent); }
+.affil-sep { margin: 0 8px; }
+
+.paper-links {
+  display: flex;
+  gap: 12px;
+  justify-content: center;
+}
+
+.badge-link {
+  text-decoration: none; color: var(--bg-ink); font-weight: bold;
+  padding: 8px 18px; border: 1px solid var(--line); border-radius: 20px;
+  background: white; transition: all 0.2s; display: flex; align-items: center; gap: 8px;
+  font-size: 0.9rem;
+}
+.badge-link:hover { border-color: var(--accent); color: var(--accent); }
+
+/* ===== Leaderboard Hero ===== */
+.leaderboard-hero {
+  text-align: center;
+  padding: 10px 0 30px;
+}
+
+.hero-divider {
+  width: 60px;
+  height: 3px;
+  background: var(--line);
+  margin: 0 auto 28px;
+  border-radius: 2px;
+}
+
+.leaderboard-title {
+  font-family: "Lato", sans-serif;
+  font-size: 2.1rem;
+  font-weight: 700;
+  color: var(--bg-ink);
+  margin: 0 0 8px;
+}
+.leaderboard-title i { color: var(--accent); margin-right: 6px; }
+
+.leaderboard-subtitle {
+  font-size: 1.1rem;
+  color: var(--muted);
+  margin: 0 0 24px;
+}
+
+/* Stat Bar */
+.stat-bar {
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  gap: 24px;
+  background: white;
+  border: 1px solid var(--line);
+  border-radius: 12px;
+  padding: 16px 40px;
+  width: fit-content;
+  margin: 0 auto;
+  box-shadow: var(--shadow);
+}
+
+.stat-item {
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  gap: 2px;
+}
+
+.stat-num {
+  font-family: "Lato", sans-serif;
+  font-size: 1.5rem;
+  font-weight: 700;
+  color: var(--accent);
+}
+
+.stat-label {
+  font-size: 0.78rem;
+  color: var(--muted);
+  text-transform: uppercase;
+  letter-spacing: 0.5px;
+}
+
+.stat-divider {
+  width: 1px;
+  height: 32px;
+  background: var(--line);
+}
+
+/* ===== Info Section ===== */
+.info-section { margin-bottom: 36px; }
+.info-grid {
+  display: grid; grid-template-columns: repeat(2, 1fr); gap: 20px;
+}
+@media (max-width: 800px) {
+  .info-grid { grid-template-columns: 1fr; }
+}
+
+.info-card {
+  background: white; padding: 24px; border-radius: 12px;
+  border: 1px solid var(--line); box-shadow: var(--shadow);
+}
+.info-card h3 {
+  margin: 0 0 12px; color: var(--accent);
+  display: flex; align-items: center; gap: 10px;
+  font-size: 1.2rem;
+}
+.info-card p {
+  font-size: 1.00rem;
+  line-height: 1.65;
+  color: var(--bg-ink);           /* Changed to black, no longer using gray */
+  margin: 0 0 10px;
+}
+.info-card p:last-child { margin-bottom: 0; }
+.info-card code {
+  background: #f4f0ec; padding: 2px 6px; border-radius: 4px; font-size: 0.85em;
+  font-family: ui-monospace, SFMono-Regular, Menlo, monospace; color: var(--accent);
+}
+
+.info-list {
+  list-style: none; padding: 0; margin: 8px 0 12px;
+}
+.info-list li {
+  font-size: 0.93rem;
+  line-height: 1.6;
+  color: var(--bg-ink);           /* Changed to black */
+  padding: 5px 0 5px 16px;
+  position: relative;
+}
+.info-list li::before {
+  content: "";
+  position: absolute;
+  left: 0; top: 13px;
+  width: 6px; height: 6px;
+  border-radius: 50%;
+  background: var(--accent);
+}
+.info-list.compact li { padding: 3px 0 3px 16px; }
+.info-list.compact li::before { top: 11px; }
+
+/* ===== Main Navigation (Tabs) ===== */
+.main-nav {
+  display: flex; justify-content: center; gap: 10px; margin-bottom: 30px;
+  background: rgba(255,255,255,0.6); padding: 10px; border-radius: 50px;
+  width: fit-content; margin-left: auto; margin-right: auto;
+  border: 1px solid var(--line);
+}
+.nav-btn {
+  border: none; background: transparent; padding: 10px 24px;
+  border-radius: 30px; font-size: 1rem; font-weight: bold; color: var(--muted);
+  cursor: pointer; transition: all 0.2s; display: flex; align-items: center; gap: 8px;
+  font-family: "Lato", sans-serif;
+}
+.nav-btn.active { background: var(--accent); color: white; box-shadow: 0 2px 10px rgba(203, 90, 45, 0.3); }
+.nav-btn:hover:not(.active) { background: var(--accent-light); color: var(--accent); }
+
+/* ===== Sub Tabs ===== */
+.sub-tabs-container { display: flex; justify-content: flex-start; gap: 20px; margin-bottom: 20px; border-bottom: 2px solid var(--line); padding-bottom: 10px; }
+.sub-tab-btn {
+  background: none; border: none; font-size: 1.1rem; font-weight: bold;
+  color: var(--muted); cursor: pointer; padding: 5px 10px;
+  border-bottom: 3px solid transparent; transition: all 0.2s;
+  font-family: "Lato", sans-serif;
+}
+.sub-tab-btn.active { color: var(--accent); border-bottom-color: var(--accent); }
+
+/* ===== Filters Toolbar ===== */
+.filters-toolbar {
+  display: flex; gap: 20px; margin-bottom: 20px; background: white;
+  padding: 15px; border-radius: 10px; border: 1px solid var(--line); align-items: center; flex-wrap: wrap;
+}
+.filter-group { display: flex; align-items: center; gap: 10px; }
+.filter-group label { font-weight: bold; font-size: 0.9rem; color: var(--muted); }
+.filter-group input, .filter-group select {
+  padding: 8px 12px; border: 1px solid var(--line); border-radius: 6px; outline: none; font-family: "Lato", sans-serif;
+}
+
+/* ===== Tables ===== */
+.table-container { overflow-x: auto; background: white; border-radius: 12px; border: 1px solid var(--line); box-shadow: var(--shadow); }
+table { width: 100%; border-collapse: collapse; min-width: 1000px; }
+
+th, td {
+  padding: 12px;
+  text-align: center;
+  border-bottom: 1px solid #f0f0f0;
+  font-size: 0.95rem;
+  white-space: nowrap;
+}
+
+th {
+  background: var(--header-bg); padding: 15px 12px;
+  border-bottom: 1px solid var(--line); color: var(--muted); font-size: 0.9rem;
+  cursor: pointer; user-select: none;
+}
+th.align-left { text-align: left; }
+
+/* Highlight table header for current sort column */
+th.sortable.active-sort {
+  background: #fff5eb;
+  color: var(--accent);
+  font-weight: 700;
+}
+
+tr:last-child td { border-bottom: none; }
+tr:hover td { background: #fafafa; }
+
+.rank-col { font-weight: bold; font-size: 1.1rem; width: 60px; }
+
+.method-col {
+  text-align: left;
+  min-width: 220px;
+  max-width: 350px;
+  white-space: normal;
+}
+
+.method-name { font-weight: 700; color: var(--bg-ink); text-decoration: none; font-size: 1rem; line-height: 1.4; display: block; }
+.org-name { font-size: 0.8rem; color: #666; margin-top: 4px; }
+
+.track-tag { font-size: 0.75rem; padding: 3px 8px; border-radius: 4px; font-weight: bold; text-transform: uppercase; }
+.track-tag.Standard { background: #e3f2fd; color: #1565c0; }
+.track-tag.Open { background: #fce4ec; color: #c2185b; }
+
+/* ---- Score columns: default all black ---- */
+td.score-cell {
+  font-weight: 600;
+  color: var(--bg-ink);
+  font-size: 0.95rem;
+}
+
+/* Highlight score in current sort column as orange */
+td.score-cell.active-col {
+  font-weight: 700;
+  color: var(--accent);
+  background: #fff8f5;
+  font-size: 1.05rem;
+}
+
+/* ===== Submit Section ===== */
+.submit-container { display: flex; justify-content: center; }
+.submit-card {
+  background: white; width: 100%; max-width: 700px; padding: 32px;
+  border-radius: 16px; border: 1px solid var(--line); box-shadow: var(--shadow);
+}
+.submit-card h2 {
+  margin-top: 0;
+  display: flex; align-items: center; gap: 10px;
+}
+.submit-card h2 i { color: var(--accent); }
+.submit-card h4 { margin: 20px 0 10px; color: var(--bg-ink); }
+
+/* Submit Flow Diagram */
+.submit-flow {
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  gap: 12px;
+  margin: 24px 0;
+  padding: 20px;
+  background: #faf8f5;
+  border-radius: 12px;
+  border: 1px solid var(--line);
+}
+.flow-step {
+  display: flex;
+  align-items: center;
+  gap: 12px;
+  flex: 1;
+}
+.flow-icon {
+  width: 44px; height: 44px;
+  border-radius: 50%;
+  background: var(--accent);
+  color: white;
+  display: flex; align-items: center; justify-content: center;
+  font-size: 1.1rem;
+  flex-shrink: 0;
+}
+.flow-text {
+  font-size: 0.85rem;
+  line-height: 1.4;
+  color: var(--bg-ink);
+}
+.flow-text strong { display: block; font-size: 0.9rem; }
+.flow-arrow {
+  color: var(--line);
+  font-size: 1.2rem;
+  flex-shrink: 0;
+}
+
+.field-desc p {
+  font-size: 0.9rem; color: var(--bg-ink); margin: 6px 0; line-height: 1.5;
+}
+.field-desc code {
+  background: #f4f0ec; padding: 2px 6px; border-radius: 4px; font-size: 0.85em;
+  font-family: ui-monospace, SFMono-Regular, Menlo, monospace; color: var(--accent);
+}
+
+.code-block {
+  background: #faf8f5; padding: 16px; border-radius: 8px;
+  font-size: 0.85rem; overflow-x: auto; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; color: #333;
+  border: 1px solid #e8e2da;
+  white-space: pre;
+  line-height: 1.55;
+}
+
+.upload-form { margin-top: 16px; }
+.upload-form input[type="file"] {
+  width: 100%; padding: 10px; border: 2px dashed var(--line); border-radius: 8px;
+  background: #faf8f5; cursor: pointer; font-family: "Lato", sans-serif;
+  box-sizing: border-box;
+}
+
+.btn.primary {
+  background: var(--accent); color: white; border: none; padding: 12px 24px;
+  border-radius: 8px; font-weight: bold; cursor: pointer; width: 100%; margin-top: 16px;
+  transition: background 0.2s;
+  font-family: "Lato", sans-serif; font-size: 1rem;
+  display: flex; align-items: center; justify-content: center; gap: 8px;
+}
+.btn.primary:hover { background: #b14a1f; }
+.btn.primary:disabled { background: #ccc; cursor: not-allowed; }
+
+/* Submit Result Messages */
+#submit-result {
+  margin-top: 20px;
+  padding: 16px;
+  border-radius: 8px;
+  font-size: 0.93rem;
+  line-height: 1.6;
+}
+#submit-result.success {
+  background: #e8f5e9;
+  border: 1px solid #a5d6a7;
+  color: #2e7d32;
+}
+#submit-result.error {
+  background: #fce4ec;
+  border: 1px solid #ef9a9a;
+  color: #c62828;
+}
+#submit-result a {
+  color: var(--accent);
+  font-weight: bold;
+}
+
+/* ===== Citation ===== */
+.citation-section {
+  margin-top: 48px;
+  text-align: center;
+}
+.citation-section h3 {
+  color: var(--muted);
+  font-size: 1rem;
+  margin-bottom: 12px;
+  display: flex; align-items: center; justify-content: center; gap: 8px;
+}
+.citation-section h3 i { color: var(--accent); }
+.citation-block {
+  text-align: left;
+  max-width: 800px;
+  margin: 0 auto;
+  font-size: 0.82rem;
+}
+
+/* ===== Footer ===== */
+.footer { text-align: center; padding: 40px; color: var(--muted); font-size: 0.9rem; }
+
+/* ===== Responsive ===== */
+@media (max-width: 640px) {
+  .paper-title { font-size: 1.4rem; }
+  .leaderboard-title { font-size: 1.8rem; }
+  .stat-bar { flex-wrap: wrap; padding: 16px 20px; gap: 16px; }
+  .stat-divider { display: none; }
+  .paper-authors { font-size: 0.85rem; }
+  .paper-links { flex-wrap: wrap; }
+  .submit-flow { flex-direction: column; gap: 16px; }
+  .flow-arrow { transform: rotate(90deg); }
+  .ruc-logo { height: 48px; }
+}

templates/index.html

@@ -0,0 +1,310 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <title>DISBench Leaderboard</title>
+    <!-- Favicon: RUC Logo -->
+    <link rel="icon" type="image/svg+xml" href="{{ url_for('static', filename='ruc-logo.png') }}" />
+    <link rel="stylesheet" href="{{ url_for('static', filename='styles.css') }}" />
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css" />
+  </head>
+  <body>
+    <div class="page">
+      
+      <!-- Paper Header -->
+      <header class="paper-header">
+        <!-- RUC Logo -->
+        <!-- <div class="institution-logo">
+          <img src="{{ url_for('static', filename='ruc-logo.png') }}" alt="Renmin University of China" class="ruc-logo" />
+        </div> -->
+
+        <h1 class="paper-title">DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories</h1>
+        <p class="paper-authors">
+          Chenlong Deng<sup>1</sup>, Mengjie Deng<sup>1</sup>, Junjie Wu<sup>2</sup>, Dun Zeng<sup>2</sup>, Teng Wang<sup>2</sup>, Qingsong Xie<sup>2</sup>, Jiadeng Huang<sup>2</sup>,
+          Shengjie Ma<sup>1</sup>, Changwang Zhang<sup>2</sup>, Zhaoxiang Wang<sup>2</sup>, Jun Wang<sup>2</sup>, Yutao Zhu<sup>1</sup>, Zhicheng Dou<sup>1</sup>
+        </p>
+        <p class="paper-affiliations">
+          <span class="affil"><sup>1</sup> Gaoling School of Artificial Intelligence, Renmin University of China</span>
+          <span class="affil-sep">&middot;</span>
+          <span class="affil"><sup>2</sup> OPPO Research Institute</span>
+        </p>
+        <div class="paper-links">
+          <a class="badge-link" href="https://arxiv.org/abs/2602.10809" target="_blank"><i class="fas fa-file-pdf"></i> Paper</a>
+          <a class="badge-link" href="https://github.com/RUC-NLPIR/DeepImageSearch" target="_blank"><i class="fab fa-github"></i> Code</a>
+          <a class="badge-link" href="https://huggingface.co/datasets/RUC-NLPIR/DISBench" target="_blank"><i class="fas fa-database"></i> Dataset</a>
+        </div>
+      </header>
+
+      <!-- Leaderboard Title -->
+      <section class="leaderboard-hero">
+        <div class="hero-divider"></div>
+        <h2 class="leaderboard-title"><i class="fas fa-ranking-star"></i> 🏆 DISBench Leaderboard</h2>
+        <p class="leaderboard-subtitle">Track and compare multimodal agents on the DeepImageSearch task</p>
+      </section>
+
+      <main>
+        
+        <!-- Info Cards -->
+        <section class="info-section">
+          <div class="info-grid">
+
+            <div class="info-card">
+              <h3><i class="fas fa-lightbulb"></i> What is DeepImageSearch and DISBench?</h3>
+              <p>
+                <strong>DeepImageSearch</strong> represents a paradigm evolution in image retrieval, advancing from independent image matching to <strong>corpus-level contextual reasoning over visual histories</strong>. 
+                People capture thousands of photos over the years, forming rich episodic memories where information is distributed across temporal sequences rather than confined to single snapshots. 
+                Many real-world queries over such episodic memories cannot be resolved by evaluating each image independently. 
+                The target images can only be identified by exploring and reasoning over the entire image corpus. 
+                This <strong>corpus-level contextual reasoning</strong> makes <strong>agentic capabilities essential rather than auxiliary</strong>.
+              </p>
+              <p>
+                <strong>DISBench</strong> is the <strong>first benchmark</strong> designed for this task. 
+                Given a user's photo collection and a natural language query, 
+                agents must autonomously plan search trajectories, discover latent cross-image associations, 
+                and chain scattered visual evidence through multi-step exploration to return the exact set of qualifying images. 
+                The benchmark covers two reasoning patterns: 
+                <strong>Intra-Event</strong> queries that require locating a target event via contextual clues and then filtering within it, 
+                and <strong>Inter-Event</strong> queries that demand scanning across multiple events to find recurring elements under temporal or spatial constraints.
+              </p>
+            </div>
+
+            <div class="info-card">
+              <h3><i class="fas fa-book-open"></i> How to Read the Leaderboard</h3>
+              <p>
+                <strong>Champion List</strong> shows top results per track. Use the sub-tabs to switch between:
+              </p>
+              <ul class="info-list">
+                <li><span class="track-tag Standard">Standard</span> Pre-processing is limited to encoding images into embeddings for building a retrieval index. No additional pre-computation (e.g., captioning, graph construction) is allowed. Tests agentic reasoning over raw visual data.</li>
+                <li><span class="track-tag Open">Open</span> Arbitrary pre-processing is permitted (captioning, knowledge graph construction, structured indexing, etc.). Tests system-level upper bounds with full engineering freedom.</li>
+              </ul>
+              <p>
+                <strong>Full Analysis</strong> lets you compare across tracks and filter by agent framework, backbone model, or retriever. Click any score column header to sort and highlight.
+              </p>
+            </div>
+
+            <div class="info-card">
+              <h3><i class="fas fa-ruler-combined"></i> Evaluation Metrics</h3>
+              <p>
+                All metrics are computed at the <strong>set level</strong>: models must predict the exact set of target images for each query.
+              </p>
+              <ul class="info-list">
+                <li><strong>EM (Exact Match)</strong>: the predicted set must be identical to the ground truth (no extra, no missing).</li>
+                <li><strong>F1 Score</strong>: harmonic mean of precision and recall over the predicted vs. ground-truth image sets.</li>
+              </ul>
+              <p>
+                Scores are reported across three dimensions:
+                <strong>Overall</strong> (all queries),
+                <strong>Intra-Event</strong> (locate a specific event, then filter targets within it), and
+                <strong>Inter-Event</strong> (scan across multiple events to find recurring elements under temporal/spatial constraints).
+              </p>
+            </div>
+
+            <div class="info-card">
+              <h3><i class="fas fa-cloud-arrow-up"></i> How to Submit</h3>
+              <p>
+                Prepare a <code>.json</code> file with two fields: <code>meta</code> (your method info) and <code>predictions</code> (your model outputs). 
+                Go to the <strong>Submit</strong> tab, upload the file, and the system will automatically create a 
+                <strong>Pull Request</strong> on the Space repository for review.
+              </p>
+              <p>
+                After maintainers merge your PR, the evaluation script will compute scores and update the leaderboard.
+              </p>
+              <p>Required fields in <code>meta</code>:</p>
+              <ul class="info-list compact">
+                <li><code>method_name</code>: display name for your method</li>
+                <li><code>agent_framework</code>, <code>backbone_model</code>, <code>retriever_model</code></li>
+                <li><code>track</code>: <code>"Standard"</code> or <code>"Open"</code></li>
+              </ul>
+              <p>See the <strong>Submit</strong> tab for the full JSON template and format details.</p>
+            </div>
+
+          </div>
+        </section>
+
+        <!-- Main Navigation -->
+        <nav class="main-nav">
+          <button class="nav-btn active" onclick="switchMainTab('leaderboard')">
+            <i class="fas fa-trophy"></i> Champion List
+          </button>
+          <button class="nav-btn" onclick="switchMainTab('full-metrics')">
+            <i class="fas fa-chart-bar"></i> Full Analysis
+          </button>
+          <button class="nav-btn" onclick="switchMainTab('submit')">
+            <i class="fas fa-upload"></i> Submit
+          </button>
+        </nav>
+
+        <!-- Champion List View -->
+        <section id="view-leaderboard" class="view-section active">
+          <div class="sub-tabs-container">
+            <button class="sub-tab-btn active" data-track="Standard">Standard Track</button>
+            <button class="sub-tab-btn" data-track="Open">Open Track</button>
+          </div>
+          
+          <div class="table-container">
+            <table id="champion-table">
+              <thead>
+                <tr>
+                  <th class="rank-col">Rank</th>
+                  <th class="method-col align-left">Method</th>
+                  <th>Agent</th>
+                  <th>Backbone</th>
+                  <th>Retriever</th>
+                  <th class="sortable active-sort" data-key="overall_em">Overall EM ↓</th>
+                  <th class="sortable" data-key="overall_f1">Overall F1</th>
+                  <th class="sortable" data-key="intra_em">Intra EM</th>
+                  <th class="sortable" data-key="intra_f1">Intra F1</th>
+                  <th class="sortable" data-key="inter_em">Inter EM</th>
+                  <th class="sortable" data-key="inter_f1">Inter F1</th>
+                </tr>
+              </thead>
+              <tbody></tbody>
+            </table>
+          </div>
+        </section>
+
+        <!-- Full Analysis View -->
+        <section id="view-full-metrics" class="view-section" style="display: none;">
+          
+          <div class="filters-toolbar">
+            <div class="filter-group">
+              <label>Track:</label>
+              <select id="filter-track"><option value="all">All Tracks</option><option value="Standard">Standard</option><option value="Open">Open</option></select>
+            </div>
+            <div class="filter-group">
+              <label>Agent:</label>
+              <input type="text" id="filter-agent" placeholder="e.g. ImageSeeker">
+            </div>
+            <div class="filter-group">
+              <label>Backbone:</label>
+              <input type="text" id="filter-backbone" placeholder="e.g. Gemini">
+            </div>
+            <div class="filter-group">
+              <label>Retriever:</label>
+              <input type="text" id="filter-retriever" placeholder="e.g. CLIP">
+            </div>
+          </div>
+
+          <div class="table-container">
+            <table id="full-table">
+              <thead>
+                <tr>
+                  <th class="rank-col">Rank</th>
+                  <th class="method-col align-left">Method</th>
+                  <th class="track-col">Track</th>
+                  <th>Agent</th>
+                  <th>Backbone</th>
+                  <th>Retriever</th>
+                  <th class="sortable active-sort" data-key="overall_em">Overall EM ↓</th>
+                  <th class="sortable" data-key="overall_f1">Overall F1</th>
+                  <th class="sortable" data-key="intra_em">Intra EM</th>
+                  <th class="sortable" data-key="intra_f1">Intra F1</th>
+                  <th class="sortable" data-key="inter_em">Inter EM</th>
+                  <th class="sortable" data-key="inter_f1">Inter F1</th>
+                </tr>
+              </thead>
+              <tbody></tbody>
+            </table>
+          </div>
+        </section>
+
+        <!-- Submit View -->
+        <section id="view-submit" class="view-section" style="display: none;">
+          <div class="submit-container">
+            <div class="submit-card">
+              <h2><i class="fas fa-code-pull-request"></i> Submit via Pull Request</h2>
+              <p>
+                Upload a <code>.json</code> file containing your method metadata and predictions. 
+                The system will create a <strong>Pull Request</strong> on the 
+                <a href="https://huggingface.co/spaces/" target="_blank">Space repository</a>. 
+                Maintainers will review, run evaluation, and merge your results into the leaderboard.
+              </p>
+
+              <div class="submit-flow">
+                <div class="flow-step">
+                  <div class="flow-icon"><i class="fas fa-upload"></i></div>
+                  <div class="flow-text"><strong>1. Upload</strong><br>Submit your JSON file below</div>
+                </div>
+                <div class="flow-arrow"><i class="fas fa-arrow-right"></i></div>
+                <div class="flow-step">
+                  <div class="flow-icon"><i class="fas fa-code-pull-request"></i></div>
+                  <div class="flow-text"><strong>2. PR Created</strong><br>A Pull Request is opened automatically</div>
+                </div>
+                <div class="flow-arrow"><i class="fas fa-arrow-right"></i></div>
+                <div class="flow-step">
+                  <div class="flow-icon"><i class="fas fa-check-circle"></i></div>
+                  <div class="flow-text"><strong>3. Review &amp; Merge</strong><br>Maintainers evaluate and publish scores</div>
+                </div>
+              </div>
+              
+              <h4>JSON Format</h4>
+              <pre class="code-block">{
+  "meta": {
+    "method_name": "My-Agent",
+    "organization": "My-Org",
+    "project_url": "https://github.com/...",
+    "agent_framework": "ImageSeeker",
+    "backbone_model": "Gemini-3-Pro",
+    "retriever_model": "Qwen3-VL-Embedding-8B",
+    "track": "Standard"
+  },
+  "predictions": {
+    "query_001": ["photo_id_1", "photo_id_2"],
+    "query_002": ["photo_id_5"],
+    ...
+  }
+}</pre>
+              
+              <h4>Field Descriptions</h4>
+              <div class="field-desc">
+                <p><code>meta.method_name</code>: Display name shown on the leaderboard (required)</p>
+                <p><code>meta.organization</code>: Your team or organization name (optional, not displayed)</p>
+                <p><code>meta.project_url</code>: Link to paper, code, or project page (optional)</p>
+                <p><code>meta.agent_framework</code>: Agent framework used (e.g. ReAct, ImageSeeker)</p>
+                <p><code>meta.backbone_model</code>: Main LLM/VLM backbone (e.g. GPT-4o, Gemini-3-Pro)</p>
+                <p><code>meta.retriever_model</code>: Retrieval model used (e.g. Qwen3-VL-Embedding-8B)</p>
+                <p><code>meta.track</code>: Must be <code>"Standard"</code> or <code>"Open"</code></p>
+                <p><code>predictions</code>: A dict mapping each query ID to a list of predicted photo IDs</p>
+              </div>
+
+              <form id="submit-form" class="upload-form">
+                <input type="file" name="file" id="submit-file" accept=".json" required>
+                <button type="submit" class="btn primary" id="submit-btn">
+                  <i class="fas fa-paper-plane"></i> Submit &amp; Create PR
+                </button>
+              </form>
+
+              <!-- Submission Result -->
+              <div id="submit-result" style="display: none;"></div>
+            </div>
+          </div>
+        </section>
+
+      </main>
+      
+      <!-- Citation -->
+      <section class="citation-section">
+        <h3><i class="fas fa-quote-left"></i> Citation</h3>
+        <pre class="code-block citation-block">@misc{deng2026deepimagesearchbenchmarkingmultimodalagents,
+  title={DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories}, 
+  author={Chenlong Deng and Mengjie Deng and Junjie Wu and Dun Zeng and Teng Wang and Qingsong Xie and Jiadeng Huang and Shengjie Ma and Changwang Zhang and Zhaoxiang Wang and Jun Wang and Yutao Zhu and Zhicheng Dou},
+  year={2026},
+  eprint={2602.10809},
+  archivePrefix={arXiv},
+  primaryClass={cs.CV},
+  url={https://arxiv.org/abs/2602.10809}
+}</pre>
+      </section>
+
+      <footer class="footer">
+        <p>DISBench Leaderboard &middot; Powered by Hugging Face Spaces</p>
+      </footer>
+    </div>
+
+    <script>
+      window.SERVER_DATA = {{ data | tojson | default([]) }};
+    </script>
+    <script src="{{ url_for('static', filename='script.js') }}"></script>
+  </body>
+</html>