spreadsheethurts/backend

Wizard

Wizard is a systematic framework for analyzing typecasting behaviors in spreadsheet software. Through feature engineering, we discover and characterize type conversion patterns, then construct explainable decision trees that faithfully reproduce the typecasting behaviors of three major spreadsheets: Microsoft Excel, LibreOffice Calc, and Google Sheets.

Built on this foundation, we provide two practical applications:

• Counterintuitive Behavior Detection: Discover edge cases where spreadsheet behavior defies user expectations by sampling from decision paths, applying mutations, then verified by LLMs and experts.
• Compatibility Analysis: Identify inconsistencies between different spreadsheets and generate portability guides.

Explore interactively on our website.

⚠️ Important Update

It is now 2026, and previous eval scripts may produce incorrect results due to date year inference. The dataset was generated in 2025. When inferring years from dates (e.g., "Jan31" → Datetime(2025,1,31)), the system calculates relative to the current date. We've added PLDI_EVALUATION=1 environment variable to all evaluation scripts to handle this year discrepancy correctly.Please re-download the repository to get the latest scripts.

Quick Start

Prerequisites

• Docker and Docker Compose: Required for setting up the Redis and PostgreSQL databases.
• uv: Required to manage this project: curl -LsSf https://astral.sh/uv/install.sh | sh

Installation

# Clone repository
git clone https://github.com/spreadsheethurts/spreadsheethurts.git
cd spreadsheethurts
# Note: Ensure you have the latest version for 2026 compatibility

# Install dependencies 
uv sync

Dataset Setup

To set up the environment and prepare the datasets for evaluation, follow these steps:

# Run the following in the spreadsheethurts project root
# 1. Download dataset from Huggingface
mkdir -p volume/redis/data
wget https://huggingface.co/spreadsheethurts/dataset/resolve/main/dump.rdb -O volume/redis/data/dump.rdb

# 2. Install Redis to store dataset and postgresql to store application intermediate and final results
docker compose up -d 

# 3. Verify dataset loading (wait ~30s on first run)
# Expected: Approximately 7M, 6M, and 10M keys distributed across DBs
docker exec -it spreadsheethurts-redis-1 redis-cli info keyspace

Note: For counterintuitive app which use LLMs , you'll need to set up your API key. Copy .env.example to .env and fill your API key:

cp .env.example .env
# Open .env and replace with your actual provider (Gemini-2.5-flash is Recommended)

Project Layout

src/wizard/
├── features/               # Discovered typecasting features for each spreadsheet
│   ├── calc/
│   ├── excel/
│   └── gsheet/
├── classifier.py           # Equivalence class checking
├── argumentation/          # Data argumentation
├── app/
│   ├── rule/               # Decision tree construction for each spreadsheet
│   │   ├── calc/
│   │   ├── excel/
│   │   └── gsheet/
│   ├── counterintuitives/  # Intra-spreadsheet application
│   └── compatibility/      # Inter-spreadsheet application

Evaluation & Validation

Typecasting

Objective: Validate that our decision trees faithfully reproduce actual spreadsheet typecasting behavior

CLI Tool: bin/typecast_eval.py

export PYTHONPATH=src

# Basic evaluation
uv run bin/typecast_eval.py calc --size 1000

# Parallel processing for large datasets (adjust batch-size according to your CPUs, larger batch-size for better CPUs)
uv run bin/typecast_eval.py excel --parallel --batch-size 10000

# Re-validate from previous discrepany report
uv run bin/typecast_eval.py gsheet --from-report

Counterintuitive App

Objective: Discover counterintuitive edge cases in spreadsheet typecasting behavior

This app contains two stages:

• stage 1: Sample inputs from decision paths and generate mutation pairs
• stage 2: Ask LLMs to detect counterintuitive behaviors

CLI Tool: bin/counterintuitive.py

export PYTHONPATH=src

# stage 1: Generate candidates (adjust batch-size according to your CPUs)
uv run bin/counterintuitive.py generate calc --batch-size 10000
# Populates the 'mutation_records' table in the counterintuitive database.

# stage 2: Analyze (online mode with real-time LLM)
uv run bin/counterintuitive.py analyze calc --online --max-concurrent 50
# Populates the 'analysis_results' table and 'counterintuitive' view.

# One-shot: Run both stages
uv run bin/counterintuitive.py run-all calc --batch-size 10000

# View database statistics
uv run bin/counterintuitive.py stats

Compatibility Analysis App

Objective: Identify inconsistencies between spreadsheets and generate portability guides

The compatibility analysis app contains three stages:

• stage 1: Collect typecasting data (e.g., decision path, outcome) using the common input set from their data argumentations for specified spreadsheets and save results to SQL database
• stage 2a: Inconsistency analysis on the stage 1 table, comparing two spreadsheets for recognition discrepancies (i.e., one typecasts to number, the other to text) and interpretive (semantic) discrepancies (i.e., both as number, but with different semantics or values)
• stage 2b: Portability analysis on the stage 1 table

CLI Tool: bin/compat.py

export PYTHONPATH=src

# stage 1: Collect typecasting data
uv run bin/compat.py collect calc excel --redis-db 14 --parallel --batch-size 10000
# Populates the 'typecasting_results' table in the compatibility database.

# stage 2a: Find discrepancies
uv run bin/compat.py diff calc excel
# Populates the 'interp_discrepancies', 'interp_discrepancy_instances', 'rec_discrepancies', and 'rec_discrepancy_instances' tables.

# stage 2b: Find safe intersection
uv run bin/compat.py intersect calc excel gsheet
# Populates the 'portability_summary' and 'safe_inputs' tables in the compatibility database.

# View statistics
uv run bin/compat.py stats