agents-last-exam/agents-last-exam huggingface dataset:
Title: Retail ETL Warehouse Cleanup
Domain: computing_math — Data & Analytics Engineering

Summary: Build a cleaned SQLite star-schema warehouse plus two truthful JSON sidecars from messy retail CSV, JSON, and TSV source files on a Linux VM.

Task prompt:
You are working on a Linux VM to build a cleaned SQLite warehouse from messy retail source files.

Visible task files:
- `base/input/task_prompt.md`
- `base/input/target_schema_spec.json`
- `base/input/output_contract.json`
- `base/input/raw_transactions/`
- `base/input/raw_customers/customers.json`
- `base/input/raw_products/product_catalog.tsv`
- `base/input/runtime_env/`
- `base/software/README.txt`
- `base/software/python_etl_env.sh`

What you must do:
1. Read the staged task prompt, schema spec, and output contract.
2. Build the final SQLite warehouse and truthful JSON sidecars from the staged raw inputs.
3. Write these three deliverables at the output-directory root:
   - `base/output/warehouse.db`
   - `base/output/data_quality_report.json`
   - `base/output/warehouse_summary.json`
4. If you use the staged helper wrapper, its runtime scratch may live under `base/output/_runtime`.

Rules:
- Do not modify files under `input/`.
- Keep the final deliverables at the output-directory root.
- Write valid SQLite / JSON outputs only.


The agent must:
Read the staged prompt, schema spec, output contract, and messy retail source files under `input/`.
Build a cleaned SQLite warehouse with deduplication, null handling, standardization, surrogate keys, and a full `dim_dates` table.
Write `output/warehouse.db`, `output/data_quality_report.json`, and `output/warehouse_summary.json`.
Keep the JSON sidecars truthful to the produced warehouse.

Software: Python 3.10
uv
pandas
SQLite

Input files:
name: task_prompt.md
format: Markdown
path: input/task_prompt.md
description: Agent-visible task brief for the ETL workflow.
name: target_schema_spec.json
format: JSON
path: input/target_schema_spec.json
description: Semantic warehouse schema contract and transformation requirements.
name: output_contract.json
format: JSON
path: input/output_contract.json
description: Required deliverable filenames and JSON key structure.
name: raw_transactions
format: CSV directory
path: input/raw_transactions
description: Six monthly transaction CSV files with schema drift and dirty rows.
name: customers.json
format: JSON
path: input/raw_customers/customers.json
description: Retail customer records including unresolved ghost-customer rows.
name: product_catalog.tsv
format: TSV
path: input/raw_products/product_catalog.tsv
description: Product catalog with supplier, category, and boolean normalization issues.
name: runtime_env
format: uv Python environment
path: input/runtime_env
description: Optional locked Python environment staged for the agent.
name: python_etl_env.sh
format: Shell wrapper
path: software/python_etl_env.sh
description: Optional helper wrapper that runs the staged Python environment and keeps scratch under `output/_runtime/`.
Title: Repair Standalone DiT Pipeline CFG Alignment
Domain: computing_math — AI Engineering, Safety & CS Research

Summary: Repair a buggy standalone DiT sampling pipeline so its classifier-free guidance behavior matches the intended reference implementation while preserving the one-file edit workflow.

Task prompt:
You are repairing a standalone Diffusers-based DiT pipeline on a Linux VM.

Task directory:
- `base`

Visible files:
- Buggy starter pipeline: `base/input/pipeline_dit.py`
- Raw sampling harness context: `base/input/sample.py`
- Raw run script context: `base/input/run_sample.sh`
- Benchmark-owned prompt: `base/input/task_prompt.md`

Your task:
1. Read `base/input/task_prompt.md` first.
2. Repair the standalone starter file at `base/input/pipeline_dit.py`.
3. Keep the fix localized to the pipeline file.
4. Save exactly one final file to:
   `base/output/pipeline_dit.py`

Do not modify files under `input/`.

## Required Pipeline Behavior

- `DiTPipeline.__call__` must expose `cfg_on_3_channels: bool = False`.
- With classifier-free guidance active, `True` applies guidance only to the first three predicted channels; `False` applies it to all latent channels. Preserve all remaining predicted channels.
- Support schedulers with or without `scale_model_input`.
- Preserve learned/learned-range variance outputs, and pass only the denoising half when a fixed-variance scheduler receives a prediction with twice the latent-channel count.
- Keep the submitted module standalone; it must import successfully outside the original Diffusers package tree.


The agent must:
Read the benchmark-owned task prompt and inspect the buggy standalone `pipeline_dit.py` plus the raw context files `sample.py` and `run_sample.sh`.
Repair the standalone DiT pipeline so the guidance behavior matches the intended reference behavior.
Keep the fix localized to the pipeline file rather than replacing the broader workflow.
Write exactly one final file to `output/pipeline_dit.py`.

Software: Python
Diffusers
PyTorch

Input files:
name: pipeline_dit.py
format: Python
path: input/pipeline_dit.py
description: Buggy standalone starter pipeline.
name: sample.py
format: Python
path: input/sample.py
description: Raw sampling harness context from the submission.
name: run_sample.sh
format: Shell
path: input/run_sample.sh
description: Raw launch script context from the submission.
name: task_prompt.md
format: Markdown
path: input/task_prompt.md
description: Benchmark-owned solve-time instructions.
Title: Ghidra Malware Config Extraction
Domain: computing_math — Cybersecurity & Digital Forensics

Summary: Reverse engineer a packed Windows malware sample in Ghidra and recover the configuration fields requested by the visible JSON schema.

Task prompt:
You are a malware analyst working on a Windows VM.

## Your Task
Use Ghidra to reverse engineer the staged executable and recover the malware configuration requested by the visible schema.

## Visible Files
- Sample to analyze: `base\input\sample.exe`
- Output schema: `base\input\output_schema.json`
- Launch Ghidra from: `base\software\launch_ghidra.bat`

## What You Must Do
1. Launch the staged task-local Ghidra runtime.
2. Create a local non-shared project and import `sample.exe`.
3. Let Ghidra analyze the binary, inspect the listing and decompiler, and trace the configuration-decryption logic.
4. Recover the packer, C2, obfuscation, and architecture details requested by the schema.
5. Save exactly one schema-valid JSON file to `base\output\malware_config.json`.

## Output Requirements
- Filename must be `malware_config.json`
- Write the file under `base\output`
- Follow the visible schema in `output_schema.json`


The agent must:
Launch the staged task-local Ghidra runtime from `software/launch_ghidra.bat`.
Import `input/sample.exe` into a local non-shared Ghidra project.
Recover the packer, C2, obfuscation, and architecture details requested by the visible schema.
Save exactly one schema-valid `output/malware_config.json`.

Software: Ghidra 11.3
JDK 21

Input files:
name: Packed malware sample
format: Windows PE
path: input/sample.exe
description: The executable the agent must reverse engineer in Ghidra.
name: Visible output schema
format: JSON
path: input/output_schema.json
description: Defines the required top-level sections, nested fields, and constrained enum values.
name: Ghidra launcher
format: BAT
path: software/launch_ghidra.bat
description: Starts the staged task-local Ghidra 11.3 runtime with the pinned JDK 21 bundle.
Title: Go Game Reconstruction 1
Domain: computing_math — Sports

Summary: Reconstruct a professional 19x19 Go game inside Sabaki from a final board image, five known opening moves, and game metadata, then export the result as SGF.

Task prompt:
You are reconstructing a professional 19x19 Go game in Sabaki on Ubuntu.

## Your Task
Use the final board image plus the known metadata to reconstruct the game move by move inside Sabaki, then export the reconstructed game as SGF.

## Visible Inputs
- Final board image: `base/input/input-board-position.png`
- Sabaki AppImage: `base/software/sabaki-v0.52.2-linux-x64.AppImage`

## Known Constraints
- Total moves: `168`
- Rules: `Chinese rules`
- Result: `White wins by resignation`
- Move 1: `B at R4`
- Move 2: `W at Q16`
- Move 3: `B at C4`
- Move 4: `W at C16`
- Move 5: `B at E3`

## What You Must Do
1. Launch Sabaki from the staged AppImage.
2. Use Sabaki's GUI to enter the game move by move.
3. Do not use a Go engine or a web browser.
4. Export the SGF into `base/output/`.
5. Prefer the exact filename `reconstructed_game.sgf`.

Only files under `base/output/` count as your output.

The agent must:
Inspect the staged final-board image under `input/`.
Launch the staged Sabaki AppImage on the Ubuntu VM and use the GUI rather than bypassing the task with a browser or Go engine.
Reconstruct the move sequence using the visible board image, five fixed opening moves, and the provided metadata.
Export one SGF into `output/`, preferably as `reconstructed_game.sgf`.

Software: Sabaki v0.52.2
Python
sgfmill

Input files:
name: Final board image
format: PNG image
path: input/input-board-position.png
description: The final 19x19 board state shown to the agent.
name: Sabaki runtime
format: Linux AppImage
path: software/sabaki-v0.52.2-linux-x64.AppImage
description: Official Sabaki v0.52.2 runtime artifact staged on the Ubuntu VM.
Title: Finite Abelian Extension Classification
Domain: computing_math — Mathematical & Operations Research

Summary: Solve a staged finite abelian group extension classification problem in GAP and produce the exact canonical JSON answer. The benchmark contains 6 variants with different abelian groups H and search ranges for m.

Task prompt:
You are working on a Linux VM.

## Your Task
Solve the staged finite abelian group extension classification problem for variant `h_4_4_4_m_1_8` ((Z/4)^3, m=1..8).

## Visible Files
- Variant config: `h_4_4_4_m_1_8/input/config.json`
- Mathematical task specification: `h_4_4_4_m_1_8/input/task_specification.md`
- GAP launcher: `h_4_4_4_m_1_8/software/run_gap.sh`

You may also use `/usr/bin/gap`, Python, shell scripts, or text editors available on the VM.

## What You Must Do
1. Read `h_4_4_4_m_1_8/input/config.json` and `h_4_4_4_m_1_8/input/task_specification.md` carefully.
2. Enumerate every finite abelian group `G` in the required short exact sequences across the full `m` search range.
3. Determine the `product_type` classification for every valid extension group.
4. Save exactly one JSON file at `h_4_4_4_m_1_8/output/results.json`.

## Required Output Schema
Write one JSON object with these top-level keys:
- `total_extensions`
- `extensions`
- `non_product_type_count`
- `non_product_type`

Each object in `extensions` and `non_product_type` must contain:
- `m`
- `G_invariant_factors`
- `G_order`
- `product_type`

Use invariant factors in ascending order with the divisibility condition satisfied. Order `extensions` by `(m, G_invariant_factors)`.

## Output Rules
- Save your final deliverable only under `h_4_4_4_m_1_8/output`.
- Do not modify files under `h_4_4_4_m_1_8/input`.


The agent must:
Read the staged `input/config.json` and `input/task_specification.md` for the active variant.
Use GAP to enumerate every finite abelian group `G` that fits into the required short exact sequences across the full `m` search range.
Determine the `product_type` classification for every valid extension group.
Write exactly one canonical `output/results.json` with the required top-level keys and per-extension schema.
Keep `G_invariant_factors` in ascending invariant-factor form and order `extensions` by `(m, G_invariant_factors)`.
Do not modify the staged input files.

Software: GAP
Python

Input files:
name: config.json
format: JSON
path: input/config.json
description: Variant parameters defining `H_invariant_factors`, `H_order`, and the inclusive `m_search_range`.
name: task_specification.md
format: Markdown
path: input/task_specification.md
description: Variant-specific mathematical contract and the required JSON schema.
name: run_gap.sh
format: Shell script
path: software/run_gap.sh
description: Stable task-local launcher for GAP on the selected Ubuntu VM.
Title: Kubernetes Payment API Root Cause Analysis
Domain: computing_math — Data & Analytics Engineering

Summary: Analyze static Kubernetes incident artifacts for a flapping payment-api deployment and produce a grounded JSON root-cause report.

Task prompt:
You are the on-call SRE for a Kubernetes production incident.

## Task Directory
`base`

## Visible Inputs
- Task prompt: `base/input/task_prompt.md`
- Cluster state capture: `base/input/cluster_state.txt`
- Current Deployment and HPA YAML: `base/input/deployment.yaml`
- Crashing pod log: `base/input/failing_pod.log`

## Your Task
Analyze the supplied static evidence and write a complete root-cause analysis
report for the `payment-api` incident, including why the HPA reports memory
utilization above target. Ground every root cause in literal
evidence from the visible files, list affected Kubernetes resources, propose a
prioritized remediation plan, and state whether rollback is safe.

## Required Output
Write exactly one JSON file:

```text
base/output/root_cause_analysis.json
```

The JSON must include these top-level keys:

```text
incident_metadata, root_causes, affected_resources, remediation_plan, summary
```

### Schema details

**`incident_metadata`** — object with at least:
- `safe_to_rollback` (boolean): whether rolling back to the previous ReplicaSet revision is safe.

**`root_causes`** — array of objects, each with at least:
- `type` (string): short classification label, e.g. `"OOMKilled"`, `"liveness_probe_too_aggressive"`, `"missing_metrics_port_env"`.
- `severity` (string): one of `"critical"`, `"high"`, `"medium"`, `"low"`.
- `description` (string): explanation of the root cause.
- `evidence` (array of strings): each item must be a verbatim text string copied from one of the visible input files (not a file-name citation — the actual text that appears in the file).

**`affected_resources`** — array of objects, each with `kind`, `name`, and `namespace`.

**`remediation_plan`** — array of objects, each with at least an `action` (string).

**`summary`** — string summarizing the incident and recommended path forward.

## Constraints
- Treat `base/input` as read-only.
- Keep generated files inside `base/output`.
- Do not use external web sources or invent resource names, metrics, commit SHAs, or timestamps.


The agent must:
Read the task prompt and three static incident artifacts.
Identify Kubernetes root causes and cite literal evidence from the input files.
List affected resources and prioritized remediation steps.
Write output/root_cause_analysis.json.

Software: Python

Input files:
name: task_prompt.md
format: md
path: input/task_prompt.md
description: Sanitized task instructions.
name: cluster_state.txt
format: txt
path: input/cluster_state.txt
description: Captured kubectl pod, event, top, deployment, and HPA output.
name: deployment.yaml
format: yaml
path: input/deployment.yaml
description: Currently applied Deployment and HPA manifests.
name: failing_pod.log
format: txt
path: input/failing_pod.log
description: Logs from a crashing payment-api pod across restart cycles.
Title: Checkpoint Consolidation V2
Domain: computing_math — AI Engineering, Safety & CS Research

Summary: Infer the non-standard TP/PP/EP checkpoint layout from staged framework code and shard files, then reconstruct a single HuggingFace-compatible `model.safetensors` artifact.

Task prompt:
You are working on a Linux checkpoint-consolidation task.

## Task Directory
`base`

## Visible Inputs
- Checkpoint shards: `base/input/checkpoints`
- Framework source: `base/input/framework`
- Reference model code: `base/input/reference_model/model.py`
- Reference model config: `base/input/reference_model/config.json`
- Self-check input IDs: `base/input/reference_output/input_ids.pt`
- Self-check target logits: `base/input/reference_output/logits.pt`
- Expected checkpoint key names: `base/input/reference_output/expected_keys.json`
- Detailed instructions: `base/input/task_instructions.md`
- Optional runtime manifest: `base/input/runtime_env/pyproject.toml` and `base/input/runtime_env/uv.lock`

## Variant Facts
- Variant: `base` (Base)
- Visible shard count: `8`
- Submission-stated parallel layout: `2-way Tensor Parallel x 2-way Pipeline Parallel x 2-way Expert Parallel`

## Your Task
1. Inspect the visible framework code and shard filenames to infer the checkpoint layout.
2. Reconstruct a single-device checkpoint compatible with the staged reference model.
3. Save exactly one file at `base/output/model.safetensors`.

## Rules
- Treat `base/input` as read-only.
- You may self-check by loading the staged reference model and comparing logits on `base/input/reference_output/input_ids.pt`.
- If you want a task-local Python environment, run `uv sync --frozen --project base/input/runtime_env`.
- Write your final artifact only under `base/output`.


The agent must:
Inspect the staged framework code and shard filenames to infer the checkpoint layout.
Reconstruct one single-device checkpoint compatible with the staged reference model.
Save exactly one file named `model.safetensors` under the active variant's `output/` directory.
Produce a checkpoint that matches the hidden oracle structure and reproduces the hidden logits within tolerance.

Software: Python
PyTorch
safetensors
uv

Input files:
name: Checkpoint shards
format: PyTorch .pt files
path: input/checkpoints/
description: The staged TP/PP/EP shard files for the active variant.
name: Framework source
format: Python
path: input/framework/
description: The visible source of truth for the custom checkpoint layout.
name: Reference model
format: Python + JSON
path: input/reference_model/
description: The staged HuggingFace-style model definition and config.
name: Reference output
format: PyTorch tensors + JSON
path: input/reference_output/
description: Visible self-check tensors and expected key names for the active variant.
name: Runtime manifest
format: uv Python environment
path: input/runtime_env/
description: Optional pinned task-local runtime manifest.
Title: Newyear Keygen2
Domain: computing_math — Cybersecurity & Digital Forensics

Summary: Reverse engineer the bundled program, recover the key-generation logic, and write the correct answer for the staged UID.

Task prompt:
Provided UID:
- `20252025`

Goal:
Reverse the bundled Windows crackme and compute the correct password for the provided UID.

Inputs:
- Executable to analyze: base\input\crackme.exe

Output:
- Save EXACTLY one UTF-8 text file to: base\output\key.txt
- The file must contain exactly one line in the form `flag{...}`
- Do not add commentary, labels, or extra blank lines

Important:
- The valid password depends on the current UTC time.
- Use the most recent UTC half-hour boundary, i.e. the current UTC timestamp rounded down to a multiple of 1800 seconds.

Verification:
- Score is 1.0 only if the output matches the password for this UID using the evaluation-time UTC half-hour slot.

The agent must:
Reverse the bundled crackme.
Use the provided UID `20252025`.
Compute the password for the most recent UTC half-hour slot.
Write exactly one line to `output/key.txt`.

Software: Python

Input files:
name: crackme.exe
format: Windows PE
path: input/crackme.exe
description: agent-visible crackme
Title: OS Log Permission Guard
Domain: computing_math — Infrastructure Engineering & Cloud Operations

Summary: Safely update sandboxed Linux log-file permissions from a filesystem snapshot while preserving protected system-owner, active-writer, and non-log files.

Task prompt:
You are working on a Linux VM.

## Task Directory
`base`

## Visible Inputs
- Filesystem snapshot: `base/input/fs_snapshot.tar.gz`
- Ownership metadata: `base/input/ownership.csv`
- Initial permission metadata: `base/input/permissions.csv`
- Active-writer exclusion list: `base/input/active_writers.json`
- Workspace setup helper: `base/input/setup_workspace.sh`
- Detailed instructions: `base/input/task_instructions.md`

## Your Task
Update the sandboxed `/var/logs` file permissions while respecting the safety constraints in the metadata files.

Use this workflow:
1. Read `base/input/task_instructions.md`.
2. Run `base/input/setup_workspace.sh "base/output"` to create a writable workspace at `base/output/sandbox_fs`.
3. Change every regular `.log` file under `base/output/sandbox_fs/var/logs` to mode `444`, except files owned by `syslog` in `ownership.csv` and files listed in `active_writers.json`.
4. Do not modify non-`.log` files. Do not change file contents.
5. Do not change ownership/group metadata in your final state.
6. Save final metadata to `base/output/final_state.json`.

The active-writer list is a provided safety record for this benchmark. Treat listed files as actively written and leave them unchanged; do not terminate or interfere with any processes.

## Output Contract
`base/output/final_state.json` must be valid JSON with a top-level `files` list or mapping. Include one entry for each file listed in `ownership.csv`, with path, type, owner, group, and final mode.

Do not write outputs outside `base/output`.

The agent must:
Read the staged task instructions and metadata.
Create a writable sandbox workspace under `output/sandbox_fs`.
Change eligible `.log` files to mode `444` while leaving protected files unchanged.
Write `output/final_state.json` with final metadata for every listed file.

Software: Bash
GNU coreutils
tar
Python

Input files:
name: fs_snapshot.tar.gz
format: tar.gz
path: input/fs_snapshot.tar.gz
description: Sandbox filesystem snapshot containing `/var/logs` files.
name: ownership.csv
format: csv
path: input/ownership.csv
description: Owner/group metadata source of truth.
name: permissions.csv
format: csv
path: input/permissions.csv
description: Initial file mode source of truth.
name: active_writers.json
format: json
path: input/active_writers.json
description: Declarative list of files to treat as actively written and leave unchanged.