# Sabnock Docker Benchmark Report

Total tasks: **108**

## Splits
- `train`: 72
- `challenge`: 12
- `seed`: 12
- `validation`: 12

## Difficulty
- `easy`: 63
- `medium`: 36
- `hard`: 9

## Ecosystems
- `python`: 44
- `node`: 19
- `compose`: 9
- `frontend`: 9
- `go`: 9
- `linux`: 9
- `security`: 8
- `rust`: 1

## Failure Types
- `apt_update_missing`: 9
- `build_secret_leak`: 9
- `compose_bind_mount_masks_dependencies`: 9
- `compose_readiness`: 9
- `container_network_binding`: 9
- `dockerignore_artifact_excluded`: 9
- `lockfile_missing`: 9
- `missing_build_context_file`: 9
- `multistage_artifact_mismatch`: 9
- `native_dependency_missing`: 9
- `poetry_root_install_order`: 9
- `python_src_layout_import`: 9

## Recommended Leaderboard Metrics

- pass_at_1: fraction of tasks where the first submitted patch passes verification
- patch_apply_rate: fraction of patches that apply cleanly
- median_runtime_seconds: median verification wall time
- mean_patch_lines: mean number of changed patch lines
- docker_pass_rate: fraction passing Docker-backed verification