recaptcha-classifier (RC1)
RC1 is a pair of small, fast image recognition models that select the matching cells in reCAPTCHA v2 image grids. The 3×3 model classifies each of the nine cells separately, while the 4×4 model processes the whole grid at once. Both run locally on a CPU with ONNX Runtime. On my $2 VPS, the 3×3 model processes about 9 grids per second using two CPU threads.
Inference in this demo ran on my $2 VPS. The left window shows a 4×4 grid and the right window shows a 3×3 grid.
Models
| Model | Backbone | Input | Supported targets |
|---|---|---|---|
grid9.onnx |
MobileNetV4 (mobilenetv4_conv_medium.e250_r384_in12k_ft_in1k) |
Nine cells from a 3×3 grid, each resized to 224×224 | bicycle, bus, car, crosswalk, fire hydrant, motorcycle*, parking meter*, stairs*, traffic light* |
grid16.onnx |
ConvNeXt Tiny (convnext_tiny.dinov3_lvd1689m) |
One 4×4 grid image, resized to 448×448 | bicycle, bus, chimney*, crosswalk, fire hydrant, motorcycle, parking meter*, stairs, taxi*, tractor, traffic light |
grid16.onnx is built with DINOv3 and is distributed under the
DINOv3 License, not the MIT License. See License.
* These targets are rare. Across all splits, the 3×3 dataset has 14 grids for motorcycle, 7 for traffic light, 2 for parking meter, and 1 for stairs. The 4×4 dataset has 7 grids for parking meter, 13 for chimney, and 108 for taxi.
Each model outputs a probability for every cell. The inference script selects
the cells at or above one tuned threshold per model: 0.475 for grid9.onnx
and 0.45 for grid16.onnx. The threshold is stored in the model file.
Evaluation
Accuracy of the released ONNX models on recaptcha-67k. Exact means every cell prediction matched the reference label.
| Model | Split | Grids | Exact | Cell precision | Cell recall | Cell F1 |
|---|---|---|---|---|---|---|
grid9.onnx |
review |
300 | 81.0% | 0.978 | 0.948 | 0.963 |
grid9.onnx |
test (random 600) |
600 | 82.5% | 0.976 | 0.958 | 0.967 |
grid16.onnx |
review |
282 | 63.5% | 0.915 | 0.939 | 0.927 |
grid16.onnx |
test (random 600) |
600 | 65.7% | 0.944 | 0.948 | 0.946 |
These numbers measure agreement with model-generated labels, not live challenge pass rates:
- The
testlabels come from the same labelers RC1 was trained on: Qwen for 3×3 and SAM 3 for 4×4. - The
reviewlabels were checked by Claude Opus 5.5, not by humans. - The original machine labels exactly matched Claude’s review on 81.3% of 3×3 grids and 70.6% of 4×4 grids.
Among 4×4 targets with at least 10 reviewed grids, exact agreement was lowest for stairs (10/18), traffic lights (29/48), and motorcycles (52/86).
Running inference
Requires Python 3.11 or newer. From this directory, install the dependencies and run the prediction script:
python -m pip install onnxruntime numpy pillow
python predict_onnx.py --model weights/grid16.onnx --image challenge.jpg --target buses
Use weights/grid9.onnx for 3×3 grids and weights/grid16.onnx for 4×4 grids.
The script prints JSON with a probability for each cell and a selected_cells
list. Cells are numbered from 0, left to right and then top to bottom.
To limit the number of CPU threads, pass --threads. For all options, run
python predict_onnx.py --help.
Benchmarks
Approximate CPU throughput once the model is loaded. Times include image loading and preprocessing.
| Machine | CPU threads | 3×3 grids/sec | 4×4 grids/sec |
|---|---|---|---|
| Apple M2 Pro | 4 | ~10 | ~7 |
| $2 VPS | 2 | ~9 | ~4 |
Training and export
Training requires Python 3.11, uv, and a local copy of the recaptcha-67k dataset. From this directory:
# Install dependencies
uv sync --locked
# Prepare the dataset
uv run python train.py prepare --data /path/to/recaptcha-67k --output runs/rc
# Train the 3×3 and 4×4 models
uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 9
uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 16 --backbone convnext_tiny.dinov3_lvd1689m
# Export to ONNX
uv run --group export python train.py export --checkpoint runs/rc/grid9/best.pt
# Run tests
uv run --group export python -m unittest test_training
Notes:
- Training uses a GPU if one is available. On CUDA, add
--ampto reduce memory usage. - The
preparestep keeps the dataset's existing splits and leaves out theexcludedsplit. - All training outputs are written locally to
runs/. - For dataset and training options, run
python train.py --helpor add--helpto any subcommand.
Roadmap
- Improve 4×4 accuracy, which trails both the 3×3 model and the SAM 3 labels it was trained on.
- Improve chimney and parking meter recognition with more training data.
- Add training data and model support for bridges.
Disclaimer
RC1 was created for educational and research purposes. It is not affiliated with or endorsed by Google or reCAPTCHA. Users are responsible for complying with all applicable laws and terms of service.
License
RC1's files are under three licenses. NOTICE.md has the details.
| Files | License |
|---|---|
| Source code and documentation | MIT |
weights/grid9.onnx |
MIT, plus Apache-2.0 for its pretrained MobileNetV4 backbone |
weights/grid16.onnx |
DINOv3 License only |
grid16.onnx is fine-tuned from Meta's
DINOv3 ConvNeXt Tiny
(built with DINOv3). The DINOv3 License requires derivatives to be distributed
under its own terms, so the MIT License does not apply to this file. Anyone who
uses or redistributes it must follow the DINOv3 License, including its trade
control and use restrictions.
Third-party imagery in the training dataset and demo is outside the scope of these licenses.
Model tree for bxding/recaptcha-classifier
Base model
timm/convnext_tiny.dinov3_lvd1689m