DOI: 10.5281/zenodo.17971721
title: VIPER_code

description: Breaking Visual Reasoning CAPTCHAs with VIPER: A Structured Vision-Language Attack Framework

VIPER_code is an academic and practical Python framework designed for the analysis and attack of Visual Reasoning CAPTCHAs (VRC). It implements a structured, modular vision-language attack pipeline that supports a range of CAPTCHA types — including six major VRC datasets and various non-VRC challenges. The framework enables automated evaluation and experimentation with large language models (LLMs) to break CAPTCHAs in a highly configurable manner.

File & Directory Structure

First, unzip all the compressed files.






File/Folder
Description




main.py
Main entry for experiment orchestration and accuracy testing. Supports command-line arguments to select LLM providers and datasets dynamically.


environment_variable.py
LLM API and environment configuration. You must edit this file before running any experiment.


r1_and_r2_main.py
Script for running “r1” (Direct VLM) and “r2” (YOLO+VLM) series experiments. Configurable via command-line arguments.


solvers/
Modular solver scripts organized by LLM provider.


├── gpt/
Solvers using OpenAI GPT models (e.g., gpt/xiaodun.py).


├── deepseek/
Solvers using DeepSeek models (e.g., deepseek/geetest.py).


├── grok/
Solvers using Grok models.


└── kimi/
Solvers using Kimi models.


r1_r2/
Contains the core logic scripts for r1 and r2 experiments (tools_r1.py, tools_r2.py).


tools_*.py
Other utility scripts.


best_28.pt
Pretrained model weights (PyTorch). May be needed for certain experiments.


data_process/
Contains scripts and raw/intermediate data for transforming datasets.


dataset/
Houses all datasets, including six VRC datasets, their labels, prompts, and evaluation scripts.


other_captcha/
Scripts and results for non-VRC experiments.


result/
Experimental result files (Excel), automatically named based on the experiment parameters.


recaptchaTest/
Specific test scripts for reCAPTCHA.


yolo_training/
YOLO training code and dataset .


requirements.txt
Python dependencies list.


README.md
This documentation.





Directions for Use

1. Configure the Environment





Install Dependencies:

pip install -r requirements.txt




Edit environment_variable.py: Set your LLM API keys and confirm dataset paths.



2. Run Main Experiments

The central script main.py now uses command-line arguments for configuration.

Syntax:

python main.py --provider <PROVIDER_NAME> --dataset <DATASET_NAME>

Examples:

# Run GPT-4o on Xiaodun datasetpython main.py --provider gpt --dataset xiaodun# Run DeepSeek on Geetest datasetpython main.py --provider deepseek --dataset geetest

Supported Providers: gpt, deepseek, grok, kimi Supported Datasets: xiaodun, geetest, shumei, dingxiang, netease, vtt

The script will automatically load the corresponding solver from solvers/{provider}/{dataset}.py, run the evaluation, and save the results to result/{provider}_{dataset}.xlsx.

3. Run r1 / r2 Experiments

Use r1_and_r2_main.py for specialized ablation studies (r1: Direct VLM, r2: YOLO + VLM without filtering).

Syntax:

python r1_and_r2_main.py --mode <r1|r2> --dataset <DATASET_NAME>

Examples:

# Run R1 (Direct VLM) experiment on Xiaodunpython r1_and_r2_main.py --mode r1 --dataset xiaodun# Run R2 (YOLO+VLM) experiment on Shumeipython r1_and_r2_main.py --mode r2 --dataset shumei

4. Datasets and Output





Datasets must be placed in /dataset.




Results are automatically saved in /result with filenames like gpt_xiaodun.xlsx or r1_xiaodun.xlsx.



YOLO Training

The project includes a yolo_training/ directory to reproduce detection model training used in final solver experiments. To train a YOLO model, go into that directory and run:

# change to the yolo_training directorycd yolo_training# install ultralytics (once)pip install ultralytics# start training (assuming train.py exists)python train.py

Note: The dataset inside yolo_training/ was not fully used for other parts of the project. For our experiments we strictly followed the paper's sample counts and used an 800/200/200 split (train/val/test), i.e., only those numbers of samples were employed for training and evaluation to ensure comparability.

Example Project Tree

VIPER_code/├── main.py├── environment_variable.py├── r1_and_r2_main.py├── requirements.txt├── solvers/│   ├── gpt/│   │   ├── xiaodun.py│   │   └── ...│   ├── deepseek/│   └── ...├── r1_r2/│   ├── tools_r1.py│   └── tools_r2.py├── dataset/└── result/

Reproducibility & Recommendations





Python 3.8+ environment recommended.




Some experiments (especially those using YOLO) require a CUDA-enabled GPU with a functional PyTorch setup.




Always check environment_variable.py before executing scripts to ensure the correct API keys are active.