qwen3-1.7b-sampling64-predictor

Question-only success probability predictor: LoRA plus a trained scalar sequence-classification head. Output is sigmoid(logit), not generated text. The score head is included in adapter_model.safetensors (modules_to_save).

Target: Qwen3-1.7B non-thinking sampling64 individual-solution success rate.

Checkpoints and evaluation

Each epoch was selected by minimum development soft BCE, never test accuracy. See selected_checkpoints.json for per-seed epochs, metrics and SHA256. All available selected seeds are retained; the first seed in the example is a fixed example, not the best test seed. Three-seed metrics describe separate models, not an ensemble. Do not mix adapters across base models or revisions. Base revision is pinned in each predictor_metadata.json, with original base weight SHA256. Base weights are downloaded separately and are not redistributed.

Inference

Use Python 3.12; original training used Python 3.12.13, torch 2.11.0+cu128, Transformers 5.17.0 and PEFT 0.21.2. Full package versions: environment_expected.json. Install the appropriate CUDA build of torch and the pinned requirements in a separate environment. Download this repository and run its explicit entrypoint:

from huggingface_hub import snapshot_download
snapshot_download("hellobye9290/qwen3-1.7b-sampling64-predictor", local_dir="predictor")

cd predictor
python predict.py --checkpoint . --queries example_questions.json --output predictions.json

Queries must contain only problem_id and problem. Gold solutions/labels are rejected. --device cpu uses float32; CUDA uses BF16, batch size 2 by default. The input uses the original non-thinking chat template and prompt suffix; more than 4,096 prompt tokens raises an error instead of silently truncating. The script downloads the pinned base model and verifies weight checksums. For other seeds, pass the corresponding seed subfolder as --checkpoint. The portable entrypoint has CPU structural checks; example_reference.json is from saved GPU predictions and is not a new GPU reproduction of this entrypoint.

Interpretation and provenance

This is an experimental predictor of a specific observed solution policy, not a fine-tuned answer generator. Lower prediction error does not establish improved final EGB answer accuracy. Observed success fractions contain sampling noise; calibration can be worse than a constant baseline despite improved BCE. Small EGB splits exhibited overfitting. See the linked dataset for policy selection overlap, split dependencies and source limitations.

Base model: Qwen/Qwen3-1.7B; its upstream terms apply. Dataset: hellobye9290/math-success-sampling64.