JingweiNi/ReProbe-Qwen3-1.7B-DAPO-block-layer16

Qwen3-1.7B DAPO block-layer16 ReProbe

This repository contains a step-level uncertainty head for Qwen/Qwen3-1.7B. It scores block-segmented reasoning steps from decoder layer 16. Lower sigmoid probabilities indicate lower predicted step error risk.

The published weights are the pre-selected epoch-2 checkpoint from training seed 43. Seed 43 is the median-performing block_layer16 run across seeds 42, 43, and 44, and was selected for publication to avoid cherry-picking the highest dev seed.

Files

• weights.pth: PyTorch state dictionary for the uncertainty head.
• config.yaml: loader-compatible head architecture and feature-extractor configuration.
• metadata.json: machine-readable training, metric, and provenance contract.
• SHA256SUMS: package digests.

This is not a standalone language model. It requires the pinned Qwen3-1.7B base model and the uncertainty-head implementation from uncertainty4reasoning.

Base model and architecture

Field	Value
Base model	Qwen/Qwen3-1.7B
Base revision	70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
Segmentation	Block reasoning steps
Hidden-state layer	Decoder layer 16 of 28
Head type	step_reasoning
Head dimension	768
Head layers / attention heads	2 / 8
Dropout	0.1
Future-token masking	Enabled
Training seed / selected epoch	43 / 2

Loading

from transformers import AutoModelForCausalLM
from luh import AutoUncertaintyHead

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-1.7B",
    revision="70d244cc86ccca08cf5af4e1e306ecf908b1ad5e",
    torch_dtype="auto",
)
uhead = AutoUncertaintyHead.from_pretrained(
    "JingweiNi/ReProbe-Qwen3-1.7B-DAPO-block-layer16",
    base_model,
)
uhead.eval()

Correct inference also requires the training-compatible chat template, stored token IDs, block-step boundaries, completion offsets, and claim masks. See bestofn_optimized/score_qwen3_reprobe_3arm.py in the linked code repository for the exact fused-scoring contract. Do not retokenize a trace and assume its step masks still align.

Direct checkpoint metrics

The checkpoint was selected by external-dev PR-AUC with early-stopping patience 3. Dev labels are a fixed synthetic-proxy set produced by a Qwen3.6-27B judge.

Metric	Seed 43, epoch 2
PR-AUC	0.410679
PR-AUC / prevalence	4.046096
ROC-AUC	0.799337
Thresholded F1	0.310571

Across seeds 42, 43, and 44, block_layer16 obtained PR-AUC 0.408683 +/- 0.007027 (sample standard deviation). Layer 16 had a higher block-step dev PR-AUC than the last layer in all three seeds, but every per-seed question-bootstrap interval included zero. This is consistent directional evidence, not statistically decisive superiority.

Training data contract

Split	Finite step labels	Positive labels	Positive rate
Train	10,000	1,320	13.20%
Dev	2,000	203	10.15%

Train and dev question/source fingerprints were disjoint. The labels are synthetic step-correctness judgments, not human annotations or native DAPO ground truth.

Best-of-8 evidence and aggregation

The uploaded seed-43 weights were not the checkpoint used in the downstream Best-of-8 experiment. That frozen test used the same recipe's seed-42 epoch-2 checkpoint:

• primary pre-registered max aggregation: 71.00% (213/300);
• shortest-response baseline: 70.33% (211/300);
• exact random-selection expectation: 66.08%;
• layer16 minus shortest: +0.67 percentage points, 95% paired bootstrap interval [-2.33, +3.67].

After opening the test result, a post-hoc mean reduction reached 73.33% (220/300) with the seed-42 checkpoint. Its gain over seed-42 max was +2.33 points with interval [-0.33, +5.33]. This reduction comparison is exploratory and must not be treated as a validated setting for the uploaded seed-43 head.

Limitations

• The checkpoint is tied to Qwen3-1.7B at the pinned revision, decoder layer 16, and block segmentation.
• Training/dev supervision is synthetic and judge-dependent.
• The 300-question Best-of-8 test set has been consumed and must not be reused for checkpoint or aggregation selection.
• The experiment does not establish that layer 16 generally beats the last layer or that ReProbe reliably beats shortest-response selection.
• Calibrate or validate the trajectory aggregation on fresh data before using the head for selection in another domain.

Integrity

The published weights.pth must have SHA-256 29ebb9df957beae16a3440f4466be22ac8e451d26282af5e738c4a2b6e48b512. The package's SHA256SUMS file covers the weights, config, model card, and metadata.

License

Apache-2.0. The base model is also published under Apache-2.0; users remain responsible for complying with the base model's terms and applicable law.