Niarfe/qwen2.5-7b-positional-reasoning

Qwen2.5-7B Positional Reasoning

A QLoRA fine-tune of Qwen2.5-7B-Instruct that solves grade-school math word problems by committing to a prediction before each step is computed, rather than narrating a solution after the fact.

1. Introduction

Large language models solve multi-step arithmetic word problems by generating a chain of reasoning, but that chain is produced after the model has effectively settled on an answer — the reasoning narrates a conclusion rather than deriving one. This matters because a narrated chain gives little signal about where it went wrong: an error sits buried somewhere in a paragraph of fluent prose, undetectable until the final number is checked.

This model was fine-tuned on GSM8K reformatted into a structure called Positional Reasoning Format: a chain of typed nodes where each node must commit to a PREDICT field describing what the next step will accomplish, before that step runs. The design borrows from Hawkins & Ahmad's account of the cortex as a prediction machine, in which intelligence arises from continuous prediction and error correction rather than passive pattern completion. Every reasoning step becomes a checkpoint, so a wrong prediction is a localized, detectable failure instead of a buried one.

A second, independent question was investigated in parallel. The format includes a filter node type intended to handle irrelevant information — a known weakness of LLMs, and one this project documented directly when Llama-3-8B folded an unrelated quantity into its arithmetic simply because the number appeared in the problem. Roughly 17% of training examples carry an injected distractor plus an explicit filter step. This was evaluated as a separate experiment on its own test set, and unlike the main result it came out inconclusive — see Limitations. The two questions are reported separately throughout because they were tested separately; the GSM8K result below does not depend on the distractor result.

Main result: on the full 1,319-problem GSM8K test split, accuracy rose from 75.8% to 81.7% (+5.9 points) over the base model — 78 additional problems solved, a paired McNemar p = 2.4 × 10⁻⁶. For scale, that gain is larger than the 4.2-point gap separating Llama-3-8B-Instruct from Qwen2.5-7B-Instruct, and it was obtained from 556 training examples in 44 minutes on a single NVIDIA T4. General capability declined slightly but measurably (LogiQA2 −2.5, ARC-Challenge −1.0, MMLU −0.5). That combination is the interesting part: a math-specialized model of the same size and family reaches higher raw GSM8K accuracy but sacrifices roughly eleven times more general capability to get there (see Evaluation).

Stated up front: this study establishes that fine-tuning on 556 examples reformatted this way improves GSM8K accuracy. It does not establish that the prediction mechanism is what caused the improvement — no ablation separates it from structured formatting, longer outputs, or ordinary in-domain fine-tuning. That limitation is discussed in Limitations.

2. Data

All data derives from GSM8K. Because this project investigated two separate questions, the materials are described separately below. Everything used for training came from GSM8K's train split; everything used for evaluation came from its test split. The two splits are disjoint by construction, so no test problem was ever trained on.

2a. Training data — the primary (prediction) track

Built from the GSM8K train split (7,473 problems available):

Stage	Result
Selection: every 8th example (deterministic, no seed needed)	935 candidates
Reformatted into Positional Reasoning Format by Claude Opus 4.8, few-shot prompted with 3 hand-authored gold examples	573 processed
Six mechanical validation checks	556 accepted, 17 rejected (97%)
Train/validation split, 90/10, seed=42	500 train / 56 validation

The remaining 362 candidates were never processed. The 500-600 example target had been fixed in advance during project planning, the run met it, and the reformatting API budget was exhausted at roughly the same point — so the cost constraint did not, in the end, bind on the plan. Scaling the corpus is nonetheless the most obvious lever for improving on these results, particularly for the filter behavior described in 2b, which had only ~95 supporting examples.

The six checks are implemented in scripts/validator.py, included in this repository. The most important is the look-ahead detector: a PREDICT field may not contain a number that has not already been established by the question or an earlier OBSERVE/STATE. Without it, the reformatter could silently solve each problem first and back-fill plausible-sounding predictions — which would defeat the entire premise of the format.

The 90/10 split above applies to all 556 training examples, including the distractor-carrying ones described next.

2b. Distractor materials — the secondary (filter) track

Two separate artifacts, both built by injecting exactly one irrelevant quantity into a problem while leaving the correct answer unchanged:

Artifact	Source split	Size	Purpose
Distractor training examples	train (subset of the 556 above)	~95 (every 6th selected, ~17%)	Teach the filter node behavior
Distractor test set	test	40	Measure whether that behavior transferred

The training subset is deliberately partial: clean problems keep clean chains, so filtering is learned as a conditional skill rather than a ritual applied to every problem.

The 40-problem test set is paired — each problem is stored in both its original and distractor-injected form, so each item serves as its own control. That pairing is what permits the McNemar analysis reported in Limitations. Note that these injections were LLM-generated and manually reviewed rather than drawn from a published benchmark; a standard dataset for this purpose (GSM-IC, from the same work cited in the introduction) would have offered a larger sample and results comparable to published numbers.

3. Methodology

QLoRA was not really optional here: full fine-tuning of a 7B model does not fit the 16GB VRAM of the NVIDIA T4 this was trained on, so 4-bit quantization plus low-rank adapters was the only feasible route. The hyperparameters, however, were chosen deliberately in response to a failed earlier attempt.

An earlier coursework experiment applied LoRA to Llama-3.2-1B at lr=1e-5 targeting only q_proj/v_proj, and produced a nominal accuracy gain that masked worse underlying reasoning on inspection. That result is suggestive rather than conclusive — at 1B parameters, weak multi-step arithmetic is partly a capacity limitation and cannot be attributed to hyperparameters alone — but it motivated a hypothesis worth testing directly. Following Hu et al. (2021) and Dettmers et al. (2023), the hypothesis was that 1e-5 sits well below the ~1e-4–3e-4 range recommended for adapters, and that attention-only adaptation under-powers multi-step reasoning. Three combinations tested this directly:

Combo	Learning rate	Target modules	Final eval loss	Eval token accuracy
A	1e-5	q_proj, v_proj	0.718	83.2%
B	1e-4	q_proj, v_proj	0.191	94.2%
C (released)	2e-4	all-linear	0.138	95.6%

The diagnosis held monotonically: fixing the learning rate alone closed most of the gap, and widening to all linear projections closed the rest. Combo C is the model published here.

Full configuration for reproducibility:

Base model	Qwen/Qwen2.5-7B-Instruct
Quantization	4-bit NF4, double quantization, fp16 compute
LoRA	r=64, lora_alpha=64, lora_dropout=0.05, bias="none"
Target modules	all-linear (q,k,v,o,gate,up,down_proj) — 161M trainable params (2.08%)
Learning rate	2e-4
Epochs	2 (126 optimizer steps)
Batch	1 × grad-accum 8
Optimizer	paged_adamw_8bit, gradient checkpointing enabled
Max sequence length	1600
Seed	42
Trainer	TRL SFTTrainer, prompt/completion format (completion-only loss)
Training script	scripts/train_qlora.py (included in this repo)
Hardware	1 × NVIDIA T4 (16GB), ~44 minutes

4. Evaluation

Table 1 — Headline comparison. Five models on four benchmarks. Rows 1-4 are off-the-shelf models benchmarked here rather than quoted from published results; row 4 is also the base this model was fine-tuned from. Best score per column in bold.

Model	GSM8K (test split)	LogiQA2	ARC-Challenge	MMLU
Mistral-7B-Instruct-v0.3	47.1%	40.6%	58.6%	59.3%
Meta-Llama-3-8B-Instruct	71.6%	30.9%	54.6%	62.9%
Qwen2.5-Math-7B-Instruct	89.6%	30.3%	41.2%	53.0%
Qwen2.5-7B-Instruct (base)	75.8%	43.8%	54.0%	71.0%
This model	81.7%	41.3%	53.0%	70.5%

Every row was produced by the same script — scripts/benchmark_suite.py, included in this repository — wrapping lm-evaluation-harness. Identical tasks, identical 4-bit NF4 quantization, no sampling cap (GSM8K n=1,319; LogiQA2 n=1,572; ARC-Challenge n=1,172; MMLU n=14,042), and no chat template applied to any model. Only the --model argument differed between rows.

Why these benchmarks. GSM8K is the testing split of the training data and measures the target skill directly. The other three probe whether the fine-tune damaged general ability: LogiQA2 for logical reasoning, ARC-Challenge for science reasoning, and MMLU as a broad catastrophic-forgetting canary at its full 14,042-example size.

Why these comparison models. Mistral-7B-Instruct-v0.3 is the standard general-purpose 7B instruct peer. Llama-3-8B-Instruct is the model whose documented distractor failure originally motivated this project's filter-node design. Qwen2.5-Math-7B-Instruct is the toughest available comparison — the math-specialized sibling of this model's own base — which asks directly: why fine-tune a general model when a purpose-built math model already exists?

Reading the results honestly. This model leads GSM8K among the general-purpose models, but most of its margin over Mistral and Llama-3 comes from the base model choice, not from this fine-tune: Qwen2.5-7B-Instruct already scored 75.8%. The fine-tune's own contribution is +5.9 points. This model also does not win every column and does not claim to — Mistral leads ARC-Challenge, and the untuned base leads LogiQA2 and MMLU. Leading only on the task actually fine-tuned for is the expected profile for a task-specific adapter.

Table 2 — Effect of the fine-tune: base vs. this model only. This table isolates rows 4 and 5 of Table 1 — the only pair where everything except the fine-tuning is held constant, and therefore the only comparison that measures what the training actually did. Because both were evaluated on identical problem sets, the appropriate test is paired (McNemar) rather than a comparison of independent proportions. "Net" is problems gained minus problems lost.

Benchmark	n	Base correct	This model	Net	McNemar p
GSM8K	1,319	1,000	1,078	+78	2.4 × 10⁻⁶
LogiQA2	1,572	689	650	−39	0.011
MMLU	14,042	9,971	9,896	−75	0.006
ARC-Challenge	1,172	601	588	−13	0.298 (n.s.)

The GSM8K improvement is the strongest effect measured in this project by roughly three orders of magnitude. The LogiQA2 and MMLU declines are also genuine rather than noise, and are reported as such; the ARC-Challenge change is not statistically distinguishable from zero.

One caveat worth stating plainly: counted as raw problems rather than rates, this model gains 78 and loses 127 across the other three benchmarks. That framing is misleading — MMLU is more than ten times the size of GSM8K, so summing raw counts lets benchmark size, not capability, drive the conclusion — but the numbers are given here so readers can judge for themselves. Measured as rates, the target-task gain (+7.8% relative) exceeds every individual decline (−5.7%, −1.9%, −0.8%).

Table 3 — Specialization tradeoff: this model vs. the math specialist. Both are derived from the same Qwen2.5-7B family and scale, so each is measured as its change from the shared base (row 4 of Table 1). "Cost" sums the declines across LogiQA2, ARC-Challenge and MMLU in percentage points.

	GSM8K gain	Cost across the other three	Gain per point sacrificed
This model	+5.9	4.0	1.46
Qwen2.5-Math-7B-Instruct	+13.8	44.3	0.31

Qwen2.5-Math wins on raw GSM8K accuracy and should be preferred if grade-school arithmetic is the only thing that matters. But it pays with an 18-point MMLU collapse. This model buys a smaller gain at roughly one-eleventh the cost to general capability — which is the actual claim being made here: predict-before-observe fine-tuning delivers a real target-task improvement while leaving the model broadly intact.

5. Usage and Intended Uses

This is a LoRA adapter, not a standalone model. Loading it requires the base model plus PEFT:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct", device_map="auto", torch_dtype="float16"
)
model = PeftModel.from_pretrained(base, "Niarfe/qwen2.5-7b-positional-reasoning")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

SYSTEM = ("You solve math word problems using positional reasoning: a chain of typed nodes, "
          "each committing to a prediction before the next step is computed. "
          "End with the final answer in <answer></answer> tags.")

messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "Zhang is twice as old as Li. Li is 12 years old. "
                                "Zhang's brother Jung is 2 years older than Zhang. How old is Jung?"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=900, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Intended uses. Grade-school arithmetic word problems where an auditable reasoning trace matters more than raw speed — tutoring tools that need to show their work, systems that verify intermediate steps, or research into structured reasoning formats. The typed-node output is designed to be parsed programmatically, so each step can be checked in isolation.

Not intended for. Competition-level or higher mathematics, non-mathematical reasoning, or production use without human review. See Limitations.

6. Prompt Format

Use the standard Qwen2.5 chat template with a system message requesting positional reasoning. The user turn is the raw word problem — no worked examples required.

<|im_start|>system
You solve math word problems using positional reasoning: a chain of typed nodes, each committing to a prediction before the next step is computed. End with the final answer in <answer></answer> tags.<|im_end|>
<|im_start|>user
Zhang is twice as old as Li.  Li is 12 years old.  Zhang's brother Jung is 2 years older than Zhang.  How old is Jung?<|im_end|>
<|im_start|>assistant

7. Expected Output Format

The model emits a numbered chain of typed nodes — entry, filter, resolve, combine, check, branch — followed by the final answer in <answer> tags. The entry node declares STATE and GOAL; every later node carries MOVE, OBSERVE, STATE, and PREDICT. Each PREDICT commits to what the next node will do, and the final node's PREDICT is always goal resolved.

<node id="0" type="entry">
  STATE: Li's age known (12). Zhang's age unknown (twice Li). Jung's age unknown (2 more than Zhang).
  GOAL: Find Jung's age.
  PREDICT: Next node will resolve Zhang's age using the twice-Li relationship.
</node>

<node id="1" type="resolve">
  MOVE: Apply Zhang = Li x 2.
  OBSERVE: 2 x 12 = 24
  STATE: Li's age known (12). Zhang's age known (24). Jung's age unknown.
  PREDICT: Next node will resolve Jung's age by adding 2 to Zhang's age.
</node>

<node id="2" type="resolve">
  MOVE: Apply Jung = Zhang + 2.
  OBSERVE: 24 + 2 = 26
  STATE: Jung's age known (26). Goal resolved.
  PREDICT: goal resolved
</node>

<answer>26</answer>

When a problem contains irrelevant information, a filter node appears at position 1 and explicitly discards it before any arithmetic begins. Given "Jana is 5 inches taller than Kelly, and Kelly is 3 inches shorter than Jess. Their coach Diego is 68 inches tall. If Jess is 72 inches tall, how tall is Jana?" — the first two nodes are:

<node id="0" type="entry">
  STATE: Jana = Kelly + 5. Kelly = Jess - 3. Diego's height mentioned (68 inches). Jess known (72 inches). Kelly unknown. Jana unknown.
  GOAL: Find Jana's height.
  PREDICT: Next node will filter the given information against the goal.
</node>

<node id="1" type="filter">
  MOVE: Evaluate all given information against the stated goal.
  OBSERVE: Jess's height (72): relevant. Kelly = Jess - 3: relevant. Jana = Kelly + 5: relevant. Diego's height (68): NOT relevant to Jana's height — discard.
  STATE: Active scope: Jess (72), Kelly = Jess - 3, Jana = Kelly + 5. Discarded: Diego's 68 — not referenced by goal.
  PREDICT: Next node will resolve Kelly's height using the shorter-than-Jess relationship.
</node>

The remaining nodes then proceed as normal. Note the design weakness discussed in Limitations: the filter node names the discarded quantity, so "Diego" and "68" remain in context for every subsequent step rather than being removed from it.

8. Limitations

The central mechanism is unverified — the accuracy gain cannot be attributed to prediction. This is the most important caveat on this page. The format constrains the order tokens are emitted in; it does not constrain the computation that produces them. A model that has already internally settled on an answer can emit a fluent PREDICT field describing the upcoming step as pure narration, and nothing measured here would distinguish that from a genuine prior commitment. The look-ahead validator described in Data policed the training corpus — it guaranteed the reformatter never leaked results into PREDICT fields — but it says nothing about the trained model's behavior at inference time.

The problem compounds at the level of attribution. At least four explanations are consistent with the +5.9 point gain, and this study cannot separate them:

1. The predict-before-observe mechanism does something real;
2. Any consistent structured format aids problem decomposition;
3. The node format emits substantially more tokens than plain chain-of-thought, and additional generated tokens improve accuracy independently of their content;
4. Fine-tuning on 556 in-domain GSM8K training problems would improve GSM8K test accuracy in any format.

The controlled experiment that would settle this was not run: fine-tuning the same base model, on the same 556 problems, with the same hyperparameters, but using GSM8K's original solution text instead of the reformatted chains. If that ablation also reached ~81%, the format contributed nothing and the gain is simply in-domain fine-tuning. Absent that comparison, the honest claim is narrow — fine-tuning on 556 examples reformatted this way improves GSM8K by 5.9 points — not that prediction-before-computation is the cause.

There is also a suggestive counter-signal in the two-call result below. If PREDICT were a genuine, separable commitment, handing it to a fresh context should be unproblematic — any executor could carry it out. Accuracy instead fell 25 points, which is at least consistent with the prediction being entangled with the model's own rolling state rather than functioning as an independent commitment. (An out-of-distribution formatting effect explains the same observation, so this is not decisive either way.) A cheap discriminating test, not yet run: supply a deliberately incorrect PREDICT and observe whether the model follows it into a wrong answer (the field steers computation) or silently ignores it and answers correctly (the field is decorative).

Mild but genuine catastrophic forgetting. General capability declined: LogiQA2 −2.5, ARC-Challenge −1.0, MMLU −0.5 points. Two of the three are statistically real, not sampling noise — paired McNemar gives p = 0.011 for LogiQA2 (39 net problems lost) and p = 0.006 for MMLU (75 net lost), both on full-size benchmarks. The ARC-Challenge change (p = 0.298) is not distinguishable from zero. The effect is small in absolute terms but should not be dismissed: this adapter does make the model slightly worse at everything other than the task it was trained on.

The distractor-filtering skill was not demonstrated. The training data deliberately includes ~95 examples with a filter node, and a held-out 40-problem distractor set was built to test whether that skill transferred. It produced no detectable improvement — but the test was underpowered, and the result is inconclusive rather than negative. A paired McNemar test gives p = 0.607 for the base model and p = 0.180 for this one; the apparent difference between them amounts to two problems out of forty, and several problems flip to correct when an irrelevant sentence is added, which cannot be a real effect. A conclusive test would need several hundred problems per condition. There is also a design weakness: the filter node names the distractor, but those tokens remain in context for every subsequent step, so attention can still reach them. A stronger approach would restate the problem with the distractor removed and solve the restatement.

Poor robustness to an externally supplied reasoning prefix. When inference is split into two calls — one to produce the entry node, a second in a fresh context to continue from it — accuracy drops 25 points on a 100-problem GSM8K subset (65.0% → 40.0%), a statistically solid effect at roughly 3.7 standard errors. The model was trained exclusively on complete, self-generated chains, so it appears to depend on continuity with its own prior context and does not gracefully continue someone else's reasoning.

Small training set. 556 examples, roughly 7% of the GSM8K train split, reformatted by an LLM rather than written by humans. The reformatting was mechanically validated but not exhaustively hand-reviewed.

Quantized throughout. Both training and all reported evaluations used 4-bit NF4 quantization. Behavior at full precision was not measured and may differ from the numbers above.

Domain-narrow. Trained and evaluated only on grade-school arithmetic word problems. Nothing here demonstrates transfer to algebra, geometry, competition mathematics, or non-mathematical reasoning — and the OOD benchmark results suggest the fine-tune slightly reduces general reasoning rather than improving it.

Citation and provenance

Built for UVA DS 5002: How to Train Your LLM (Summer 2026). Training data derived from GSM8K (Cobbe et al., 2021, arXiv:2110.14168); base model Qwen2.5-7B-Instruct; method after QLoRA (Dettmers et al., 2023) and LoRA (Hu et al., 2021).