marinemee/physics-sft-flexibility

Domain-Focused Fine-Tuning and Behavioral Flexibility

Research question

Does domain-focused supervised fine-tuning reduce a language model's ability to adapt physics explanations to different audiences, and can mixed-audience training mitigate that reduction?

Experimental setup

• Base model: Qwen3-1.7B (unmodified baseline).
• Fine-tuned conditions: expert-only LoRA SFT and mixed-audience LoRA SFT.
• 50 training examples per fine-tuned condition; the datasets differ in token counts.
• Evaluation: 10 physics concepts × 3 audience-specific prompts × 3 model conditions = 90 responses.
• Generation: thinking disabled; matched sampling settings, with larger output caps needed for some baseline completions.
• Ratings: blinded ChatGPT judge, integer 1–5; physics correctness on every response and one audience-specific metric per response.
• Scores are model-generated judgments, not expert-validated ground truth.

Overall results

Condition Physics correctness Audience behavior (pooled across prompt types) Baseline 4.73 4.67 Expert SFT 3.77 3.10 Mixed SFT 4.00 3.47

Main comparison

Mixed-audience SFT versus expert-only SFT, beginner adaptability:

• Mean paired difference: +0.80 points on the 1–5 rating scale.
• 95% paired percentile bootstrap CI (20,000 resamples): [+0.30, +1.30].
• Two-sided exact paired sign-flip p-value: 0.0312 (10 matched prompts; exploratory, unadjusted).

The baseline had the highest mean scores on the measured outcomes. Mixed-audience SFT had a relative advantage over expert-only SFT on beginner adaptability but did not recover baseline performance. These data do not establish general causal claims about fine-tuning or flexibility. Statistical methods Each comparison pairs the same prompt across conditions. Mean paired differences use 20,000 paired bootstrap resamples for percentile 95% confidence intervals. Two-sided sign-flip tests are exact for the 10-prompt audience-specific contrasts. The 30-prompt correctness contrasts use 100,000 Monte Carlo sign-flip draws, with a plus-one correction, to avoid enumerating 2^30 permutations. No multiple-comparison correction is applied; p-values are exploratory. Individual ratings are ordinal and the small, fixed prompt set limits inference.

Figures

Audience adaptation

Physics correctness

Paired effects and confidence intervals

Limitations

• Small set of 10 physics concepts and 30 matched prompts.
• Single ChatGPT judge; no validated physics-expert assessment.
• Different token counts and audience compositions in fine-tuning sets.
• Large baseline-versus-adapter response-length differences; the judge may respond to length or style.
• Only one fine-tuning run per condition; model and decoding stochasticity remain.
• Many exploratory comparisons, without multiplicity adjustment.

Files

• condition_summary.csv: overall descriptive statistics.
• audience_summary.csv: audience-specific means.
• paired_statistical_tests.csv: paired differences, bootstrap intervals, and sign-flip p-values.
• figures/: publication-style plots.

The original generated responses, adapters, training examples, and full evaluation scores should be retained separately for reproduction. The private condition key should be withheld during any new blinded judging.

Key Findings

The baseline outperformed both fine-tuned models. The original Qwen3-1.7B achieved the highest scores across all evaluation metrics, including physics correctness (4.73/5). Surprisingly, expert-focused fine-tuning did not improve expert rigor; instead, its score dropped from 4.60 to 2.90. This suggests that specialization through a small fine-tuning dataset does not necessarily improve domain-specific performance.

Mixed-audience training helped preserve beginner adaptability. Mixed SFT scored 3.80/5 compared with Expert SFT's 3.00/5, a difference of +0.80 points (unadjusted p = 0.031). However, it showed only a modest improvement in expert rigor and no improvement in misconception handling. The benefits of training-data diversity were therefore limited to certain behaviors.

Lower scores may not necessarily indicate knowledge loss. The fine-tuned models produced substantially shorter responses than the baseline. Their lower ratings could reflect reduced explanatory depth rather than weaker physics knowledge. Our evaluation cannot yet distinguish between these possibilities.

Questions Raised

Did fine-tuning change what the models knew, or how they communicated it? Since the fine-tuned models generated shorter explanations, evaluating factual knowledge separately from explanation quality could help distinguish knowledge degradation from changes in response style.

Why did expert-focused training fail to improve expert rigor? The result raises questions about the influence of dataset size, training duration, and optimization settings. Would additional examples or different training configurations produce better expert-level explanations?

How much does training-data composition influence behavioral flexibility? Mixed training improved beginner adaptability but not misconception handling. This raises the possibility that different audience-specific capabilities require different proportions or types of training examples.

Would these findings hold across other models and evaluators? Testing multiple model families, training seeds, and independent judges would help establish whether the observed differences reflect general patterns or characteristics of this particular experiment.