Qwen3-8B-NBPO

Nash Bargaining Preference Optimization on a Qwen3-8B backbone, at the step size and budget the method's own Nash product selects. It leads all four objectives, the minimum surplus and the average surplus against every baseline on this panel.

Trained from Qwen3-8B, which is also the reference policy and the initialisation. Qwen3 signals direct answering through an empty <think></think> block in its chat template, and the training pipeline additionally needs the generation prompt to be a strict prefix of the rendered conversation. This model was trained under a template that emits the empty block unconditionally, satisfying both; without it the model reasons aloud and most generations terminate inside the trace, which corrupts the preference signal. Use the tokenizer shipped here rather than the stock Qwen3-8B one.

Objective-wise held-out surplus over the reference (population scale, 100 prompts, prompted Qwen3-32B oracle, swap-averaged over both presentation orders):

objective	surplus
helpfulness	+0.0719
truthfulness	+0.0180
honesty	+0.0427
instruction following	+0.0307
minimum	+0.0180
average	+0.0408

Generations for every arm in the paper are at promotion/nbpo-benchmark-generations.

Released under the Qwen3 licence.