Qwen3-8B-HTMNPO-helpfulness

Single-objective corner of the weight simplex: all weight on helpfulness.

Trained from Qwen3-8B, which is also the reference policy and the initialisation. Qwen3 signals direct answering through an empty <think></think> block in its chat template, and the training pipeline additionally needs the generation prompt to be a strict prefix of the rendered conversation. This model was trained under a template that emits the empty block unconditionally, satisfying both. Without it the model reasons aloud and most generations end mid-trace, which corrupts the preference signal. Use the tokenizer shipped here rather than the stock Qwen3-8B one.

Every arm in this release shares one pool, one optimizer and the same budget, and differs only in how the four objectives are aggregated. Held-out surplus over the reference (population scale, 100 prompts, prompted Qwen3-32B oracle, swap-averaged):

objective	surplus
helpfulness	-0.0019
truthfulness	-0.0009
honesty	-0.0012
instruction following	+0.0018
minimum	-0.0019
average	-0.0006

For comparison, the bargaining solution on this panel reaches a minimum of +0.0180 and an average of +0.0408: promotion/Qwen3-8B-NBPO. Generations for every arm are at promotion/nbpo-benchmark-generations.

Released under the Qwen3 licence.