Llama-3.1-8B-Utilitarian-baseline

Utilitarian aggregation baseline: a fixed equal weighting of the four objectives, the natural scalarisation control for a bargaining rule.

Trained from meta-llama/Llama-3.1-8B-Instruct, which is also the reference policy and the initialisation. Four objectives are scored on UltraFeedback prompts by a prompted Qwen3-32B preference oracle, each pair queried in both presentation orders and swap-averaged. Every arm in this release shares one pair set, one optimizer and a 300-step budget, and differs only in how the objectives are aggregated.

Held-out surplus over the reference on the 657-prompt panel (population scale):

objective	surplus
instruction following	-0.0639
truthfulness	-0.0765
honesty	-0.0705
helpfulness	+0.0268
minimum	-0.0765

For comparison, the bargaining solution reaches a minimum of +0.0391 on the same panel: promotion/Llama-3.1-8B-NBPO-600step. Generations for every arm are at promotion/nbpo-benchmark-generations.

Built with Llama. Use is subject to the Llama 3.1 Community License.