NeurIPS 2025

Stochastically Dominant Peer Prediction
Yichi Zhang
DIMACS, Rutgers University
yz1636@dimacs.rutgers.edu
Shengwei Xu
University of Michigan, Ann Arbor
shengwei@umich.edu
David Pennock
DIMACS, Rutgers University
dpennock@dimacs.rutgers.edu
Grant Schoenebeck
University of Michigan, Ann Arbor
schoeneb@umich.edu
Abstract
Eliciting reliable human feedback is essential for many machine learning tasks,
such as learning from noisy labels and aligning AI systems with human prefer-
ences. Peer prediction mechanisms incentivize truthful reporting without ground
truth verification by scoring agents based on correlations with peers. Traditional
mechanisms, which ensure that truth-telling maximizes the expected scores in
equilibrium, can elicit honest information while assuming agents’ utilities are
linear functions of their scores. However, in practice, non-linear payment rules
are usually preferred, or agents’ utilities are inherently non-linear.
We propose stochastically dominant truthfulness (SD-truthfulness) as a stronger
guarantee: the score distribution of truth-telling stochastically dominates all other
strategies, incentivizing truthful reporting for a wide range of monotone utility func-
tions. Our first observation is that no existing peer prediction mechanism naturally
satisfies this criterion without strong assumptions. A simple solution—rounding
scores into binary lotteries—can enforce SD-truthfulness, but often degrades sensi-
tivity, a key property related to fairness and statistical efficiency. We demonstrate
how a more careful application of rounding can better preserve sensitivity. Further-
more, we introduce a new enforced agreement (EA) mechanism that is theoretically
guaranteed to be SD-truthful in binary-signal settings and, under mild assump-
tions, empirically achieves the highest sensitivity among all known SD-truthful
mechanisms.
1
Introduction
Information elicitation studies how to design mechanisms that incentivize honest and high-effort
human feedbac...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000057. Conference bucket: NeurIPS.