  Out of the participants in a randomized experiment with anticipated
heterogeneous treatment effects, is it possible to identify which ones have a
positive treatment effect, even though each has only taken either treatment or
control but not both? While subgroup analysis has received attention, claims
about individual participants are more challenging. We frame the problem in
terms of multiple hypothesis testing: we think of each individual as a null
hypothesis (the potential outcomes are equal, for example) and aim to identify
individuals for whom the null is false (the treatment potential outcome
stochastically dominates the control, for example). We develop a novel
algorithm that identifies such a subset, with nonasymptotic control of the
false discovery rate (FDR). Our algorithm allows for interaction -- a human
data scientist (or a computer program acting on the human's behalf) may
adaptively guide the algorithm in a data-dependent manner to gain high
identification power. We also propose several extensions: (a) relaxing the null
to nonpositive effects, (b) moving from unpaired to paired samples, and (c)
subgroup identification. We demonstrate via numerical experiments and
theoretical analysis that the proposed method has valid FDR control in finite
samples and reasonably high identification power.
