Suppose I am lecturing to 200 students, half of whom are Pakistani and half are Korean. During my talk, my mind unaccountably wanders to the question of which is the taller group. I can't stop to calculate the relevant group means, but I can find the tallest person in the room and assume that he came from the tallest group. In fact, this crude method works pretty well. One does much better by picking the group with two of the top three.
Similar algorithms are often useful in evaluating public policies.
Yet outlier-based reasoning is dangerous when it neglects two important points. First, programs that work for some populations might fail for others. You shouldn't abolish prison furloughs for pickpockets after Willy Horton goes on a rampage. This is especially important in evaluating affirmative action. Race and gender preferences often look foolish at the extremes of the income distribution, but they provide significant, if less visible, benefits for many people near the middle.