The debate over school vouchers is one example. In recent years, several thousand students in failing public schools have used government-financed vouchers or private scholarships to attend private schools. But skeptics have pointed out that it is hard to assess whether vouchers actually improve achievement, because those self-selecting students who use them may be smarter or more motivated than those who do not. Mr. Peterson was able to use random sampling in a recent study of privately financed vouchers in New York because the program was oversubscribed, and vouchers were awarded by lottery. He found that, after a year or two in private schools, voucher recipients did score better on achievement tests than the unsuccessful applicants who stayed in public schools.
Some scholars say that the experimental approach is resisted for ideological reasons.
After most approaches flunked and a method called Direct Instruction came out on top, he says, the education field essentially ignored the results. Even quantitative researchers differ over the relative worth of randomized experiments. The prime exhibit for the method is a Tennessee study from the late 1980s. That research appeared to show that reducing classes to 15 to 17 students in the early grades raises achievement, particularly among low-income students. Critics say that such studies must be fairly large to be useful, and hence are quite expensive.