A CRITICAL LOOK AT THE MATCHING TECHNIQUE IN EXPERIMENTATION

THE INADEQUACIES of matching or pairing sub jects as a means of increasing precision in experi mentation and statistical analysis are pointed out by Stanley and Beeman (8). McNemar (6: 386), while in favor of the technique under certain conditions, cautions against placing absolute confidence in matching. Walker and Lev (9) and Cox (2) call at tention to the impossibility of making g e n e r a 1 i za tions beyond the matched samples. Peters and Van Voorhis (7) make a strong case for m ate hing without warning the unwary researcher of its attendant pit falls. Other authors on research design, such as Finney (4) and F?d?rer (3), only touch upon the sub ject tangentially in connection with discussion on re stricted randomization in the randomized blocks design. This paper will recapitulate the usual objections to matching as well as the methods for overcoming them as suggested in textbooks on research design and statistics. Further, this paper will examine, from the statistical point of view, the error terms for thej and F tests used in the matched groups de sign. The examination should give insight into the nature and consequences of the judgmental manipu lation of subjects for the purpose of equating exper imental groups on a given variable. The major objections to matching are: 1) match ing reduces the number of subjects available for an experiment, thereby lowering precision in the sta tistical test of significance; 2) matching subj ects from an available pool of supply violates an impor tant assumption basic to all statistical tests of sig nificance, namely, the random distribution of ex perimental error; 3) matching subjects from two populations known to differ on the matching variable fails to consider the effects of possible regression of the matched subjects toward their respective pop ulation means on a second testing and, therefore, does not produce truly equivalent groups with re spect to the matching variable; and 4) matching re sults in samples that are no longer representative of the original populations from which the subjects have been drawn. With regard to objection 1), that matching re duces the number of subjects available for an exper iment, McNemar (6: 385) suggests that the attrition of subjects can be reduced by matching distribu tions rather than subjects. In this way many sub jects who would have to be eliminated in the individ ual pairing method can be retained in the sample for the experiment. With reference to objection 2), that matching vi olates the assumption of random distribution of ex perimental error, Campbell and Stanley (1) suggest that instead of matching subjects, subjects be divid ed into homogeneous subgroups on the matching var iable and within each subgroup the subjects be ran domly assigned to treatments. This is, of course, the technique used in the complete randomized blocks design discussed in most books on experimen tal design. This writer suggests that the same pur pose can be achieved if the paired subjects are in dependently and randomly assigned, pair by pair, to experimental groups. By using randomization with in each pair as a block, the experimenter can be con fident that the assumption of random distribution of experimental error has been met. Both McNemar (6) and Peters and Van Voorhis (7) have provided an answer to objection 3) that matching subjects from two populations known to dif fer on the matching variable fails to consider pos sible regression effects and therefore does not pro duce truly equivalent groups. The authors suggest that, rather than match the subjects on the original scores on the control variable, tlie subjects be matched on their regressed scores based on regres sion equations of the form