A Large-Scale Study of Specific Reading Comprehension Disability. (2009)

The results from a single study rarely make a substantial contribution to existing knowledge. This is true for all disciplines whether they are characterized by carefully controlled experimental studies in laboratories or less carefully controlled studies in the field. To further complicate matters, results often cannot be replicated because of modest sample sizes and over-simplified designs. Given the complexity and subtlety of most phenomena of interest, such as the study of specific reading comprehension disability, another approach is required.

One solution to this problem is a meta-analysis of the literature on a given topic. Meta-analysis is a method for quantitatively combining the results of many studies (Cooper, Hedges, & Valentine, 2009). A successful meta-analysis a) summarizes existing studies in terms of an average weighted effect size; b) determines whether the observed variability in results across studies is more than would be expected due to chance, and (c) identifies variables that predict or explain variability in results across studies. An advantage of meta-analysis is that the studies incorporated usually have different investigators, different conditions, and different measures. Consequently, a strong result from a meta-analysis is unlikely to be specific to a given investigator, situation, or measure.

Of course, all methodologies, including meta-analysis, have disadvantages as well as advantages. For example, one disadvantage of meta-analysis is that comparing different studies can be like comparing apples to oranges. A second disadvantage is that the meta-analysis is completely dependent on the quality of the studies that have been carried out and included in the analysis (i.e., “garbage in, garbage out”). A third disadvantage is that because the studies have already been completed, limited flexibility exists in the kinds of questions that can be answered and the researcher has little power to design a study that will best address a given question. Finally, as a result of inconsistent inclusion of basic statistics required for doing meta-analysis in published studies, it may not be possible to include a substantial number of relevant studies in the meta-analysis.

A complementary approach for obtaining results that can substantially advance existing knowledge is a large-scale study. A large-scale study may include thousands or hundreds of thousands of participants as opposed to a hundred or less. Large samples yield estimates of effects with high precision and provide the power to detect subtle effects. Because large-scale studies typically are designed by a research team, the study can be designed specifically to answer the question of interest.

Large-scale studies complement meta-analyses in that the strengths of large-scale studies often address the weaknesses of meta-analysis. Conversely, some of the strengths of meta-analysis are weaknesses of large-scale studies. For example, large-scale studies tend to be carried out by a limited number of researchers using a given set of measures and conditions.

The purpose of the present study was to investigate the construct of specific reading comprehension disability using a large scale. The data was supplied by Florida’s Progress Monitoring and Reporting Network (PMRN) database. Understanding the educational needs of students who comprehend poorly is critical for developing programs that teachers can use to improve reading comprehension for struggling readers.