8 Because the methods were applied in different villages that had some distance among them, it **is** highly unlikely that there were contamination effects from one classroom to another.

# EvaRuadtng the llmpact of the linstructioimal lInterventions-lData Challenges

Because of delays and administrative difficulties, only 15 centers could be used in the<sup>study,</sup> but the number of participants was much larger than the 250 planned (Appendix 2, Table<sup>1).Reading</sup> tests and a questionnaire were administered to a total of 425 learners. Of these, 199 were<sup>enrolledin</sup> classrooms conducted by OSEO staff and 227 were enrolled in classrooms conducted by<sup>INA staff.</sup> Field research in relatively remote rural areas inevitably runs into the unexpected. OSEO classes were underway in early February, but government-financed (INA) classes<sup>did not startuntil</sup> the end of March. The INA leamers received only three months of instruction by the time the rains arrived in June, while the OSEO leamers had five months of instruction.

Five leamers from each class were to be chosen at tested at random using CAAS. Although three batteries were provided for each notebook frequent electrical blackouts and perhaps<sup>the</sup><sup>**450**C</sup> heat in the villages (where there was no electricity) contributed to the deterioration of the batteries. Each would last 20 minutes or so, and they often ran out in the middle of a test, losing the<sup>results.</sup> Thus, researchers could not test the number of leamers expected. Overall, only 37 learners could be assessed through the CAAS in pre- and post-tests, and these were not distributed evenly in the various classes. Therefore, the analyses of these data were not included in this report.

Near the end of the course, learners answered a questionnaire that collected demographic information (see Table 2 for a summary), and they completed tests (in their native language) that measured their competence at identifying letters, syllables, words, and sentence length material. Obtaining demographic data from leamers also posed some problems. About 30 learners stated<sup>that</sup> they had never before attended literacy class or school, but displayed rudimentary reading<sup>skills</sup> during the pretest. Apparently literacy classes are a social function in the villages, and residents did not want to be excluded.

Because the experimental conditions were imperfect, the data were "noisy" with more error variance than expected. Efforts were made to understand the effects of the treatments and disentangle them from related variables. The observational data collected (including videotaped sessions) provided qualitative information to supplement the statistical analyses.

# Results of the linterventdon Efforts

Analyses of variance were carried out, comparing the various treatment groups in OSEO, INA, and control classes. Performance on the tests administered near the end of phase one<sup>training</sup> indicated that learners enrolled in courses sponsored by OSEO performed better than did learners enrolled in government-sponsored courses. Given that the government courses had lasted a shorter period, this result was not surprising.

The results on the paper-and-pencil tests also showed that the learners receiving any of the new instructional interventions tended to perform better on the tests than did the learners enrolled in the control classes. There were 20 possible comparnsons between treatment and control classes; the treatment groups scored higher than the control group in 18 of the 20. Moreover, in 13 of these 18 comparisons the advantage for the treatment group over the control group was statistically significant. In the two comparisons favoring the control group over the treatment group, the difference was not statistically significant. Specifically, analysis of variance results were: