Recall comprehension, on the other hand, proved very difficult for the children. We found it to be highly disorienting for students who are used to repeating textually what the teacher or the material says. Most students in the sample tried to remember the passage word by word and many gave up after they could not go pass “Once upon a time.” Strictly, the real problem here is not so much that this comprehension measure assesses inappropriate skills –for it responds to curricular objectives– but rather the fact that in many public schools, teaching tends to encourage repetition rather than inferential or critical comprehension, as seen in classroom observations during the study (Banco Mundial, 2008). In any case, the recall comprehension measure tells us more about the gap between actual teaching practices and theoretical curricular guidelines, than any other single measure used in this study; yet, it is not the most appropriate for establishing the relationship between fluency and comprehension, as we will later discuss.

Regarding test administration, the team of evaluators was well trained and could control most administration conditions, that is: that children took the oral test in a private area with only the evaluators present, that they took the written test in silence and without interference of the teacher or any other school staff, and that they were given the time required to finish the tests. Test instructions were standard and written down for the evaluators to repeat. The evaluators personally made sure that classrooms and students were selected randomly and following the preestablished procedures. However, test administration could have been influenced by cultural, linguistic and racial differences between the evaluators and some of the students. In the future, it is suggested that at least one of the evaluators speaks the first language of the children being evaluated and is familiar with their culture, and that the tests are administered in both Spanish and the native language for the early grades.

Another problem in this study probably arises from the mismatch between the tests’ original target population –students at the end of second-grade– and the sample population –third-grade students in the second quarter of classes. This could explain, to some extent, why students could easily answer the first items of the written and the Q&A comprehension in the oral test. Fortunately, this is not an issue for future use of these tests, as the census evaluations are carried on at the end of the year and designed for each grade according to curricular standards. As for the oral test, it can be efficiently used to track progress among students in second and third grade during the school year by selecting passages with different lengths and complexity levels. If these oral measures are taken by the teacher or parents of the student, and if bad performance does not result in punishment of the child, then the contextual factors that influenced our sample could be overcome: the students would be familiar with their evaluator, could take the test in their native language, and eventually would grow familiar with this technique.

# **5.2. Nature of the relationship between oral fluency and written comprehension**<sup>**46**</sup>

From the correlation results, it is possible to say that fluency explains 23 percent of the variance in written comprehension at the individual level and 32 percent at the school level in Junin, and that the probability of this result being due to chance is virtually nonexistent. This is considered a good effect size, and the effect of fluency on written comprehension is significant even when controlling for key individual and school characteristics, such as age, gender, first language, preschool education, type of school and location. In fact, it is by far the single most important variable in explaining performance on the written test. Furthermore, the fluency test tends to

> 46 It is quite interesting that the correlation between fluency in the oral test and comprehension in the written test correlate higher than the oral comprehension measures correlate with oral fluency or written comprehension, but this has been discussed earlier, so here we will focus on the relationship between oral fluency and written comprehension.

46