UNKAGE BETWEEN TEACHER QUALITY , STUDENT ACHIEVEMENT , AND COGNITIVE SKILLS : A RULE-SPACE MODEL 1

The topic of teacher credentials and student performance is revisi ted in an international setting using the TIMSS-99 data. The lack of consistent positive link between credentials and performance can be explained via three routes: measurement problem of "teacher quality" input, measurement problem of "student outcomes", and the production function form that is assumed to link the input and the output. Although there is a small literature focusing on student outcome measurement problems, suggesting the use of cognitive achievements rather than test scores, in most cases those cognitive measures are nothing but math and science scores. This study contributes to the literature by borrowing from the measurement and psychometrics theories to decompose single scores into three categories of cognitive abilities. The hypothesis is that teachers may play a crucial role in the development of some student cognitive skills while not in the others. Using a "rule-space" model, this article identifies three cognitive skills: the process skill, the reading skill and the mathematical think skill. The study finds that: (1) in general teacher credentials have no effect on any type of cognitive skill development as well as on the test score, and (2) the within-teacher variance of student performance is much larger than betweenteacher variance in Japan and Korea, whereas the reverse is true in the US and the Netherlands. The phenomenon of "private tutoring" is quoted as an explanation of this pattern. 0191-491X/04/$ see front matter © 2004 Published by Elsevier Science Ltd. doi: 10. l 016/j.stueduc. 2004.09. 002 206 T. Xin et al. / Studies in Educational Evaluation 30 (2004) 205-223 There are extensive studies on the relationship between school resources, student achievement and adult success (e.g., Burtless, 1996; Card & Krueger, 1992; Case & Deaton; 1999, Cawley, Heckman, Lochner & Vytlacil, 2000; Currie & Thomas, 2001). In the cconolnics of education and education and social policy literature, the production function approach is usually adopted to describe education as a production process comparable to factory manufacturing, with school resources and family support as the input and achievement scores and future earnings as the output. Within the production function framework, educators, parents and students seek to maximize the productivity in transforming input into output by achieving two efficiencies: technical efficiency and allocative efficiency. Technical efficiency abstractly refers to the latest technology available for production, and allocative efficiency is defined in strict economic terms that resources should be allocated to each input to such a point that the marginal cost of that input equalizes its marginal product. We often see three types of studies using the production function approach: the impact of school resources on achievement and cognitive skills, the impact of achievement and cognitive skills on adult success (usually measured by earnings), and the impact of school resources on adult success. Yet how strong the links are is far from conclusive. Despite Card and Krueger's (1992) finding that labor market outcome is related with school resources, the lack of relationship with school resources is more generally true for recent studies of earnings than earlier investigations, while more recent studies have tended to find stronger effects of cognitive skills on earnings (Hanushek, 1997). Typically a one-standard-deviation difference in cognitive scores is associated with about a 3 to 4% difference in earnings, after controlling for race, gender, educational level, and experiences (Levin, 2001). Background The relationship between school resources and student performance is an issue of even more controversy. The debate started since the Coleman Report (Coleman et al., 1966), which finds predominant importance of family background and little significance of school factors in deciding school performance using education production function analysis. Teacher salary accounting for the largest portion of school budgets (Belfield & Levin, 2002), teacher quality has been the focus of discussion in school resource efficiency studies. Large amounts of research have been done and several important meta-analyses arrive at very different conclusions on the basis of empirical results (Hanushek, 1997, 1986; Hedges, Laine, & Greenwald, 1994a, 1994b). The Hanushek versus Hedges debate is the most famous one that still keeps updating (Hanushek, 2003; Hanushek & Luque, 2003). In 2003 there are also summarizing studies that specifically focus on the effectiveness of teacher quality (Rice, 2003; Wayne & Young, 2003). These latest teacher quality reviews still cannot reach an agreement on the existence of a teacher quality student performance relationship. Not being able to find any conclusive positive effect of teacher quality on student perforlnance poses questions for policy makers on the wisdom of spending so much money based on a teacher-pay system that is mainly based on observable credential measures such as teacher's highest degree, experience, previous test scores and teaching certificate. Researchers try to explain the puzzle from three perspectives: measurement of input, measurement of output, and the form of link between the two. T. Xin et al. / Studies in Educational Evaluation 30 (2004) 205-223 207 Measurement error biases production function estimates towards zero in regression analysis. Teacher quality is very difficult to quantify. Are qualified teachers, as judged by observable credentials, also likely to be quality teachers? Teacher credentials are the most dircctly available measures that do not account for teachers' personality, within classroom activity and many other aspects of teaching. In fact, teacher quality is only weakly linked with teacher pay, which in turn is based on observable and quantifiable teacher attributes (Ballou & Podgursky, 2000). A direct examination of principal's teacher rating, which is mainly based on teacher credentials, and students' rating of teachers finds very low correlations (Mumane & Cohen, 1986). 2 The measurement of output is also problematic. Most individual level studies use test scores as the only outcome. Scores are easily availaNe. They are an important output of schools, and they can somewhat predict students' future labor market success. Therefore score is an important and very realistic approximation of the quality of education. But score is not the whole story: Score is too narrow a measure of outcome because there are other abilities that cannot be measured by academic testing. Score is also too broad because it embodies many latent cognitive abilities that work together to produce the score. Finally, there are studies trying to explain the non-existence of resource effect on student performance by exploring the production function form that relates input to output. The format of education production function is unknown. In most empirical analysis production function is assumed to take linear additive tbrmat. Some researchers suggested more flexible functional forms like the transcendental logarithm function and found significant school resource effect (Figlio, 1999). Existing studies traditionally focus on cognitive development as a measure of output (e.g., Hanushek, 1997; Murnane, 1975; Summers & Wolfe, 1977). Cognitive development, in turn° is usually measured by test scores. Using scores as the dependent variable, current studies reached mixed conclusions on the effect of teacher's degree level and major field, experience, and certificate status. Degree levels are found to have both significant positive effect (e.g., Ferguson & Ladd, 1996) and significant negative effect (e.g., Ehrenberg & Brewer, 1994) on student scores. But in most cases, without controlling for the degree major field, the relationship is indeterminate (Wayne & Young, 2003). With degree major included, however, it is found that having a math-major teacher positively contributes to high school students' math scores (Goldhaber & Brewer, 2000). On the other hand, according to Wayne and Young (2003), findings from existing studies on teacher experience effect are difficult to interpret for many reasons. For example, experience captures the teacher labor market surplus or shortage at the time of hiring, and therefore its effect is hard to generalize. Finally, it is found that if teachers have certification in teaching mathelnatics, students' math score is on average better than when teachers are not certified or certified in a different subject (Gotdhaber & Brewer, 2000). When teacher certification is examined without controlling for certified field, no determinate effect can be detected. This study contributes to the existing teacher quality literature by focusing on hnproving the measurement of student outcomes. It is admitted that measuring teacher quality by credentials is very crude. But credentials are feasible features based on which policy makers and administrators can make large-scale employment decisions and payment plans. By watching more closely at, say, classroom activities to judge teacher quality may have no realistic applications beyond experiments and academic discussions. Governments at different levels still have to rely on teacher's degree, teaching certificate, experience and 208 T. Xin et al. / Studies in Educational Evaluation 30 (2004) 205-223 other credentials to make administrative and policy decisions. Therefore, this study decides to use these traditional measures of teacher quality. On the other hand, even though many studies claim to use ability as the outcome measure, those ability indices are simply math or science scores (Levin, 2001). TIMSS tests students on different fields of knowledge and various problem-solving capabilities using a variety of question formats. This offers us an opportunity to apply a "Rule-Space Model" to decompose student scores into achi