Can Multifactor Models of Teaching Improve Teacher Effectiveness Measures?

NCLB waiver requirements have led to development of teacher evaluation systems, in which student growth is a significant component. Composite teacher evaluation scores commonly sum up the results of measurements made using several instruments. We hypothesize that, across different measures, there is more than one underlying factor and each measure can be decomposed into distinct factors. By performing factor analysis on the disaggregated evaluation data (observation components and survey items), we can identify several orthogonal factors, of which only one is associated with student test performance. We use teacher evaluation data collected by the Measures of Effective Teaching (MET) project as a model of state teacher evaluation system, which includes a value-added score as a measure of student performance, an observational rubric, and a student survey. We find that one possible model of the latent data structure has three factors, of which only one is associated with value-added scores. This factor is also strongly associated with observation and survey items that deal with classroom control. The second factor is associated with such aspects of pedagogy as classroom dialog, questioning techniques, etc. The third factor is associated with items related to the notions of teacher sensitivity to students’ well-being and includes mostly student survey items. Those factors can be interpreted as reflecting “effective,�? “constructive,�? and “positive�? dimensions of teaching respectively. Each of them is an important and independent input into child development, while only first of them leads to achievement gains measurable in the short run. The importance for policy of identifying orthogonal underlying factors is, first, that it provides precise knowledge of what exactly the evaluation system is measuring. This knowledge can be used by policy-makers to make an informed decision on how to combine the aspects of teaching into a single “teacher utility�? function or, alternatively, if a multidimensional (matrix) evaluation system should be used. Second, it makes clear that teacher effectiveness consists of more than just the ability to promote student growth as measured by test scores. Additional factors may be weighted differently, for example, in identifying a teacher to become a mentor vs. to become a principal, which may require interpersonal capabilities unrelated to promoting student growth. Third, evaluating teachers using several independent factor scores may help target resources (such as professional development) more accurately. NCLB waiver requirements have led to development of teacher evaluation systems, in which student growth is a significant component. Composite teacher evaluation scores commonly sum up the results of measurements made using several instruments. We hypothesize that, across different measures, there is more than one underlying factor and each measure can be decomposed into distinct factors. By performing factor analysis on the disaggregated evaluation data (observation components and survey items), we can identify several orthogonal factors, of which only one is associated with student test performance. We use teacher evaluation data collected by the Measures of Effective Teaching (MET) project as a model of state teacher evaluation system, which includes a value-added score as a measure of student performance, an observational rubric, and a student survey. We find that one possible model of the latent data structure has three factors, of which only one is associated with value-added scores. This factor is also strongly associated with observation and survey items that deal with classroom control. The second factor is associated with such aspects of pedagogy as classroom dialog, questioning techniques, etc. The third factor is associated with items related to the notions of teacher sensitivity to students’ well-being and includes mostly student survey items. Those factors can be interpreted as reflecting “effective,�? “constructive,�? and “positive�? dimensions of teaching respectively. Each of them is an important and independent input into child development, while only first of them leads to achievement gains measurable in the short run. The importance for policy of identifying orthogonal underlying factors is, first, that it provides precise knowledge of what exactly the evaluation system is measuring. This knowledge can be used by policy-makers to make an informed decision on how to combine the aspects of teaching into a single “teacher utility�? function or, alternatively, if a multidimensional (matrix) evaluation system should be used. Second, it makes clear that teacher effectiveness consists of more than just the ability to promote student growth as measured by test scores. Additional factors may be weighted differently, for example, in identifying a teacher to become a mentor vs. to become a principal, which may require interpersonal capabilities unrelated to promoting student growth.