We are concerned about more than just pupil achievement in judging teacher quality. There is not complete agreement on what should be counted as pupil achievement. There is variation within the ranks of teachers who show high pupil achievement, some teachers go considerably beyond with additional performance that we care about. Finally, we can obtain other important information about teacher quality, such as pupil reports, parent perceptions, and peer review of instructional materials. No single data source works for every teacher.
Good teachers are good for different constellations of reasons.
There are some excellent teachers for whom we just cannot good, defensible gain data. Teachers teach in different contexts and settings. All of these are arguments to vary data by individual teacher. It is necessary that these people do not have direct relationships with the teachers they judge. That is, decisions to recognize teachers for specific performances, on specific measures, for specific reasons themselves are informed, expert subjectivity and not a special kind of objective decision making.