Several of these concerns are addressed clearly in the Faculty Handbook or other material disseminated from the Provost's Office. We strongly recommend strict adherence to those materials. Our major concern centers upon our current student teaching evaluations. Many departments do not report the group of instructors or courses that define departmental means, standard deviations, and z-scores. Without these facts, the magnitudes of means, and the signs and sizes of z-scores, necessary for comparative analysis, are uninformative.
Many departments also misinterpret z-scores.
When creating summary tables of quantitative course evaluations, departments should explain their choices and copy the data carefully. When less than half of enrolled students complete course evaluations, the results should be treated as unreliable. Some departments do not make such distinctions, some exclude certain courses from summary reports for no obvious reason and some make transcription errors when creating summary tables. If quantitative teaching evaluations are to be useful in promotion and tenure evaluations, departments must, at a minimum, identify and report comparator groups, accurately report and interpret z-scores, and explain their summary tables. Either improvement will reduce burdens in the peer review process.