To show why, consider the following two-paragraph quotation from William McKeachie, an expert with more than 50 years of research on student evaluations. I argued that anything that existed, existed in a quantifiable fashion. Thus it was appropriate to use quantifiable ratings. I now feel that that was a mistake. Not that qualities of teaching are not quantifiable.
Numbers are often useful.
The fault is not in the numbers, but rather in their use. Once numbers are assigned, faculty promotion committees begin to make comparisons between teachers and assume that if one number is larger than another, there is a real difference between the teachers to whom the numbers have been assigned. Moreover faculty members are supplied with norms indicating the average ratings on each item. The remaining 4 respondents provided answers that did not fit within one of the two categories. For tenure and promotion decisions, most respondents did not clearly indicate how student ratings were employed for decision making.