Nevertheless, the findings raise serious concerns that merit further study. Meanwhile, a variety of sources, including many of the legislators responsible for funding state-supported universities, are calling for increased use of student evaluations. Administrators, faced with a glut of data, often find creative ways to reduce it to meaningless numbers. I encountered one who insisted that it sufficed to consider only the question on overall effectiveness, because he had once seen a report that, on average, the average on this question equalled the average of all other questions. He persisted in this policy even in cases for which it was patently false.
Advocates often cite a few superficial studies in support of the reliability of student evaluations.
However, other studies give a more complex picture. Moreover, survey results can be extremely sensitive to the wording of questions and the circumstances under which the survey is given. But simply bashing student evaluations serves little purpose. Some types of student feedback are useful, and many advocates of student evaluations are motivated by a genuine concern about the quality of education. However, many experienced faculty question the reliability of student evaluations as a measure of teaching effectiveness and worry that they may have counter-productive effects, such as contributing to grade inflation, discouraging innovation, and deterring instructors from challenging students.