Actual practice rarely looks anything like the procedure that I have just described. We all know of tenure cases in which senior faculty members have argued and voted against candidates whose work they have not even read. We all know of cases in which the choice of outside evaluators has been designed to produce a particular result or of cases in which laudatory evaluations have been ignored or even destroyed. And if most of our colleagues are now careful about making remarks that betray crude biases, more-subtle versions of those same prejudices are often unspoken factors in the way decisions turn out. Even the most conscientious among us are often inexpert, and senior scholars judging the work of junior colleagues with different interests must make decisions driven as strongly by intuition as by specific knowledge.
Conflicting methodologies pose even more-vexing difficulties.
Our colleagues in the sciences may have a slightly easier time with this issue, since new techniques and theories in their disciplines, once tested and established, tend to drive out older methods. I have reason to doubt it. In fact, I think scholars now often present what amounts to a difference of opinion about method as an objective judgment of quality. To these already troubling problems we may add the fact that tenure decisions invite the reviewers to act as prophets. Discussion often focuses on what the candidate is likely to do over the next few decades, providing an opening for astonishingly speculative and inferential statements.