Evaluation forms should have room for comments, or ask for comments on the back of the form or on a separate sheet. A paper competition chair serves as the final arbiter. It's particularly important not to reject controversial work automatically. The chair is in a unique position to observe judges' lack of consensus on controversial scholarship. If numerical scoring is used, be sure it is fair.
Most divisions use some form of numerical scoring for paper judges.
Thus, you cannot simply sum the numerical scores and use the summed numbers to make judgments of quality. The best solution is to use standardized scoring. This is not hard to do, and it helps to compensate for a judge's idiosyncratic scoring tendencies. If you use rankings, then each judge must evaluate the same number of papers, and papers should be randomly assigned to judges. If the division has a theme, selection should not be biased by the theme.