A well written criteria document should tell prospective award submitters exactly what needs to be true in order for them to win your award. Thus, an awards program with a well written criteria will flow like clockwork and will be a pleasure both for submitters and awardmasters alike. Everything is spelled out and everything is understood. A set of well written criteria makes the awardmasters job much easier. You see, all you need to do is compare the site against your criteria.
Nothing is left to chance and there is little room for doubt.
Well, sit down and think about what your awards program is all about. Personally, when I came up with my programs I began from the theme and worked down. Let's say I wanted to award good science fiction sites. That's the theme. Now it's possible to design some criteria around that.