Use count directly

Also ensure that we're dealing with an Enumerable and not an
ActiveRecord collection (in which case count with a block won't work
properly).
Prefer positive logic to negative