Not scanning classes (or superclasses) that can't be instantiated (as that's a requirement for step defs and hooks). This was causing Cucumber to class load and scan all methods (including loading annotations) of super classes of any anonymous class defined in the step definitions. Should fix the premature class initialization that @zabawaba99 was seeing when running his tests.