Fix: Ensure rake task correctly identifies directories with specs in them

see https://github.com/metricfu/metric_fu/issues/57

It was returning files in the spec/ folder since it never checked if the `dirs` were directories

I don't see any tests for the rake task.  Where should I make one?