Feature/improve find and scan code

Part of fixing #856 and #858. I'm stilling missing the follow config option, but rest should be more or less done, or at least ready for review.
Find will traverse it, but files in hidden directories will be ignored. But think I'll code up the "predicate" based idea for this to bring back the old behavior properly.