Allow generic file inclussion.
Moved the crawlers into a separate dir.
Auto include crawlers.