Further modularisation and code cleanup.
Resolved issues with crawl date handling, and turned off some features.
More tricky features reconfigd