Now creating one MapDB database per crawlers, to avoid concurency
issues.  Fixed a few other thread issues and provided better logging.