Methods for concurrent crawling added; untested though; depth monitoring is still missing
All methods for concurrent crawling added but not currently active and tested.