Added a semaphore which limits the number of simultaneous crawls which may occur