Make multi thread crawling possible.