Fixed deadlock bug in parallel web crawler, works now