Now only unique PDF URLs are output
Added checked_url hash which urls are checked against to prevent duplicate work, fixed urls_to_crawl hash so that it no longer contains multiple hashes and parent urls (bug)