Fix crawl duplicate url.