Implemented duplicate URL detection.