Ignore duplicate urls
Improved url handling