adding auxiliary function to check url file dups