A first approach to fight again link duplicates

we store every link in a global hash called 'links', if the link is already stored we ignore it, if not we proceed