Added ability to filter out duplicate links.