Weed out duplicate URLs from the resulting list.