retrieve unique set of links for inbound link generation