Process the JSON retrieved from the HB website (remove duplicates etc).