sanitize spider results properly
unique spider results