added search dedup and sanitization