Fix 'Contact Details' for non-contacts

When searching contacts, if the list has only the item you are searching
for, and you long-press, some of the options that appear (contact
details, invite to existing) can cause issues. This removes those items
in that scenario.