Yes, you can set up access permissions for a contact so that only certain users can view or edit the contact in the Office 365 Admin Center.