Have a separate flow for invitations when the invitee is an existing user.  Send a slightly different email, and make the invitation page have different text as well as the ability to confirm/decline right inline, without having to log in or sign up.