Remove delivered-to addresses from list of recipients

This is necessary so we don't add delivered-to addresses to the list of addressee/collaborators.