More strictly validate recipient fields.

Previously we'd accept things like

"to": [{"email": ["hello@example.com"]}],

which we shouldn't.