Only try to deliver a failure message if there was a from address.