Update mail settings configuration change.

Prevents issues getting opened up all over the place about it.
Fixes #5367