I personally think the exclamation mark looks a little odd and it also isn't consistent with the rest of our errors/messages.

Furthermore, it's possible that the exclamation mark conveys something different in other languages.