Show the user if there was an error during provider setup.

Also:
- add slot decorator where needed
- remove incorrect trigger docstring
- factor out setting the same message into an existing helper

[Closes #5424]