Ensure that the app config does not get mutated by addons.

Closes #2131.

Thank you to @rlivsey and @ef4 for idenitifying this.