(#15529) Reinitialize app defaults before all and each test

Puppet settings were cleared after the first test, but never reinitialized. Mostly this caused problems for module testing, since they have to be able to load from the modulepath, but it also caused order dependent spec failures for puppet.

Reinitializing of settings could be made faster, but we are planning on rewriting settings so I don't want to spend a lot of time optimizing that.