This is actually called every time a settings value is changed: https://github.com/puppetlabs/puppet/blob/master/lib/puppet/settings.rb#L760