(PE- 6014) Create NOOP CA service

When you load up Ruby it actually does create some things by default. I just overrode all the settings I could find that does this, but there could be more in the future.  I'll just put that `with-puppet-conf` function in a central place and reuse it.