(PE- 6014) Create NOOP CA service

Same here, now you're pointing into our source tree as the directory for the master config, which would cause the master to potentially write some files there, which would require us to re-add the `.gitignore` stuff that was pointing into our source tree.  I don't think we want to do that.  I think I have examples of how to solve this elsewhere, let me find something...