(PE- 6014) Create NOOP CA service

Is all this jruby stuff really necessary? Or just copy-paste from the other test?