Project: talendesb
Issue TESB-4473: tESBConsumer SSL truststore configuration does not work in TESB container
* I've created a consumer job which is intended to invoke a service using HTTPS transport. * In the consumer job I've enabled the check box for "Trust server with SSL" and ** configured the location of the "TrustStore file" to be found relative to the container working directory at "./etc/keystore.jks" ** as well as a proper TrustStore password to access that keystore * I've published the consumer job to the artifact repository (Archiva) using the "Publish" command in the tooling * I've executed the consumer job by installing the consumer feature within the TESB container As result of executing the consumer job within the TESB container I'm experiencing the problem, that the configured keystore.jks file cannot be found - resulting in the following root cause: {color:red} {code} sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target {code} {color} The complete stacktrace is: {code} 15:52:11,545 | WARN | pool-9-thread-1 | ache.cxf.common.logging.LogUtils 372 | - - | Interceptor for {http://services.test.esb.talend.org/ufp/ufpService}ufpServiceHttpsProvider#{http://services.test.esb.talend.org/ufp/ufpService}doRequestResponse has thrown exception, unwinding now org.apache.cxf.interceptor.Fault: Could not send Message. org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:64)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:535)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:465)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:368)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:321)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:341)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:327)[141:org.apache.cxf.bundle:2.5.0] org.talend.esb.job.controller.internal.RuntimeESBConsumer.sendDocument(RuntimeESBConsumer.java:213)[176:org.talend.esb.job.controller:5.0.1] org.talend.esb.job.controller.internal.RuntimeESBConsumer.invoke(RuntimeESBConsumer.java:201)[176:org.talend.esb.job.controller:5.0.1] unifiedplatform.httpsconsumerdorequestresponse_0_1.HttpsConsumerDoRequestResponse.tFixedFlowInput_1Process(HttpsConsumerDoRequestResponse.java:1801)[191:HttpsConsumerDoRequestResponse:0.1.0] unifiedplatform.httpsconsumerdorequestresponse_0_1.HttpsConsumerDoRequestResponse.runJobInTOS(HttpsConsumerDoRequestResponse.java:2254)[191:HttpsConsumerDoRequestResponse:0.1.0] org.talend.esb.job.controller.internal.SimpleJobTask$JobRunner.run(SimpleJobTask.java:80)[176:org.talend.esb.job.controller:5.0.1] java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)[:1.6.0_29] java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)[:1.6.0_29] java.util.concurrent.FutureTask.run(FutureTask.java:138)[:1.6.0_29] org.talend.esb.job.controller.internal.SimpleJobTask.run(SimpleJobTask.java:62)[176:org.talend.esb.job.controller:5.0.1] java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_29] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_29] java.lang.Thread.run(Thread.java:680)[:1.6.0_29] Caused by: javax.net.ssl.SSLHandshakeException: SSLHandshakeException invoking https://provider:9078/services/tesb.test.participants.osgi.ufp.provider/soap/StandardTestServiceProvider/https/: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)[:1.6.0_29] sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)[:1.6.0_29] sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)[:1.6.0_29] java.lang.reflect.Constructor.newInstance(Constructor.java:513)[:1.6.0_29] org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1431)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1416)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:649)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)[141:org.apache.cxf.bundle:2.5.0]	... 19 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)[:1.6] com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)[:1.6] com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)[:1.6] com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)[:1.6] com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)[:1.6] com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)[:1.6] com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)[:1.6] com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)[:1.6] com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)[:1.6] com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)[:1.6] com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)[:1.6] com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)[:1.6] sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)[:1.6] sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:166)[:1.6] sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1014)[:1.6.0_29] sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:230)[:1.6] org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleHeadersTrustCaching(HTTPConduit.java:1368)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.onFirstWrite(HTTPConduit.java:1310)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:42)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)[141:org.apache.cxf.bundle:2.5.0] org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1388)[141:org.apache.cxf.bundle:2.5.0]	... 22 more Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:323)[:1.6.0_29] sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:217)[:1.6.0_29] sun.security.validator.Validator.validate(Validator.java:218)[:1.6.0_29] com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:126)[:1.6] com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:209)[:1.6] com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:249)[:1.6] com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1185)[:1.6]	... 38 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:174)[:1.6.0_29] java.security.cert.CertPathBuilder.build(CertPathBuilder.java:238)[:1.6.0_29] sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:318)[:1.6.0_29]	... 44 more 15:52:11,552 | INFO | pool-9-thread-1 | internal.SimpleJobTask$JobRunner 81 | - - | Job HttpsConsumerDoRequestResponse finished, return code is 1 {code} As result of configuring SSL in the consumer job the following code is been generated: {code:java} config_tESBConsumer_1.setKeyStoreFile(System.getProperty("javax.net.ssl.keyStore")); config_tESBConsumer_1.setKeyStoreType(System.getProperty("javax.net.ssl.keyStoreType")); config_tESBConsumer_1.setKeyStorePwd(System.getProperty("javax.net.ssl.keyStorePassword")); System.setProperty("java.protocol.handler.pkgs", "com.sun.net.ssl.internal.www.protocol"); System.setProperty("javax.net.ssl.trustStore", "./etc/keystore.jks"); System.setProperty("javax.net.ssl.trustStorePassword", "password"); {code} Hence it tries to reset the system wide ssl configuration in order to configure the keystore for the current consumer job execution. In my view there are two major problems with this approach: {panel} h4. 1. I don't think it is appropriate to "silently" reconfigure the system wide keystore configuration just in order to run a consumer job. A running TESB container can have many purposes, not only running several consumer jobs. There might be other bundles running which need the system wide SSL configuration as it is - or as the user has configured it himself. Using this approach a consumer job would *silently* change the SSL configuration for the whole VM and does not even restore the previous configuration (which also would not be a proper solution in my view). This might break other functionality running in this container. Also you could think of problems when running consumer jobs with different configuration in parallel. h4. 2. This approach setting the ssl system properties is not working when running the consumer job within the TESB container anyway. :) After some (long) analysis - finally using jconsole and taking a look into the list of current system properties in the running consumer VM after having executed the consumer job - I've found out that the system properties are definitelly not set! Either the code is not been executed - or there is any security mechanism in Karaf disabling the possibility to set these properties programatically. I don't know. As the path to the keystore file has not been set, the system uses the default Java truststore. Based on this the handshake with the correctly configured provider using Karaf HTTPS configuration fails, as the required certificate cannot be found in the default truststore on consumer side. A corresponding handshake exception is also visible on provider side: {code} javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)[:1.6] at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:136)[:1.6] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1806)[:1.6] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:986)[:1.6] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)[:1.6] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)[:1.6] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)[:1.6] at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:659)[59:org.eclipse.jetty.server:7.4.5.v20110725] at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)[53:org.eclipse.jetty.util:7.4.5.v20110725] at java.lang.Thread.run(Thread.java:680)[:1.6.0_29] {code} As a _*workaround*_ I've set the same properties (_javax.net.ssl.trustStore_ and _javax.net.ssl.trustStorePassword_) with the same configuration (path and password) as "-D" parameter in the TESB start script "trun" and have executed the same consumer job feature once again. The result was a successful request-response message exchange in this case. {panel} I think the proper and better approach for HTTPS configuration on consumer side with CXF is to _*create / generate a corresponding XML html-conduit configuration using spring/blueprint*_. As additional advantage this approach would also enable a setting which is not available with the normal Java SSL configuration: _*disabling the host verification*_ - which is a _*very helpful*_ feature for many usages and is been implemented by all containers (including Karaf) - except pure old Java itself. :) Such a solution would enable us to add a corresponding check box in the component. With the current approach (setting the SSL properties) such a feature is not available. Please note that using *tSetKeystore* component instead of directly configuring SSL within the tESBConsumer component does not fix the problem as the approach of the generated code is the same. => Find attached the exported consumer job as example. The settings for trust store and password are only meant for the stand alone case. It's anyway not the job of the job /route / service designer to specify the location of the trust store in the container. This is the responsibility of an admin. We should rather provide a way to specify these properties container wide and use them for all data service clients. These properties are: - WSDL port name for which the configuration is valid - location of the trust store and corresponding password - location of the key store and corresponding password - alias for the certificate - host verification on / off Of course it should be possible to specify configuration for several ports. As usual part of this item is to add documentation for this feature.
Story points: 3