lookfirst/jmxtrans — Issue #10: Multiple threads per query caused concurrent issues
Author: securez
Created: 2012-02-01 10:40:47+00:00
State: open

My platform:
jmxtrans running on Linux x86, talking to some WL Servers, when i discover the issue, i have 5 threads per query, this causes in the log this kind of errors:

...
[30 ene 2012 14:11:30] [WorkerThread-LastJob] 2665 DEBUG (com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 4 query threads
[30 ene 2012 14:11:30] [pool-1-thread-4] 2698 ERROR (com.googlecode.jmxtrans.util.JmxUtils$ProcessQueryThread:163) - Error executing query
java.lang.IllegalStateException: Cannot register two instances of WorkContextAccessController
at weblogic.workarea.spi.WorkContextAccessController.(WorkContextAccessController.java:24)
at weblogic.workarea.spi.WorkContextAccessController$1.(WorkContextAccessController.java:65)
at weblogic.workarea.spi.WorkContextAccessController.getAccessController(WorkContextAccessController.java:65)
at weblogic.workarea.spi.WorkContextAccessController.isAccessAllowed(WorkContextAccessController.java:38)
at weblogic.workarea.WorkContextLocalMap.put(WorkContextLocalMap.java:50)
at weblogic.workarea.WorkContextLocalMap.put(WorkContextLocalMap.java:62)
at weblogic.workarea.WorkContextMapImpl.put(WorkContextMapImpl.java:52)
at weblogic.management.context.JMXContextHelper$DefaultJMXContextAccessImpl.putJMXContext(JMXContextHelper.java:89)
at weblogic.management.context.JMXContextHelper.putJMXContext(JMXContextHelper.java:36)
at weblogic.management.remote.common.RMIConnectionWrapper.initializeJMXContext(ClientProviderBase.java:1020)
at weblogic.management.remote.common.RMIConnectionWrapper.getMBeanInfo(ClientProviderBase.java:1030)
at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getMBeanInfo(RMIConnector.java:1052)
at com.googlecode.jmxtrans.util.JmxUtils.processQuery(JmxUtils.java:181)
at com.googlecode.jmxtrans.util.JmxUtils$ProcessQueryThread.run(JmxUtils.java:161)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
[30 ene 2012 14:11:30] [pool-1-thread-1] 2756 DEBUG (com.googlecode.jmxtrans.util.JmxUtils:195) - Executing queryName: com.bea:Location=AdminServer,Name=AdminServer,ServerRuntime=AdminServer,Type=JVMRuntime from query: Query
..

This seems that jmxconnection is not thread safe, i solve with only one thread per query, this causes me, a lot of headache becaouse some of the metrics are lost and not reach the writters.

This can happen with the WL RMI/JMX connector, i test it with different protocols (rmi, iiop, t3), all the same.