Atmosphere/atmosphere — Issue #188: [runtime] Possible deadlock on unsubscribe
Author: jfarcand
Created: 2012-02-17 13:37:07+00:00
State: open
Labels: 0.9.0

Hi Jeanfrancois,

during some testing I ran into the a deadlock between threads in Atmosphere. The situation occurred on Glassfish 3.0.1 using Atmosphere 0.8.5 in combination with atmosphere-gwt. I don't have a lot of information on the issue, neither was I able to reproduce it (as is often with threading issues). The applications in use has a single async connection to which multiple models in the client can subscribe to receive push messages from the server. When the application is closed it tells the server to unsubscribe the models, there the error seems to occur.

From the dump:
Found one Java-level deadlock:
=============================
"http-thread-pool-8080-(9)":
waiting to lock monitor 0x00000000089fa5f8 (object 0x00000000feff88b0, a java.util.LinkedList),
which is held by "Thread-24"
"Thread-24":
waiting to lock monitor 0x00000000090ad200 (object 0x00000000feff89e8, a org.atmosphere.cpr.AtmosphereResourceImpl),
which is held by "http-thread-pool-8080-(9)"

The issues occurred only once (for now) but I thought to inform you about it, perhaps it contains some useful information for you.

Regards,
Chris

atmosphere_deadlock.txt

2012-02-17 13:15:13
Full thread dump Java HotSpot(TM) 64-Bit Server VM (19.1-b02 mixed mode):

"AMXConfigLoader.AMXConfigLoaderThread" daemon prio=6 tid=0x000000000a3be800 nid=0x112c waiting on condition [0x0000000014a3f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e0d0b8a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at org.glassfish.admin.mbeanserver.PendingConfigBeans.take(PendingConfigBeans.java:96)
at org.glassfish.admin.amx.impl.config.AMXConfigLoader$AMXConfigLoaderThread.doRun(AMXConfigLoader.java:582)
at org.glassfish.admin.amx.impl.config.AMXConfigLoader$AMXConfigLoaderThread.run(AMXConfigLoader.java:533)

"ComplianceMonitor.ValidatorThread" daemon prio=6 tid=0x00000000101d6000 nid=0xe8 waiting on condition [0x000000001443f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000fee6ee48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.doRun(ComplianceMonitor.java:240)
at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.run(ComplianceMonitor.java:227)

"JMX server connection timeout 173" daemon prio=6 tid=0x00000000101d5800 nid=0x908 in Object.wait() [0x00000000122bf000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fee6f650> (a [I)
at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)
- locked <0x00000000fee6f650> (a [I)
at java.lang.Thread.run(Thread.java:662)

"RMI TCP Connection(idle)" daemon prio=6 tid=0x00000000101d4800 nid=0xd24 waiting on condition [0x000000001111f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e1218ad0> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"RMI TCP Connection(idle)" daemon prio=6 tid=0x00000000079d2000 nid=0x17a4 waiting on condition [0x000000001101f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e1218ad0> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"Ejb-Timer-Thread-6" daemon prio=10 tid=0x00000000101d4000 nid=0x14a4 waiting on condition [0x0000000010f1f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e338a920> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"Ejb-Timer-Thread-5" daemon prio=10 tid=0x00000000101d3000 nid=0x7b0 waiting on condition [0x000000000e43f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e338a920> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"Ejb-Timer-Thread-4" daemon prio=10 tid=0x00000000079d4800 nid=0x164c waiting on condition [0x000000000d33f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e338a920> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"http-thread-pool-8080-(10)" daemon prio=10 tid=0x00000000101d2800 nid=0x1364 in Object.wait() [0x000000000e23f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1b27ed0> (a java.lang.Object)
at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
- locked <0x00000000e1b27ed0> (a java.lang.Object)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:662)

"http-thread-pool-8080-(9)" daemon prio=10 tid=0x00000000101d1800 nid=0x12bc waiting for monitor entry [0x000000000e03e000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.catalina.connector.AsyncContextImpl.notifyAsyncListeners(AsyncContextImpl.java:403)
- waiting to lock <0x00000000feff88b0> (a java.util.LinkedList)
at org.apache.catalina.connector.Request$6.resumed(Request.java:3689)
at org.apache.catalina.connector.Request$6.resumed(Request.java:3684)
at com.sun.grizzly.tcp.Response$ResponseAttachment.invokeCompletionHandler(Response.java:964)
at com.sun.grizzly.tcp.Response.resume(Response.java:767)
at org.apache.catalina.connector.Request.asyncComplete(Request.java:3782)
at org.apache.catalina.connector.AsyncContextImpl.complete(AsyncContextImpl.java:223)
at org.atmosphere.container.Servlet30CometSupport.action(Servlet30CometSupport.java:164)
at org.atmosphere.container.Servlet30CometSupport.action(Servlet30CometSupport.java:63)
at org.atmosphere.cpr.AtmosphereResourceImpl.resume(AtmosphereResourceImpl.java:220)
- locked <0x00000000feff89e8> (a org.atmosphere.cpr.AtmosphereResourceImpl)
at org.atmosphere.gwt.server.AtmosphereGwtHandler.disconnect(AtmosphereGwtHandler.java:319)
at org.atmosphere.gwt.server.AtmosphereGwtHandler.doServerMessage(AtmosphereGwtHandler.java:254)
at org.atmosphere.gwt.server.AtmosphereGwtHandler.onRequest(AtmosphereGwtHandler.java:180)
at org.atmosphere.cpr.AsynchronousProcessor.action(AsynchronousProcessor.java:219)
at org.atmosphere.cpr.AsynchronousProcessor.suspended(AsynchronousProcessor.java:154)
at org.atmosphere.container.Servlet30CometSupport.service(Servlet30CometSupport.java:87)
at org.atmosphere.cpr.AtmosphereServlet.doCometSupport(AtmosphereServlet.java:1217)
at org.atmosphere.cpr.AtmosphereServlet.doPost(AtmosphereServlet.java:1176)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:279)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:325)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:239)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:662)

"http-thread-pool-8080-(8)" daemon prio=10 tid=0x00000000101d1000 nid=0xdb8 in Object.wait() [0x000000000c1ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1b27ed0> (a java.lang.Object)
at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
- locked <0x00000000e1b27ed0> (a java.lang.Object)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:662)

"pool-45-thread-1" prio=6 tid=0x00000000101d0000 nid=0x1530 waiting on condition [0x0000000018c7f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ea050140> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"pool-47-thread-1" prio=6 tid=0x00000000101cf800 nid=0x10e4 waiting on condition [0x0000000018b7f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ea04efa8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"Session4022899239086758656" daemon prio=10 tid=0x0000000007a8f800 nid=0x1444 in Object.wait() [0x000000001924f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000eca0ee28> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at com.sun.messaging.jmq.jmsserver.service.imq.SessionListener.process(IMQDirectService.java:2890)
- locked <0x00000000eca0ee28> (a java.lang.Object)
at com.sun.messaging.jmq.jmsserver.service.imq.SessionListener.run(IMQDirectService.java:2857)
at java.lang.Thread.run(Thread.java:662)

"http-thread-pool-8080-(2)" daemon prio=10 tid=0x0000000007a8c800 nid=0xf08 in Object.wait() [0x000000001874f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1b27ed0> (a java.lang.Object)
at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
- locked <0x00000000e1b27ed0> (a java.lang.Object)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:662)

"JDWP Command Reader" daemon prio=6 tid=0x0000000007a8c000 nid=0x176c runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"ContainerBackgroundProcessor[StandardEngine[com.sun.appserv].StandardHost[server].StandardContext[/ht-print]]" daemon prio=10 tid=0x000000000eda6800 nid=0x1648 waiting on condition [0x000000001814f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1774)
at java.lang.Thread.run(Thread.java:662)

"ContainerBackgroundProcessor[StandardEngine[com.sun.appserv].StandardHost[server].StandardContext[/ht-service]]" daemon prio=10 tid=0x000000000eda6000 nid=0xf80 waiting on condition [0x000000001804f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1774)
at java.lang.Thread.run(Thread.java:662)

"Atmosphere-AsyncWrite-0" daemon prio=10 tid=0x000000000eda5000 nid=0x9bc waiting on condition [0x0000000017f4f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ea04ee58> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
at org.atmosphere.cpr.DefaultBroadcaster$3.run(DefaultBroadcaster.java:738)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"Atmosphere-BroadcasterConfig-0" daemon prio=10 tid=0x000000000eda4800 nid=0x10f4 waiting on condition [0x0000000017e4f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ea04e8b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
at org.atmosphere.cpr.DefaultBroadcaster$2.run(DefaultBroadcaster.java:485)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"pool-46-thread-1" prio=6 tid=0x000000000eda3800 nid=0xa28 waiting on condition [0x0000000017d4f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ea04f3e0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"ContainerBackgroundProcessor[StandardEngine[com.sun.appserv].StandardHost[server].StandardContext[/ht]]" daemon prio=10 tid=0x000000000eda3000 nid=0x14b8 waiting on condition [0x0000000017c4f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1774)
at java.lang.Thread.run(Thread.java:662)

"ContainerBackgroundProcessor[StandardEngine[com.sun.appserv].StandardHost[server].StandardContext[/ht-gmsgws]]" daemon prio=10 tid=0x000000000eda2800 nid=0x12c4 waiting on condition [0x0000000017b4f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1774)
at java.lang.Thread.run(Thread.java:662)

"connector-timer-proxy" daemon prio=10 tid=0x000000000f882000 nid=0xe3c in Object.wait() [0x000000001764f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e95056c0> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00000000e95056c0> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"Timer-2" daemon prio=10 tid=0x000000000f881000 nid=0xcc8 in Object.wait() [0x000000001754f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e9505220> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x00000000e9505220> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"derby.rawStoreDaemon" daemon prio=10 tid=0x000000000f880800 nid=0x9e4 in Object.wait() [0x000000001744f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e937e250> (a org.apache.derby.impl.services.daemon.BasicDaemon)
at org.apache.derby.impl.services.daemon.BasicDaemon.rest(Unknown Source)
- locked <0x00000000e937e250> (a org.apache.derby.impl.services.daemon.BasicDaemon)
at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)

"Timer-1" daemon prio=10 tid=0x000000000f87f800 nid=0x15d8 in Object.wait() [0x000000001734f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e94b8278> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x00000000e94b8278> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"derby.antiGC" daemon prio=2 tid=0x000000000f87e800 nid=0x169c in Object.wait() [0x000000001724f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e94b7078> (a org.apache.derby.impl.services.monitor.AntiGC)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.services.monitor.AntiGC.run(Unknown Source)
- locked <0x00000000e94b7078> (a org.apache.derby.impl.services.monitor.AntiGC)
at java.lang.Thread.run(Thread.java:662)

"Session4022899239073338112" daemon prio=10 tid=0x000000000f87c000 nid=0x16d8 in Object.wait() [0x000000001493f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e8e9d6d8> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at com.sun.messaging.jmq.jmsserver.service.imq.SessionListener.process(IMQDirectService.java:2890)
- locked <0x00000000e8e9d6d8> (a java.lang.Object)
at com.sun.messaging.jmq.jmsserver.service.imq.SessionListener.run(IMQDirectService.java:2857)
at java.lang.Thread.run(Thread.java:662)

"Session4022899239073075968" daemon prio=10 tid=0x000000000f87d000 nid=0x9f4 in Object.wait() [0x000000000e33f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e86aab18> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at com.sun.messaging.jmq.jmsserver.service.imq.SessionListener.process(IMQDirectService.java:2890)
- locked <0x00000000e86aab18> (a java.lang.Object)
at com.sun.messaging.jmq.jmsserver.service.imq.SessionListener.run(IMQDirectService.java:2857)
at java.lang.Thread.run(Thread.java:662)

"ContainerBackgroundProcessor[StandardEngine[com.sun.appserv].StandardHost[server].StandardContext[/ejb-timer-service-app]]" daemon prio=10 tid=0x000000000a3bd800 nid=0xbec waiting on condition [0x000000001483f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1774)
at java.lang.Thread.run(Thread.java:662)

"Session4022899239069997312" daemon prio=10 tid=0x00000000079ca000 nid=0x16ec in Object.wait() [0x0000000010e1f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e5ab6458> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at com.sun.messaging.jmq.jmsserver.service.imq.SessionListener.process(IMQDirectService.java:2890)
- locked <0x00000000e5ab6458> (a java.lang.Object)
at com.sun.messaging.jmq.jmsserver.service.imq.SessionListener.run(IMQDirectService.java:2857)
at java.lang.Thread.run(Thread.java:662)

"ClusterServiceListener" daemon prio=10 tid=0x00000000079c4800 nid=0x106c runnable [0x0000000010d1f000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0x00000000e5c6bfe0> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at com.sun.messaging.jmq.jmsserver.multibroker.fullyconnected.ClusterServiceListener.run(ClusterImpl.java:1903)

"MessageBusCallbackDispatcher" daemon prio=10 tid=0x00000000079c7000 nid=0x1488 in Object.wait() [0x0000000010c1f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e5cd0258> (a java.util.LinkedList)
at java.lang.Object.wait(Object.java:485)
at com.sun.messaging.jmq.jmsserver.multibroker.CallbackDispatcher.run(CallbackDispatcher.java:566)
- locked <0x00000000e5cd0258> (a java.util.LinkedList)

"jms_ACCEPT" daemon prio=10 tid=0x00000000079cc800 nid=0x900 runnable [0x0000000010b1f000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0x00000000e5c6aa98> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at com.sun.messaging.jmq.jmsserver.net.tcp.TcpProtocol.accept(TcpProtocol.java:281)
at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPService.run(IMQIPService.java:574)
at java.lang.Thread.run(Thread.java:662)

"admin_ACCEPT" daemon prio=10 tid=0x00000000079c8800 nid=0x1380 runnable [0x000000000e13f000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0x00000000e5c6b730> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at com.sun.messaging.jmq.jmsserver.net.tcp.TcpProtocol.accept(TcpProtocol.java:281)
at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPService.run(IMQIPService.java:574)
at java.lang.Thread.run(Thread.java:662)

"RMI RenewClean-[172.16.7.230:60199]" daemon prio=6 tid=0x00000000079c7800 nid=0x318 in Object.wait() [0x000000000da3f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e5ce05b8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x00000000e5ce05b8> (a java.lang.ref.ReferenceQueue$Lock)
at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:516)
at java.lang.Thread.run(Thread.java:662)

"RMI TCP Accept-0" daemon prio=10 tid=0x00000000079d4000 nid=0xa04 runnable [0x000000000d23f000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0x00000000e5ce0950> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
at java.lang.Thread.run(Thread.java:662)

"Broker Monitor" daemon prio=10 tid=0x00000000079d3800 nid=0x13ec in Object.wait() [0x000000000c6ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e5cd16b0> (a java.util.Collections$SynchronizedSet)
at java.lang.Object.wait(Object.java:485)
at com.sun.messaging.jmq.jmsserver.core.cluster.BrokerConsumers.run(MultibrokerRouter.java:1673)
- locked <0x00000000e5cd16b0> (a java.util.Collections$SynchronizedSet)
at java.lang.Thread.run(Thread.java:662)

"MQTimer-Thread" daemon prio=10 tid=0x000000000755a800 nid=0x1374 in Object.wait() [0x000000000c3ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e5cdee78> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00000000e5cdee78> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"p: thread-pool-1; w: 3" daemon prio=10 tid=0x000000000755f800 nid=0x1740 in Object.wait() [0x000000000c2ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e47342e8> (a com.sun.corba.ee.impl.orbutil.threadpool.WorkQueueImpl)
at com.sun.corba.ee.impl.orbutil.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:120)
- locked <0x00000000e47342e8> (a com.sun.corba.ee.impl.orbutil.threadpool.WorkQueueImpl)
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:506)

"Thread-31" prio=10 tid=0x00000000079d2800 nid=0x1290 in Object.wait() [0x0000000010a1f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e532c498> (a com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive)
at java.lang.Object.wait(Object.java:485)
at com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive.run(Util.java:827)
- locked <0x00000000e532c498> (a com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive)

"p: thread-pool-1; w: 2" daemon prio=10 tid=0x00000000079d1000 nid=0x1184 runnable [0x000000001091f000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0x00000000e5422b68> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:261)
at com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.getAcceptedSocket(SocketOrChannelAcceptorImpl.java:141)
at com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:111)
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:492)
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:528)

"p: thread-pool-1; w: 1" daemon prio=10 tid=0x00000000079d0800 nid=0x14d8 runnable [0x000000000e83f000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0x00000000e5421f58> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:261)
at com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.getAcceptedSocket(SocketOrChannelAcceptorImpl.java:141)
at com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:111)
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:492)
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:528)

"ContainerBackgroundProcessor[StandardEngine[com.sun.appserv]]" daemon prio=10 tid=0x00000000079cf800 nid=0x13b8 waiting on condition [0x000000000e73f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1774)
at java.lang.Thread.run(Thread.java:662)

"ContainerBackgroundProcessor[StandardEngine[com.sun.appserv].StandardHost[server].StandardContext[]]" daemon prio=10 tid=0x00000000079cf000 nid=0x15ac waiting on condition [0x000000000e63f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1774)
at java.lang.Thread.run(Thread.java:662)

"Timer-0" daemon prio=10 tid=0x00000000079ce000 nid=0x8b8 in Object.wait() [0x000000000e53f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e338b0c0> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00000000e338b0c0> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"http-thread-pool-8080-(1)" daemon prio=10 tid=0x000000000887b800 nid=0x1634 in Object.wait() [0x000000000c0ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1b27ed0> (a java.lang.Object)
at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
- locked <0x00000000e1b27ed0> (a java.lang.Object)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:662)

"{felix.fileinstall.poll=5000, felix.fileinstall.bundles.new.start=true, service.pid=org.apache.felix.fileinstall.00feed5e-64d5-4d6f-90f0-7640715a4805, felix.fileinstall.dir=C:\Develop\servers\sges-v3\glassfish\domains\domain1/autodeploy/bundles/, felix.fileinstall.filename=org.apache.felix.fileinstall-autodeploy-bundles.cfg, service.factorypid=org.apache.felix.fileinstall, felix.fileinstall.debug=1}" daemon prio=6 tid=0x000000000755e000 nid=0x13c4 in Object.wait() [0x000000000de3f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1ae3c90> (a org.apache.felix.fileinstall.internal.DirectoryWatcher)
at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:210)
- locked <0x00000000e1ae3c90> (a org.apache.felix.fileinstall.internal.DirectoryWatcher)

"Thread-27" daemon prio=10 tid=0x00000000079be000 nid=0x570 runnable [0x000000000baff000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:273)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:255)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:136)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00000000e1b00c40> (a sun.nio.ch.Util$2)
- locked <0x00000000e1b00c30> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000e1b009a8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"Thread-25" daemon prio=10 tid=0x00000000079c1000 nid=0x13a8 in Object.wait() [0x000000000b6ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1b12608> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
- locked <0x00000000e1b12608> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.start(Controller.java:843)
at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
- locked <0x00000000e1b003d8> (a java.lang.Object)
at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

"SCR Component Actor" daemon prio=6 tid=0x000000000755e800 nid=0x1568 in Object.wait() [0x000000000df3f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1aedd58> (a java.util.LinkedList)
at java.lang.Object.wait(Object.java:485)
at org.apache.felix.scr.impl.ComponentActorThread.run(ComponentActorThread.java:63)
- locked <0x00000000e1aedd58> (a java.util.LinkedList)

"Thread-24" daemon prio=10 tid=0x000000000755d000 nid=0x1430 waiting for monitor entry [0x000000000dd3f000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.atmosphere.cpr.AtmosphereResourceImpl.cancel(AtmosphereResourceImpl.java:679)
- waiting to lock <0x00000000feff89e8> (a org.atmosphere.cpr.AtmosphereResourceImpl)
at org.atmosphere.cpr.AsynchronousProcessor.cancelled(AsynchronousProcessor.java:505)
- locked <0x00000000ea96b2d0> (a org.apache.catalina.connector.RequestFacade)
at org.atmosphere.container.Servlet30CometSupport$CometListener.onError(Servlet30CometSupport.java:197)
at org.apache.catalina.connector.AsyncContextImpl.notifyAsyncListeners(AsyncContextImpl.java:427)
- locked <0x00000000feff88b0> (a java.util.LinkedList)
at org.apache.catalina.connector.Request$6.cancelled(Request.java:3699)
at org.apache.catalina.connector.Request$6.cancelled(Request.java:3684)
at org.apache.catalina.connector.Request$RequestAttachment.handleSelectedKey(Request.java:3970)
at com.sun.grizzly.SelectorHandlerRunner.handleSelectedKey(SelectorHandlerRunner.java:274)
at com.sun.grizzly.SelectorHandlerRunner.handleSelectedKeys(SelectorHandlerRunner.java:258)
at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:195)
at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"{felix.fileinstall.poll=5000, felix.fileinstall.bundles.new.start=true, felix.fileinstall.dir=C:\Develop\servers\sges-v3\glassfish/modules/autostart/, felix.fileinstall.debug=1}" daemon prio=6 tid=0x00000000079bd800 nid=0xf58 in Object.wait() [0x000000000b8ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1ae51e8> (a org.apache.felix.fileinstall.internal.DirectoryWatcher)
at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:210)
- locked <0x00000000e1ae51e8> (a org.apache.felix.fileinstall.internal.DirectoryWatcher)

"Thread-22" daemon prio=10 tid=0x00000000079bf800 nid=0x2bc in Object.wait() [0x000000000b5ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1b2bfb8> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
- locked <0x00000000e1b2bfb8> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.start(Controller.java:843)
at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
- locked <0x00000000e1b25d90> (a java.lang.Object)
at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

"Configuration Updater" daemon prio=6 tid=0x000000000755c800 nid=0xc3c in Object.wait() [0x000000000dc3f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1aea710> (a java.util.LinkedList)
at java.lang.Object.wait(Object.java:485)
at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:67)
- locked <0x00000000e1aea710> (a java.util.LinkedList)

"telnetconsole.Listener" daemon prio=6 tid=0x000000000755b800 nid=0x1c4 runnable [0x000000000db3f000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0x00000000e1b8eb48> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at org.apache.felix.shell.remote.Listener$Acceptor.run(Listener.java:137)
at java.lang.Thread.run(Thread.java:662)

"GlassFish Kernel Main Thread" prio=6 tid=0x0000000007559800 nid=0x1788 in Object.wait() [0x000000000d93f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1b8c210> (a com.sun.enterprise.v3.server.AppServerStartup$1)
at java.lang.Object.wait(Object.java:485)
at com.sun.enterprise.v3.server.AppServerStartup$1.run(AppServerStartup.java:146)
- locked <0x00000000e1b8c210> (a com.sun.enterprise.v3.server.AppServerStartup$1)

"AutoDeployer" daemon prio=6 tid=0x000000000755b000 nid=0x1534 in Object.wait() [0x000000000c9ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1bc3758> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00000000e1bc3758> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"RMI RenewClean-[172.16.7.230:8686]" daemon prio=6 tid=0x0000000007559000 nid=0xa64 in Object.wait() [0x000000000d83f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1e802d0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x00000000e1e802d0> (a java.lang.ref.ReferenceQueue$Lock)
at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:516)
at java.lang.Thread.run(Thread.java:662)

"RMI Scheduler(0)" daemon prio=6 tid=0x0000000008882800 nid=0x958 waiting on condition [0x000000000d73f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e1218c70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"transaction-manager" daemon prio=6 tid=0x0000000008882000 nid=0x17ec in Object.wait() [0x000000000d63f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1c09590> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x00000000e1c09590> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"GC Daemon" daemon prio=2 tid=0x0000000008881000 nid=0x888 in Object.wait() [0x000000000d53f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e125c0b8> (a sun.misc.GC$LatencyLock)
at sun.misc.GC$Daemon.run(GC.java:100)
- locked <0x00000000e125c0b8> (a sun.misc.GC$LatencyLock)

"RMI Reaper" prio=6 tid=0x000000000887f800 nid=0x1410 in Object.wait() [0x000000000d43f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e125c0c8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x00000000e125c0c8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)
at java.lang.Thread.run(Thread.java:662)

"RMI TCP Accept-8686" daemon prio=6 tid=0x0000000008880800 nid=0x708 runnable [0x000000000cccf000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0x00000000e13e0218> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
at java.lang.Thread.run(Thread.java:662)

"pool-13-thread-1" daemon prio=6 tid=0x000000000887e800 nid=0x13f0 waiting on condition [0x000000000d13f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e0d0bcb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"pool-12-thread-1" daemon prio=6 tid=0x000000000887d800 nid=0xd34 waiting on condition [0x000000000d03f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e13fb130> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"pool-10-thread-1" daemon prio=6 tid=0x000000000887c000 nid=0x139c waiting on condition [0x000000000ce3f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e0d0baa0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"DynamicReloader" daemon prio=6 tid=0x00000000079cc000 nid=0xe54 in Object.wait() [0x000000000c8ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e13fb960> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00000000e13fb960> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"Grizzly" daemon prio=10 tid=0x00000000079cb000 nid=0xfa4 waiting on condition [0x000000000cbcf000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e13ff6a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"http-thread-pool-4848-(2)" daemon prio=10 tid=0x00000000079ca800 nid=0x170c in Object.wait() [0x000000000c7ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e13fa4a0> (a java.lang.Object)
at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
- locked <0x00000000e13fa4a0> (a java.lang.Object)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:662)

"http-thread-pool-4848-(1)" daemon prio=10 tid=0x00000000079c5800 nid=0x17b8 in Object.wait() [0x000000000bfff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e13fa4a0> (a java.lang.Object)
at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
- locked <0x00000000e13fa4a0> (a java.lang.Object)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:662)

"Thread-16" daemon prio=10 tid=0x00000000079c4000 nid=0x1288 runnable [0x000000000beff000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:273)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:255)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:136)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00000000e185a470> (a sun.nio.ch.Util$2)
- locked <0x00000000e185a460> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000e185a1d8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"Thread-14" daemon prio=10 tid=0x00000000079c3800 nid=0x1150 in Object.wait() [0x000000000bdff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1861440> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
- locked <0x00000000e1861440> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.start(Controller.java:843)
at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
- locked <0x00000000e1859c00> (a java.lang.Object)
at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:129)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

"Thread-13" daemon prio=10 tid=0x00000000079c2800 nid=0xedc runnable [0x000000000bcff000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:273)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:255)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:136)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00000000e197f600> (a sun.nio.ch.Util$2)
- locked <0x00000000e197f5f0> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000e197f368> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"Thread-11" daemon prio=10 tid=0x00000000079c2000 nid=0x17a8 in Object.wait() [0x000000000bbff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e19c68d0> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
- locked <0x00000000e19c68d0> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.start(Controller.java:843)
at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
- locked <0x00000000e197ed90> (a java.lang.Object)
at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:129)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

"Thread-8" daemon prio=10 tid=0x00000000079c0800 nid=0x4a4 runnable [0x000000000b9ff000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:273)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:255)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:136)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00000000e1828718> (a sun.nio.ch.Util$2)
- locked <0x00000000e1828708> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000e1400bb0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

"Thread-5" daemon prio=10 tid=0x00000000079bf000 nid=0xe4c in Object.wait() [0x000000000b7ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e1404180> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
- locked <0x00000000e1404180> (a java.util.concurrent.atomic.AtomicInteger)
at com.sun.grizzly.Controller.start(Controller.java:843)
at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
- locked <0x00000000e1401038> (a java.lang.Object)
at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

"Thread-1" daemon prio=6 tid=0x0000000007f3b800 nid=0x5cc waiting on condition [0x000000000b03f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e0d92300> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
at com.sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:510)
at com.sun.enterprise.server.logging.GFFileHandler$3.run(GFFileHandler.java:255)

"pool-2-thread-1" daemon prio=6 tid=0x000000000a099800 nid=0x1130 waiting on condition [0x0000000008f4f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e0dc2138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)

"FelixPackageAdmin" daemon prio=6 tid=0x00000000079de000 nid=0xe64 in Object.wait() [0x0000000008e4f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e096cfd0> (a org.apache.felix.framework.PackageAdminImpl)
at java.lang.Object.wait(Object.java:485)
at org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:314)
- locked <0x00000000e096cfd0> (a org.apache.felix.framework.PackageAdminImpl)
at java.lang.Thread.run(Thread.java:662)

"FelixStartLevel" daemon prio=6 tid=0x00000000079dd800 nid=0x15e0 in Object.wait() [0x0000000008d4f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e084e8f8> (a java.util.ArrayList)
at java.lang.Object.wait(Object.java:485)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:242)
- locked <0x00000000e084e8f8> (a java.util.ArrayList)
at java.lang.Thread.run(Thread.java:662)

"FelixDispatchQueue" prio=6 tid=0x0000000007494800 nid=0x17dc in Object.wait() [0x0000000007c7f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e03c0ec0> (a java.util.ArrayList)
at java.lang.Object.wait(Object.java:485)
at org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:931)
- locked <0x00000000e03c0ec0> (a java.util.ArrayList)
at org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)
at org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:106)
at java.lang.Thread.run(Thread.java:662)

"DestroyJavaVM" prio=6 tid=0x000000000056f800 nid=0x17e0 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"OSGi Framework Launcher" prio=6 tid=0x0000000006e90800 nid=0x10c0 in Object.wait() [0x000000000737e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e087fe10> (a org.apache.felix.framework.util.ThreadGate)
at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:79)
- locked <0x00000000e087fe10> (a org.apache.felix.framework.util.ThreadGate)
at org.apache.felix.framework.Felix.waitForStop(Felix.java:794)
at org.apache.felix.main.Main.main(Main.java:299)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.glassfish.bootstrap.ASMainFelix$1.run(ASMainFelix.java:116)
at java.lang.Thread.run(Thread.java:662)

"Low Memory Detector" daemon prio=6 tid=0x0000000006387000 nid=0x1718 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=10 tid=0x0000000006384000 nid=0x1760 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x0000000006378800 nid=0x14d4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"JDWP Event Helper Thread" daemon prio=6 tid=0x0