Elastic Agent stuck at pending
Asked by Frederick Myers:

Our Windows elastic agents get stuck at "Pending". I've looked through the various related questions on this site and none of the solutions work for us. When I left work on a Friday, everything seemed to be working fine (and had been for a couple of weeks) but on Monday everything is hosed up. I can ping our bamboo url fine and the Bamboo user is not set to have an expiring password.

Log

```
2015-08-31 16:44:23,585 FATAL [Thread-1] [AgentBootstrap] Exiting due to fatal exception.java.lang.reflect.InvocationTargetException	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)	at java.lang.reflect.Method.invoke(Method.java:597)	at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:28)	at java.lang.Thread.run(Thread.java:662)Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'buildAgentController' defined in class path resource [applicationContextRemoteAgent.xml]: Cannot resolve reference to bean 'agentQueueAccessor' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentQueueAccessor' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot resolve reference to bean 'connectionFactory' while setting bean property 'connectionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:336)	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)	at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:646)	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:140)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:703)	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)	at org.springframework.context.support.ClassPathXmlApplicationContext.&lt;init&gt;(ClassPathXmlApplicationContext.java:139)	at org.springframework.context.support.ClassPathXmlApplicationContext.&lt;init&gt;(ClassPathXmlApplicationContext.java:105)	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:227)	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:70)	at com.atlassian.bamboo.agent.elastic.client.ElasticAgent.start(ElasticAgent.java:31)	... 6 moreCaused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentQueueAccessor' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot resolve reference to bean 'connectionFactory' while setting bean property 'connectionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:336)	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1456)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1197)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)	... 25 moreCaused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:290)	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)	at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:646)	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:140)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)	... 35 moreCaused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:278)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:276)	... 47 moreCaused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:164)	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:125)	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:270)	... 52 moreCaused by: java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable	at com.google.common.io.Closeables.close(Closeables.java:80)	at com.google.common.io.Closeables.closeQuietly(Closeables.java:99)	at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.ping(BambooActiveMQConnectionFactory.java:191)	at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.validateUri(BambooActiveMQConnectionFactory.java:145)	at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.setBrokerURL(BambooActiveMQConnectionFactory.java:63)	at org.apache.activemq.ActiveMQConnectionFactory.&lt;init&gt;(ActiveMQConnectionFactory.java:142)	at org.apache.activemq.ActiveMQConnectionFactory.&lt;init&gt;(ActiveMQConnectionFactory.java:138)	at org.apache.activemq.ActiveMQSslConnectionFactory.&lt;init&gt;(ActiveMQSslConnectionFactory.java:75)	at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.&lt;init&gt;(BambooActiveMQConnectionFactory.java:53)	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:148)	... 54 more
```


Answers:
author: Przemek Bruski
body: Are you using a custom image?

Half a year ago, Atlassian announced deprecation of Java 7 (<https://confluence.atlassian.com/display/BAMBOO/End+of+support+announcements+for+Bamboo>) . Support for Java 7 has been removed this weekend. All images have to be running Java 8.

comments: author: Glen Sonnenberg
body: Apparently I'm still running Java 6 and it was working until this weekend. :( How do I go about updating my image to Java 8? Any updates I need to apply?


author: Glen Sonnenberg
body: Looking into this further the system has Java 7 installed also. I manually installed Java 8 and am rebooting the instance. Hopefully that will fix it? I can create a new AMI and try starting it from Bamboo if that's required. If not, I'm kind of hosed. Any thoughts on what I need to do to get this custom image current?


author: Frederick Myers
body: Yeah, it is a custom image so that must be the issue. Now trying to figure out how to get it upgraded. Updated to JDK 1.8 but still no luck


author: Frederick Myers
body: Download latest version of the atlassian-bamboo-elastic-image and grabbed the updated agent code from there. Also had to manually edit the .bat file since I was getting command not found errors on java.exe even though JAVA_HOME was defined. After that, build agents are being found and running. Thanks!


author: Glen Sonnenberg
body: After I installed Java 8, I got mine working by editing the /etc/profile.d/bamboo.sh file to set up the correct $JAVA_HOME variable. I hope that helps.


author: Glen Sonnenberg
body: I ran into the same problem this morning. Is there a way to update the custom image? I logged into it and did a "sudo yum update" and am now creating a new AMI. Will this resolve the problem?


Tags: cloud