Bamboo Elastic Agent failed to connect
Asked by Sam:

I'm trying to connect elastic agents to my Bamboo Server and the agent is stuck in a pending state.

**Agent Logs**

```
WARN [AgentRunnerThread] [ResourceResolver] Unable to find resource bamboo-capabilities.properties
INFO [AgentRunnerThread] [DefaultAgentCapabilityManagerImpl] Failed to load bamboo-capabilities.properties
WARN [AgentRunnerThread] [FailoverTransport] Transport (tcp://localhost:4527?wireFormat.maxInactivityDuration=300000) failed, attempting to automatically reconnect
java.io.IOException: Wire format negotiation timeout: peer did not send his wire format.
#011at com.sun.proxy.$Proxy40.registerAgent(Unknown Source) ~[?:?]
#011at com.atlassian.bamboo.agent.elastic.client.ElasticAgentRegistrationBean.registerAgent(ElasticAgentRegistrationBean.java:25) ~[atlassian-bamboo-agent-remote-9.0.1.jar:?]
WARN [ActiveMQ Task-2] [FailoverTransport] Failed to connect to [tcp://localhost:4527?wireFormat.maxInactivityDuration=300000] after: 1 attempt(s) with Wire format negotiation timeout: peer did not send his wire format., continuing to retry.
```

**Bamboo Server**

Broker client URL: failover:(tcp://ip_address:54663?wireFormat.maxInactivityDuration=300000\&socket.verifyHostName=false)?initialReconnectDelay=15000\&maxReconnectAttempts=10


Answers:
author: Eduardo Alvarenga
body: Hello [@Sam](/t5/user/viewprofilepage/user-id/5124975)

Welcome to the **Atlassian Community**!

Is the *ip_address:54663* reachable from your Bamboo Server? It may also be the case of another process listening on port 54663 and preventing ActiveMQ from working.

Please check this KB article for more tips:

* [Bamboo Elastic Agents stuck on pending state with wire format negotiation timeout error](https://confluence.atlassian.com/bamkb/bamboo-elastic-agents-stuck-on-pending-state-with-wire-format-negotiation-timeout-error-1142228158.html)

Cheers,

Eduardo Alvarenga  
Atlassian Support APAC

**--please don't forget to Accept the answer if the reply is helpful--**

comments: author: Sam
body: Hi Eduardo, thank you for the reply. What do you mean by ip_address is reachable from the Bamboo Server? Are you referring to Agent is able to ping the Bamboo Server? When I ran this command **netstat -an \| grep 54633** on Bamboo Server, I don't see anything that is LISTEN on that port. Also, I did try the fix in the KB article you sent with no luck.


author: Eduardo Alvarenga
body: Hello [@Sam](/t5/user/viewprofilepage/user-id/5124975)

ActiveMQ will listen to port TCP/54663 on every interface on the server by default. The "Broker Client URL" is the URL the Agents will connect to, whilst the "Broker URL" controls where Bamboo will listen to that service along with other options. It defaults to 0.0.0.0:54663, meaning every interface and every IP address.

Please validate that property as well.

Any changes to those properties require you to restart the complete application.

Regards,

Eduardo Alvarenga  
Atlassian Support APAC

**--please don't forget to Accept the answer if the reply is helpful--**


author: Sam
body: For the Broker URL: nio://0.0.0.0:54663?wireFormat.maxInactivityDuration=300000 and Broker client URL: failover:(tcp://192.168.5.8:54663?wireFormat.maxInactivityDuration=300000\&socket.verifyHostName=false)?initialReconnectDelay=15000\&maxReconnectAttempts=10

What I don't understand is why the agent is tying to connect to "tcp://localhost:4527"? Shouldn't it be connecting to port 54663?


author: Eduardo Alvarenga
body: Hi [@Sam](/t5/user/viewprofilepage/user-id/5124975)

Once the Elastic Agent is started by the Server, it starts its tunnel listener on port TCP/26224 and waits for a connection from the Server. Once the server is in, a tunnel is established and the Elastic Agent starts a TCP forward service on localhost:4527 sending data back to the broker running on the server.

Remote Agents will connect to the ActiveMQ JMS Broker directly on port TCP/54663 whilst Elastic Agents will use the Tunnel.

More information here:

* [Elastic Bamboo Security](https://confluence.atlassian.com/bamboo/elastic-bamboo-security-289277195.html)

Best regards,

Eduardo Alvarenga  
Atlassian Support APAC

**--please don't forget to Accept the answer if the reply is helpful--**


author: Sam
body: Ok, so if all my settings and security groups are all correct, what could be causing the agent from not connecting to the Bamboo Server?

In the logs I see the following error: Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean': Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue \[queue://com.atlassian.bamboo.registrationQueue\]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception


author: Eduardo Alvarenga
body: Hello [@Sam](/t5/user/viewprofilepage/user-id/5124975)

Can you please post the complete exception?


author: Sam
body: .


author: Sam
body: .


author: Sam
body: .


author: Sam
body: .


author: Sam
body: .


author: Sam
body: .


author: Sam
body: .


author: Sam
body: .


author: Sam
body: Complete exception:

Caused by: javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exceptionCaused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean': Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue \[queue://com.atlassian.bamboo.registrationQueue\]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception

Caused by: org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue \[queue://com.atlassian.bamboo.registrationQueue\]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception

at com.sun.proxy.$Proxy40.registerAgent(Unknown Source) \~\[?:?\]

at com.atlassian.bamboo.agent.elastic.client.ElasticAgentRegistrationBean.registerAgent(ElasticAgentRegistrationBean.java:25) \~\[atlassian-bamboo-agent-remote-9.0.1.jar:?\]

at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) \~\[?:?\]

at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) \~\[?:?\]

at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) \~\[?:?\]

at java.lang.reflect.Method.invoke(Method.java:566) \~\[?:?\]

Caused by: javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception


author: Sam
body: ```
java.io.IOException: Wire format negotiation timeout: peer did not send his wire format.
Caused by: javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception
INFO [ActiveMQ Connection Executor: unconnected] [PooledConnectionFactory$1$1] Expiring connection ActiveMQConnection {id=ID:ip-172-31-13-69-41155-1679501420985-1:1,clientId=ID:ip-172-31-13-69-41155-1679501420985-0:1,started=false} on IOException: Wire format negotiation timeout: peer did not send his wire format.
ERROR [AgentRunnerThread] [ElasticAgentRegistrationBean] org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue [queue://com.atlassian.bamboo.registrationQueue]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception
WARN [AgentRunnerThread] [AbstractApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean': Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue [queue://com.atlassian.bamboo.registrationQueue]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception
java.lang.reflect.InvocationTargetException
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean': Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue [queue://com.atlassian.bamboo.registrationQueue]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception
Caused by: org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue [queue://com.atlassian.bamboo.registrationQueue]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception
at org.springframework.jms.remoting.JmsInvokerClientInterceptor.convertJmsInvokerAccessException(JmsInvokerClientInterceptor.java:434)
Caused by: javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception
Caused by: javax.jms.JMSException: Disposed due to prior exception
Caused by: org.apache.activemq.transport.TransportDisposedIOException: Disposed due to prior exception
Caused by: java.io.IOException: Wire format negotiation timeout: peer did not send his wire format.
FATAL [AgentRunnerThread] [RemoteAgentBootstrap] Exiting due to fatal exception.
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:27) ~[atlassian-bamboo-agent-bootstrap-9.0.1.jar:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
```


author: Eduardo Alvarenga
body: Hey [@Sam](/t5/user/viewprofilepage/user-id/5124975)

Thank you for sharing the complete stack trace.

Bamboo should be listening on ***0.0.0.0:54663**. Can* you confirm if that port is open for any hosts? Perhaps a local firewall rule on the Bamboo server will not allow a connection?

The "***netstat -tl \| grep 54663***" should return something like the below on the Bamboo server:

```
tcp        0      0 localhost:54663         0.0.0.0:*               LISTEN     
```

If, even after checking that you still can't make the Elastic Agents work, I advise you to open a ticket with Atlassian at <https://support.atlassian.com/contact> so we can help you with an in-deep investigation.

Best regards,

Eduardo Alvarenga  
Atlassian Support APAC

**--please don't forget to Accept the answer if the reply is helpful--**


author: Sam
body: When I ran "**netstat -tl \| grep 54663**" on the Bamboo server it returned nothing. But when I ran "**netstat -tl \| grep 8085**" it returned the following:

```
tcp6        0      0 [::]:8085        [::]:*               LISTEN   
```

Do I need to restart the Bamboo server or a service for it to start listening on port "**54663"?**


author: Eduardo Alvarenga
body: Hello [@Sam](/t5/user/viewprofilepage/user-id/5124975)

The missing output for port 54663 is an indicator that ActiveMQ is not listening on your server.

Please restart Bamboo looking to make ActiveMQ available again.  
Eduardo Alvarenga  
Atlassian Support APAC

**--please don't forget to Accept the answer if the reply is helpful--**


author: Sam
body: So from what you're saying port 54663 needs to be listening for the agent to work? In the logs I see the following "tunnelserver:26224-1-thread-2\] \[LocalAcceptor\] Accepted a local connection on port 46593" When I ran "**netstat -tl \| grep 26224"** on the Bamboo server it shows

```
tcp6        0      0 [::]:26244        [::]:*               LISTEN   
tcp6        0      0 [::]:26244        [::]:*               ESTABLISHED
```


author: Eduardo Alvarenga
body: Hello [@Sam](/t5/user/viewprofilepage/user-id/5124975)

This is the expected LISTEN:

* **Server:** *0.0.0.0:54663* (when Bambo Server is up)
* **Elastic Agent:** *0.0.0.0:26224* (after initial bootstrap)

Eduardo Alvarenga  
Atlassian Support APAC