Is automatic rollback supported for Bamboo Agents as well as automatic upgrade?
Asked by JesseSilverman:

We seem to see automatic upgrade of remote agents working fine in test for our Bamboo upgrade from 5.10.3 to 6.4.0 -- however, simulating the "Oh no, we will need to rollback!" scenario by pointing already upgraded agents to our production server instance has been consistently failing. Maybe simulating the rollback by just pointing agents configured against an upgraded test server back to the production server isn't quite valid.

Is automatic rollback of agents something that is actually supported in general? We do have a lot of agents on a lot of machines, so if we missed any problems in testing and had to rollback all of the agent upgrades manually it could be a lot of pain and some downtime.

If this works in general we will open a support ticket for our specific problems, if not we will just invest especially heavy effort in testing before moving production.


Answers:
author: Przemek Bruski
body: That depends on the actual question:

Does it work? Yes, I am not aware of a single situation where rollback compatibility was broken (since the rollout of automated upgrades years ago).

Is it supported? - no.

comments: author: JesseSilverman
body: I appreciate that. We have enough agents that we want to see in advance that a rollback wouldn't mean full fresh re-installs on all machines. In this case we are trying to simulate the actual rollback by pointing the agent(s) back at the production instance after having been upgraded by and running with the upgraded test instance, which I mention in case that isn't identical to an actual in-place rollback of the server.

The errors we see are at the wrapper level, and look like this:

STATUS \| wrapper \| 2018/03/19 16:36:02 \| --\> Wrapper Started as Service  
STATUS \| wrapper \| 2018/03/19 16:36:02 \| Java Service Wrapper Community Edition 3.3.0  
STATUS \| wrapper \| 2018/03/19 16:36:02 \| Copyright (C) 1999-2008 Tanuki Software, Inc. All Rights Reserved.  
STATUS \| wrapper \| 2018/03/19 16:36:02 \| <http://wrapper.tanukisoftware.org>  
STATUS \| wrapper \| 2018/03/19 16:36:02 \|   
STATUS \| wrapper \| 2018/03/19 16:36:03 \| Launching a JVM...  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| WrapperManager: Initializing...  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,416 INFO \[WrapperSimpleAppMain\] \[AgentBootstrap\] Starting Agent Bootstrap using Java 1.8.0_144 from Oracle Corporation. Default charset: windows-1252, file name encoding: Cp1252  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,417 INFO \[WrapperSimpleAppMain\] \[AgentBootstrap\] Agent bootstrap using endpoint: <https://MyProdV5_10_3.mydomain.com:8443/agentServer/>  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,424 INFO \[WrapperSimpleAppMain\] \[RemoteAgentHomeLocatorForBootstrap\] Using agent home located at \[C:\\bamboo-agent-home-5.10.3\]  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,424 INFO \[WrapperSimpleAppMain\] \[RemoteAgentHomeLocatorForBootstrap\] Using agent home located at \[C:\\bamboo-agent-home-5.10.3\]  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,467 INFO \[WrapperSimpleAppMain\] \[AgentUuidInitializer\] Found agent UUID '31052ca0-86cd-4c3e-be6d-dbce966d594d' in agent config file   
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,474 INFO \[WrapperSimpleAppMain\] \[AgentContext\] Requesting fingerprint, url: https://MyProdV5_10_3.mydomain.com:8443/agentServer/GetFingerprint.action?hostName=VMDEV6154.olf.com\&version=3\&agentUuid=31052ca0-86cd-4c3e-be6d-dbce966d594d  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,478 INFO \[WrapperSimpleAppMain\] \[RemoteAgentHttpClientFactory\] Creating a new HTTP client...  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,766 INFO \[WrapperSimpleAppMain\] \[RemoteAgentHttpClientFactory\] Allowing non-matching certificates in HTTPClient  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,808 INFO \[WrapperSimpleAppMain\] \[RemoteAgentHttpClientFactory\] Allowing non-matching certificates in HTTPClient  
INFO \| jvm 1 \| 2018/03/19 16:36:03 \| 2018-03-19 16:36:03,860 INFO \[WrapperSimpleAppMain\] \[AgentContext\] Authenticating against Bamboo server at https://MyProdV5_10_3.mydomain.com:8443/agentServer/  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,045 INFO \[WrapperSimpleAppMain\] \[AgentContext\] Successfully authenticated with Bamboo server  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,054 INFO \[WrapperSimpleAppMain\] \[BootstrapUpdater\] Checking if bootstrap C:\\bamboo_agent\\bamboo-agent-5.10.3.jar is up to date, using url: https://MyProdV5_10_3.mydomain.com:8443/agentServer/GetJar?name=bootstrap-with-deps\&checksum=3304565565  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,057 WARN \[WrapperSimpleAppMain\] \[BootstrapUpdater\] Unable to update bootstrap: HTTP/1.1 410 Gone, continuing with the existing bootstrap.  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,057 INFO \[WrapperSimpleAppMain\] \[AgentContext\] Bootstrap Type: \[REMOTE\], agent class: com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,057 INFO \[WrapperSimpleAppMain\] \[AgentContext\] Preparing for a classloader switch  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,058 INFO \[WrapperSimpleAppMain\] \[RemoteAgentHomeLocatorForBootstrap\] Using agent home located at \[C:\\bamboo-agent-home-5.10.3\]  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,077 INFO \[WrapperSimpleAppMain\] \[ClasspathBuilder\] Content caching is enabled for high performance bootstrap.  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,077 INFO \[WrapperSimpleAppMain\] \[ClasspathBuilder\] Syncing C:\\bamboo-agent-home-5.10.3 with https://MyProdV5_10_3.mydomain.com:8443/agentServer/bootstrap/classpath/manifest  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,082 ERROR \[WrapperSimpleAppMain\] \[ClasspathBuilder\] An error has occurred when retrieving classpath data: HTTP/1.1 404 Not Found  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| Exiting due to fatal exception.  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| com.atlassian.bamboo.agent.bootstrap.RemoteAgentHttpException: Response: \[HTTP/1.1 404 Not Found\] received after manifest retrieval request.  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.terminateWithError(ClasspathBuilder.java:213)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.getJarList(ClasspathBuilder.java:202)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.sync(ClasspathBuilder.java:85)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.getClassLoader(ClasspathBuilder.java:58)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.AgentContext.newClassLoader(AgentContext.java:329)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.AgentContext.run(AgentContext.java:93)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.AgentBootstrap.run(AgentBootstrap.java:111)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.AgentBootstrap.main(AgentBootstrap.java:50)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at java.lang.reflect.Method.invoke(Method.java:498)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:237)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at java.lang.Thread.run(Thread.java:748)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| 2018-03-19 16:36:04,084 FATAL \[WrapperSimpleAppMain\] \[AgentBootstrap\] Exiting due to fatal exception.  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| com.atlassian.bamboo.agent.bootstrap.RemoteAgentHttpException: Response: \[HTTP/1.1 404 Not Found\] received after manifest retrieval request.  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.terminateWithError(ClasspathBuilder.java:213)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.getJarList(ClasspathBuilder.java:202)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.sync(ClasspathBuilder.java:85)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.getClassLoader(ClasspathBuilder.java:58)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.AgentContext.newClassLoader(AgentContext.java:329)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.AgentContext.run(AgentContext.java:93)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.AgentBootstrap.run(AgentBootstrap.java:111)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at com.atlassian.bamboo.agent.bootstrap.AgentBootstrap.main(AgentBootstrap.java:50)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at java.lang.reflect.Method.invoke(Method.java:498)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:237)  
INFO \| jvm 1 \| 2018/03/19 16:36:04 \| at java.lang.Thread.run(Thread.java:748)  
STATUS \| wrapper \| 2018/03/19 16:36:06 \| \<-- Wrapper Stopped