Git with ssh key fails on one agent
Asked by Timothy Meunier:

Using git with an SSH private key for authentication fails on one specific agent. This seems to coinside with an upgrade of the Bamboo server from 4.2 to 5.1.1 It works on all other agents. I can checkout on the offending agent with the key from the console. It seems to have something to do with the SSH proxy that is used by Bamboo to protect the repository key. The agent is running Ubuntu 12.

The error "Private key cannot be read" is confusing because of the proxy. What key? Where is it looking for it? Is the error on the server or on the agent? And since this works for all other agents, It would seem to rule out the repository key, and the Bamboo server.

```
imple	11-Oct-2013 09:49:57	Cleaned source directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW'...simple	11-Oct-2013 09:49:57	Creating local git repository in '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW/.git'.simple	11-Oct-2013 09:49:57	/usr/bin/git initsimple	11-Oct-2013 09:49:57	Initialized empty Git repository in /home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW/.git/error	11-Oct-2013 09:49:57	Cannot fetch branch '(unresolved) master' from 'ssh://user@server/project/repo.git' to source directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW'. Cannot decode connection paramssimple	11-Oct-2013 09:49:57	Updating source code to revision: 2dddbe824e96e5a5cbe9149472b1d1f91e5cc2dcsimple	11-Oct-2013 09:49:57	/usr/bin/git log -1 --encoding=UTF-8 --format=%H HEADerror	11-Oct-2013 09:49:57	Cannot fetch branch '(unresolved) master' from 'ssh://user@server/project/repo.git' to source directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW'. Cannot decode connection paramssimple	11-Oct-2013 09:49:57	Warning: failed to checkout source code to directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW', trying to recover...simple	11-Oct-2013 09:49:57	Cleaned source directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW'...simple	11-Oct-2013 09:49:57	Creating local git repository in '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW/.git'.simple	11-Oct-2013 09:49:57	/usr/bin/git initsimple	11-Oct-2013 09:49:57	Initialized empty Git repository in /home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW/.git/error	11-Oct-2013 09:49:57	Cannot fetch branch '(unresolved) master' from 'ssh://user@server/project/repo.git' to source directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW'. Cannot decode connection paramserror	11-Oct-2013 09:49:57	Error occurred while running Task 'Checkout Default Repository(1)' of type com.atlassian.bamboo.plugins.vcs:task.vcs.checkout.error	11-Oct-2013 09:49:57	java.lang.RuntimeException: com.atlassian.bamboo.repository.RepositoryException: Cannot fetch branch '(unresolved) master' from 'ssh://user@server/project/repo.git' to source directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW'. Cannot decode connection paramserror	11-Oct-2013 09:49:57		at com.atlassian.bamboo.executor.RetryingTaskExecutor.rerun(RetryingTaskExecutor.java:132)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.executor.RetryingTaskExecutor.runTask(RetryingTaskExecutor.java:88)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:191)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:176)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.vcs.task.VcsCheckoutTask.execute(VcsCheckoutTask.java:170)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:220)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.task.TaskExecutorImpl.executePreparationTasks(TaskExecutorImpl.java:73)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:74)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:189)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:102)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:108)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.build.pipeline.concurrent.NamedThreadFactory$2.run(NamedThreadFactory.java:55)error	11-Oct-2013 09:49:57		at java.lang.Thread.run(Thread.java:662)error	11-Oct-2013 09:49:57	Caused by: com.atlassian.bamboo.repository.RepositoryException: Cannot fetch branch '(unresolved) master' from 'ssh://user@server/project/repo.git' to source directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW'. Cannot decode connection paramserror	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.git.NativeGitOperationHelper.fetch(NativeGitOperationHelper.java:389)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.git.GitRepository.retrieveSourceCode(GitRepository.java:361)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.git.GitRepository.retrieveSourceCode(GitRepository.java:293)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.vcs.task.VcsCheckoutTask.fillWorkingDirFromVcs(VcsCheckoutTask.java:248)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.vcs.task.VcsCheckoutTask.access$100(VcsCheckoutTask.java:54)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.vcs.task.VcsCheckoutTask$2.call(VcsCheckoutTask.java:175)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.vcs.task.VcsCheckoutTask$2.call(VcsCheckoutTask.java:171)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.executor.RetryingTaskExecutor.rerun(RetryingTaskExecutor.java:108)error	11-Oct-2013 09:49:57		... 12 moreerror	11-Oct-2013 09:49:57	Caused by: com.atlassian.bamboo.repository.RepositoryException: Cannot decode connection paramserror	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.git.NativeGitOperationHelper.adjustRepositoryAccess(NativeGitOperationHelper.java:201)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.git.NativeGitOperationHelper.fetch(NativeGitOperationHelper.java:353)error	11-Oct-2013 09:49:57		... 19 moreerror	11-Oct-2013 09:49:57	Caused by: java.io.IOException: Couldn't register Bouncy Castle as a JCE provider. Private key cannot be readerror	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.ssh.ProxyConnectionDataBuilderImpl.withKeyFromReader(ProxyConnectionDataBuilderImpl.java:90)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.ssh.ProxyConnectionDataBuilderImpl.withKeyFromString(ProxyConnectionDataBuilderImpl.java:67)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.ssh.ProxyConnectionDataBuilderImpl.withKeyFromString(ProxyConnectionDataBuilderImpl.java:23)error	11-Oct-2013 09:49:57		at com.atlassian.bamboo.plugins.git.NativeGitOperationHelper.adjustRepositoryAccess(NativeGitOperationHelper.java:178)error	11-Oct-2013 09:49:57		... 20 moreerror	11-Oct-2013 09:49:57	Failed to prepare the build 'TJM-test - LEADTEST - Framework #33 (TJM-LTST-FW-33)'
```


Answers:
author: Timothy Meunier
body: Good call on it being a problem loading Bouncy Castle. Turned out to be a problem with an NCSS/PKCS11 configuration with java. I didn't realize some java files are linked into /etc so effect all java installs (including the java running the Bamboo agent). The build hadn't run on the agent for a few days after the changes so I didn't make the connection.

Thanks!


author: Przemek Bruski
body: That error message is misleading. It actually means "Couldn't register Bouncy Castle as a JCE provider. It won't be possible to read your private key."

We have to find out why BC can't be instantiated. Could you grep your agent log for occurrences of the word SecurityUtils ?

comments: author: Timothy Meunier
body: Kelly:  
There is a built in git implementation in Bamboo. It has some limitations, so I'd like to use a native git installation.  

"Note that Bamboo comes with its own built-in Git implementation. However, you need to use native Git to be able to use symbolic links, submodules, automatic branch detection and automatic merging - these are not supported by the built-in Git."  

https://confluence.atlassian.com/display/BAMBOO/Git  

Przemek:  
I find no mention of 'SecurityUtils' in the agent log. Do I have to turn up the log level?


author: Kelly Schoenhofen
body: I think your error is starting much earlier, when it's complaining after a git init

"Cannot fetch branch '(unresolved) master' from 'ssh://user@server/project/repo.git' to source directory '/home/build/bamboo-agent-home/xml-data/build-dir/TJM-LTST-FW'. Cannot decode connection params"

That error - cannot decode connection params - that's an Atlassian error from their source code, not git, I believe. They throw it when an ssh connection fails and it's not a cipher failure.

I don't know enough about how this is setup, but your build/build folder seems very confused about its connection. Since it looks like a straightforward linux system this is running on, it will fall back on the id_rsa key in the .ssh folder in the home directory of the user executing this, unless you've overridden the key location (and it doesn't appear that you have). So that might be the source of the stack trace at the bottom.

comments: author: Timothy Meunier
body: Yes, '(unresolved)' should be something like 'refs/heads/master'. So maybe it's the local 'git init' operation? But the init should have no need to know about the remote repo. When does the remote part get filled in?

The key to access the repo does exist on the agent and is configured properly to be used from the command line. So if it falls back to that, it should work. Maybe the fallback isn't given full access to te environment to work the same way as when at the console...

I'll look closer at the local git funtionality.


author: Timothy Meunier
body: If I remove git as a capability of the agent, it uses the internal git and that works. Not sure why it can't use the system git on this one machine.


author: Kelly Schoenhofen
body: When you say you remove git as a capability of the agent, then how are you doing your git commands? As a script?


Tags: bamboo-administration