Problem with git submodules in Bamboo coupled with local cache
Asked by yogesh_devi:

Dear Bamboo experts,

I have a Bamboo build plan that uses git submodules . The build is failing with a message as follows . The problem seems to be that bamboo agent is trying to clone submodules from local cache and clone is failing because that operation needs a password . See log below

```
Already on 'dev/mainrepo'simple	27-May-2019 12:32:10	/usr/bin/git remote set-url origin ssh://d5fcbb24-a632-4e2e-948f-489ca955c411@127.0.0.2:41106/project/mainrepo.gitsimple	27-May-2019 12:32:10	/usr/bin/git submodule update --init --recursivesimple	27-May-2019 12:32:11	Cloning into 'submodule1'...simple	27-May-2019 12:32:11	Warning: Permanently added '[127.0.0.2]:41106' (RSA) to the list of known hosts.simple	27-May-2019 12:32:11	Permission denied (password).simple	27-May-2019 12:32:11	fatal: Could not read from remote repository.
```

If I try to do the same operation from agent command line I see following message which tells me that the passwordless auth is NOT set-up for my local cache.

> ```
> git submodule update --init --recursive
> Cloning into '/home/bamboo/bamboo-agent-home/xml-data/build-dir/BUILDNAME/submodule1'...
> 12ff7ca1-452e-4836-920e-8d44bc@127.0.0.1's password:?
>
>
>
> ```
>
> Please note that my bamboo agent account ssh key is already uploaded to bitbucket and if I try to checkout and do submodule update from the git repository on bitbucket everything works fine ( without prompting me for password). It asks me for password though when I run the build from bamboo because then it tries to clone from local cache on agent
>
> Does someone know the way I can set-up passwordless auth for bamboo local cache.
>
> Alternatively I am okay to turn off the caching.
>
> I tried to turn off caching from Bamboo gui by un-checking the checkbox for caching repositories. However Bamboo seems to cache regardless of the checkbox if submodules are involved
>
> I read <https://jira.atlassian.com/browse/BAM-18265> -- still trying to figure what I could do though


Answers:
author: yogesh_devi
body: This problem remains unresolved till date - but i devised following work around

# I created a task that does this sequence to explicitly update submodules

git remote remove origin  
git remote add origin ${bamboo.repository.git.repositoryUrl}  
git remote set-url origin ${bamboo.repository.git.repositoryUrl}  
git remote -v  
/usr/bin/git submodule deinit --force .  
/usr/bin/git submodule update --init --recursive


author: Kaspars Laizans
body: Had same issue.  
1. Generate ssh keys on bamboo server under bamboo user (not atlassian software), add them to your ssh-keytool  
$ ssh-keygen -m PEM -t rsa -C bamboo@myserver

$ eval $(ssh-agent)

$ ssh-add \~/.ssh/id_rsa

2. copy private key to (GUI) Bamboo/Config/Shared Credentials

3. Add pubkey to Bitbucket GUI to all the submodule projects that host submodule repositories

4. Create GIT NOT BITBUCKET linked repo, as mentioned by Daniel above. This forces Bamboo to use system git which supports submodules, in stead of bundled whatever gutted git version they have. Select shared SSH credentials you entered above


author: Daniel Santos
body: Hi [@yogesh_devi](/t5/user/viewprofilepage/user-id/737128)

Thank you for sharing a detailed problem description. That is very helpful.  

Can you confirm you are using the Git repository type and not Bitbucket one to configure your source repository in Bamboo?  

The Git option should be able to use the submodules with your local SSH keys if the option **\[x\]Use submodules** is checked in the repository **advanced options** configuration.  

<br />

comments: author: yogesh_devi
body: hi [@Daniel Santos](/t5/user/viewprofilepage/user-id/502125) Thank for responding. I am using "Bitbucket" repository . How to use "Git repository" as you mention ?


author: Daniel Santos
body: You will keep using the Bitbucket source repository, but in Bamboo side, you will use a different method for linking the repository. Instead of using the Bitbucket linking you will use the Git one as shown below:

![Linked_repositories_-_Atlassian_Bamboo_-_Vivaldi.png](https://community.atlassian.com/t5/image/serverpage/image-id/57372i3595DCC4C1F63963/image-dimensions/370x230?v=v2 "Linked_repositories_-_Atlassian_Bamboo_-_Vivaldi.png"):  
More details on how to configure can be found here: [Git - Atlassian Documentation](https://confluence.atlassian.com/bamboo/git-289277007.html).  
Let me know if you have other questions in the meantime.


author: ianburrowssa
body: Hi,

Did this end up working? I have created a git repo as described above. Used an SSH URl and used my private key, tested the connection which was successful but submodule cloning still does not work.


author: Anil Unnikrishnan
body: I am facing the same issue as well.


Tags: git

submodule