Can we have multiple Bamboo YAML specs and services in one repository?
Asked by Lars Kristian:

Currently we are using the [monorepo strategy](https://en.wikipedia.org/wiki/Monorepo) where we have the source code for multiple services in one repository.

When looking into YAML build plans, it seems like it's limited to one spec pr repository, ref. <https://confluence.atlassian.com/bamboo/tutorial-bamboo-specs-yaml-stored-in-bitbucket-server-941616819.html>
> It's important to save your Bamboo Specs YAML definition in the ${repo-home}/bamboo-specs/bamboo.yml or bamboo.yaml file under the repository root.

Ideally we would want to gather the bamboo.yaml and source code for the different services into different folders and then trigger builds only if there are any changes in that specific service's folder.

Is this doable? I couldn't seem to find any documentation/questions related to monorepo and YAML builds.


Answers:
author: Victor Debone
body: Hello [@Lars Kristian](/t5/user/viewprofilepage/user-id/3103285), welcome to the community :)

The feature you are looking for will be released in the next version of Bamboo.

On Bamboo 6.9, you will be able to declare multiple plans and deployments projects, using one or more files. You can try and experiment with it now, with the EAP version of 6.9 (link).

If you want to share more details, give some feedback on your current usage (or future) of Bamboo Specs with YAML, feel free to send a message at vdebone@atlassian.com :)

Victor

comments: author: Lars Kristian
body: Thanks for the quick reply [@Victor Debone](/t5/user/viewprofilepage/user-id/1892075)!

What I imagine is something like this

* Jobs
  * JobA
    * docs/
    * src/
    * scripts/
    * readme.md
    * bamboo.yaml
  * ...
* Services
  * ServiceA
    * docs/
    * src/
    * scripts/
    * readme.md
    * bamboo.yaml
  * ServiceB
    * docs/
    * src/
    * scripts/
    * readme.md
    * bamboo.yaml
  * ...
* ...
* readme.md

How would the build plans be visualized/organized in Bamboo 6.9? Given that the monorepo above is branched, would you still have one build plan based on the YAML project.plan.key, where the steps then might differ from branch to branch if changes are made to bamboo.yaml? Or would you have copies of the build plan per branch which might lead to overloading the build dashboard? (I'm hoping the former since one usually creates a branch to create or make changes on one service, and there would be no point in creating duplicates of all the build plans. The build dashboard might not even be navigatable.)

Any information on when the 6.9 version is scheduled for GA? I don't think there's a possibility of getting the EAP installed for now, unfortunately. (The "link" above is not a link at the moment btw.)


author: Victor Debone
body: Great!

I will answer them in reversed order.

To make it clear, the EAP version **is not for production** :) My suggestion of EAP was only if you wanted to check the new YAML first-hand! And for the release date, we don't keep public facing dates for releases.

On the YAML part, plan branches differing will not be part of this release. Bamboo Specs on YAML (and will for 6.9) only accepts the master branch \`bamboo.yml\`. Definitely this is under our radar, it is high priority for us, and you can follow its progress on [this suggestion.](https://jira.atlassian.com/browse/BAM-19620)

For the organization of the repo, this is what will be possible:

```
Repository
??? your-service-a
?   ??? files
?   ??? ...
??? your-service-b
?   ??? files
?   ??? ...
??? bamboo-specs
    ??? bamboo.yml
    ??? your-service-a.yml
    ??? your-service-b.yml
```

Would that work for you? (Apart from the lack of differing branches configurations)

Victor


author: Lars Kristian
body: Thanks,

For the repository structure, would that mean the high-level bamboo.yml have some kind of mapping saying which path belongs to the different Bamboo yml-files? So if there are changes in the path, trigger pipeline X. (For instance there is no point in running builds for documentation changes, only application code.)

```
your-service-a/service-a-sourcecode => your-service-a.yml

your-service-b/sourceCode => your-service-b.yml
```

You are saying that the YAML build plan will only accept the master branch's bamboo.yml. Does that mean we can't even duplicate the plan configuration to a new one where the branch is explicitly set to the feature branch if we need to work on the YAML build plan(s)? We are only allowed to do that directly in the master branch?


author: Victor Debone
body: If I understood your first question correctly, yes. You will have a single bamboo.yml file that will include all the others files, for organisation purposes, and they are required to be inside the \`bamboo-specs\` folder. This is how it might look:

```
---
!include your-service-a.yml

---
!include your-service-b.yml

...
```

And from each of these files you will be defining either a plan or a deployment.

On the second question, yes. Bamboo 6.9 will still only support the master's branch \`bamboo.yml\`, and again, this is a high priority suggestion for us and you can follow it's progress under [BAM-19620](https://jira.atlassian.com/browse/BAM-19620)

Did that help? :)

Victor


author: Lars Kristian
body: Thanks, that clear things up.

The only thing I'm not seeing is how the relationship is set up between the yaml file and the source code location in terms of triggering a build.

From [spec-yaml](https://docs.atlassian.com/bamboo-specs-docs/6.8.1/specs-yaml.html) it should be simple to have relative paths pointing from "your-service-a.yml" to for instance the directory where "your-service-a" resides for the scripts, test results etc.

But where do you define the trigger path? In other CI tools they call this "path filter" or similar. It usually means: "ok, if something is changed/added/modified in path x/y/z then the build definition that should be run is z.yml". I'm assuming that all the plans aren't triggered if just one file in a certain service directory is changed?


author: Victor Debone
body: Glad that I could help!

For the "Path filter" feature, I have just created a feature request [BAM-20422](https://jira.atlassian.com/browse/BAM-20422). We currently do not support this granularity of change detection natively.

It's not 100% clear what is your urgency on that, but I also want to leave it stated that we have plugin points at change detection and writing a plugin to cover this gap would be an alternative.

Did that help?


author: Lars Kristian
body: Thanks for all your help,

To summarize, our use case isn't really supported at this time, but will likely be with 6.9 + BAM-20422. So for this question itself, I'll close it as answered.

As to the urgency, we'll look into workarounds and/or other tooling. If you can link to documentation regarding the plugin point you mentioned, that might help as well, but I'm not sure we have the time to go that route.


author: Matthew Lieder
body: [@Victor Debone](/t5/user/viewprofilepage/user-id/1892075) did this get dropped from 6.9? We're on 6.9 now and I'm not seeing it mentioned in documentation or working when I try.


author: Victor Debone
body: [@Matthew Lieder](/t5/user/viewprofilepage/user-id/702751) which feature are you specifically trying out that is not working?


author: Matthew Lieder
body: [@Victor Debone](/t5/user/viewprofilepage/user-id/1892075) my initial reading of this thread seemed to indicate that putting "plan-1.yml", "plan-2.yml", etc. inside a bamboo-specs folder would be all that's needed in v6.9 to having multiple plans. After further digging I found hidden inside the "Understanding YAML in Specs" section at the bottom of the docs (<https://docs.atlassian.com/bamboo-specs-docs/6.9.0/specs.html?yaml#understanding-yaml-in-specs>) details on how to import specs from the root spec (bamboo.yml) and also, more importantly to me, how to combine multiple specs into one file (separating them by ---). So, while the most straightforward scenario (multiple simple spec files) doesn't work, there are workable alternatives. After re-reading this thread I realized you were in fact referring to the !import method yourself too. Sorry about misreading that!


author: Victor Debone
body: Glad that you found the solution!

Indeed the documentation is misleading, specially in this regard, and feedback was already taken into consideration for updating it.


Tags: spec

yaml