PR and tag handling in shared pipelines
Asked by Fran?ois Collins:

Did anyone get pull request or tag pipelines to work with imported pipelines?

I'm trying to consolidate duplicated logic from many repos into a common pipeline. I have copied the pipeline into a new repository, flattened the hierarchy (each branch in branches: now is a top-level pipeline, and so on) and referenced those pipelines in a test repo.

The specific branches I set up (develop, release, master) are triggering the imported pipeline correctly, but using the same pattern with pull-request or tags does not trigger. Is this an expected limitation?

So the referencing bitbucket-pipelines.yml looks like this:

```
pipelines:

  #Upon commits

  default:

    import: microservice-pipeline-templates:master:default

  branches:

    develop:

      import: microservice-pipeline-templates:master:develop-branch

    release/*:

      import: microservice-pipeline-templates:master:release-branch

    hotfix/*:

      import: microservice-pipeline-templates:master:hotfix-branch

    master:

      import: microservice-pipeline-templates:master:master-branch

  

  #Upon pull requests

  pull-requests:

    default:

      import: microservice-pipeline-templates:master:default-pr

    release/*:

      import: microservice-pipeline-templates:master:release-pr

    hotfix/*:

      import: microservice-pipeline-templates:master:hotfix-pr

    master:

      import: microservice-pipeline-templates:master:master-pr

  tags:

    'QA-*':

      import: microservice-pipeline-templates:master:qa-tag

    'UAT-*':

      import: microservice-pipeline-templates:master:uat-tag

    'PROD-*':

      import: microservice-pipeline-templates:master:prod-tag
```

And the imported pipeline looks something like this:  

```
export: true

image: atlassian/default-image:4

clone:

? depth: full

definitions:

? caches:

? ? sonar: ~/.sonar/cache??

? ? maven: ~/.m2/repository??

??

? services:

? ? docker:

? ? ? memory: 512

? ? docker-with-more-memory:

? ? ? memory: 2048

? ? ? type: docker

? ? docker-with-large-memory:

? ? ? memory: 5120

? ? ? type: docker

??

? steps:

? ? - step: &step1

     {...}

    - step: &step2

     {...}

    - step: &step3

     {...}

    - step: &step4

     {...}

    - step: &step5

     {...}

? ??

? pipelines:

? ? default:

? ? ? - step: *step1

? ? develop-branch:

? ? ? - step: *step1

? ? ? - step: *step2

      - step: *step3

? ? release-branch:

? ? ? - step: *step1

? ? ? - step: *step2

      - step: *step4

? ??

? ? default-pr:

? ? ? - step: *step1

? ? ? - step: *step2

? ? release-pr:

? ? ? - step: *step1

? ? ? - step: *step4

? ? qa-tag:

? ? ? - step: *step5

? ? uat-tag:

? ? ? - step: *step5

? ? prod-tag:

? ? ? - step: *step5
```

What am I doing wrong? The documentation only mentions limitations about environment variables, images and individual steps... This is a deal breaker for me, if tags don't work it's unusable.


Answers:
author: Theodora Boudale
body: Hi Fran?ois,

For pull-requests pipelines to be triggered, the referencing bitbucket-pipelines.yml needs to exist on the source branch of the PR. If you are creating a pull request with source branch develop, and the yml file in develop doesn't have these definitions, then the pipeline won't get triggered. If you edit the yml file on develop branch and copy-paste the referencing bitbucket-pipelines.yml, the build should get triggered.

For pipelines running on tags, the referencing bitbucket-pipelines.yml needs to exist on the commit where you are adding a tag. If the referencing bitbucket-pipelines.yml exists on master branch, and you add a tag to an older commit of master where this version of yml does not exist, the build will not get triggered. Similarly, if you add a tag to a commit of another branch where the referencing bitbucket-pipelines.yml doesn't exist, the pipeline won't get triggered.

You can add the referencing bitbucket-pipelines.yml to all of the referencing repo's branches to ensure that the builds will get triggered for tags on newer commits and for existing and new pull requests.

Please feel free to reach out if you have any questions!

Kind regards,  
Theodora

comments: author: Fran?ois Collins
body: Hi Theodora,

The bitbucket-pipelines.yml already existed on all branches, I replaced the existing local step definitions with the import references and that's when it stopped working. As I mentioned earlier, the branch triggers are working correctly, it's only the PR and tags that don't work. And if I modify the bitbucket-pipelines.yml for PR on develop branch to put back the local step definition, it triggers the local step.


author: Theodora Boudale
body: Hi Fran?ois,

Thank you for your reply.

I see in your referencing bitbucket-pipelines.yml **pull-requests** definitions for **default** , **release/\*** , **hotfix/\*** , and **master** . Is there an additional definition in **pull-requests** for **develop** ? Or for **'\*\*'**?

The **default** definition you have in **pull-requests** is not going to match the rest of the branches (other than release/\*, hotfix/\*, and master), it is going to look for a PR with a source branch named **default** . If you intended to match the rest of the branches (other than release/\*, hotfix/\*, and master) and if there is no pull-requests definition for develop, you will need to replace **default** with **'\*\*'** in the pull-requests definition.

With regards to tags, are you adding the tags to commits where this new referencing bitbucket-pipelines.yml exists, or to older commits that did not have these definitions?

Kind regards,  
Theodora


author: Fran?ois Collins
body: Thanks.

For tags, they are added to new commits. I had refactored the pipeline in a feature branch, then merged to develop, and they were not working. Your emphasis on imports existing in the referencing pipeline had me wondering, so I redid an experiment. I created another branch from develop (rather than committing changes directly to develop), added the \*\* PR trigger, and merged it back to develop.

The PR trigger worked, and interestingly, without further changes, the tag trigger started working as well. I'm not sure what happened, it looks like if someone is trying to uptake pipeline templates on an existing repository, tags won't work until you merge a branch where the origin of the branch already contained the imported references... not very intuitive.


author: Theodora Boudale
body: Hi Fran?ois,

It's good to hear that the imported pipelines are working now. It's difficult to say what exactly was happening before, the most likely reason is that the necessary definitions were not there, but we would need to check the content of the repo to investigate and verify.

If you would like our support team to look into this you can create a ticket and provide more details (for example, specific tags that did not trigger builds). If we have a support ticket open, the engineer working on it will be able to access the repo and investigate further.

You can create a ticket via <https://support.atlassian.com/contact/#/>, in "What can we help you with?" select "Technical issues and bugs" and then Bitbucket Cloud as product. When you are asked to provide the workspace URL, please make sure you enter the URL of the workspace that is on a paid billing plan to proceed with ticket creation.

Please feel free to reach out if you have any questions.

Kind regards,  
Theodora


Tags: bitbucket-cloud

cloud