Can you skip a pipeline step if a cache is found (do the step if a cache is not found)
Asked by ian.weir:

Basically was wondering if there is a way to determine if you can skip a step based on a cache before it's present? The build setup is the slowest part of the stage so my current solution doesn't save as much time as it could.  

Current solution is to to use an if/then block where it checks if the cached folder has any contents. If has contents it skips the rest of the script otherwise it continues on. It still saves a bit of time but still wastes time as well.

Ending the script early saves about 1 minutes but skipping the step altogether could save an additional two minutes.


Answers:
author: Theodora Boudale
body: Hi Ian and welcome to the community!

It is not possible to skip a step based on whether a cache is found.

Would you mind sharing why you are looking to do so? The purpose of using a cache is that subsequent builds will use it instead of downloading dependencies from the internet, so I'm wondering why you are defining a cache if you don't want to use it and why is the presence of a cache relevant to whether your build starts or not.

At the moment it is possible to use a condition in a step based on the files changed, specifically, start a step only if certain files of the repo are changed:

* <https://support.atlassian.com/bitbucket-cloud/docs/step-options/#Condition>

Kind regards,  
Theodora

comments: author: Marcel Daneck
body: In our case we have an Angular Project.

For the pipeline to run, we need to execute yarn install.

After yarn install, we can execute Lint, Test and Build in parallel, but all need the node_modules folder.

We could solve it with an artifact, but there are steps for deployment, which do need the node_modules.

So we use one single step for yarn install to create the cache, before all other steps start.

We use the cache-key feature to create a node_modules-cache for each change of the yarn-lock file. After the cache is created, yarn install does not need to be executed, if the cache for this specific yarn-lock file exists.

But currently we still need to execute it, download the node_modules cache and the yarn cache, which takes \~1 minute. Just for yarn install finish in 8 seconds and no upload of the cache.

If we could skip the step if the cache with the correct hash does exist, we would save this time.

We could use the condition to only create the cache if there are changes in the yarn.lock, but the caches get deleted after one week. So if we dont change the yarn-lock every week, the build will crash.

The only solution I could think of, is a scheduled pipeline which deletes the cache and builds it again, which still could fail and break all subsequent builds.

Skipping if the cache is present would be much more straight forward.


author: Theodora Boudale
body: Hi Marcel,

Thank you for the detailed use case. I went ahead and created a feature request in our issue tracker:

* <https://jira.atlassian.com/browse/BCLOUD-22954>

You can add your vote to it (by selecting the **Vote for this issue** link) to express your interest, as the number of votes helps the development team and product managers better understand the demand for new features. You are more than welcome to leave any feedback, and you can also add yourself as a watcher (by selecting the **Start watching this issue** link) if you'd like to get notified via email on updates.

In case I have misunderstood something about your use case, please feel free to let me know and I will adjust the description in the feature request.

> After yarn install, we can execute Lint, Test and Build in parallel, but all need the node_modules folder.
>
> We could solve it with an artifact, but there are steps for deployment, which do not need the node_modules.

Artifact downloads can be disabled for a certain step if that is an issue:

* <https://support.atlassian.com/bitbucket-cloud/docs/step-options/#Artifact-downloads>

However, if the steps after the first one are parallel, you would still need the first step to generate the artifacts, as parallel steps cannot consume artifacts generated by a step in the same parallel set. The solution with artifacts could work if you don't use parallel steps.

Kind regards,  
Theodora


Tags: bitbucket-cloud

cloud