Caching poetry on Bitbucket Pipelines
Asked by Matteo Verit?:

I'm trying to setup my bitbucket_pipelines.yml file to make it possible for all the steps to have Poetry already installed, so that running \`poetry run \[command\]\` should never return \`bash: poetry: command not found\`. So far I made it work with the following configuration:  

```
# This is an example Starter pipeline configuration
# Use a skeleton to build, test and deploy using manual and parallel steps
# -----
# You can specify a custom docker image from Docker Hub as your build environment.

image: python:3.12

definitions:
  caches:
    poetry-cache:
      key:
        files:
          - poetry.lock
      path: .venv

pipelines:
  default:
    - parallel:
      - step:
          name: 'Build and Test'
          caches:
            - poetry-cache
          script:
            - pipe: atlassian/poetry-cli-setup:0.2.0
            - ./setup-poetry.sh
            - poetry config virtualenvs.in-project true
            - poetry install
            - poetry run pytest
      - step:
          name: 'Lint'
          caches:
            - poetry-cache
          script:
            - pipe: atlassian/poetry-cli-setup:0.2.0
            - ./setup-poetry.sh
            - poetry install
            - poetry run ruff check
```

As you can see, the only way I found was to reinstall Poetry in each step. It's already good enough that \`poetry install\` is fast because it uses the cache and in fact ends up installing nothing, however I wonder if there's a better way for poetry itself to make it available to all steps without having to install it in each.


Answers:
author: Joshua Tang
body: Hi Matteo,

We have just released a new version of `poetry-cli-setup:0.2.2` with some changes to allow caching of the Poetry installation. Similar to how you're caching the virtual environment, you can define an extra cache for the Poetry installation:

```
definitions:
  caches:
    poetry-cli-cache:
      key:
        files:
          - bitbucket-pipelines.yml
      path: /opt/poetry
```

Then you can include this cache in your individual steps, where the Poetry setup step would reuse the cached installation:

```
- step:
    name: 'Lint'
    caches:
      - poetry-cache
      - poetry-cli-cache
    script:
      - pipe: atlassian/poetry-cli-setup:0.2.2
      - ./setup-poetry.sh
      - poetry install
      - poetry run ruff check
```

This example is also documented on the README of the pipe: <https://bitbucket.org/atlassian/poetry-cli-setup/src/master/>

Hope this helps!

Kind regards,  
Joshua

comments: author: Matteo Verit?
body: Thank you [@Joshua Tang](/t5/user/viewprofilepage/user-id/4551530)! It worked splendidly.

And thanks [@Ben](/t5/user/viewprofilepage/user-id/4537226) for the additional help!


author: jbsilva
body: Could you provide a full example?  
When I removed \`./setup-poetry.sh\` from my second step I got \`bash: poetry: command not found\`.


author: Ben
body: Hi [@jbsilva](/t5/user/viewprofilepage/user-id/5268080)

Please refer to the pipe documentation for a full example, the error message you've received generally means that the command you are executing can't be executed as a setup step is missing (ie the bash script you removed):  
<https://bitbucket.org/atlassian/poetry-cli-setup/src/master/>

Cheers!

- Ben (Bitbucket Cloud Support)


author: jbsilva
body: By full example I mean one with at least two steps.


author: Ben
body: Hi Matteo,

You can potentially use artifacts to achieve this - so basically you could figure out where poetry is installed in the context of your build, then compress it as a zip - pass it to subsequent steps as an artifact, unzip it and execute poetry commands:

* <https://community.atlassian.com/t5/Bitbucket-questions/Is-there-a-way-of-sharing-data-between-steps-in-bitbucket/qaq-p/777977>

Other than that, I have reached out to our pipes development team to figure out the best place to raise suggestions.

Cheers!

- Ben (Bitbucket Cloud Support)


Tags: bitbucket-cloud

bitbucket-pipelines

cache

cloud