How to update the cache automatically
Asked by Mike Prince:

I'm fairly new to CI and even newer to bitbucket CI. What I want to do is cache the conda virtual environment that I create, update it incrementally as we add new packages to the environment.yml file, but also be able to newly create since it apparently gets dropped each week.

My file looks like as follows, and it works to a degree, but with two main issues

```
image: continuumio/anaconda3

pipelines:
  default:
    - step:
        name: Build
        caches:
          - condacache
        script:
          # - conda env create -f environment.yml 
          - conda activate vlm_venv
          - conda env update -f environment.yml
          - python tests/tests.py

definitions:
  caches:
    condacache: /opt/conda/envs/vlm_venv?
```

<br />

**++Issue #1++**   
Note that the first bullet in the "script" section is commented out. This is due to the fact that the env has already been created and cached, so there's no need to recreate it. It actually results in this error if that line is uncommented:

```
CondaValueError: prefix already exists: /opt/conda/envs/vlm_venv
```

But what happens in 1 week when condacache no longer exists? How can run that command IFF condacache does not exist? Or should I not be concerned, because once we're at 1 week, it will replace the dropped cache with the latest build?

++**Issue #2**++

As I add packages to environment.yml, the \`conda env update\` command properly installs them, but since condacache already exists, it doesn't update with the new updated environment containing these new packages. Rather, the build teardown process notes the following:

    You already have a 'condacache' cache so we won't create it again
    Cache "condacache": Skipping upload for existing cache

So, the question here is how can I replace the condacache either (1) every time or (2) whenever packages were added? Either would work, I suppose?   

Thanks in advance.


Answers:
author: Justin Thomas
body: [@Mike Prince](/t5/user/viewprofilepage/user-id/4333863) I am not an expert in python development, but I don't think we need to worry about *Issue #1*since the environment will be created if it doesn't exist.

***Issue #2*** is a bit tricky since we need to invalidate the cache when *environment.yml* is updated. You can use the [cache clear pipe](https://bitbucket.org/product/features/pipelines/integrations?p=atlassian/bitbucket-clear-cache) to achieve it.

```
image: continuumio/anaconda3

pipelines:
  default:
    - step:
        name: Clear cache if environments updated
         condition:
             changesets:
                 includePaths:
                   - "environment.yml"  
        script:
          - pipe: atlassian/bitbucket-clear-cache:3.1.1
             variables:
                 BITBUCKET_USERNAME: $BITBUCKET_USERNAME
                 BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
                 CACHES: ["condacache"]
    - step: 
        name: Build
        caches:
          - condacache
        script:
          - conda env create -f environment.yml 
          - conda activate vlm_venv
          - conda env update -f environment.yml
          - python tests/tests.py
```

There is an [open feature request](https://jira.atlassian.com/browse/BCLOUD-16314) to invalidate the cache when dependencies are updated, which should make it easier to invalidate the cache without a pipe. Please vote for that issue.

I hope the above steps should resolve your issue, please let me know if you have further questions.


Tags: bitbucket-cloud

cache

cloud

python