Is cosmos db emulater supported on bitbucket pipeline?
Asked by s.barbuto:

Hi,

the scenario we are working on, uses cosmos db as database on a C# .NET application. The integration tests - locally - use the cosmos db emulator spun up through docker. It's necessary to reproduce the same behavior on the pipeline when the PR is submitted.

```
?
```

```
image: mcr.microsoft.com/dotnet/sdk:6.0

definitions:
  services:
   cosmos-db:
     image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
     variables:
        AZURE_COSMOS_EMULATOR_PARTITION_COUNT: '10'
        AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE: 'true'
        AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE: '127.0.0.1'
steps:
 - step: &stepTests
    name: Run Tests
    services:
     - cosmos-db
    script:
       dotnet test
```

The db's connection string points to 127.0.0.1 on the port 8081  

```
"ConnectionString": "AccountEndpoint=https://127.0.0.1:8081/;AccountKey=secretKey"
```

The result of it, is that connections are refused and consequently tests fail. As side note, the cosmos db tab - during the execution - seems stuck, it hangs and shows "Starting".

![Screenshot from 2022-11-07 09-32-01.png](/t5/image/serverpage/image-id/224384iCD3B75F8DFCEF222/image-size/large?v=v2&px=999 "Screenshot from 2022-11-07 09-32-01.png")![Screenshot from 2022-11-07 09-34-00.png](/t5/image/serverpage/image-id/224385i4A4F5942F1E90F6D/image-size/large?v=v2&px=999 "Screenshot from 2022-11-07 09-34-00.png")

Questions here are:

Is the service correctly configured? Does the connection string point to the right address?

Is cosmos db emulator supported by the pipeline?

Thanks in advance.

Best regards

Sante


Answers:
author: Theodora Boudale
body: Hi Sante,

We haven't tested Pipelines with every database service available and we don't have a guide on how to use cosmos db.

I have tried running a build in Pipelines based on the bitbucket-pipelines.yml file you posted here. In the script of my step I used the following command instead to check if the service is running:

```
- curl -k https://127.0.0.1:8081/_explorer/emulator.pem
```

In the beginning, I was seeing errors in the output of curl. Services in Pipelines have no mechanism to wait for service startup, so I added the command **sleep 30** before the curl command, in case the service needed more time to start up, but I was still seeing a "Connection refused" error.

I checked the documentation here <https://learn.microsoft.com/en-us/azure/cosmos-db/linux-emulator?tabs=sql-api%2Cssl-netstd21> and I saw that 3 GB of memory is required for the service container, so I adjusted the memory used for the service in Pipelines. After that change, and with a sleep command at the beginning of my script, the curl command started succeeding.

This is the bitbucket-pipelines.yml file I used:

```
image: mcr.microsoft.com/dotnet/sdk:6.0

pipelines: 
  default: 
    - step:
        size: 2x
        script: 
          - sleep 30 
          - curl -k https://127.0.0.1:8081/_explorer/emulator.pem
        services: 
          - cosmos-db

definitions: 
  services: 
    cosmos-db: 
      image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator  
      memory: 3072
      variables: 
        AZURE_COSMOS_EMULATOR_PARTITION_COUNT: '10'
        AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE: 'true'
        AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE: '127.0.0.1'
```

I would suggest trying to run your build by allocating 3072 MB to the service (as above) and also adding a sleep command at the beginning of the step.

Instead of using a sleep command, another option would also be to write a script that checks if the connection to <https://127.0.0.1:8081> succeeds. If it does, exit the script so the rest of the commands in the step's script are executed. If not, use a sleep command for e.g. 10 seconds, then try again (e.g. 3 more times) and if it fails after these 3 times, then exit the script with exit code 1 so that the build fails and doesn't continue.

I would also recommend using size: 2x for the step, if you allocate 3072 MB of memory to the service. Regular steps (without size: 2x) have 4096 MB of memory in total. The build container is given 1024 MB of the total memory, so the remaining memory for a regular step is 3072 MB. If all of that is given to the service container, then there may not be enough memory left for the build to run.

With size: 2x the step will have 8192 MB of memory in total. The build container is given 1024 MB, we allocate 3072 MB to the service and the rest remains available for the build. Please keep in mind that 2x pipelines will use twice the number of build minutes.

Please feel free to let me know how it goes.

Kind regards,  
Theodora


Tags: bitbucket-cloud

cloud