Question on accessing to GPU in Pipeline Runners
Asked by Simon Lee:

We are trying to incorporate tests into the Pipeline Runners. The software requires access to the GPU. We tried briefly with "docker run --gpus all" and it looks like accessing to the device is not allowed. Is this a hard restricton in Pipeline, or is there a way to get around it.


Answers:
author: Theodora Boudale
body: Hi [@Simon Lee](/t5/user/viewprofilepage/user-id/4379745) and welcome to the community.

This is a limitation in Pipelines.

Our development team has been exploring the option to allow access to GPUs, as you can see from one of Justin Thomas's replies here:

* <https://community.atlassian.com/t5/Bitbucket-Pipelines-articles/Bitbucket-Pipelines-Runners-is-now-in-open-beta/ba-p/1691022>

Kind regards,  
Theodora

comments: author: Kristinepetrosyan
body: [@Theodora Boudale](/t5/user/viewprofilepage/user-id/818086) any updates on this?


author: Theodora Boudale
body: Hi [@Kristinepetrosyan](/t5/user/viewprofilepage/user-id/4491450),

I don't have any update on this yet, you can add yourself as a watcher in the following feature request (by selecting the Start watching this issue link) if you'd like to get notified via email on updates:

* <https://jira.atlassian.com/browse/BCLOUD-21459>

I would also recommend adding your vote (by selecting the Vote for this issue link).

Kind regards,  
Theodora


author: Luu Quang Thang
body: I can use Bitbucket pipeline with GPU access by following the steps in root machine (which running Runner):

- [Installing the NVIDIA Container Toolkit --- NVIDIA Container Toolkit 1.14.3 documentation](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)

- Set nvidia to docker default runtime (It likes --runtime=docker). Edit /etc/docker/daemon.json

    {  "default-runtime": "nvidia",   "runtimes": {      "nvidia": {          "path": "/usr/bin/nvidia-container-runtime",          "runtimeArgs": []      }  }}

Restart the `docker` service by typing in `sudo systemctl restart docker`

`- Enable container accesses GPUs by the environment variable NVIDIA_VISIBLE_DEVICES (`[Specialized Configurations with Docker --- NVIDIA Container Toolkit 1.14.3 documentation](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/docker-specialized.html)`)`

    # build a image has default support GPU and will use it for pipeline
    # in Dockerfile
    # ubuntu is an example base image

    FROM ubuntu 
    ENV NVIDIA_VISIBLE_DEVICES=all

- Test

\` docker run --rm ubuntu:gpu nvidia-smi \`

- Update Bitbucket pipeline

```
pipelines: 
  default: 
    - step: 
      image: 
        name: ubuntu:gpu
```


author: Thebester
body: Integrating tests into Pipeline Runners is like adding a dash of magic to your workflow, but GPU access hiccups can be a buzzkill. We gave "docker run --gpus all" a whirl, but it seems the device isn't playing nice. Wondering if this is a Pipeline hard stop or a puzzle with a workaround.

Might be worth a visit to <https://anycompares.com/> - they often have nifty tricks up their sleeve for overcoming such hurdles. Your GPU might just need a different dance move!


author: a.stashevsky
body: I would also like to advocate for GPU access in bitbucket CI/CD processes. [@Theodora Boudale](/t5/user/viewprofilepage/user-id/818086)

comments: author: Theodora Boudale
body: Hi [@a.stashevsky](/t5/user/viewprofilepage/user-id/4992861),

Thank you for your comment. Please make sure to add your vote in the following feature request (by selecting the Vote for this issue link in that ticket), if you haven't done so already:

* <https://jira.atlassian.com/browse/BCLOUD-21459>

You are also more than welcome to leave any feedback there. <https://jira.atlassian.com/> is our public issue tracker, and it's where our product managers monitor the demand for new features.

Kind regards,  
Theodora


Tags: bitbucket-cloud

bitbucket-pipelines

cloud