Pipe trigger-pipeline failure on self-hosted runner
Asked by Elias Balasis:

The atlassian/trigger-pipeline:5.0.0 pipe  
produces the following output under a self-hosted runner:

    docker container run \

       --volume=/opt/atlassian/pipelines/agent/build:/opt/atlassian/pipelines/agent/build \

       --volume=/usr/local/bin/docker:/usr/local/bin/docker:ro \

       --volume=/opt/atlassian/pipelines/agent/ssh:/opt/atlassian/pipelines/agent/ssh:ro \

       --volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes \

       --volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/trigger-pipeline:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/trigger-pipeline \

       --workdir=$(pwd) \

       --label=org.bitbucket.pipelines.system=true \

       --env=BITBUCKET_STEP_TRIGGERER_UUID="$BITBUCKET_STEP_TRIGGERER_UUID" \

       --env=BITBUCKET_REPO_FULL_NAME="$BITBUCKET_REPO_FULL_NAME" \

       --env=BITBUCKET_GIT_HTTP_ORIGIN="$BITBUCKET_GIT_HTTP_ORIGIN" \

       --env=BITBUCKET_PROJECT_UUID="$BITBUCKET_PROJECT_UUID" \

       --env=BITBUCKET_REPO_IS_PRIVATE="$BITBUCKET_REPO_IS_PRIVATE" \

       --env=BITBUCKET_WORKSPACE="$BITBUCKET_WORKSPACE" \

       --env=BITBUCKET_REPO_OWNER_UUID="$BITBUCKET_REPO_OWNER_UUID" \

       --env=BITBUCKET_BRANCH="$BITBUCKET_BRANCH" \

       --env=BITBUCKET_REPO_UUID="$BITBUCKET_REPO_UUID" \

       --env=BITBUCKET_PROJECT_KEY="$BITBUCKET_PROJECT_KEY" \

       --env=BITBUCKET_REPO_SLUG="$BITBUCKET_REPO_SLUG" \

       --env=CI="$CI" \

       --env=BITBUCKET_REPO_OWNER="$BITBUCKET_REPO_OWNER" \

       --env=BITBUCKET_STEP_RUN_NUMBER="$BITBUCKET_STEP_RUN_NUMBER" \

       --env=BITBUCKET_BUILD_NUMBER="$BITBUCKET_BUILD_NUMBER" \

       --env=BITBUCKET_GIT_SSH_ORIGIN="$BITBUCKET_GIT_SSH_ORIGIN" \

       --env=BITBUCKET_PIPELINE_UUID="$BITBUCKET_PIPELINE_UUID" \

       --env=BITBUCKET_COMMIT="$BITBUCKET_COMMIT" \

       --env=PIPELINES_JWT_TOKEN="$PIPELINES_JWT_TOKEN" \

       --env=BITBUCKET_STEP_UUID="$BITBUCKET_STEP_UUID" \

       --env=BITBUCKET_CLONE_DIR="$BITBUCKET_CLONE_DIR" \

       --env=BITBUCKET_DOCKER_HOST_INTERNAL="$BITBUCKET_DOCKER_HOST_INTERNAL" \

       --env=DOCKER_HOST="tcp://host.docker.internal:2375" \

       --env=BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes" \

       --env=BITBUCKET_PIPE_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/trigger-pipeline" \

       --env=BITBUCKET_APP_PASSWORD="$BUILD_TRIGGER_PIPELINE_PASSWORD" \

       --env=BITBUCKET_USERNAME="$BUILD_TRIGGER_PIPELINE_USERNAME" \

       --env=DEBUG="true" \

       --env=REF_NAME="develop" \

       --env=REF_TYPE="branch" \

       --env=REPOSITORY="project" \

       --add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL" \

       bitbucketpipelines/trigger-pipeline:5.0.0

    Unable to find image 'bitbucketpipelines/trigger-pipeline:5.0.0' locally

    5.0.0: Pulling from bitbucketpipelines/trigger-pipeline

    5eb5b503b376: Pulling fs layer

    5c69ac0246d0: Pulling fs layer

    5adfeb420f7d: Pulling fs layer

    149ca41c402a: Pulling fs layer

    81a4c47d8175: Pulling fs layer

    2d2a1ca0f69b: Pulling fs layer

    0600a5d33135: Pulling fs layer

    2566fc56a1b4: Pulling fs layer

    839a8a969a56: Pulling fs layer

    149ca41c402a: Waiting

    81a4c47d8175: Waiting

    2d2a1ca0f69b: Waiting

    0600a5d33135: Waiting

    2566fc56a1b4: Waiting

    839a8a969a56: Waiting

    5c69ac0246d0: Verifying Checksum

    5c69ac0246d0: Download complete

    149ca41c402a: Download complete

    5adfeb420f7d: Verifying Checksum

    5adfeb420f7d: Download complete

    5eb5b503b376: Verifying Checksum

    5eb5b503b376: Download complete

    2d2a1ca0f69b: Verifying Checksum

    2d2a1ca0f69b: Download complete

    81a4c47d8175: Verifying Checksum

    81a4c47d8175: Download complete

    839a8a969a56: Verifying Checksum

    839a8a969a56: Download complete

    2566fc56a1b4: Verifying Checksum

    2566fc56a1b4: Download complete

    0600a5d33135: Verifying Checksum

    0600a5d33135: Download complete

    5eb5b503b376: Pull complete

    5c69ac0246d0: Pull complete

    5adfeb420f7d: Pull complete

    149ca41c402a: Pull complete

    81a4c47d8175: Pull complete

    2d2a1ca0f69b: Pull complete

    0600a5d33135: Pull complete

    2566fc56a1b4: Pull complete

    839a8a969a56: Pull complete

    Digest: sha256:029e80242186e4668d334622085a0519ebbced0df99a7d540d70870ff9daa15c

    Status: Downloaded newer image for bitbucketpipelines/trigger-pipeline:5.0.0

    docker: Error response from daemon: OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:352: getting the final child's pid from pipe caused: EOF: unknown.

    time="2022-04-16T23:33:30Z" level=error msg="error waiting for container: context canceled"

The OS of the Docker host of the self-hosted runner is "CentOS Linux release 7.9.2009 (Core)"

Here is the Docker output:

`time="2022-04-17T00:31:14.107652154Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"``time="2022-04-17T00:31:14.107927182Z" level=warning msg="Binding to IP address without --tlsverify is insecure and gives root access on this machine to everyone who has access to your network." host="tcp://0.0.0.0:2375"``time="2022-04-17T00:31:14.107949224Z" level=warning msg="Binding to an IP address, even on localhost, can also give access to scripts run in a browser. Be safe out there!" host="tcp://0.0.0.0:2375"``time="2022-04-17T00:31:15.108244661Z" level=warning msg="Binding to an IP address without --tlsverify is deprecated. Startup is intentionally being slowed down to show this message" host="tcp://0.0.0.0:2375"``time="2022-04-17T00:31:15.108329086Z" level=warning msg="Please consider generating tls certificates with client validation to prevent exposing unauthenticated root access to your network" host="tcp://0.0.0.0:2375"``time="2022-04-17T00:31:15.108343834Z" level=warning msg="You can override this by explicitly specifying '--tls=false' or '--tlsverify=false'" host="tcp://0.0.0.0:2375"``time="2022-04-17T00:31:15.108353352Z" level=warning msg="Support for listening on TCP without authentication or explicit intent to run without authentication will be removed in the next release" host="tcp://0.0.0.0:2375"``time="2022-04-17T00:31:30.344011958Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"``time="2022-04-17T00:31:30.344669283Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"``time="2022-04-17T00:31:30.351624299Z" level=warning msg="failed to load plugin io.containerd.internal.v1.opt" error="mkdir /opt/containerd: read-only file system"``time="2022-04-17T00:31:30.441725614Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge 151336 2 br_netfilter,ebtable_broute\nstp 12976 1 bridge\nllc 14552 2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter 22256 0 \nbridge 151336 2 br_netfilter,ebtable_broute\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"``time="2022-04-17T00:31:31Z" level=info msg="Pipelines plugin request authorization." allowed=false method=HEAD plugin=pipelines uri=/_ping``time="2022-04-17T00:31:31.574249641Z" level=error msg="AuthZRequest for HEAD /_ping returned error: authorization denied by plugin pipelines: "``time="2022-04-17T00:31:31Z" level=info msg="Pipelines plugin request authorization." allowed=true method=GET plugin=pipelines uri=/_ping``time="2022-04-17T00:31:31Z" level=info msg="Container create request." ArgsEscaped=false AttachStderr=true AttachStdin=false AttachStdout=true ExposedPorts="map[]" Healthcheck="<nil>" Labels="map[org.bitbucket.pipelines.system:true]" MacAddress= NetworkDisabled=false OnBuild="[]" OpenStdin=false StdinOnce=false StopSignal= StopTimeout="<nil>" Tty=false plugin=pipelines``time="2022-04-17T00:31:31Z" level=info msg="Container create request." AutoRemove=false BlkioDeviceReadBps="[]" BlkioDeviceReadIOps="[]" BlkioDeviceWriteBps="[]" BlkioDeviceWriteIOps="[]" BlkioWeight=0 BlkioWeightDevice="[]" CPUCount=0 CPUPercent=0 CPUPeriod=0 CPUQuota=0 CPURealtimePeriod=0 CPURealtimeRuntime=0 CPUShares=0 CapAdd="[]" CapDrop="[]" Cgroup= CgroupParent= ConsoleSize="[0 0]" ContainerIDFile= CpusetCpus= CpusetMems= DNS="[]" DNSOptions="[]" DNSSearch="[]" DeviceCgroupRules="[]" Devices="[]" ExtraHosts="[host.docker.internal:172.17.0.4]" GroupAdd="[]" IOMaximumBandwidth=0 IOMaximumIOps=0 Init="<nil>" IpcMode= Isolations= KernelMemory=0 Links="[]" LogConfig="{ map[]}" MaskedPaths="[]" Memory=0 MemoryReservation=0 MemorySwap=0 MemorySwappiness=0xc0002bc1b0 Mounts="[]" NanoCPUs=0 NetworkMode=default OomKillDisable=0xc0002bc1ba OomScoreAdj=0 PidMode= PidsLimit=0xc0002bc1c0 PortBindings="map[]" Privileged=false PublishAllPorts=false ReadOnlyPaths="[]" RestartPolicy="{no 0}" Runtime= SecurityOpt="[]" ShmSize=0 StorageOpt="map[]" Sysctls="map[]" Ulimits="[]" UsernsMode= VolumeDriver= VolumesFrom="[]" plugin=pipelines``time="2022-04-17T00:31:31Z" level=info msg="Container create request." EndpointsConfig="map[]" plugin=pipelines``time="2022-04-17T00:31:31Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri=/v1.40/containers/create``time="2022-04-17T00:31:31.589400568Z" level=error msg="Handler for POST /v1.40/containers/create returned error: No such image: bitbucketpipelines/trigger-pipeline:5.0.0"``time="2022-04-17T00:31:31Z" level=info msg="Pipelines plugin request authorization." allowed=true method=GET plugin=pipelines uri=/v1.40/info``time="2022-04-17T00:31:31Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri="/v1.40/images/create?fromImage=bitbucketpipelines%2Ftrigger-pipeline&tag=5.0.0"``time="2022-04-17T00:31:31.628410860Z" level=warning msg="Error getting v2 registry: Get `[http://localhost:5000/v2/:](http://localhost:5000/v2/)` dial tcp 127.0.0.1:5000: connect: connection refused"``time="2022-04-17T00:31:41Z" level=info msg="Container create request." ArgsEscaped=false AttachStderr=true AttachStdin=false AttachStdout=true ExposedPorts="map[]" Healthcheck="<nil>" Labels="map[org.bitbucket.pipelines.system:true]" MacAddress= NetworkDisabled=false OnBuild="[]" OpenStdin=false StdinOnce=false StopSignal= StopTimeout="<nil>" Tty=false plugin=pipelines``time="2022-04-17T00:31:41Z" level=info msg="Container create request." AutoRemove=false BlkioDeviceReadBps="[]" BlkioDeviceReadIOps="[]" BlkioDeviceWriteBps="[]" BlkioDeviceWriteIOps="[]" BlkioWeight=0 BlkioWeightDevice="[]" CPUCount=0 CPUPercent=0 CPUPeriod=0 CPUQuota=0 CPURealtimePeriod=0 CPURealtimeRuntime=0 CPUShares=0 CapAdd="[]" CapDrop="[]" Cgroup= CgroupParent= ConsoleSize="[0 0]" ContainerIDFile= CpusetCpus= CpusetMems= DNS="[]" DNSOptions="[]" DNSSearch="[]" DeviceCgroupRules="[]" Devices="[]" ExtraHosts="[host.docker.internal:172.17.0.4]" GroupAdd="[]" IOMaximumBandwidth=0 IOMaximumIOps=0 Init="<nil>" IpcMode= Isolations= KernelMemory=0 Links="[]" LogConfig="{ map[]}" MaskedPaths="[]" Memory=0 MemoryReservation=0 MemorySwap=0 MemorySwappiness=0xc00020ec50 Mounts="[]" NanoCPUs=0 NetworkMode=default OomKillDisable=0xc00020ec5a OomScoreAdj=0 PidMode= PidsLimit=0xc00020ec60 PortBindings="map[]" Privileged=false PublishAllPorts=false ReadOnlyPaths="[]" RestartPolicy="{no 0}" Runtime= SecurityOpt="[]" ShmSize=0 StorageOpt="map[]" Sysctls="map[]" Ulimits="[]" UsernsMode= VolumeDriver= VolumesFrom="[]" plugin=pipelines``time="2022-04-17T00:31:41Z" level=info msg="Container create request." EndpointsConfig="map[]" plugin=pipelines``time="2022-04-17T00:31:41Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri=/v1.40/containers/create``time="2022-04-17T00:31:41Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri="/v1.40/containers/4c0889b472a10dca81112acbe78688dfe1b2e56175b6aa1da8b30721f47e35f7/attach?stderr=1&stdout=1&stream=1"``time="2022-04-17T00:31:41Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri="/v1.40/containers/4c0889b472a10dca81112acbe78688dfe1b2e56175b6aa1da8b30721f47e35f7/wait?condition=next-exit"``time="2022-04-17T00:31:41Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri=/v1.40/containers/4c0889b472a10dca81112acbe78688dfe1b2e56175b6aa1da8b30721f47e35f7/start``time="2022-04-17T00:31:41.646831237Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/4c0889b472a10dca81112acbe78688dfe1b2e56175b6aa1da8b30721f47e35f7 pid=447``time="2022-04-17T00:31:41.693423617Z" level=error msg="copy shim log" error="read /proc/self/fd/12: file already closed"``time="2022-04-17T00:31:41.696219545Z" level=error msg="stream copy error: reading from a closed fifo"``time="2022-04-17T00:31:41.698250208Z" level=error msg="stream copy error: reading from a closed fifo"``time="2022-04-17T00:31:41.754263331Z" level=error msg="4c0889b472a10dca81112acbe78688dfe1b2e56175b6aa1da8b30721f47e35f7 cleanup: failed to delete container from containerd: no such container"``time="2022-04-17T00:31:41.754315337Z" level=error msg="Handler for POST /v1.40/containers/4c0889b472a10dca81112acbe78688dfe1b2e56175b6aa1da8b30721f47e35f7/start returned error: OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:352: getting the final child's pid from pipe caused: EOF: unknown"``time="2022-04-17T00:31:41.754974125Z" level=error msg="Handler for POST /v1.40/containers/4c0889b472a10dca81112acbe78688dfe1b2e56175b6aa1da8b30721f47e35f7/start returned error: OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:352: getting the final child's pid from pipe caused: EOF: unknown"`


Answers:
author: Elias Balasis
body: The solution is use of custom docker-in-docker image (see <https://support.atlassian.com/bitbucket-cloud/docs/configure-your-runner-in-bitbucket-pipelines-yml/#Custom-docker-in-docker-image>) hosted on a private "insecure" Docker registry and using the docker://\<DockerAccountName\>/\<ImageName\>:\<version\> syntax (see <https://support.atlassian.com/bitbucket-cloud/docs/write-a-pipe-for-bitbucket-pipelines/>)


author: Caroline R
body: Hi, [@Elias Balasis](/t5/user/viewprofilepage/user-id/761714),

I saw that you created a ticket with our support related to the same question, so to avoid any possible misunderstandings or miscommunication that may arise from discussing the same matter on two different platforms, let's focus on the ticket instead.

Once the ticket is resolved, feel free to share the resolution here to help other community members that may have the same issue.

Kind regards,  
Caroline


author: Elias Balasis
body: see <https://getsupport.atlassian.com/servicedesk/customer/portal/11/BBS-184378>


author: Elias Balasis
body: This feels like a bug, can someone raise this with Atlassian? as I am still on the free tier and I don't have permission to raise bugs.


Tags: bitbucket-cloud

bitbucket-pipelines

cloud