"Connection reset by peer in connection to bitbucket.org:443" in Pipelines
Asked by Rudolf Byker:

I'm using a self-hosted runner, version 1.369 (the latest at the time of writing). It was working well for a long time, but recently, without any infrastructure changes, I started getting this error on all steps:
> We couldn't clone the repository. Try rerunning the pipeline.

In the "build setup", I see this output:

```
+ umask 000
+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="..." https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...
fatal: unable to access 'https://bitbucket.org/.../....git/': OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443 
Attempt 1 of 6 failed.
Retrying in 1 seconds
Cloning into '/opt/atlassian/pipelines/agent/build'...
fatal: unable to access 'https://bitbucket.org/.../....git/': OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443 
Attempt 2 of 6 failed.
Retrying in 2 seconds
```

etc...

The setup {#toc-hId--522405798}
-------------------------------

This runner is in kubernetes on a K3s cluster. It's running on a development node which is a physical machine in my house. I manage this cluster, and nothing changed in the network.

The deployment for my runner instance is as follows:

```
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: bbp-runner
  name: runner
  labels:
    app: bbp-runner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bbp-runner
  template:
    metadata:
      labels:
        app: bbp-runner
        accountUuid: foo
        runnerUuid: bar
    spec:
      nodeSelector:
        env: prod
      containers:
        - name: bitbucket-k8s-runner
          image: docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner
          imagePullPolicy: Always
          env:
            - name: ACCOUNT_UUID
              value: "{foo}"
            - name: RUNNER_UUID
              value: "{bar}"
            - name: OAUTH_CLIENT_ID
              valueFrom:
                secretKeyRef:
                  name: runner-oauth-credentials
                  key: oauthClientId
            - name: OAUTH_CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  name: runner-oauth-credentials
                  key: oauthClientSecret
            - name: WORKING_DIRECTORY
              value: "/tmp"
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: docker-containers
              mountPath: /var/lib/docker/containers
              readOnly: true
            - name: var-run
              mountPath: /var/run
        - name: docker-in-docker
          image: docker:20.10.7-dind
          securityContext:
            privileged: true
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: docker-containers
              mountPath: /var/lib/docker/containers
            - name: var-run
              mountPath: /var/run
      volumes:
        - name: tmp
          emptyDir: {}
        - name: docker-containers
          emptyDir: {}
        - name: var-run
          emptyDir: {}
```

The relevant containers are

```
docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner
```

and

```
docker:20.10.7-dind
```

I followed this guide <https://janosmiko.com/blog/2021-09-08-bitbucket-pipelines-runners-in-k8s/> when I set it up a few months ago.


Answers:
author: Syahrul
body: Hey [@Rudolf Byker](/t5/user/viewprofilepage/user-id/340335)

G'day.

The error below means the TLS handshake failed to establish the connection.

```
OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443 "
```

This can happen for multiple reasons, but generally, it's caused by the local network.

I suggest checking with your network team to see if there's any recent change in your Firewall/proxy.

That said, you can also investigate this by running a few tests, such as:

1. Try cloning a repository from your Runner server with verbose logs enable to get more information  

   ```
   GIT_TRACE_PACKET=1 GIT_TRACE=1 GIT_CURL_VERBOSE=1 <git command_here>
   ```

2. Run Curl to see if there's any issue with TLS connection from your server  

   ```
   curl -v https://bitbucket.org
   ```

If both tests failed with the same runner error, then most likely, the connection is being blocked on your Firewall/Proxy level.

I hope this helps.

Cheers,  
Syahrul

comments: author: Rudolf Byker
body: Thanks for your time.

I added some info about the infrastructure setup to the original post above.

I get a shell in the pod by doing this:

```
kubectl -n bbp-runner exec -it deploy/runner -- /bin/sh
```

The \`docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner\` image does not contain \`git\` or \`curl\`, but I could add it with \`apk\`, and then test what you suggested:

```
apk add curl git openssh
```

#### curl {#toc-hId-178746509}

```
curl -v https://bitbucket.org
```

This works fine. I get a 200 response with lots of HTML.

#### git {#toc-hId--1628707954}

```
git clone git@bitbucket.org:rscdevelopers/living_word_vue.git
```

This works fine as well. I get the RSA key fingerprint, and then permission denied as expected, since I don't have the keys set up. This is enough to verify that TLS is working.


author: Syahrul
body: Hey [@Rudolf Byker](/t5/user/viewprofilepage/user-id/340335)

Thanks for the update.

We may need to investigate this further, so I went ahead and created a support ticket on your behalf. Please review it [here.](https://support.atlassian.com/requests/BBS-193900/)

One of our Support should get back to you within your timezone shortly.

Cheers,  
Syahrul


Tags: bitbucket-cloud

bitbucket-pipelines

cloud