Permission denied (publickey) even though public key is registered
Asked by DerHabicht:

Story time.  

I attempt to clone a repository. No big deal, I've done this a million times:

```
$ git clone git@bitbucket.org:[my-username]/[repo-i-want].git
Cloning into '[repo-i-want]'...
git@bitbucket.org: Permission denied (publickey).
fatal: Could not read from remote repository.
```

Huh. I could have sworn that I added my SSH public key to Bitbucket ages ago.

```
$ ssh -T git@bitbucket.org
git@bitbucket.org: Permission denied (publickey).
```

Huh. Guess not.  
So, I go to Bitbucket, click on my cute-little-overly-cliche-gopher-profile-id picture in the lower-left corner, click "Personal Settings," navigate my way on down to SSH keys and I see my public key there. Well, really all I know is that there is a key there and it's supposedly descriptive name in my settings is claiming to be the key I need to use. Maybe it's just an old key, I'll just replace it.

```
$ cat ~/.ssh/id_rsa.pub | xclip -sel clip
```

Paste that into Bitbucket and:

```
$ ssh -T git@bitbucket.org
git@bitbucket.org: Permission denied (publickey).
```

Well, that's just rude. Maybe the id_rsa.pub file in my .ssh directory isn't actually the public key to my private key (seems like a stretch, but I guess it *could* happen, right?)

```
$ ssh-keygen -y -f ~/.ssh/id_rsa | xclip -sel clip
```

Paste that in. No joy. Maybe I need a new key. I evacuate my current creds to safety and then try:

```
$ ssh-keygen
[default answer to all prompts]
$ cat ~/.ssh/id_rsa.pub | xclip -sel clip
```

And, you guessed it, still no joy. I mean, maybe something could go wrong in the copy/paste process, right?

```
$ diff ~/.ssh/id_rsa.pub [file created from the key copied off of Bitbucket]
$
```

Yeah, okay, that was a stretch anyway.

Time to hit the Googles. I find advice to use ssh-add. All that does is tell SSH to add that key to the list of keys it should try. I'm trying to use id_rsa which is the default that the config already looks for, so this is redundant. Plus:

```
$ ssh -T -i ~/.ssh/id_rsa git@bitbucket.org
git@bitbucket.org: Permission denied (publickey).
```

So, that wouldn't do anything anyway. The other "advice" I find is to use HTTPS which isn't a solution, it's a workaround (one that I'm using right now so I can move forward with my project, but it meant disabling my MFA which I'm not super happy about). The other advice I have been able to find basically tells me to rehash the things I did above.

Clearly, I'm missing something here and it's probably painfully obvious but I'm just not seeing it. Anyone have any clarity on this?


Answers:
author: Theodora Boudale
body: Hi [@DerHabicht](/t5/user/viewprofilepage/user-id/3764543) ,

**1.** Could you please copy-paste the output of the following command, executed from the same machine you've been working on?

```
ssh -Tvvv git@bitbucket.org
```

This will give us more verbose output and possibly an indication of what is going wrong.

**2.** What are the permissions of the .ssh directory, the private and public key?

**3.** Is there a file named *config* in the .ssh directory, along with the SSH keys? If so, could you let us know if there is a section for 'Host bitbucket.org' in that file, and what options are set there?

Kind regards,  
Theodora

comments: author: DerHabicht
body: Sorry I didn't answer sooner. I thought I was going to get an email when I got a reply. Must've landed in my spam folder and I just didn't notice. Anyway, here are the answers to your questions:

1. I've pushed the dump to the end of my reply so you don't have to scroll past it to get to the rest of my answers. I did that step early on in all of this, but didn't see anything terribly enlightening. Could have missed something, though.
2. The perms on .ssh and the private key are fine. The same key is also used for SSH access to some other servers/services with no problem. The public key is not used by the SSH client, so its perms don't matter (it would be kinda silly to have to keep the public key locked down). For reference, though:

   ```
   $ ls -al | grep .ssh
   drwx------. 1 [MY_USERNAME] [MY_GROUP] 76 Dec 31 17:15 .ssh
   $ ls .ssh
   -rw-------. 1 [MY_USERNAME] [MY_GROUP] 3381 Dec 4 12:51 id_rsa
   -rw-rw-r--. 1 [MY_USERNAME] [MY_GROUP] 742 Dec 24 14:42 id_rsa.pub
   ```

3. There are no host-specific configs for my client.  

   The dump for answer 1:  

   ```
   $ ssh -Tvvv git@bitbucket.org > log.txt
   OpenSSH_8.4p1, OpenSSL 1.1.1i FIPS 8 Dec 2020
   debug1: Reading configuration data /etc/ssh/ssh_config
   debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0
   debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
   debug2: checking match for 'final all' host bitbucket.org originally bitbucket.org
   debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'
   debug2: match not found
   debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
   debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
   debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
   debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
   debug1: configuration requests final Match pass
   debug1: re-parsing configuration
   debug1: Reading configuration data /etc/ssh/ssh_config
   debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0
   debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
   debug2: checking match for 'final all' host bitbucket.org originally bitbucket.org
   debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'
   debug2: match found
   debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
   debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
   debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
   debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
   debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '[SSH_DIR]/known_hosts'
   debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '[SSH_DIR]/known_hosts2'
   debug2: resolving "bitbucket.org" port 22
   debug2: ssh_connect_direct
   debug1: Connecting to bitbucket.org [104.192.141.1] port 22.
   debug1: Connection established.
   debug1: identity file [SSH_DIR]/id_rsa type 0
   debug1: identity file [SSH_DIR]/id_rsa-cert type -1
   debug1: identity file [SSH_DIR]/id_dsa type -1
   debug1: identity file [SSH_DIR]/id_dsa-cert type -1
   debug1: identity file [SSH_DIR]/id_ecdsa type -1
   debug1: identity file [SSH_DIR]/id_ecdsa-cert type -1
   debug1: identity file [SSH_DIR]/id_ecdsa_sk type -1
   debug1: identity file [SSH_DIR]/id_ecdsa_sk-cert type -1
   debug1: identity file [SSH_DIR]/id_ed25519 type -1
   debug1: identity file [SSH_DIR]/id_ed25519-cert type -1
   debug1: identity file [SSH_DIR]/id_ed25519_sk type -1
   debug1: identity file [SSH_DIR]/id_ed25519_sk-cert type -1
   debug1: identity file [SSH_DIR]/id_xmss type -1
   debug1: identity file [SSH_DIR]/id_xmss-cert type -1
   debug1: Local version string SSH-2.0-OpenSSH_8.4
   debug1: Remote protocol version 2.0, remote software version conker_c123b90d72-dirty conker-3003
   debug1: no match: conker_c123b90d72-dirty conker-3003
   debug2: fd 4 setting O_NONBLOCK
   debug1: Authenticating to bitbucket.org:22 as 'git'
   debug3: hostkeys_foreach: reading file "[SSH_DIR]/known_hosts"
   debug3: record_hostkey: found key type RSA in file [SSH_DIR]/known_hosts:9
   debug3: load_hostkeys: loaded 1 keys from bitbucket.org
   debug3: order_hostkeyalgs: prefer hostkeyalgs: rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
   debug3: send packet: type 20
   debug1: SSH2_MSG_KEXINIT sent
   debug3: receive packet: type 20
   debug1: SSH2_MSG_KEXINIT received
   debug2: local client KEXINIT proposal
   debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-c
   debug2: host key algorithms: rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,sk-ssh-ed25519@openssh.com
   debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes128-gcm@openssh.com,aes128-ctr
   debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes128-gcm@openssh.com,aes128-ctr
   debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
   debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
   debug2: compression ctos: none,zlib@openssh.com,zlib
   debug2: compression stoc: none,zlib@openssh.com,zlib
   debug2: languages ctos: 
   debug2: languages stoc: 
   debug2: first_kex_follows 0 
   debug2: reserved 0 
   debug2: peer server KEXINIT proposal
   debug2: KEX algorithms: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
   debug2: host key algorithms: ssh-dss,ssh-rsa
   debug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,chacha20-poly1305@openssh.com
   debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,chacha20-poly1305@openssh.com
   debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha2-256,hmac-sha1,hmac-sha1-96
   debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha2-256,hmac-sha1,hmac-sha1-96
   debug2: compression ctos: none
   debug2: compression stoc: none
   debug2: languages ctos: 
   debug2: languages stoc: 
   debug2: first_kex_follows 0 
   debug2: reserved 0 
   debug1: kex: algorithm: curve25519-sha256@libssh.org
   debug1: kex: host key algorithm: ssh-rsa
   debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
   debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
   debug1: kex: curve25519-sha256@libssh.org need=64 dh_need=64
   debug1: kex: curve25519-sha256@libssh.org need=64 dh_need=64
   debug3: send packet: type 30
   debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
   debug3: receive packet: type 31
   debug1: Server host key: ssh-rsa SHA256:zzXQOXSRBEiUtuE8AikJYKwbHaxvSc0ojez9YXaGp1A
   debug3: hostkeys_foreach: reading file "[SSH_DIR]/known_hosts"
   debug3: record_hostkey: found key type RSA in file [SSH_DIR]/known_hosts:9
   debug3: load_hostkeys: loaded 1 keys from bitbucket.org
   debug3: hostkeys_foreach: reading file "[SSH_DIR]/known_hosts"
   debug3: record_hostkey: found key type RSA in file [SSH_DIR]/known_hosts:9
   debug3: load_hostkeys: loaded 1 keys from 104.192.141.1
   debug1: Host 'bitbucket.org' is known and matches the RSA host key.
   debug1: Found key in [SSH_DIR]/known_hosts:9
   debug3: send packet: type 21
   debug2: set_newkeys: mode 1
   debug1: rekey out after 134217728 blocks
   debug1: SSH2_MSG_NEWKEYS sent
   debug1: expecting SSH2_MSG_NEWKEYS
   debug3: receive packet: type 21
   debug1: SSH2_MSG_NEWKEYS received
   debug2: set_newkeys: mode 0
   debug1: rekey in after 134217728 blocks
   debug1: Will attempt key: [SSH_DIR]/id_rsa RSA SHA256:s99U/lIHVlJRtezk6Vkj36shW8w7cY9dsOXf4DrtSkM agent
   debug1: Will attempt key: [SSH_DIR]/id_dsa 
   debug1: Will attempt key: [SSH_DIR]/id_ecdsa 
   debug1: Will attempt key: [SSH_DIR]/id_ecdsa_sk 
   debug1: Will attempt key: [SSH_DIR]/id_ed25519 
   debug1: Will attempt key: [SSH_DIR]/id_ed25519_sk 
   debug1: Will attempt key: [SSH_DIR]/id_xmss 
   debug2: pubkey_prepare: done
   debug3: send packet: type 5
   debug3: receive packet: type 6
   debug2: service_accept: ssh-userauth
   debug1: SSH2_MSG_SERVICE_ACCEPT received
   debug3: send packet: type 50
   debug3: receive packet: type 51
   debug1: Authentications that can continue: publickey
   debug3: start over, passed a different list publickey
   debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
   debug3: authmethod_lookup publickey
   debug3: remaining preferred: keyboard-interactive,password
   debug3: authmethod_is_enabled publickey
   debug1: Next authentication method: publickey
   debug1: Offering public key: [SSH_DIR]/id_rsa RSA SHA256:s99U/lIHVlJRtezk6Vkj36shW8w7cY9dsOXf4DrtSkM agent
   debug1: send_pubkey_test: no mutual signature algorithm
   debug1: Trying private key: [SSH_DIR]/id_dsa
   debug3: no such identity: [SSH_DIR]/id_dsa: No such file or directory
   debug1: Trying private key: [SSH_DIR]/id_ecdsa
   debug3: no such identity: [SSH_DIR]/id_ecdsa: No such file or directory
   debug1: Trying private key: [SSH_DIR]/id_ecdsa_sk
   debug3: no such identity: [SSH_DIR]/id_ecdsa_sk: No such file or directory
   debug1: Trying private key: [SSH_DIR]/id_ed25519
   debug3: no such identity: [SSH_DIR]/id_ed25519: No such file or directory
   debug1: Trying private key: [SSH_DIR]/id_ed25519_sk
   debug3: no such identity: [SSH_DIR]/id_ed25519_sk: No such file or directory
   debug1: Trying private key: [SSH_DIR]/id_xmss
   debug3: no such identity: [SSH_DIR]/id_xmss: No such file or directory
   debug2: we did not send a packet, disable method
   debug1: No more authentication methods to try.
   git@bitbucket.org: Permission denied (publickey).
   ```


author: Theodora Boudale
body: Hi [@DerHabicht](/t5/user/viewprofilepage/user-id/3764543) ,

Thank you for your reply.

Looking at the verbose output, the line that indicates the issue is the following:

```
debug1: send_pubkey_test: no mutual signature algorithm
```

after the public key id_rsa is offered.

Are you using Fedora 33? (If not, could you please let us know what OS you are using?)

I see similar issues reported by other users who upgraded to Fedora 33 as this version has some changes on OpenSSH connections.

To work around this, you can create a config file in the .ssh folder and add the following:

```
Host bitbucket.org
    Hostname bitbucket.org
    PubkeyAcceptedKeyTypes +rsa-sha2-256,rsa-sha2-512
```

Another option would be to generate a new SSH key pair with encryption algorithm Ed25519 or ECDSA and use that instead. The algorithm is specified with the -t option, so you can use e.g. the command

```
ssh-keygen -t ecdsa
```

Please feel free to let me know how it goes and if you need further assistance.

Kind regards,  
Theodora


author: DerHabicht
body: The host config you posted worked like a charm. Thank you very much!

And, as a matter of fact, I am using Fedora 33. And, come to think of it, I think this is the first time I've tried to pull from Bitbucket since the upgrade. I guess they did some fiddling with the default client config. I'll have to educate myself on that.

Once again, thank you for your help!


author: Theodora Boudale
body: That's good to hear! You are very welcome and I'm glad to have helped!


Tags: atl-gruland

bitbucket-cloud

cloud