setup a piple line with sftp copy file
Asked by Mohamed Bakr:

I have a pipe line with 2 steps , i have create ssh and added public key to my server

```
- step:
    name: deployment
    runs-on:
     - 'self.hosted'
    script:
    - pipe: atlassian/ssh-run:0.4.0
      variables:
       SSH_USER: 'ubuntu'
       SERVER: 172.0.1.195
       MODE: 'command'
       DEBUG: 'true'
       COMMAND: 'echo ${BITBUCKET_COMMIT::7} > a.txt'
    - pipe: atlassian/sftp-deploy:0.5.11
      variables:
        USER: 'ubuntu'
        SERVER: '172.0.1.195'
        REMOTE_PATH: '/home/ubuntu'
        DEBUG: 'true'
```

but when i tried to execute this pipeline ssh pass and sftp or scp always fail with the following logs

Runner matching labels:  
- linux  
- self.hosted  
Runner name: ta3meed-sandbox-runner  
Runner labels: self.hosted, linux  
Runner version:  
current: 1.333  
latest: 1.336

<br />

\[93mThe version of this runner is outdated. Upgrade to the latest version (1.336). \[0m  
+ umask 000

+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="deployment-pipeline" --depth 50 https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR  
Cloning into '/opt/atlassian/pipelines/agent/build'...

+ git reset --hard bc6100786f45d22fc8ef283281d49687c8e08409  
HEAD is now at bc61007 add debug

+ git config user.name bitbucket-pipelines

+ git config user.email commits-noreply@bitbucket.org

+ git config push.default current

+ git config http.${BITBUCKET_GIT_HTTP_ORIGIN}.proxy <http://localhost:29418/>

+ git remote set-url origin <http://bitbucket.org/$BITBUCKET_REPO_FULL_NAME>

+ git reflog expire --expire=all --all

+ echo ".bitbucket/pipelines/generated" \>\> .git/info/exclude

+ chmod 777 $BUILD_DIR

Images used:  
build: atlassian/default-image@sha256:3a09dfec7e36fe99e3910714c5646be6302ccbca204d38539a07f0c2cb5902d4  
docker: docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-docker-daemon@sha256:03bfa3875b810fe280239175f742a20c500d9b1f181c83a3b05cdc04b1cfb12c

+ docker container run \\  
--volume=/opt/atlassian/pipelines/agent/build:/opt/atlassian/pipelines/agent/build \\  
--volume=/usr/local/bin/docker:/usr/local/bin/docker:ro \\  
--volume=/opt/atlassian/pipelines/agent/ssh:/opt/atlassian/pipelines/agent/ssh:ro \\  
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes \\  
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/ssh-run:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/ssh-run \\  
--workdir=$(pwd) \\  
--label=org.bitbucket.pipelines.system=true \\  
--env=BITBUCKET_STEP_TRIGGERER_UUID="$BITBUCKET_STEP_TRIGGERER_UUID" \\  
--env=BITBUCKET_REPO_FULL_NAME="$BITBUCKET_REPO_FULL_NAME" \\  
--env=BITBUCKET_GIT_HTTP_ORIGIN="$BITBUCKET_GIT_HTTP_ORIGIN" \\  
--env=BITBUCKET_PROJECT_UUID="$BITBUCKET_PROJECT_UUID" \\  
--env=BITBUCKET_REPO_IS_PRIVATE="$BITBUCKET_REPO_IS_PRIVATE" \\  
--env=BITBUCKET_WORKSPACE="$BITBUCKET_WORKSPACE" \\  
--env=BITBUCKET_REPO_OWNER_UUID="$BITBUCKET_REPO_OWNER_UUID" \\  
--env=BITBUCKET_BRANCH="$BITBUCKET_BRANCH" \\  
--env=BITBUCKET_REPO_UUID="$BITBUCKET_REPO_UUID" \\  
--env=BITBUCKET_PROJECT_KEY="$BITBUCKET_PROJECT_KEY" \\  
--env=BITBUCKET_REPO_SLUG="$BITBUCKET_REPO_SLUG" \\  
--env=CI="$CI" \\  
--env=BITBUCKET_REPO_OWNER="$BITBUCKET_REPO_OWNER" \\  
--env=BITBUCKET_STEP_RUN_NUMBER="$BITBUCKET_STEP_RUN_NUMBER" \\  
--env=BITBUCKET_BUILD_NUMBER="$BITBUCKET_BUILD_NUMBER" \\  
--env=BITBUCKET_GIT_SSH_ORIGIN="$BITBUCKET_GIT_SSH_ORIGIN" \\  
--env=BITBUCKET_PIPELINE_UUID="$BITBUCKET_PIPELINE_UUID" \\  
--env=BITBUCKET_COMMIT="$BITBUCKET_COMMIT" \\  
--env=PIPELINES_JWT_TOKEN="$PIPELINES_JWT_TOKEN" \\  
--env=BITBUCKET_STEP_UUID="$BITBUCKET_STEP_UUID" \\  
--env=BITBUCKET_CLONE_DIR="$BITBUCKET_CLONE_DIR" \\  
--env=BITBUCKET_DOCKER_HOST_INTERNAL="$BITBUCKET_DOCKER_HOST_INTERNAL" \\  
--env=DOCKER_HOST="tcp://host.docker.internal:2375" \\  
--env=BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes" \\  
--env=BITBUCKET_PIPE_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/ssh-run" \\  
--env=COMMAND="echo ${BITBUCKET_COMMIT::7} \> a.txt" \\  
--env=DEBUG="true" \\  
--env=MODE="command" \\  
--env=SERVER="172.0.1.195" \\  
--env=SSH_USER="ubuntu" \\  
--add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL" \\  
bitbucketpipelines/ssh-run:0.4.0  
Unable to find image 'bitbucketpipelines/ssh-run:0.4.0' locally  
0.4.0: Pulling from bitbucketpipelines/ssh-run  
31603596830f: Pulling fs layer  
c507f8e37a23: Pulling fs layer  
7eca4d8c4c7b: Pulling fs layer  
c21a713484c2: Pulling fs layer  
c21a713484c2: Waiting  
7eca4d8c4c7b: Verifying Checksum  
7eca4d8c4c7b: Download complete  
31603596830f: Download complete  
c507f8e37a23: Verifying Checksum  
c507f8e37a23: Download complete  
31603596830f: Pull complete  
c21a713484c2: Verifying Checksum  
c21a713484c2: Download complete  
c507f8e37a23: Pull complete  
7eca4d8c4c7b: Pull complete  
c21a713484c2: Pull complete  
Digest: sha256:2aa2f6517fb364da07a18f8b1e55b651c1db0361f7a850b79914c8da663f709e  
Status: Downloaded newer image for bitbucketpipelines/ssh-run:0.4.0  
\[36mINFO: Executing the pipe...\[0m  
\[36mINFO: Using default ssh key\[0m  
\[36mINFO: Executing command on 172.0.1.195\[0m  
ssh -A -tt -i /root/.ssh/pipelines_id -o StrictHostKeyChecking=no -p 22 ubuntu@172.0.1.195 bash -c 'echo bc61007 \> a.txt'  
Warning: Permanently added '172.0.1.195' (ECDSA) to the list of known hosts.  
Connection to 172.0.1.195 closed.  
\[32m? Execution finished.\[0m

+ docker container run \\  
--volume=/opt/atlassian/pipelines/agent/build:/opt/atlassian/pipelines/agent/build \\  
--volume=/usr/local/bin/docker:/usr/local/bin/docker:ro \\  
--volume=/opt/atlassian/pipelines/agent/ssh:/opt/atlassian/pipelines/agent/ssh:ro \\  
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes \\  
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/sftp-deploy:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/sftp-deploy \\  
--workdir=$(pwd) \\  
--label=org.bitbucket.pipelines.system=true \\  
--env=BITBUCKET_STEP_TRIGGERER_UUID="$BITBUCKET_STEP_TRIGGERER_UUID" \\  
--env=BITBUCKET_REPO_FULL_NAME="$BITBUCKET_REPO_FULL_NAME" \\  
--env=BITBUCKET_GIT_HTTP_ORIGIN="$BITBUCKET_GIT_HTTP_ORIGIN" \\  
--env=BITBUCKET_PROJECT_UUID="$BITBUCKET_PROJECT_UUID" \\  
--env=BITBUCKET_REPO_IS_PRIVATE="$BITBUCKET_REPO_IS_PRIVATE" \\  
--env=BITBUCKET_WORKSPACE="$BITBUCKET_WORKSPACE" \\  
--env=BITBUCKET_REPO_OWNER_UUID="$BITBUCKET_REPO_OWNER_UUID" \\  
--env=BITBUCKET_BRANCH="$BITBUCKET_BRANCH" \\  
--env=BITBUCKET_REPO_UUID="$BITBUCKET_REPO_UUID" \\  
--env=BITBUCKET_PROJECT_KEY="$BITBUCKET_PROJECT_KEY" \\  
--env=BITBUCKET_REPO_SLUG="$BITBUCKET_REPO_SLUG" \\  
--env=CI="$CI" \\  
--env=BITBUCKET_REPO_OWNER="$BITBUCKET_REPO_OWNER" \\  
--env=BITBUCKET_STEP_RUN_NUMBER="$BITBUCKET_STEP_RUN_NUMBER" \\  
--env=BITBUCKET_BUILD_NUMBER="$BITBUCKET_BUILD_NUMBER" \\  
--env=BITBUCKET_GIT_SSH_ORIGIN="$BITBUCKET_GIT_SSH_ORIGIN" \\  
--env=BITBUCKET_PIPELINE_UUID="$BITBUCKET_PIPELINE_UUID" \\  
--env=BITBUCKET_COMMIT="$BITBUCKET_COMMIT" \\  
--env=PIPELINES_JWT_TOKEN="$PIPELINES_JWT_TOKEN" \\  
--env=BITBUCKET_STEP_UUID="$BITBUCKET_STEP_UUID" \\  
--env=BITBUCKET_CLONE_DIR="$BITBUCKET_CLONE_DIR" \\  
--env=BITBUCKET_DOCKER_HOST_INTERNAL="$BITBUCKET_DOCKER_HOST_INTERNAL" \\  
--env=DOCKER_HOST="tcp://host.docker.internal:2375" \\  
--env=BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes" \\  
--env=BITBUCKET_PIPE_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/sftp-deploy" \\  
--env=DEBUG="true" \\  
--env=REMOTE_PATH="/home/ubuntu" \\  
--env=SERVER="172.0.1.195" \\  
--env=USER="ubuntu" \\  
--add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL" \\  
bitbucketpipelines/sftp-deploy:0.5.11  
Unable to find image 'bitbucketpipelines/sftp-deploy:0.5.11' locally  
0.5.11: Pulling from bitbucketpipelines/sftp-deploy  
31603596830f: Already exists  
9e006a152763: Pulling fs layer  
8c449c3c01f1: Pulling fs layer  
1bd77f40acd7: Pulling fs layer  
8c449c3c01f1: Verifying Checksum  
8c449c3c01f1: Download complete  
1bd77f40acd7: Verifying Checksum  
1bd77f40acd7: Download complete  
9e006a152763: Verifying Checksum  
9e006a152763: Download complete  
9e006a152763: Pull complete  
8c449c3c01f1: Pull complete  
1bd77f40acd7: Pull complete  
Digest: sha256:9fcd9a8ec73daf0ec51461079b1853fb6b5f93970842f6d80e64160f4b5535ff  
Status: Downloaded newer image for bitbucketpipelines/sftp-deploy:0.5.11  
+ SFTP_DEBUG_ARGS=-v  
+ \[\[ -z '' \]\]  
+ info 'Using SSH.'  
\[36mINFO: Enabling debug mode.\[0m  
\[36mINFO: Using SSH.\[0m  
+ echo -e '\\e\[36mINFO: Using SSH.\\e\[0m'  
+ setup_ssh_dir  
+ INJECTED_SSH_CONFIG_DIR=/opt/atlassian/pipelines/agent/ssh  
+ IDENTITY_FILE=/opt/atlassian/pipelines/agent/ssh/id_rsa_tmp  
+ KNOWN_HOSTS_FILE=/opt/atlassian/pipelines/agent/ssh/known_hosts  
+ mkdir -p /root/.ssh  
+ touch /root/.ssh/authorized_keys  
+ \[\[ -z '' \]\]  
+ '\[' '' '!=' '' '\]'  
+ '\[' '!' -f /opt/atlassian/pipelines/agent/ssh/id_rsa_tmp '\]'  
+ debug 'Using default ssh key'  
+ \[\[ true == \\t\\r\\u\\e \]\]  
+ echo -e '\\e\[37mDEBUG: Using default ssh key\\e\[0m'  
+ cp /opt/atlassian/pipelines/agent/ssh/id_rsa_tmp /root/.ssh/pipelines_id  
\[37mDEBUG: Using default ssh key\[0m  
+ '\[' '!' -f /opt/atlassian/pipelines/agent/ssh/known_hosts '\]'  
+ cat /opt/atlassian/pipelines/agent/ssh/known_hosts  
+ '\[' -f /root/.ssh/config '\]'  
+ \[\[ -z '' \]\]  
+ echo 'IdentityFile \~/.ssh/pipelines_id'  
+ chmod -R go-rwx /root/.ssh/  
+ run_pipe  
\[36mINFO: Starting SFTP deployment to 172.0.1.195:/home/ubuntu...\[0m  
\[37mDEBUG: Executing echo "mput /opt/atlassian/pipelines/agent/build/\*" \| sftp -b - -rp -v ubuntu@172.0.1.195:/home/ubuntu\[0m  
+ info 'Starting SFTP deployment to 172.0.1.195:/home/ubuntu...'  
+ echo -e '\\e\[36mINFO: Starting SFTP deployment to 172.0.1.195:/home/ubuntu...\\e\[0m'  
+ set +e  
+ \[\[ -z '' \]\]  
+ debug Executing echo '"mput' '/opt/atlassian/pipelines/agent/build/\*"' '\|' sftp -b - -rp -v ubuntu@172.0.1.195:/home/ubuntu  
+ \[\[ true == \\t\\r\\u\\e \]\]  
+ echo -e '\\e\[37mDEBUG: Executing echo "mput /opt/atlassian/pipelines/agent/build/\*" \| sftp -b - -rp -v ubuntu@172.0.1.195:/home/ubuntu\\e\[0m'  
+ echo 'mput /opt/atlassian/pipelines/agent/build/\*'  
+ sftp -b - -rp -v ubuntu@172.0.1.195:/home/ubuntu  
OpenSSH_7.9p1, OpenSSL 1.1.1g 21 Apr 2020  
debug1: Reading configuration data /root/.ssh/config  
debug1: Reading configuration data /etc/ssh/ssh_config  
debug1: Connecting to 172.0.1.195 \[172.0.1.195\] port 22.  
debug1: Connection established.  
debug1: identity file /root/.ssh/pipelines_id type -1  
debug1: identity file /root/.ssh/pipelines_id-cert type -1  
debug1: Local version string SSH-2.0-OpenSSH_7.9  
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.9p1 Ubuntu-3  
debug1: match: OpenSSH_8.9p1 Ubuntu-3 pat OpenSSH\* compat 0x04000000  
debug1: Authenticating to 172.0.1.195:22 as 'ubuntu'  
debug1: SSH2_MSG_KEXINIT sent  
debug1: SSH2_MSG_KEXINIT received  
debug1: kex: algorithm: curve25519-sha256  
debug1: kex: host key algorithm: ecdsa-sha2-nistp256  
debug1: kex: server-\>client cipher: chacha20-poly1305@openssh.com MAC: \<implicit\> compression: none  
debug1: kex: client-\>server cipher: chacha20-poly1305@openssh.com MAC: \<implicit\> compression: none  
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY  
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:cBkkJ0ADEjHdkR6jqj+XaNobKRAyefvZ4jp/Ub3mPtA  
Host key verification failed.

Connection closed.  
Connection closed  
\[31m? Deployment failed.\[0m  
+ STATUS=255  
+ set -e  
+ \[\[ 255 == \\0 \]\]  
+ fail 'Deployment failed.'  
+ echo -e '\\e\[31m? Deployment failed.\\e\[0m'  
+ exit 255  
Searching for files matching artifact pattern .bitbucket/pipelines/generated/pipeline/pipes/\*\*

Searching for test report files in directories named \[test-reports, TestResults, test-results, surefire-reports, failsafe-reports\] down to a depth of 4  
Finished scanning for test reports. Found 0 test report files.  
Merged test suites, total number tests is 0, with 0 failures and 0 errors.


Answers:
author: Theodora Boudale
body: Hi [@Mohamed Bakr](/t5/user/viewprofilepage/user-id/4988401) and welcome to the community!

If you open the repository with this pipeline on Bitbucket website, go to its **Repsoitory settings \> SSH keys** , is the address of your server in the **Known hosts** section?

The error "Host key verification failed" indicates that it may not be there.

If it is not there, can you add it and select **Fetch** to fetch the host's fingerprint?  
After it is successfully added, please run this pipeline again and let me know how it goes.

If you still see errors, please copy-paste the new log so we can further investigate.

Kind regards,  
Theodora

comments: author: Mohamed Bakr
body: Thanks Theodora for your update , but I am running this pipeline on self-host agent and for local network we are establishing the communication to our application server that we need to copy files their,

as you see IP address is

```
172.0.1.195
```

which is reside on the same network of our agent , so it's not applicable to add it to know hosts list.


author: Theodora Boudale
body: Hi [@Mohamed Bakr](/t5/user/viewprofilepage/user-id/4988401),

Builds that use Linux runners run in a Docker container on your machine, they do not run directly on your machine.

The SSH keys and *known_hosts* file that you have on your machine with the runner are not accessible and cannot be used by the Docker container that runs your build.

* Can you please confirm how you have set up SSH?  
  Have you generated a new key pair from **Repository settings \> SSH key** ?  
  Have you added your own key pair on that page?  
  Or do you have an SSH key on the Docker image you are using?

<br />

The **\~/.ssh/known_hosts** file inside the Docker container (not the one on your machine) needs to be updated with the public key of the server you want to send files to. The easiest way to do this is by adding the address of the server on the **Known hosts** section in the **Repository settings** of this repo **\> SSH keys**.

Alternatively, you can also try passing an argument to the pipe and set *StrictHostKeyChecking* to *no*, to prevent this error without updating known_hosts. You could do this as follows:

```
- pipe: atlassian/sftp-deploy:0.5.11
  variables:
    USER: 'ubuntu'
    SERVER: '172.0.1.195'
    REMOTE_PATH: '/home/ubuntu'
    DEBUG: 'true'
    EXTRA_ARGS: '-o StrictHostKeyChecking=no'
```

Kind regards,  
Theodora


author: Mohamed Bakr
body: Thanks [@Theodora Boudale](/t5/user/viewprofilepage/user-id/818086) Extra Args solved my problem


author: Igor Stoyanov
body: [@Mohamed Bakr](/t5/user/viewprofilepage/user-id/4988401) hi. Thanks for your question.  

Since you are using self hosted runner, you should try to pass your ssh key directly to the pipe:

```
script:  - pipe: atlassian/ssh-run:0.4.1    variables:      SSH_USER: 'ec2-user'      SERVER: '127.0.0.1'      SSH_KEY: $MY_SSH_KEY      MODE: 'script'      COMMAND: 'myscript.sh' # path to a script in your repository
```

SSH_KEY is a base64 encoded alternate SSH_KEY to use instead of the key configured in the Bitbucket Pipelines admin screens (which is used by default). This should be encoded.

To encode ssh key:

**Linux**  
` ``$ base64 -w 0 < my_ssh_key`

**Mac OS X**  
` ``$ base64 < my_ssh_key`  
Best regards, Igor.

comments: author: Mohamed Bakr
body: [@Igor Stoyanov](/t5/user/viewprofilepage/user-id/4550492)

ssh Pipe is working fine with me , but ftp pipe has a problem , and [@Theodora Boudale](/t5/user/viewprofilepage/user-id/818086) helped me to resolve this issue using extra args to all unkhown host names


Tags: bitbucket-cloud

bitbucket-pipelines

cloud

scp

sftp

ssh