Copy Files from One Repo to Another Automatically with GitHub Actions
Tags: github, git, bash, devops
Published: 2023-08-15 13:57:57

---
title: Copy Files from One Repo to Another Automatically with GitHub Actions
published: true
date: 2023-08-13 00:00:00 UTC
tags: github,git,bash,devops
canonical_url: https://www.paigeniedringhaus.com/blog/copy-files-from-one-repo-to-another-automatically-with-git-hub-actions
---

[![Assembly line making identical pink shoes](https://www.paigeniedringhaus.com/static/bb9a4c12a3ec59c59c10ac75f4aaa89b/1e043/assembly-line-hero.png "Assembly line making identical pink shoes")](/static/bb9a4c12a3ec59c59c10ac75f4aaa89b/5a190/assembly-line-hero.png)

## Introduction

Welcome to the third (and final) installment in my [series](https://www.paigeniedringhaus.com/blog/use-git-hub-actions-to-automatically-publish-a-repo-subfolder-as-an-npm-library) about the many things I learned while building my first open source API library for [Blues](https://blues.io/?&utm_source=dev.to&utm_medium=web&utm_campaign=nf&utm_content=notehub-js), the Internet of Things startup I work for.

It was a good challenge and a great learning opportunity for me.

I automated as many pieces of the library's maintenance and deployment to as I could with the help of GitHub Actions workflows, one of them being: copy the `openapi.yaml` file from the Blues cloud repo, [Notehub](https://notehub.io/?&utm_source=dev.to&utm_medium=web&utm_campaign=nf&utm_content=notehub-js), whenever that file was updated, and push it into a feature branch in the [Notehub JS repo](https://github.com/blues/notehub-js) (the name of the API library).

Interacting with two different repos in one workflow might sound tricky, and at first it was, but it _is_ possible and not too bad once you see how it can be done.

**In this tutorial, I'll demonstrate how to create a GitHub Actions workflow that automatically copies a file from one repo to another and pushes the changes to another repo inside a feature branch.**

I know the use case sounds a bit specific, but it came in really handy for me, and you never know when it might for you too.

---

## Notehub JS

Before we get to the actual GitHub Actions workflows, I'll give you a little background on the [**Notehub JS**](https://github.com/blues/notehub-js) project.

> This section outlines the folder structure for the repo in case you want to explore it in GitHub, if you just want to get to the solutions, feel free to jump down to the next section.

Notehub JS is a JavaScript-based library for interacting with the native [**Notehub API**](https://dev.blues.io/reference/notehub-api/api-introduction/?&utm_source=dev.to&utm_medium=web&utm_campaign=nf&utm_content=notehub-js), and it is actually generated from the Notehub project's own `openapi.yaml` file, which follows the [OpenAPI specification](https://swagger.io/specification/) standards.

The [OpenAPI Generator CLI](https://openapi-generator.tech/docs/installation) is a tool that uses the `openapi.yaml` file to create an entire API library complete with documentation, models, endpoints, and scripts to package it up for publishing as an npm module. The end library that is published to npm is a _subfolder_ inside of the main Notehub JS repo. At the root of the project are the `openapi.yaml` file, the GitHub Actions workflows, and a few other config files.

Here's a simplified view of the [Notehub JS repo's](https://github.com/blues/notehub-js) folder structure:

```
root/
├── .github/ 
| ├── workflows/
| | ├── create-pr.yml 
| ├── PULL_REQUEST_TEMPLATE.md
├── src/ <- this is the folder generated by the OpenAPI Generator CLI
| ├── dist/
| ├── docs/
| ├── src/ 
| | ├── api/ 
| | ├── model/
| | ├── index.js 
| openapi.yaml
| config.json
| package.json
```

Based on the diagram above, the `openapi.yaml` file that the library is generated from lives at the root of the repo, and the `src/` folder is what holds all the Notehub API JavaScript code that powers the [Notehub JS library](https://www.npmjs.com/package/@blues-inc/notehub-js).

This `openapi.yaml` file gets copied from the Notehub repo when changes are made to it. And changes are made to that file whenever the Notehub API is updated with new features and functionality, so making sure that the Notehub JS library keeps up with the new additions to the original API it's based on is important.

Now that I've explained more about the Notehub JS repo and why keeping it in sync with the Notehub API is important (and depends on the `openapi.yaml` file), we can get down to the business of ensuring the file gets copied to this repo whenever it's updated.

> **Looking for more details about Notehub JS?**
> 
> Check out my [first blog post](https://www.paigeniedringhaus.com/blog/use-git-hub-actions-to-automatically-publish-a-repo-subfolder-as-an-npm-library) about how to use GitHub Actions to automatically publish new releases to npm - I do a fairly deep dive on Notehub JS there.

### Create a GitHub Actions workflow to run when the `openapi.yaml` file is updated

First things first: creating a new GitHub Actions workflow that's triggered when the `openapi.yaml` file changes in the Notehub repo.

> **Need a refresher on GitHub Actions?**
> 
> If you want a quick primer on what GitHub Actions are, I recommend you check out a previous article I wrote about them [here](https://www.paigeniedringhaus.com/blog/use-secret-environment-variables-in-git-hub-actions#what-is-github-actions).

Below is the final version of the `copy-openapi-file.yml` file, which lives inside of the `./github/workflows/` folder. I'll walk through each step of the script below.

**`.github/workflows/copy-openapi-file.yml`**

```yaml
name: Copy updated OpenAPI file

# only run this workflow when the openapi file has changed on the master branch
on: 
  push:
    branches: 
      - master
    paths:
      - 'notehub/api/openapi.yaml'

jobs:
  copy_openapi_file_to_notehub_js: 
    runs-on: ubuntu-latest
    steps:
    # check out notehub project
    - name: Check out Notehub project
      uses: actions/checkout@v3
    # check out notehub-js project using token generated in that project to successfully access it from inside the Notehub GitHub Action workflow
    - name: Check out Notehub JS project
      uses: actions/checkout@v3
      with:
        repository: blues/notehub-js
        path: ./notehub-js
        token: ${{ secrets.NOTEHUB_JS_TOKEN }}
    # make a copy the openapi file from notehub project
    - name: Copy OpenAPI file
      run: bash ./.github/scripts/copy-openapi-file.sh
      env:
        DESTINATION_PATH: ./notehub-js/
    # make a branch in notehub-js repo and push the copy of the openapi file there
    - name: Push to notehub-js repo
      run: bash ./.github/scripts/push-openapi-to-notehub-js.sh
      env:
        BRANCH: feat-openapi-update
```

Each GitHub Actions workflow needs a **`name`** , and I try to choose descriptive ones like this one: `Copy updated OpenAPI file`.

Then, I want this workflow to only run when the `openapi.yaml` file in the `master` branch of the project is changed. `master` is the branch that gets deployed to production, and in my case it's a safe assumption that when there's a change to this file in `master`, those changes are going to prod, which is when I want to copy the file over to the Notehub JS repo so it can reflect those same changes.

The code snippet below does just that.

```yaml
on: 
  push:
    branches: 
      - master
    paths:
      - 'notehub/api/openapi.yaml'
```

- [**`on`**](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#on) is how a workflow is triggered.
- [**`push`**](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushbranchestagsbranches-ignoretags-ignore) is the event that triggers the workflow.
- [**`branches`**](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-including-branches-and-tags) defines which branch names this workflow should run for.
- [**`paths`**](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-including-paths) is another filter to even more finely determine when the workflow runs.

What it boils down to is: when there's a change to the `master` branch in the repo for the `openapi.yaml` file at this particular file path, run the GitHub Actions workflow.

From there, the `copy_openapi_file_to_notehub_js` job runs. There's only one job in the **`jobs`** section of this script, but if there were multiple jobs, they'll run sequentially.

The `create_pr_repo_sync` job defines that it runs on the latest version of Ubuntu in **`runs-on`**.

And finally, I get to the **`steps`** section. Here's how it goes:

1. Check out the Notehub code so the workflow can access it using the GitHub Action [`actions/checkout@v3`](https://github.com/actions/checkout).
2. Check out the Notehub JS code so the workflow can access it as well using the GitHub Action [`actions/checkout@v3`](https://github.com/actions/checkout).
  - **NOTE** : Notice that in this second checkout action I had to include a [`with`](https://github.com/actions/checkout#checkout-multiple-repos-side-by-side) parameter that allows me to specify another repo to check out by providing a `repository`, `path`, and `token`. The `token` is a GitHub [Personal Access Token (PAT)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) required when checking out other libraries beyond the current one the workflow is operating within.
3. Run a shell script named `copy-openapi-file.sh` with the [GitHub Actions environment variable](https://docs.github.com/en/actions/learn-github-actions/variables#defining-environment-variables-for-a-single-workflow) `DESTINATION_PATH` passed to the script.
4. Run another shell script named `push-openapi-to-notehub-js.sh` with the [GitHub Actions environment variable](https://docs.github.com/en/actions/learn-github-actions/variables#defining-environment-variables-for-a-single-workflow) `BRANCH` passed to the script.

In the following sections I'll go through what those two shell scripts do in detail.

Ok, so the workflow file is done. Time to get into the details of the two scripts doing the heavy lifting.

### Write one shell script to copy the `openapi.yaml` file to the Notehub JS repo

Premade GitHub Actions can do many things, but something as specific as copying a file from one repo to another is a bit beyond them.

Luckily, GitHub Action workflows can [run shell scripts](https://docs.github.com/en/actions/learn-github-actions/essential-features-of-github-actions#adding-scripts-to-your-workflow), so if you can use a scripting language like PowerShell or Bash, you're in luck.

Here is the Bash script I wrote to copy the file from the Notehub repo to the Notehub JS repo.

**`copy_openapi_file.sh`**

```bash
#!/usr/bin/env bash
# bash boilerplate
set -euo pipefail # strict mode
readonly SCRIPT_NAME="$(basename "$0")"
readonly SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
function l { # Log a message to the terminal.
    echo
    echo -e "[$SCRIPT_NAME] ${1:-}"
}

# File to copy from Notehub
OPENAPI_FILE=./notehub/api/openapi.yaml

# if the file exists in Notehub, copy it to Notehub-JS repo
if [-f "$OPENAPI_FILE"]; then
    echo "Copying $OPENAPI_FILE"
    cp -R ./notehub/api/openapi.yaml $DESTINATION_PATH
fi

echo "OpenAPI file copied to $DESTINATION_PATH"
```

The 9 lines at the top of the file are Bash script set up, so don't worry too much about what's happening there.

The first line to pay attention to is where the `OPENAPI_FILE` variable is defined with a path to the `openapi.yaml` file in the Notehub repo.

After identifying the file to copy from the Notehub repo, a standard [Bash `if` statement](https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html) checks if the file exists in the Notehub repo (just to be safe), and if it does, it copies the file recursively using `cp -R` to the `$DESTINATION_PATH` variable that was provided by the GitHub Actions environment variable.

> In this case, the `$DESTINATION_PATH` is pointing to the root folder of the Notehub JS project where the file should be copied to, but you could specify it copy the file anywhere within the receiving repo that makes sense.

During the copying process and when the script finishes copying the file I added a few [`echo` commands](https://ss64.com/bash/echo.html) to print out useful messages in the logs to let me know things are going to plan.

And that's the first shell script that actually copies the file that will be taken from one repo to the other. I hope it's not too complicated once the actions in the script are broken down.

Now, it's time to add that file to the Notehub JS repo.

### Use a second shell script to push the copied file to a feature branch in the other repo

When I checked the [GitHub Marketplace](https://github.com/marketplace) for a ready-made action to create a new feature branch in the repo receiving the copied file, there were a few to choose from, but I wanted more control over what was happening than they offered.

I wanted an action that would:

1. Create a new branch in the Notehub JS receiving repo if it didn't exist, add the copied file, and then push that branch to GitHub.
2. Check out the already existing branch in the Notehub JS repo, overwrite any previous file changes in the branch, and then push the new file changes in that branch back to GitHub.

Here's the bash script that I came up with to handle those two scenarios.

**`push-openapi-to-notehub-js.sh`**

```bash
#!/usr/bin/env bash
# bash boilerplate
readonly SCRIPT_NAME="$(basename "$0")"
readonly SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
function l { # Log a message to the terminal.
    echo
    echo -e "[$SCRIPT_NAME] ${1:-}"
}

# move to the root the notehub-js repo
cd "./notehub-js"
echo "Open root of Notehub JS repo"

# check if there's already a currently existing feature branch in notehub-js for this branch
# i.e. the altered openapi.yaml file's already been copied there at least once before
echo "Check if feature branch $BRANCH already exists in Notehub JS"
git ls-remote --exit-code --heads origin $BRANCH >/dev/null 2>&1
EXIT_CODE=$?
echo "EXIT CODE $EXIT_CODE"

if [[$EXIT_CODE == "0"]]; then
  echo "Git branch '$BRANCH' exists in the remote repository"
  # fetch branches from notehub-js
  git fetch
  # stash currently copied openapi.yaml
  git stash
  # check out existing branch from notehub-js
  git checkout $BRANCH 
  # overwrite any previous openapi.yaml changes with current ones
  git checkout stash -- .
else
  echo "Git branch '$BRANCH' does not exist in the remote repository"
  # create a new branch in notehub-js 
  git checkout -b $BRANCH
fi

git add -A .
git config user.name github-actions
git config user.email github-actions@github.com
git commit -am "feat: Update OpenAPI file replicated from Notehub"
git push --set-upstream origin $BRANCH

echo "Updated OpenAPI file successfully pushed to notehub-js repo"
```

Once more, the first 9 lines of code are boilerplate for Bash; no comments necessary.

The first thing this script does is situate itself in the root of the Notehub JS folder just as you would when navigating around a computer via the command line: `cd "./notehub-js"`.

Once there, it checks if the feature branch environment variable passed in by the GitHub Action `$BRANCH` already exists in the Notehub JS repo with this one-liner.

```bash
git ls-remote --exit-code --heads origin $BRANCH >/dev/null 2>&1
```

Essentially, this code is:

- Getting a list of references in the remote repository with `git ls-remote`,
- Checking if env var `$BRANCH` (passed from the GitHub Action) exists _only_ in the `refs/heads` of the origin remote with `--heads`
  - (i.e. does the branch name already exist in the GitHub repo where it keeps track of the names of all of its branches),
- And returning an `--exit-code` with status `"2"` when no matching refs are found in the remote repo and a status of `"0"` if a matching ref is found.

When the exit code is returned, it's set equal to the variable `$EXIT_CODE`.

Then, an `if/else` statement checks what `$EXIT_CODE` equals.

**EXIT\_CODE = "0"**

If the exit code is `0`, that means the branch already exists in the repository; this could happen if multiple changes are made to the `openapi.yaml` file before the feature branch in Notehub JS gets merged in and deleted.

When that happens the following steps take place:

1. The branches for the Notehub JS repo are fetched via `git fetch`,
2. The current changes (the freshly copied `openapi.yaml` file) are stashed with `git stash`,
3. The existing branch of the same name as the `$BRANCH` env var is checked out locally,
4. The stashed changes are applied to that branch overwriting whatever was there before with `git checkout stash -- .`.

**EXIT\_CODE = "2"**

If `$EXIT_CODE` is equal to `2`, that means the branch doesn't already exist remotely in GitHub and the script can simply checkout a new branch with the name in `$BRANCH` via `git checkout -b $BRANCH`.

Finally, after all this is completed, a standard set of git commands follows:

1. Use `git-add -A .` to stage all changes in the working directory,
2. Configure a username and email address for who's doing the commit with `git config user.name` and `git config user.email`,
3. Add a commit message with `git commit -am "Standard commit message here"`,
4. Lastly, push it to GitHub with `git push --set-upstream origin $BRANCH`.

At this point, you should be good to go. The file should be successfully copied over to the other repository.

> **NOTE:** Inspiration for this second Bash script was initially based on this [blog post](https://remarkablemark.org/blog/2022/09/25/check-git-branch-exists-in-remote-repository/) by Remarkable Mark.

### Check that the workflow runs

The simplest way to test this new functionality is just to keep an eye out for changes in the main branch of the repo that would trigger the workflow to run and see if it works.

When you name the workflow a descriptive name, it's easy to find it amongst the various GitHub Action workflows the repo may have.

Here's a screenshot of the workflow runs of the `Copy updated OpenAPI file` job inside the GitHub repo's **Actions** page.

[![Multiple runs of a particular workflow in the GitHub Actions page of a repository](https://www.paigeniedringhaus.com/static/485b1572d0292816fc45e4be360de478/1e043/gh-actions-workflow.png "Multiple runs of a particular workflow in the GitHub Actions page of a repository")](/static/485b1572d0292816fc45e4be360de478/e9d87/gh-actions-workflow.png)

And if you want to dig in further to any of the runs to check the logs and job steps (especially handy for debugging purposes), just click on a workflow run, then the name of the job, and inside the job you can expand out each step.

Here's another screenshot of the `Push to notehub-js repo` step expanded to see exactly what the Bash script in that step is logging to the console via `echo`.

[![Details of a particular step inside of a GitHub Actions job expanded for review](https://www.paigeniedringhaus.com/static/d1cc8912c1d8339500c3d985ad375a14/1e043/gh-actions-job-details.png "Details of a particular step inside of a GitHub Actions job expanded for review")](/static/d1cc8912c1d8339500c3d985ad375a14/6edca/gh-actions-job-details.png)

And you're done! The workflow's running, the steps are successfully completing, and, most importantly, the file from the Notehub repo is successfully copied to the Notehub JS repo. Mission accomplished.

---

## Conclusion

My first foray into building an open source software library on behalf of one of my company's APIs was a unique challenge that taught me a lot. Especially as I got to leverage GitHub Action workflows to automate as many parts of the process as I could to make ongoing maintenance and upkeep easier.

One such task that I automated involved copying a file from one repo into another whenever a change was made to that particular file. And while I was able to leverage a few pre-existing GitHub Actions, I also had to write a couple custom Bash scripts for some of the trickier, more-unique-to-my-use-case steps.

Fortunately, GitHub Action workflows allows for both types of steps to be combined in a workflow, making for some very powerful, turnkey solutions that did just what I needed.

Check back in a few weeks — I’ll be writing more about JavaScript, React, IoT, or something else related to web development.

If you’d like to make sure you never miss an article I write, sign up for my newsletter here: https://paigeniedringhaus.substack.com

Thanks for reading. I hope learning to copy a file from one repo to another automatically with GitHub Actions workflows is as helpful for you as it has been for me. Enjoy!

---

## References & Further Resources

- Previous OSS article [using GH Actions to publish a package to npm](./use-git-hub-actions-to-automatically-publish-a-repo-subfolder-as-an-npm-library)
- Previous OSS article [using GH Actions to create a new PR branch](./automatically-create-a-pull-request-against-a-feature-branch-with-git-hub-actions)
- [Notehub JS](https://github.com/blues/notehub-js) GitHub repo