How to push files programatically to a repository using Octokit with Typescript
Tags: github, api, typescript, octokit
Published: 2019-12-27 02:36:22

I've always liked this word **programatically**, and from time to time I find myself using it when [searching Google](https://www.google.com/search?q=how+to+programatically&oq=how+to+programatically&aqs=chrome..69i57l2j69i59j69i60l2j69i61j69i60l2.1323j0j7&sourceid=chrome&ie=UTF-8) for a really specific problem. Often, doing something "programatically" leads to an elegant solution for some real-world problem or saves you some time automating a task.

This last week I was set up to develop a product that had dealing with GitHub _repos_ as one of its challenges. I had already worked with GitHub's API before, but mostly for Github Actions and some CI stuff, and this was a bit different, since, as its core, was Git, and Git is [not simple](https://developer.ibm.com/tutorials/d-learn-workings-git/).

Basically, I had to **create a repository** if it didn't exist and **push _n_ files** into it. At first, I started testing my guesses of how to do it using the [GitHub API](https://developer.github.com/v3/) directly and a **personal access token** as an auth method. After that, I started coding the solution using GitHub's [Octokit/REST](https://github.com/octokit/rest.js), a cool wrapper for its API that supports Typescript and it will help you a lot.

> One thing that I have to mention is that this was one of the first times that using Typescript really sped up my work. After **Step 2** I didn't stop for testing and, after finishing it, I was pretty sure that I would've missed something but the solution worked seamlessly at first try.

So, one of my first concerns was **what do I have to do to mimic a `git add . && git commit -m "My commit" && git push`** using the API? After a bit of Googling, I wrote on my board:

_**Note:** Most of the referencing here uses SHA, a **hash of the object** within Git. That "code" that every commit has and you see on `git log` is the commit's SHA._

1. Get the `ref` of the branch you wanna push files to. You'll want the **SHA of the last commit** [doc]
(https://octokit.github.io/rest.js/#octokit-routes-git-get-ref)

2. With the _commit's_ SHA, you will fetch for that _commit's_ **tree**, which is the data structure that actually holds the files. You'll want its SHA too [doc](https://octokit.github.io/rest.js/#octokit-routes-git-get-commit)

3. After this, you will need the **list of files** you want to upload, this includes the **files' paths and its contents**. With those you will **create a blob** for each file. In my case, I was dealing with `.md`s and `.yaml`s, so I've used `utf8` encode and sent each file content's as a plain text that I got from `fs.readFile(path, 'utf8')`, but there're other options for working with binaries, submodules, empty directories and symlinks. Oh, also: **don't worry about files inside directories**, it's just about sending the path on the next step for each file that GitHub will organize them accordingly (that's was one of the things I was afraid that I would've to deal with it manually) [doc](https://octokit.github.io/rest.js/#octokit-routes-git-create-blob)

4. With all the SHA's from the blobs created on **Step 3**, you'll **create the new tree** for the repo with those files. It's here where you'll **link the blobs to the paths**, and all of them together to the tree. There are some **modes** that you can read about it on the docs, but I've only used `100644` for regular files. You'll also have to **set the SHA retrieved on Step 2** as a parent tree for this one you're creating [doc](https://octokit.github.io/rest.js/#octokit-routes-git-create-tree)

5. With the tree (and its SHA) containing all the files, you'll need to **create a new commit** pointing to such tree. This will be the commit holding all of your changes for the _repo_. It will also have to have **the commit SHA got from Step 1** as its parent commit. Notice this is a common pattern on Git. It's here where you **set the commit message** too [doc](https://octokit.github.io/rest.js/#octokit-routes-git-create-commit)

6. And, finally, you **set the branch ref** to point to the commit created on the last step, using the commit's returned SHA [doc](https://octokit.github.io/rest.js/#octokit-routes-git-update-ref)

Done!

One of the problems that I've also had was that **I had to create the repository** in some cases, and there was no "initial" commit for me to work. After, I found that you can send a `auto_init` _param_ when creating a repository that GitHub will automatically create a simple README.md initializing the repo. [Dodged a bullet](https://developer.github.com/changes/2012-09-28-auto-init-for-repositories/) there.

So, let's go to the code. 

```typescript
import Octokit from '@octokit/rest'
import glob from 'globby' 
import path from 'path'
import { readFile } from 'fs-extra'

const main = async () => {
  // There are other ways to authenticate, check https://developer.github.com/v3/#authentication
  const octo = new Octokit({
    auth: process.env.PERSONAL_ACESSS_TOKEN,
  })
  // For this, I was working on a organization repos, but it works for common repos also (replace org for owner)
  const ORGANIZATION = `my-organization`
  const REPO = `my-repo`
  const repos = await octo.repos.listForOrg({
    org: ORGANIZATION,
  })
  if (!repos.data.map((repo: Octokit.ReposListForOrgResponseItem) => repo.name).includes(REPO)) {
    await createRepo(octo, ORGANIZATION, REPO)
  }
  /**
   * my-local-folder has files on its root, and subdirectories with files
   */
  await uploadToRepo(octo, `./my-local-folder`, ORGANIZATION, REPO)
}

main()

const createRepo = async (octo: Octokit, org: string, name: string) => {
  await octo.repos.createInOrg({ org, name, auto_init: true })
}

const uploadToRepo = async (
  octo: Octokit,
  coursePath: string,
  org: string,
  repo: string,
  branch: string = `master`
) => {
  // gets commit's AND its tree's SHA
  const currentCommit = await getCurrentCommit(octo, org, repo, branch)
  const filesPaths = await glob(coursePath)
  const filesBlobs = await Promise.all(filesPaths.map(createBlobForFile(octo, org, repo)))
  const pathsForBlobs = filesPaths.map(fullPath => path.relative(coursePath, fullPath))
  const newTree = await createNewTree(
    octo,
    org,
    repo,
    filesBlobs,
    pathsForBlobs,
    currentCommit.treeSha
  )
  const commitMessage = `My commit message`
  const newCommit = await createNewCommit(
    octo,
    org,
    repo,
    commitMessage,
    newTree.sha,
    currentCommit.commitSha
  )
  await setBranchToCommit(octo, org, repo, branch, newCommit.sha)
}


const getCurrentCommit = async (
  octo: Octokit,
  org: string,
  repo: string,
  branch: string = 'master'
) => {
  const { data: refData } = await octo.git.getRef({
    owner: org,
    repo,
    ref: `heads/${branch}`,
  })
  const commitSha = refData.object.sha
  const { data: commitData } = await octo.git.getCommit({
    owner: org,
    repo,
    commit_sha: commitSha,
  })
  return {
    commitSha,
    treeSha: commitData.tree.sha,
  }
}

// Notice that readFile's utf8 is typed differently from Github's utf-8
const getFileAsUTF8 = (filePath: string) => readFile(filePath, 'utf8')

const createBlobForFile = (octo: Octokit, org: string, repo: string) => async (
  filePath: string
) => {
  const content = await getFileAsUTF8(filePath)
  const blobData = await octo.git.createBlob({
    owner: org,
    repo,
    content,
    encoding: 'utf-8',
  })
  return blobData.data
}

const createNewTree = async (
  octo: Octokit,
  owner: string,
  repo: string,
  blobs: Octokit.GitCreateBlobResponse[],
  paths: string[],
  parentTreeSha: string
) => {
  // My custom config. Could be taken as parameters
  const tree = blobs.map(({ sha }, index) => ({
    path: paths[index],
    mode: `100644`,
    type: `blob`,
    sha,
  })) as Octokit.GitCreateTreeParamsTree[]
  const { data } = await octo.git.createTree({
    owner,
    repo,
    tree,
    base_tree: parentTreeSha,
  })
  return data
}

const createNewCommit = async (
  octo: Octokit,
  org: string,
  repo: string,
  message: string,
  currentTreeSha: string,
  currentCommitSha: string
) =>
  (await octo.git.createCommit({
    owner: org,
    repo,
    message,
    tree: currentTreeSha,
    parents: [currentCommitSha],
  })).data

const setBranchToCommit = (
  octo: Octokit,
  org: string,
  repo: string,
  branch: string = `master`,
  commitSha: string
) =>
  octo.git.updateRef({
    owner: org,
    repo,
    ref: `heads/${branch}`,
    sha: commitSha,
  })
```
https://gist.github.com/luciannojunior/864849a7f3c347be86862a3a43994fe0

Feel free to reach me for any questions :)