Sync GitHub Repos to Notion
Tags: cron, notion, github, temporal
Published: 2023-06-07 00:50:43

---
title: Sync GitHub Repos to Notion
published: true
description: For a company hackathon, I built an internal tool that syncs your GitHub org's list of repositories to a Notion database. Here's how it works and how I implemented it.
tags: cron, notion, github, temporal
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pux6bghkp2dptzs597p1.png
# Use a ratio of 100:42 for best results.
# published_at: 2023-06-03 04:44 +0000
---

At our last company hackathon, I built an internal tool that syncs your GitHub org's list of repositories to a [Notion database](https://www.notion.so/help/intro-to-databases). It takes this list:

[![List of organization repositories on GitHub](//images.ctfassets.net/0uuz8ydxyd9p/2Kbgml2Ezvi3O1rpOkLblh/89202c83425e50f00e5124a9ec71e48f/repos-on-github.png)](https://github.com/orgs/temporalio/repositories?type=public&language=typescript)

and makes this database in Notion:

![Notion DB with the same repositories](//images.ctfassets.net/0uuz8ydxyd9p/2U90h7mGTnofG9ZoPFU8x3/0acfba827d6287bc17d203c9ee819596/notion-db.png)

The database serves two purposes:

1. Tracking which team owns each repo, and whom to contact if you have a question. A couple of questions that come up are: 
  - "This sample repo is out of date—who is responsible for it?"
  - "A user asked about their issue/PR on this repo—which team is responsible for responding?"

  The two manually updated columns we use for this are "Owner" with team multi-select and "POC" with Person multi-select.
2. Tracking which GitHub teams or individuals have which roles on each repo. It's hard to view all that info on github.com, and having it all in one table makes it much easier. The columns we use for this are:
  - Teams admins
  - Individual admins
  - Teams with role: maintainer
  - Individuals with role: maintainer
  - Teams with role: write
  - Individuals with role: write
  - Teams with role: triage
  - Individuals with role: triage
  - Teams with role: read
  - Individuals with role: read

*Since, by default, everyone in the org has write access to all repos, org members are excluded from the "Individuals with role: write" column to reduce noise.*

## Implementation

### Sync function

The one-way sync is implemented with TypeScript code ([`temporalio/github-repo-notion-sync`](https://github.com/temporalio/github-repo-notion-sync)) that fetches data from the [GitHub API](https://docs.github.com/en/rest?apiVersion=2022-11-28) and then updates the Notion database via Notion's [client library](https://www.npmjs.com/package/@notionhq/client):

[`sync.ts`](https://github.com/temporalio/github-repo-notion-sync/blob/090f0967601157b25329dcfa148db3bc635cc2db/sync.ts#L15-L25)

```ts
export async function syncGithubToNotion() {
  const [repos, teammates] = await Promise.all([getRepos(), getTeammates()])
  const withContributors = await addContributors(repos)
  const completeRepos = await addCollaboratorsAndTeams(
    withContributors,
    teammates
  )
  const transformedRepos = transformRepos(completeRepos)

  await updateNotion(transformedRepos)
}
```

The `syncGithubtoNotion` function:

- Fetches the org's repos ([`getRepos`](https://github.com/temporalio/github-repo-notion-sync/blob/157c21bf061c06e6c527f335bdd20bbc01f7b764/github.ts#L61-L67))
- Fetches the org's members ([`getTeammates`](https://github.com/temporalio/github-repo-notion-sync/blob/157c21bf061c06e6c527f335bdd20bbc01f7b764/github.ts#L69-L74))
- Fetches each repo's contributors ([`addContributors`](https://github.com/temporalio/github-repo-notion-sync/blob/157c21bf061c06e6c527f335bdd20bbc01f7b764/github.ts#L79-L102))
- [Merges](https://github.com/temporalio/github-repo-notion-sync/blob/157c21bf061c06e6c527f335bdd20bbc01f7b764/github.ts#L104-L169) and [transforms](https://github.com/temporalio/github-repo-notion-sync/blob/157c21bf061c06e6c527f335bdd20bbc01f7b764/sync.ts#L27-L40) the above data into the format needed
- Updates the Notion db with the `notion.pages.create` and `notion.pages.update` functions ([`updateNotion`](https://github.com/temporalio/github-repo-notion-sync/blob/090f0967601157b25329dcfa148db3bc635cc2db/notion.ts#L51-L66))

### Running on a schedule

`syncGithubToNotion` runs daily at 6am. The easiest way to set that up would have been a GitHub Action that runs on a schedule:

```yml
name: run daily at 6am

on:
  schedule:
    - cron: '0 6 * * *'

jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - env:
          NOTION_TOKEN: ${{ secrets.NOTION_TOKEN }}
          GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
        run: npm start
```

However, GitHub Action crons aren't very reliable:

[![Theo and swyx tweets on reliability](//images.ctfassets.net/0uuz8ydxyd9p/2a8RtM8VFdUAF7Z2YfCBpA/45a376f71e0b2389f3166699e9b8131b/theo-swyx-tweets.png)](https://twitter.com/swyx/status/1552989710301151233)

So I used a new feature of Temporal called [Schedules](https://temporal.io/blog/temporal-v1-20-lighter-and-simpler-development-environments-and-improved#the-new-way-to-schedule-workflows-in-temporal):

[`create-or-update-schedule.ts`](https://github.com/temporalio/github-repo-notion-sync/blob/cadda19a53f15deae8b375c760ef546dc23afb61/create-or-update-schedule.ts)

```ts
await client.schedule.create({
  action: {
    type: 'startWorkflow',
    workflowType: syncGithubToNotion,
    taskQueue,
    workflowId: 'gh-notion-sync,
  },
  scheduleId: 'gh-notion-sync',
  spec: {
    calendars: [
      {
        comment: 'daily at 6am',
        hour: 6,
      },
    ],
  },
})
```

This reliably runs `syncGithubToNotion` daily at 6am, and since `syncGithubToNotion` is a [durable function](https://temporal.io/blog/building-reliable-distributed-systems-in-node), if any part of it fails, that part will automatically be retried until it succeeds, and then the function will continue. It's run so reliably that even if the machine loses power on the third line of the function, the function will continue to run starting from the third line on another machine. *Perhaps* that's more reliability than I needed for this particular internal tool, but it was a good use case for Temporal Schedules and demo of durable functions, which you can use for your application's backend code if you want it to run extremely reliably! 😃 (For an intro and explanation of how, see [Building Reliable Distributed Systems in Node](https://temporal.io/blog/building-reliable-distributed-systems-in-node).)

---

So that was my project 🤓. It's [open source](https://github.com/temporalio/github-repo-notion-sync) if you'd like to adapt it for your company. Let me know if you have any questions—I'd be happy to help out. I'm [@lorendsr](https://twitter.com/lorendsr) on Twitter and @Loren on our [community Slack](https://t.mp/slack) 💃.