github-backup-sync 0.1.0
Mirror every GitHub repository you can access into local bare git mirrors.
License: MIT
Requires Python: >=3.12

# GitHub Backup Sync

`github_backup_sync.py` mirrors every GitHub repository your account can access into bare `--mirror` clones, grouping sources and forks separately. The project was inspired after watching [ThePrimeTime's reminder about GitHub bans](https://www.youtube.com/watch?v=7gCCXCSs734), highlighting why local backups matter. Plenty of alternative backup tools exist, but I wanted something simple that relies on the GitHub CLI for authentication so I never have to juggle API tokens directly.

<details>
<summary>Table of Contents</summary>

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Requirements](#requirements)
- [Usage](#usage)
  - [CLI Help](#cli-help)
- [Layout](#layout)
- [Working with Mirrors](#working-with-mirrors)
- [Automated Backups (cron)](#automated-backups-cron)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

</details>

## Requirements

- [`uv`](https://docs.astral.sh/uv/latest/) (recommended; the shebang runs the script through `uv run`)
- GitHub CLI (`gh`) authenticated with `gh auth login`
- `git` and, optionally, `git-lfs` if you use the `--lfs` flag

The script also works inside a manual virtual environment if you install the same dependencies listed in the script header, but `uv` provides the fastest startup and dependency management.

## Usage

```bash
./github_backup_sync.py --root /path/to/mirrors [--https] [--prune] [--skip-forks] [--workers 4]
```

- `--root` points at the backup directory.
- `--https` switches from SSH to HTTPS remotes (still using the `gh` token).
- `--prune` removes local mirrors that no longer exist upstream.
- `--skip-forks` mirrors only non-fork repositories.
- `--workers` controls how many repositories sync in parallel (default: CPU-based).
- `--limit` is handy for smoke-testing with only a few repos.

The script automatically fetches a GitHub token from `gh auth token`, so you only need to keep the CLI logged in.

### CLI Help

<details>
<summary><code>github_backup_sync.py --help</code> output</summary>

<!-- CODE:BASH:START -->
<!-- echo '```text' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- ./github_backup_sync.py --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```text

 Usage: github_backup_sync.py [OPTIONS]

 Coordinate the CLI workflow for mirroring repositories.

╭─ Options ──────────────────────────────────────────────────────────────────────────────╮
│ --root              -r                        PATH                Directory that will  │
│                                                                   hold the mirror      │
│ --https                                                           Use HTTPS remotes    │
│                                                                   (token optional for  │
│                                                                   public repos)        │
│ --include-archived      --exclude-archived                        Include archived     │
│                                                                   repositories         │
│                                                                   [default:            │
│                                                                   include-archived]    │
│ --prune                                                           Remove local mirrors │
│                                                                   that no longer exist │
│                                                                   upstream             │
│ --lfs                                                             Fetch Git LFS        │
│                                                                   objects after        │
│                                                                   mirroring            │
│ --sleep                                       FLOAT RANGE         Delay between        │
│                                               [x>=0.0]            repositories in      │
│                                                                   seconds              │
│                                                                   [default: 0.0]       │
│ --limit                                       INTEGER RANGE       Process at most this │
│                                               [x>=1]              many repositories    │
│ --skip-forks                                                      Ignore forked        │
│                                                                   repositories while   │
│                                                                   mirroring            │
│ --workers           -w                        INTEGER RANGE       Max concurrent       │
│                                               [x>=1]              mirror operations    │
│                                                                   (default: based on   │
│                                                                   CPU count)           │
│ --help              -h                                            Show this message    │
│                                                                   and exit.            │
╰────────────────────────────────────────────────────────────────────────────────────────╯

```

<!-- OUTPUT:END -->

</details>

## Layout

```
<root>/source/<owner>/<repo>.git
<root>/forks/<owner>/<repo>.git
```

All repositories are bare mirrors, suitable for backup purposes.

## Working with Mirrors

Repositories under `source/` and `forks/` are bare `--mirror` clones, so they do not contain a checked-out working tree. Clone from a mirror path when you want editable files, for example:

```bash
git clone /srv/github-backups/source/basnijholt/dotfiles.git ~/dotfiles
```

You can also inspect a mirror directly without cloning by pointing Git commands at it, e.g. `git --git-dir=/srv/github-backups/source/basnijholt/dotfiles.git log --oneline`.

## Automated Backups (cron)

You can schedule a daily sync via `cron` after installing the script somewhere on your `$PATH`:

```cron
0 3 * * * /usr/bin/env uv run /opt/github-backup-sync/github_backup_sync.py --root /srv/github-backups --https --prune --workers 4 >> /var/log/github-backup-sync.log 2>&1
```

This example runs every day at 03:00, performs a prune, and logs output. Adjust the path, flags, and worker count to suit your environment.