Github and Gitlab sync
Tags: github, devops, tooling, git
Published: 2021-03-21 03:02:46

Both Github and Gitlab are great ways to host your git repositories online. They both have great tools to manage your project, be it open source or private code.
I'm not going to compare both services in depth since many guides already do so. For me, the general difference between Github and Gitlab is Gitlab is focusing more effort on CI/CD pipelines and tools for companies whereas Github is focused more on the community and creating a platform to share open-source code.
Both can be used for either scenario... which makes it really hard to choose between them!

But what if I told you you didn't have to choose O_o?

## Why I want to use both Gitlab and Github
*Just a quick disclaimer, this is the method I use. It works for me but might not work for everyone. I recommend experimenting with things for yourself, do some tests.*
*I'm sharing my experience because I feel that it could be useful for someone in the same position as me.*

I like to be able to organize my repositories into projects (or groups in Gitlab). Unfortunately, Github doesn't do this.
I also want to be able to reach as many people as possible with my repositories, Github has the largest community.
Damn it, I'm stuck, I want features from both providers.

Luckily for me, Gitlab has two things that make symbiosis between the two tools possible:
1. Github repository import 
This makes importing a repository from Github super easy
2. Repository synchronization from Gitlab -> Github (for free)
This allows me to use Gitlab as the main source of truth but also have the repository on Github. I can use Github's issue tracker, wiki and forum (Github discussion)
3. Great CI/CD (ok, I lied when I said two things)

# Sync Gitlab to Github (easy)
This one is simple. You could set up some CI/CD yourself. But Gitlab will automatically do this for you. 
1. Go to "Settings > Repository > Mirroring repositories"
2. Enter your Github repo with your username in front `https://<github username>@github.com/path/to/your/repo.git`
3. In the password field, enter your Github token
4. Select push (this requires a subscription)
5. Press Mirror repository

From now on, changes to Gitlab will be mirrored to Github

# Sync Github to Gitlab (intermediate unless you pay)
There are two ways to do this (both are equivalent just from opposite platforms)
1. Either you set up a CI/CD pipeline on Github that triggers using a webhook (on push and pull requests).
2. Either you set up a CI/CD pipeline on Gitlab that triggers using a webhook (on push and pull requests). **Turns out this is actually a premium feature... You cannot use the pull webhook without paying now**

I personally chose to do this from the Github side since the Github method is free
## Method 1 Github CI/CD
1. Create the following secrets in your Github repository
	1. `TARGET_URL` value: the URL of the Gitlab repository
	2. `TARGET_TOKEN` value: Gitlab token
	3. `TARGET_USERNAME` value: Gitlab username
2. Create a Github action for your repository with the following code:
```yml
name: GitlabSync

on:
  - push
  - delete

jobs:
  sync:
    runs-on: ubuntu-latest
    name: Git Repo Sync
    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0
    - uses: wangchucheng/git-repo-sync@v0.1.0
      with:
        # Such as https://github.com/wangchucheng/git-repo-sync.git
        target-url: ${{ secrets.TARGET_URL }}
        # Such as wangchucheng
        target-username: ${{ secrets.TARGET_USERNAME }}
          # You can store token in your project's 'Setting > Secrets' and reference the name here. Such as ${{ secrets.ACCESS\_TOKEN }}
        target-token: ${{ secrets.TARGET_TOKEN }}
```

3. Make sure the branch you are pushing to in Gitlab is not protected or allows for force push

## Method 2 Gitlab CI/CD (requires Gitlab premium)
Note that if you have a subscription with Gitlab, you can also do this in a few clicks in the same dashion as how we set up syncing from Gitlab to Github. 

Go to "Settings > Repository > Mirroring repositories"
2. Enter your Github repo with your username in front `https://<github username>@github.com/path/to/your/repo.git`
3. In the password field, enter your Github token
4. Select pull (this requires a subscription)
5. Press Mirror repository

### Manual Setup

#### Setup the webhooks
If you created the Gitlab project via the Github import tool then you can completely skip this step.

1. In your Github repository, go to "Settings > Webhooks"
2. Create a new webhook
	- Payload path: `https://gitlab.com/api/v4/projects/<gitlab project id>/mirror/pull` (the project id can be found in "Settings > General")
	- Content Type: `application/json`
	- SSL Verification: off
	- Only send for: push and pull
3. Activate

1. In the Gitlab project, go to "Settings > Webhooks"
2. select **Push events** from the repository and enter the Github repository (with a token if it's private)

#### Setup the CI/CD Variables
1. In the Gitlab project, go to "Settings > CI/CD > Variables"
2. Create a variable with key: `ACCESS_TOKEN` and value: <your **Gitlab** token> (make it hidden because this is sensitive)
	1. Create a variable with key: `REMOTE_REPOSITORY_URL` and value: `<your Github token>@<your repository URL>` (make it hidden because this is sensitive)

#### Setup the CI/CD
1. Create a pipeline by either going to "CI/CD > Editor" or by creating a file `.gitlab-ci.yml` in your project root and add the following content:
```yaml
sync-with-github:
  before_script:
    - git config --global user.name "${GITLAB_USER_NAME}"
    - git config --global user.email "${GITLAB_USER_EMAIL}"
  script:
    - git remote add github $REMOTE_REPOSITORY_URL
    - git checkout master
    - git pull origin master
    - git pull github master
    - git status
    - git push https://root:$ACCESS_TOKEN@$CI_SERVER_HOST/$CI_PROJECT_PATH.git HEAD:master
```

# Caveats
This doesn't come without downsides. Here are a few I have found while testing this setup.
- Wikis, Issues, Discussions, boards, etc. will not sync between the two hosting providers. This will only sync what is in your repository. There are ways to synchronize the Wikis ( The Wiki is essentially a hidden .git repository and both Github and Gitlab use the same Wiki format )
- It can get confusing to have two remote repositories at the same time
- They can get out of sync if multiple commits happen at two locations at once and create conflicts (I only use this on repositories I and only I use)