TIL different ways to clone multiple git repos at once
Tags: todayilearned, github, gitlab, linux
Published: 2023-05-19 20:35:51

---
title: TIL different ways to clone multiple git repos at once
published: true
date: 2023-03-22 09:03:00 UTC
tags: TIL,github,gitlab,linux
canonical_url: https://www.zufallsheld.de/2023/03/22/til-different-ways-to-clone-multiple-git-srepos-at-once/
---

As someone who frequently works with GitHub and GitLab repositories, I often need to get an overview of groups. At my workplace, we use GitLab’s group- and subgroup function to organize our team-specific and customer-specific structures.

Recently, I needed to clone all 27 repositories in our microservice-project. Doing this manually would require me to copy the URL for each repository and run “git clone \$URL” on the command line, which is time-consuming and tedious.

Fortunately, I discovered a tool called [ghorg](https://github.com/gabrie30/ghorg) that simplifies this process. With just one command, I was able to clone entire GitHub organizations or GitLab (sub)groups. Here’s the command I used:

```
ghorg clone customergroup/services --base-url=https://git.example.com --scm=gitlab --token=$TOKEN --preserve-dir -p .

```

This command clones all repositories in the group `customergroup/services`, connecting to our private GitLab instance, using the specified token `$TOKEN` to authenticate. The `--preserve-dir` option clones the repos in a directory structure that matches GitLab groups, so repository `foo` will be cloned into `customergroup/services/foo`.

```
ghorg clone customergroup/services --base-url=https://git.example.com --scm=gitlab --token=$TOKEN --preserve-dir -p .

 +-+-+-+-+ +-+-+ +-+-+-+-+-+
 |T|I|M|E| |T|O| |G|H|O|R|G|
 +-+-+-+-+ +-+-+ +-+-+-+-+-+

*************************************
* SCM : gitlab
* Type : org
* Protocol : https
* Location : ./
* Concurrency : 25
* Base URL : https://git.example.com
* Preserve Dir : true
* Config Used : none
* Ghorg version : v1.9.3
*************************************
27 repos found in customergroup/services

Success cloning repo: https://git.example.com/customergroup/services/foo.git -&gt; branch: main
Success cloning repo: https://git.example.com/customergroup/services/foo-audit-report.git -&gt; branch: master
Success cloning repo: https://git.example.com/customergroup/services/foo-tools.git -&gt; branch: main
Success cloning repo: https://git.example.com/customergroup/services/foo-base.git -&gt; branch: main
Success cloning repo: https://git.example.com/customergroup/services/foo-pages.git -&gt; branch: main
Success cloning repo: https://git.example.com/customergroup/services/foo-helm.git -&gt; branch: master
Success cloning repo: https://git.example.com/customergroup/services/foo-application-base.git -&gt; branch: master

```

But I’m also a great fan of the [gitlab-cli](https://gitlab.com/gitlab-org/cli/) and it has a similar feature, too!

Running the following command does the same as ghorg does:

```
GITLAB_HOST=git.example.com glab repo clone --group customergroup/services -p

```

The syntax is slightly different. With `--group $GROUP` you define the group you want to clone; with `-p` you preserve the group-structure.

```
&gt; GITLAB_HOST=git.example.com glab repo clone --group customergroup/services -p
Cloning into 'customergroup/services/foo'...
remote: Enumerating objects: 59, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 59 (delta 0), reused 0 (delta 0), pack-reused 54
Receiving objects: 100% (59/59), 13.36 KiB | 1.03 MiB/s, done.

```

Both tools produce the same folder structure with the cloned repositories:

```
&gt; tree -d -L 3
.
└── customergroup
    └── services
        ├── foo
        ├── foo-audit-report
        ├── foo-tools
        ├── foo-base
        ├── foo-pages
        ├── foo-helm
        └── foo-application-base

```

There are many more tools that do this, like [myrepos](https://myrepos.branchable.com/), [vcsh](https://github.com/RichiH/vcsh) or [gitlabber](https://github.com/ezbz/gitlabber), but I didn’t try these.