A Short Journey to Git Internals
Tags: git
Published: 2020-09-28 14:24:23

In this article, we’ll dive into Git internals by going through a real example. If you don’t have your terminal open already, do so, fasten your seatbelts, and let’s go! 💨

### Initializing a Git repository

You have probably already initialized an empty Git project using `git init`, but did you ever wondered what does this command do?

Let’s create an empty folder, and initialize an empty Git project. From the official [Git documentation](https://git-scm.com/docs/git-init), `git init`:

> This command creates an empty Git repository — basically a .git directory with subdirectories for `objects`, `refs/heads`, `refs/tags`, and template files. An initial `HEAD` file that references the `HEAD` of the master branch is also created.

If we inspect the folder’s content, we’ll see the following structure:

```bash
$ tree -L 1 .git/
.git/
├── HEAD
├── config
├── description
├── hooks
├── info
├── objects
└── refs
```

### Git is a key-value datastore

At its core, Git is a content-addressable filesystem. Huh? 🤔 Ok, Git is simply a key-value database. You insert any kind of content into a Git repository, for which Git will give you back a unique identifier (a key) that you can use to retrieve that content back.

Git uses the [hash-object](https://git-scm.com/docs/git-hash-object) command to store values into the database:

> Computes the object ID value for an object with specified type with the contents of the named file (which can be outside of the work tree), and optionally writes the resulting object into the object database. Reports its object ID to its standard output. When <type> is not specified, it defaults to “blob”.

“blob” is nothing but a sequence of bytes. A Git blob contains the exact data as a file, but it’s stored in the Git key-value data store, while the “actual” file is stored on the file system.

Let’s create a blob:

```bash
$ echo hello | git hash-object --stdin -w
ce013625030ba8dba906f756967f9e9ca394464a
```

We used the `-w` flag to actually write the object into the object database, and not only display it (achieved by the `--stdin` flag).

The value “hello” is the “value” in the Git data store, and the hash returned from the `hash-object` function is in this case our key. We can now do the opposite operation to read the value by its key, using the [`git-cat-file`](https://git-scm.com/docs/git-cat-file) command:

```bash
$ git cat-file -p ce013625030ba8dba906f756967f9e9ca394464a
hello
```

We can check its type using the `-t` flag:

```bash
$ git cat-file -t ce013625030ba8dba906f756967f9e9ca394464a
blob
```

git `hash-object` stores the data in the `.git/objects/` folder (AKA the object database). Let’s verify:

```bash
$ tree .git/objects/
.git/objects/
├── ce
│   └── 013625030ba8dba906f756967f9e9ca394464a
├── info
└── pack
```

The hash suffix (under the “ce” directory) is the same as the one we got back from the `hash-object` function, but it has a different prefix. Why? That’s because the parent folder name contains the first two characters of our key. Why? Because some file systems have limitations on numbers of sub-directories. So introducing this layer mitigates that problem.

Let’s save another object:

```bash
$ echo world | git hash-object --stdin -w
cc628ccd10742baea8241c5924df992b5c019f71
```

As expected, we’ll now have two directories under `.git/objects/`:

```bash
$ tree .git/objects/
.git/objects/
├── cc
│   └── 628ccd10742baea8241c5924df992b5c019f71
├── ce
│   └── 013625030ba8dba906f756967f9e9ca394464a
├── info
└── pack
```

and again, the `cc` folder, which has the key’s prefix, has the rest of the key in the contained file’s name.

### Tree Objects 🌲

The next Git object we’ll investigate is the tree. This type solves the problem of storing the filename and allows storing a group of files together.
A tree object contains entries. Each entry is the SHA-1 of the blob, or subtree with its associated mode, type, and filename. Let’s check the [`git-mktree`](https://git-scm.com/docs/git-mktree) documentation:

> Reads standard input in non-recursive `ls-tree` output format, and creates a tree object. The order of the tree entries is normalized by mktree so pre-sorting the input is not required. The object name of the tree object built is written to the standard output.


If you’re wondering about [`ls-tree`](https://git-scm.com/docs/git-ls-tree) output format, it looks like:

```
<mode> SP <type> SP <object> TAB <file>
```

Let’s now associate the two blobs above:

```bash
$ printf '%s %s %s\t%s\n' \
    100644 blob ce013625030ba8dba906f756967f9e9ca394464a hello.txt \
    100644 blob cc628ccd10742baea8241c5924df992b5c019f71 world.txt |
  git mktree
88e38705fdbd3608cddbe904b67c731f3234c45b
```

`mktree` returns a key for the newly created tree object.

At this point, we can visualize our tree as follows:

```
             88e38705fdbd3608cddbe904b67c731f3234c45b  
                                |                   
                  +-------------|------------+             
                  |                          |        
                  |                          |        
                  |                          |          
                  |                          |       
                  |                          |
                hello                       world        
            ce013625030b                 cc628ccd1074
```

Let’s view the tree’s content:

```bash
$ git cat-file -p 88e38705fdbd3608cddbe904b67c731f3234c45b
100644 blob ce013625030ba8dba906f756967f9e9ca394464a hello.txt
100644 blob cc628ccd10742baea8241c5924df992b5c019f71 world.txt
```

and of course, the `.git/objects` was updated accordingly:

```bash
$ tree .git/objects/
.git/objects/
├── 88
│   └── e38705fdbd3608cddbe904b67c731f3234c45b
├── cc
│   └── 628ccd10742baea8241c5924df992b5c019f71
├── ce
│   └── 013625030ba8dba906f756967f9e9ca394464a
├── info
└── pack
```

So far we **have not** updated our index. To do so, we use the [`git-read-tree`](https://git-scm.com/docs/git-read-tree) command:

> Reads the tree information given by <tree-ish> into the index, but does not actually **update** any of the files it “caches”. (see: [git-checkout-index[1]](https://git-scm.com/docs/git-checkout-index))

```bash
$ git read-tree 88e38705fdbd3608cddbe904b67c731f3234c45b
$ git ls-files -s
100644 ce013625030ba8dba906f756967f9e9ca394464a 0 hello.txt
100644 cc628ccd10742baea8241c5924df992b5c019f71 0 world.txt
```

Note however, that we still **don’t** have the files on our file system, since we’re writing values **directly** to the Git datastore. In order to “checkout” the files, we’ll use the `git-checkout-index` command, which copies files from the index to the working tree:


```bash
git checkout-index 0 -a
```

The `-a` stands for “all”. Now we should be able to see our files:

```bash
$ ls
hello.txt world.txt
$ cat hello.txt
hello
$ cat world.txt
world
```

### Summary 📝

In this article, we stored two files directly to the Git datastore. The files weren’t yet visible to our local filesystem. We’ve created a tree and associated the two “blobs” to it, and then we brought the files to our working directory using the git-checkout-index command.