Git under the hoods : Learn the concepts not just commands
By: Rising Programmer
Tags: Clever Programmer, Learning To Code, Git, Programming, Coding

Git under the hoods : Learn the concepts not just commands

We all developers use git to track changes in our project files so that we can roll back to previous version at any time we wish to but how many of us really understand how git works under the hood?

For example, what happens when we use git commit? What is stored between commits? Is it just a diff between the current and previous commit? If so, how is the diff encoded? Or is an entire snapshot of the repo stored each time? What really happens when we use git init ?

About 99% of developers use Git but they don’t know the answers to the questions above. But does it really matter?

As I said Learn the concepts not just commands

First, as professionals, we should be hungry to understand the tools we use, especially if we use them all the time — like Git.

But even more acutely, I’ve found that understanding how git actually works is useful in many scenarios — whether it’s resolving merge conflicts, looking to conduct an interesting rebase, or even just when something goes slightly wrong.

You’ll benefit from this post if you’re experienced enough with git to feel comfortable with commands such as git pull , git push , git add or git commit .

Still, we will start with an overview to make sure we are on the same page regarding the mechanisms of git , and specifically, the terms used throughout this post

Omer Rosenbaum has also uploaded a YouTube series covering this concepts — Watch it here.

What to expect from this article

We will understand what really happens on the other side when we use git commands.

We will start by covering objects — blobs, trees, and commits. We will then briefly discuss branches and how they are implemented. We will dive into the working directory, staging area and repository.

We will understand how these terms are related to git commands we use to create repos.

Next, will create a repository from scratch — without using git init , git add , or git commit . This will allow us to deepen our understanding of what is happening under the hood when we work with git .

By the end of this post, you will feel like you understand git . Let’s go! 😎

Git Objects — blob, tree and commit

It is very useful to think about git as maintaining a file system, and specifically — snapshots of that system in time.

A file system begins with a root directory (in UNIX-based systems, / ), which usually contains other directories (for example, /usr or /bin ). These directories contain other directories, and/or files (for example, /usr/1.txt ).

In git , the contents of files are stored in objects called blobs, binary large objects.

Blobs, on the other hand, are just contents — binary streams of data. A blob doesn’t register its creation date, its name, or anything but its contents.

Every blob in git is identified by its SHA-1 hash. SHA-1 hashes consist of 20 bytes, usually represented by 40 characters in hexadecimal form. Throughout this post we will sometimes show just the first characters of that hash.

BLOB

In git , the equivalent of a directory is a tree. A tree is basically a directory listing, referring to blobs as well as other trees.

Trees are identified by their SHA-1 hashes as well. Referring to these objects, either blobs or other trees, happens via the SHA-1 hash of the objects.

TREE

TREES

The diagram above is equivalent to a file system with a root directory that has one file at /test.js , and a directory named /docs with two files: /docs/pic.png and /docs/1.txt .

In git , a snapshot is a commit. A commit object includes a pointer to the main tree (the root directory), as well as other meta-data such as the committer, a commit message and the commit time.

In most cases, a commit also has one or more parent commits — the previous snapshot(s). Of course, commit objects are also identified by their SHA-1 hashes. These are the hashes we are used to seeing when we use git log .

COMMIT

Every commit holds the entire snapshot, not just diffs from the previous commit(s).

How can that work? Doesn’t that mean that we have to store a lot of data every commit?

Let’s examine what happens if we change the contents of a file. Say that we edit 1.txt , and add an exclamation mark — that is, we changed the content from HELLO WORLD , to HELLO WORLD! .

Well, this change would mean that we have a new blob, with a new SHA-1 hash. This makes sense, as sha1("HELLO WORLD") is different from sha1("HELLO WORLD!") .

Since we have a new hash, then the tree’s listing should also change. After all, our tree no longer points to blob 73D8A, but rather blob 62E7A instead. As we change the tree’s contents, we also change its hash.

The tree that points to the changed blob needs to change as well

And now, since the hash of that tree is different, we also need to change the parent tree — as the latter no longer points to tree CAFE7, but rather tree 24601. Consequently, the parent tree will also have a new hash.

Almost ready to create a new commit object, and it seems like we are going to store a lot of data — the entire file system, once more! But is that really necessary?

Actually, some objects, specifically blob objects, haven’t changed since the previous commit — blob F92A0 remained intact, and so did blob F00D1.

So this is the trick — as long as an object doesn’t change, we don’t store it again. In this case, we don’t need to store blob F92A0 and blob F00D1 once more. We only refer to them by their hash values. We can then create our commit object.

Blobs that remained intact are referenced by their hash values

Since this commit is not the first commit, it has a parent — commit A1337.

So to recap, we introduced three git objects:

blob — contents of a file.

contents of a file. tree — a directory listing (of blobs and trees ).

— a directory listing (of and ). commit — a snapshot of the working tree.

Let us consider the hashes of these objects for a bit. Let’s say I wrote the string git is awesome! and created a blob from it. You did the same on your system. Would we have the same hash?

The answer is — Yes. Since the blobs consist of the same data, they’ll have the same SHA-1 values.

What if I made a tree that references the blob of git is awesome! , and gave it a specific name and metadata, and you did exactly the same on your system. Would we have the same hash?

Again, yes. Since the trees objects are the same, they would have the same hash.

What if I created a commit of that tree with the commit message Hello , and you did the same on your system. Would we have the same hash?

In this case, the answer is — No. Even though our commit objects refer to the same tree, they have different commit details — time, committer etc.

Branches in Git

A branch is just a named reference to a commit.

We could always reference a commit by its SHA-1 hash, but humans usually prefer other forms to name objects. A branch is one way to reference a commit, but it’s really just that.

In most repositories, the main line of development is done in a branch called master . This is just a name, and it’s created when we use git init , making it is widely used. However, it’s by no means special, and we could use any other name we’d like.

Typically, the branch points to the latest commit in the line of development we are currently working on.

To create another branch, we usually use the git branch command. By doing that, we actually create another pointer. So if we create a branch called test , by using git branch test , we are actually creating another pointer that points to the same commit as the branch we are currently on.

Using `git branch` creates another pointer

How does git know what branch we’re currently on? It keeps a special pointer called HEAD . Usually, HEAD points to a branch, which in turns points to a commit. In some cases, HEAD can also point to a commit directly, but we won’t focus on that.

HEAD points to the branch we are currently on.

To switch the active branch to be test , we can use the command git checkout test . Now we can already guess what this command actually does — it just changes HEAD to point to test .

`git checkout test` changes where `HEAD` points

We could also use git checkout -b test before creating the test branch, which is the equivalent of running git branch test to create the branch, and then git checkout test to move HEAD to point to the new branch.

What happens if we make some changes and create a new commit using git commit ? Which branch will the new commit be added to?

The answer is the test branch, as this is the active branch (since HEAD points to it). Afterwards, the test pointer will move to the newly added commit. Note that HEAD still points to test .

Every time we use `git commit`, the branch pointer moves to the newly created commit.

So if we go back to master by git checkout master , we move HEAD to point to master again.

Now, if we create another commit, it will be added to the master branch (and its parent would be commit B2424).

How to Record Changes in Git

Usually, when we work on our source code we work from a working dir. A working dir(ectrory) (or working tree) is any directory on our file system which has a repository associated with it. It contains the folders and files of our project, and also a directory called .git.

After we make some changes, we want to record them in our repository. A repository (in short: repo) is a collection of commits, each of which is an archive of what the project’s working tree looked like at a past date, whether on our machine or someone else’s.

A repository also includes things other than our code files, such as HEAD , branches, and so on.

Unlike other, similar tools you may have used, git does not commit changes from the working tree directly into the repository. Instead, changes are first registered in something called the index, or the staging area.

Both of these terms refer to the same thing, and they are used often in git ’s documentation. We will use these terms interchangeably throughout this post.

When we checkout a branch, git populates the index with all the file contents that were last checked out into our working directory and what they looked like when they were originally checked out. When we use git commit , the commit is created based on the state of the index.

The use of the index allows us to carefully prepare each commit. For example, we may have two files with changes since our last commit in our working dir. We may only add one of them to the index (using git add ), and then use git commit to record this change only

Files in our working directory can be in one of two states: tracked or untracked.

Tracked files are files that git knows about. They either were in the last snapshot (commit), or they are staged now (that is, they are in the staging area).

Untracked files are everything else — any files in our working directory that were not in our last snapshot (commit) and are not in our staging area.

.