GIT: It Is Never Too Late to Ignore Files
By: Yvens Serpa
Tags: Git Rebase, Code, Tutorial, Git, Programming

I commonly see Git projects bloated with files and folders that should have been ignored and are now using valuable space in the repository. These files are usually auto-generated data from the IDE or the software used, such as the infamous .vs folder from Visual Studio or Unity’s Temp folder.

Auto-generated files are usually dependent on the environment (installed API and OS, for instance). Thus, it is preferable to let each user generate them themselves, instead of keeping them in the repository. For the same reason, those files are more likely to conflict in a merge operation, since they will differ for every user. While those merge conflicts can be easily solved, it is still a time-consuming task and detrimental to the stability of the repository.

While in an ideal world, .gitignore files solve that issue on their own, we are still bound to make mistakes and let some data slip. Fortunately, there are plenty of ways to remove unwanted files from the repository after they have been added and tracked by the repository.

This article describes a simple action to clear the repository and a more sophisticated alternative. Additionally, it is used the term “file”, but the techniques work just the same as for folders. When there is any difference between file and directories, it will be noted.

A Simple Step

Add the file to your .gitignore file and then delete it from your repository. To remove a file from your Git repository, use the rm command:

git rm <file>

The remove command deletes the file locally and stops tracking it. Thus, you can commit and push the change to your repository. It should now be purged.

In some cases, you might want to keep the local files and only remove them from the repository. For this, we can use the --cached flag. It will remove the file and save it, modified or not, as it is in your local repository.

git rm --cached <file>

As stated previously, you can then commit and push this action to your remote repository and not bother again with the file.

For directories, you need to add the -r flag as follows:

git rm --cached -r <folder>

A more automated alternative is to ask Git to untrack everything from the local repository, using the --cached flag to avoid deleting them by mistake and then re-add everything back. By doing so, Git will be forced to reevaluate all files according to your .gitignore file and, by the end, should stage the deletion of all ignored files that are currently being tracked. For this alternative, use the following commands in the root folder of your repository:

git rm --cached -r .

git add .

The dot ( . ) added, in the end, stands for the path of the current directory. Thus, these commands: remove everything from the repository, but do not delete the files from your computer (due to the --cached flag); recursively ( -r ); and, starting from the current directory ( . ).

After performing the second command ( git add . ) Git will track all the files again and mark the ignored files as deleted. Ideally, it will show a list of tracked changes containing only deletion operations, since the rest of the repository was not changed at all. You can then commit and push.

Cleaning History

In some cases, removing the objects from the repository is not enough. Data with sensitive information (such as passwords stored in code, e.g.) should be deleted from the Git history as if they never existed.

The same applies to large files that will continue to use repository memory, even if you remove them with the previous steps showed here. That is the case because they are still part of the history, and any user could reset to a commit in which those files exist. Thus, Git needs to keep them.

The easiest, but the limited alternative, is to use the amend flag. When committing with the amend flag, Git will replace the tip of the current branch (the latest commit) with the new one. Think of it as if you were merging a new commit with the previous one, resulting in a single new commit.

If the last commit (the tip of the current branch) was the one to add the unwanted file, you could amend it, deleting the file by using:

git commit --amend

git push -f

It is necessary to use the -f flag, that stands for "force," and ignores some of the Git checks, forcing it to operate as requested. In this case, it will force rewriting the history, removing the previous tip, and replacing it with the new commit. With that, the unwanted file will not be part of it.

Rewriting Git history is the way of completely removing files from it so that for all effects, it is as if the files were never there.

Rewriting History through Rebasing

If your files were added in older commits and now you want them to be erased from history, you need to use the rebase command. Rebase is the process of reapplying commits in a different base. The base, as the tip of the branch mentioned in the previous section, is another commit.

It is possible to interactively apply the commits into the base by using the -i flag. This flag allows the user to edit the list of commits before doing so, allowing for changes, such as reordering commits or dropping them. To perform a rebase, use the following command. The <base> variable should be the hash of the second commit before the one that added the unwanted file.

git rebase -i <base>

As an example, given the following commit order:

123ac60 Commit with code

321ab56 Commit with unwanted files

213ad23 Commit with code

413ee26 Commit with code

...

324cc15 Initial commit

If you were to rebase using the commit 213ad23 Git might not allow you to drop the commit 321ab56 since it is the first commit in the rebase operation. There are other alternatives, but the easiest one is to rebase using the previous commit, 413ee26 . Thus you can safely remove the commit 321ab56 .

In our example, use the rebase command as below:

git rebase -i 413ee26

After this command, you will be allowed to select one operation per commit, including the drop operation that deletes a commit. Depending on your configuration, using the interactive rebase command will launch Vim in your terminal. If you have no idea how to use Vim, check this link. Otherwise, you can change the default program Git uses by doing this. Do it before the rebase.

After that, the list of commits and operations will appear as below:

pick 123ac60 Commit with code

pick 321ab56 Commit with unwanted files

pick 213ad23 Commit with code

The pick operation uses the commit as it is. You should keep all other commits as pick and change the unwanted commit to drop instead. The drop command does not delete a commit but removes it from the list of commits to be applied in the new base. Since we will later change our history for the new, rebased history, the unwanted commit will not be there, as if it was deleted.

Your commit list should look like this:

pick 123ac60 Commit with code

drop 321ab56 Commit with unwanted files

pick 213ad23 Commit with code

After finishing this operation (in Vim, you have to save the file and close it), Git will interactively rebase the commits. With this, your branch is conflicting with the remote branch. That happens because your branch history differs from the remote branch, as we intended. To alter the remote branch and reflect the changes, you must perform a push -f command.

Notice that with this action, you are rewriting the branch history, and this is a destructive operation, use it with care! For the sake of security, try checking your repository locally first to see if the action harmed it anyway. You should only push the changes after being sure that everything is working.

Alternatively, you might need to keep some of the changes in that commit ( 321ab56 ) instead of dropping it entirely. For that, you can use the edit operation. During the rebasing, Git will stop when it starts to rebase the edited commit and will let you perform operations on it as if you were committing it. In our example, you could delete or untrack the unwanted file.

Your commit list using the edit command should look like this:

pick 123ac60 Commit with code

edit 321ab56 Commit with unwanted files

pick 213ad23 Commit with code

After editing the commit to remove the unwanted file, you need to ask Git to continue the rebase operation by using git rebase --continue . Everything else should proceed as explained before: finish the rebase and push -f .

Even Worse Scenario

It might be the case that your unwanted file was not only tracked and added; it was also changed and updated. Therefore, there are multiple versions of it spread through various commits in your story. Fortunately, our previous strategy still works, but there are going to be conflicts in every commit that alters the unwanted file. Grab some tea, relax, and solve them one by one.

In the end, it is never too late to ignore a file and remove it from your repository. But, keep in mind that the longer it takes to take action, the harder it might be to do it (and, potentially, the number of conflicts).

Keep your .gitignore files update to date and always check your commit meticulously before pushing them. If it fails, fall back to an --amend approach first to avoid repercussions in a later stage. In the worst case, recur to a big rebase and check everything before pushing the changes to your remote. Unit testing and a good CI are handy tools in that scenario and might save time.

Thanks for reading :)