Demystifying Rsync
Tags: linux, tools, productivity, shell
Published: 2020-02-17 18:26:53

---
title: Demystifying Rsync
published: true
description: 
tags: linux, tools, productivity, shell
cover_image: https://antjanus.com/assets/2020/02/demystifying-rsync.png
canonical_url: https://antjanus.com/digital-garden/demystifying-rsync
---

## 📃Cheatsheet

Sync one directory into another with compression:

```sh
rsync -az source/directory destination/directory
```

Sync over SSH:

```sh
rsync -az user@source-server.com:path/to/file destination
```

or 

```sh
rsync -az source/directory user@destination-server.com:destination/path
```

Sync does *not* delete existing files at destination. To make sure the source and destination folders look *identical*, use the `--delete` flag:

```sh
rsync -az --delete source/directory destination/directory
```

## 🔄 What is Rsync?

Rsync is this neat utility that commonly exists on Linux systems (and other Unix-likes). It's like a copy command but the destination (or source) can be remote. 

It *syncs* files rather than copies them so if the destination file is already present, it doesn't get overwritten. This is great because you can sync large folders several times but only the missing files will get copied.

So while you might run something like `cp -r ~/projects/dotfiles/.bashrc ~` to copy a `.bashrc` file from a folder to your home directory, rsync lets you do it in a remote fashion like so: `rsync me@my-server.com:path/to/file ~`. Or the other way around `rsync path/to/file me@my-server.com:path/to/destination`.

To me, what makes rsync complicated are the *flags*. It's always the flags that trip me up and push me to feel uncertain about the command.

## 🏳Copying with no flags

You lose a lot by not providing flags. Without flags, you can also sync one file at a time.

```sh
rsync source/file destination
```

You can also do remote copy:

```sh
rsync serverUser@serverAddressOrIP.com:path/to/file destination
```

Rsync uses `ssh` for remote connections so make sure you have access via ssh!

Copying the other way is the same deal:

```sh
rsync path/to/file serverUser@serverAddressOrIP.com:destination
```

## 🚩 Supercharging rsync with flags

![verbose rsync output](https://antjanus.com/wp-content/uploads/2020/02/verbose-rsync.png)

```sh
rsync -flags source destination
```

There are several helpful flags and if you follow any guides online, you'll most likely come across these same exact ones.  

Flags:

1. `a` - stands for `archive`. It is *necessary* for recursive copies (ie. copying whole directories)
2. `z` - the `z` flag enables compression for faster copying across networks
3. `v` - for verbose output

The `archive` option has always been confusing to me. Luckily, there's a great [server fault post detailing the archive flag](https://serverfault.com/questions/141773/what-is-archive-mode-in-rsync#141778). It's a compilation of other flags that allows recursive syncing - so directory copy/sync support. And if you're copying locally, it preserves permissions and a bunch of other stuff.

The `z` flag enables compression via [Zlib](https://en.wikipedia.org/wiki/Zlib) (at least that's what Wikipedia tells me!). In case you wanted to know, zlib uses the same compression technique as `gzip` which you can read about in my [Demystifying TAR article](https://dev.to/antjanus/demystifying-tar-351l).

The verbose output should display files copied file-by-file. A typical rsync command might look like this:

```sh
rsync -azv ./files-to-upload me@myserver.com:~/uploads
```

## 🏳️‍🌈 ️‍More flags? (OPTIONAL)

![verbose rsync partial](https://antjanus.com/wp-content/uploads/2020/02/partial-upload.png)

Don't read this section of rsync is already giving you a headspin. *But* if you ever want to refer back to this article, you can check these flags out. I wouldn't use any of these by default you might come across them in tutorials/articles or you might want to use them yourself:

1. `n` - stands for `dry run`. This will check that you actually *can* transfer those files.
2. `delete` - delete files from destination that don't exist in source
3. `P` - The `P` flag is a combination of the `--partial` and `--progress`. 

The `n` flag is handy because you can use it to make sure you're running your command correctly. Combined with a verbose output, you can see exactly what's going to happen without making it happen. I found this useful when preparing to upload a *lot* of files and not wanting to screw up where they upload or which files upload.

The `delete` flag is also super useful in that it doesn't just copy files to the destination, it ensures that files in the source directory *don't exist* in the destination. Super useful when syncing from a source that you consider the "source of truth" and want to have identical copies elsewhere...without having to delete everything and copy everything from scratch.

I don't often use the `P` flag but for many, it's the default. The `P` flag will do 2 very important things:

1. allow partial file upload + resume
2. display a progress bar

If you check the screenshot above, I ran that copy with a `P` flag. At the end, it'll let you know neat stuff about how much data you've transferred if you combine it with the `v` flag.

The wonderful part about partial file uploads is that if your connection fails, you can run the command again and it'll pick up where it left off with the partially uploaded files. Without the `P` flag, rsync would not be able to finish uploading the interrupted file.