Backing up Appwrite to Backblaze
Tags: opensource, tutorial, docker, webdev
Published: 2021-07-17 10:28:31

## Table Of Contents
  * [Introduction](#introduction)
  * [Getting Backblaze secrets](#getting-backblaze-secrets)
  * [B2 CLI basics](#b2-cli-basics)
  * [Appwrite MariaDB 👉 Backblaze](#appwrite-mariadb-to-backblaze)
  * [Appwrite Docker Volumes 👉 Backblaze](#introduction)

---

## Introduction <a name="introduction"></a>

Recently, Appwrite has released an article on how to backup and restore all data on your production server. You can read it here:
{% post https://dev.to/appwrite/appwrite-in-production-backups-and-restores-4beg %}


This article got me thinking 🤔 Sure, you can backup most MySQL databases into a simple `backup.sql` file, but what if you have multiple massive projects inside Appwrite? Your disk storage may or may not be able to hold a backup of all Appwrite volumes. Additionally, you most likely need to backup periodically and need multiple versions of the backup. So, what are your options?
- Write a complex bash script to keep last three versions
- Throw all backups to a storage provider and let them take care of it

You should go with storage providers not only because they have a retention system already in place, also due to their experience in the field as they most likely care about security a lot and use raids to ensure your data won't get lost.

I wrote this article as I was implementing this on my own production server, and I will show you everything you need to know about backing up **Appwrite** into **Backblaze**.

## Getting Backblaze secrets <a name="getting-backblaze-secrets"></a>

First things first, what is Backblaze? Backblaze is one of the most, if not most, used commercial platform to backup your data for the lowest price possible. They provide solutions to backup your personal computer or your business data. The service we will be using is **B2 Cloud Storage** as this gives our flexibility just like an actual hard drive. You don't have to worry about pricing yet because their free plan gives you 10GB of free space. To learn more about this product and its pricing, you can take a look at: https://www.backblaze.com/b2/cloud-storage.html

> "Over one exabyte of data storage under management, and counting." - Backblaze website

After creating an account and logging in (make sure to confirm your email), Backblaze may ask you to set up 2FA using your phone number for additional security.

To use Backblaze from our server, we will need:
- BUCKET_NAME
- ACCESS_KEY_ID
- APPLICATION_KEY

Backblaze uses **buckets** to store your data. In our case, we will only need one bucket as we will only back up two or three files at most (MySQL database and Appwrite Docker volumes). To create a bucket, navigate to `Bucket` and press `Create a Bucket`. Give it some name such as `appwrite` and click `Create a Bucket` again. You should see a newly created bucket on your list. Make sure to mark your `Bucket Name` as we will need this later.

If you enter `App Keys`, you can `Add a New Application Key`. For simplicity, we will ignore that, and we will use our master key to have all permissions. Simply click `Generate a New Master Application Key`, and this should give you `keyID` and `applicationKey`.

Wohoo 🎉 we have all secrets we need; let's continue!

## B2 CLI basics <a name="b2-cli-basics"></a>

To make sure everything is working, let's try some simple action on B2:
- List files in a bucket
- Upload a file
- List files again

We need to have B2 CLI installed on our machine, but this can be easily avoided thanks to a Docker image `mtronix/b2-cli:0.0.1`. This is the only image I could find that was actually working.

To list all files in our bucket, we run:
```
docker run \
  --rm \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 list-file-names <BUCKET_NAME>"
```

The output should be an empty list of files:
```
{
  "files": [],
  "nextFileName": null
}
```

Let's upload a file 😎
We need to have a file... I downloaded a cute cat image and saved it as `cat.png`. To upload the file, I run:
```
docker run \
  --rm \
  -v $PWD:/b2 \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> cat.png cat.png"
```

The successful result should look something like:
```
{
  "action": "upload",
  "fileId": "4_z78da5cd2a05db73574a90515_f11841831f8e91ca6_d20210717_m092603_c002_v0001140_t0056",
  "fileName": "cat.png",
  "size": 136021,
  "uploadTimestamp": 1626513963000
}
```

Finally, run the `list-file-names` command from above again, and you should see your file in the array of files:
```
{
  "files": [
    {
      "accountId": "8ac20d754955",
      "action": "upload",
      "bucketId": "78da5cd2a05db73574a90515",
      "contentLength": 136021,
...
```

Backblaze let you do all of this function manually on their website. Exercise for you 😲 Remove our test file from the bucket. Do it however you want - you can remove it using B2 CLI or manually on the website.

## Appwrite MariaDB 👉 Backblaze <a name="appwrite-mariadb-to-backblaze"></a>

To create an Appwrite MariaDB backup, you run this command:
```
docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u "$MYSQL_USER" -p "$MYSQL_ROOT_PASSWORD"' > ./dump.sql
```
(Source: Appwrite article mentioned earlier)

To backup this export into Backblaze, I created a script that:
1. Export MariaDB data to `dump.sql`
2. Upload `dump.sql` to Backblaze
3. Remove `dump.sql` to free up the space on the machine

This is what the script looks like:
```
docker-compose exec \
  mariadb \
  sh -c \
  'exec mysqldump --all-databases --add-drop-database -u"$MYSQL_USER" -p"$MYSQL_ROOT_PASSWORD"' > ./dump.sql ; \
  docker run \
  --rm \
  -v $PWD:/b2 \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> dump.sql dump.sql" ; \
  rm dump.sql
```

Each time you run this script, it will backup your MariaDB database into Backblaze. Backblaze can store multiple versions of a single file; just make sure to configure retention on a bucket, for example, 7 days.

## Appwrite Docker Volumes 👉 Backblaze <a name="appwrite-docker-volumes-to-backblaze"></a>

Once again, let's get our base command from Appwrite's article:

**Before running these commands is it highly recommended to shut down your Appwrite instance to ensure you get a complete backup.**

```
mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/uploads && tar cvf /backup/uploads.tar ."
```

```
mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/functions && tar cvf /backup/functions.tar ."
```

These two commands will backup two different volumes. The first one back up all files in Appwrite storage while the second one backup your function tags. Let's start with backing up uploads.

Just like with MariaDB, we will:
1. Create a backup in `backup/uploads.tar`
2. Send the file to Backblaze
3. Remove the backup file

The full command to do this is:
```
mkdir -p backup && docker run \
  --rm \
  --volumes-from "$(docker-compose ps -q appwrite)" \
  -v $PWD/backup:/backup \
  ubuntu \
  bash -c \
  "cd /storage/uploads && tar cvf /backup/uploads.tar ."  ; \
  docker run \
  --rm \
  -v $PWD:/b2 \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> backup/uploads.tar upload_backup.tar" ; \
  rm backup/uploads.tar
```

This should create a file called `upload_backup.tar` in your Backblaze bucket.

Exercise time 💪 You will need to backup Appwrite functions too! Using all knowledge from this article, you should be able to prepare a command that will do it.
If you are here only to copy&paste commands, here is the command:
{% spoiler Command to backup Appwrite function to Backblaze %}
```
mkdir -p backup && docker run \
  --rm \
  --volumes-from "$(docker-compose ps -q appwrite)" \
  -v $PWD/backup:/backup \
  ubuntu \
  bash -c \
  "cd /storage/functions && tar cvf /backup/functions.tar ."  ; \
  docker run \
  --rm \
  -v $PWD:/b2 \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> backup/functions.tar upload_functions.tar" ; \
  rm backup/functions.tar
```
{% endspoiler %}

---

That's it! I hope this article helped you backing up your Appwrite server. If you have **any** questions, feel free to join Appwrite's Discord server and chat with their **amazing** community: https://appwrite.io/discord