Backup And Restore Your SpiceDB With zed
Published: 2024-02-16 15:56:18

---
title: Backup And Restore Your SpiceDB With zed
published: true
date: 2024-02-06 12:00:00 UTC
tags: 
canonical_url: https://authzed.com/blog/how-to-backup-and-restore-spicedb
---

The new year brought some new cool features to [`zed`](https://github.com/authzed), specifically to backup and restore functionality, so we thought it was a good opportunity to showcase how to use it with your SpiceDB and what the new features look like.

Let's dive right in! 🏊‍♂️

## How SpiceDB Backup / Restore Works

The SpiceDB API exposes two APIs designed to support the ingestion and retrieval of large amounts of data, in addition to the existing CRUD APIs. These APIs are better optimized to support bulk import and export of system data and enable things like disaster recovery or bootstrapping new instances.

You can use the exposed gRPC APIs [BulkImportRelationships](https://buf.build/authzed/api/docs/main:authzed.api.v1#authzed.api.v1.ExperimentalService.BulkImportRelationships) and [BulkExportRelationships](https://buf.build/authzed/api/docs/main:authzed.api.v1#authzed.api.v1.ExperimentalService.BulkExportRelationships) to build any functionality you'd like on top of it, but if all you are looking for is a simple backup/restore CLI command to integrate into your workflows, we've got your covered: the `zed` CLI tool conveniently exposes functionality to create, restore, extract, and redact backups.

`zed` keeps all the data in an Apache Avro container which efficiently stores the schema, the snapshot revision of the backup, and relationships at that revision.

Your entry point to the wonders of SpiceDB backups is `zed backup`, which will show you the existing sub-commands

```
Create, restore, and inspect Permissions System backups

Usage:
  zed backup <filename> [flags]
  zed backup [command]

Available Commands:
  create Backup a permission system to a file
  parse-relationships Extract the relationships from a backup file
  parse-revision Extract the revision from a backup file
  parse-schema Extract the schema from a backup file
  redact Redact a backup file to remove sensitive information
  restore Restore a permission system from a file

```

- `create` will start the creation of a backup
- `restore` will restore the backup into SpiceDB, including schema and relationships
- `parse-*` command allows you to extract data from the backup file. You can pipe it into other processes!
- `redact` will take a zed backup file, and redact schema definitions, relations, permissions, caveats, and relationship data

## How to create a SpiceDB Backup

Creating a SpiceDB backup starts by installing `zed` on your machine, which is described [here](https://github.com/authzed/zed?tab=readme-ov-file#installing-the-binary).

Once the command is in place, you should define a "zed context", which defines the connection parameters to a SpiceDB instance. Let's say you have a local SpiceDB instance running:

```
zed context set dev localhost:50051 my_very_secret_preshared_key --insecure
zed use dev

```

Make sure everything is working as intended by issuing a `read schema` request:

```
zed schema read

```

Then creating a backup is as simple as running

```
zed backup create mybackup

```
<script async id="asciicast-MVtNe6hI2zqNqmrDAxXzEyihu" src="https://asciinema.org/a/MVtNe6hI2zqNqmrDAxXzEyihu.js"></script>
## How to restore a SpiceDB Backup

You probably already guessed it, but restoring a SpiceDB backup is pretty simple!

```
zed backup restore mybackup

```

The restore command will slice and write the backup data in batches of configurable size, and once it completes, it will show stats of the operation:

<script async id="asciicast-8zHnFzKLI9VdK9gCDekrTvaAl" src="https://asciinema.org/a/8zHnFzKLI9VdK9gCDekrTvaAl.js"></script>

Restoring a backup is a write-heavy operation, so zed provides some options to handle errors and conflicts

### New Options In SpiceDB Backup Restore

Depending on the type and provisioned capacity of the datastore, restoring a very large SpiceDB backup can take some time, so in the face of a network error, we may lose precious time. Not only that but it can be a hassle to deal with all the relationships provisioned on the initial run, with the system refusing to restore the backup due to the conflicting relationships.

While a restore operation can be done on a live SpiceDB system, the most common scenarios we've seen are:

- seeding a brand new SpiceDB instance
- restoring a snapshot in a different instance for troubleshooting or migration purposes

To help with these tasks, `zed` has now some new tricks up its sleeve to help you in your endeavor:

- **Handle Conflicts** : `--conflict-strategy` allows you to define how to handle the conflict that arises when trying to restore a relationship that already exists. The default until now has been to fail the restore operation, but you can now also skip those over, or write them with `touch` semantics instead of `create`. The default continues to be `fail` to honor the original behavior.
- **Automatic Retries** : Retries on serialization errors are now enabled by default. When the backing datastore returns serialization errors, `zed` will now retry with a backoff by default. It's now enabled by default, but you can go back to the original behavior of failing fast with `--disable-retries`
- **Restore Subsets of a Backup** : You can now restore a portion of your backup SpiceDB instance with a prefix filter. This is useful if you have multiple teams working on the same schema and their namespaces are separated by a prefix.

```
zed backup restore --help
Restore a permission system from a file

Usage:
  zed backup restore <filename> [flags]

Flags:
      --conflict-strategy string strategy used when a conflicting relationship is found. Possible values: fail, skip, touch (default fail)
      --disable-retries retries when an errors is determined to be retryable (e.g. serialization errors)
      --prefix-filter string include only schema and relationships with a given prefix

```
<script async id="asciicast-yt65RNMn01RiaMA609g3pN432" src="https://asciinema.org/a/yt65RNMn01RiaMA609g3pN432.js"></script>

As a bonus, `zed backup create` also supports the `--prefix-filter`, so it will only create a backup of a specific subset of relationships!

## How To Obfuscate / Redact A SpiceDB Backup

Sometimes a certain issue can only be reproduced with the state of production data. If only we could use it for your tests or benchmarking. Well, you now can! The new Backup Redaction, available from [`zed v0.16.X`](https://github.com/authzed/zed/releases/tag/v0.16.4) **takes an unredacted backup and obfuscates all the data present** in it so you can:

- share it with anyone
- check it into your SCM and use it as part of your CI
- use it for load-testing
- mirror production data in a different system without concerns over customer data

```
zed backup redact --help
Redact a backup file to remove sensitive information

Usage:
  zed backup redact <filename> [flags]

Flags:
      --redact-definitions redact definitions (default true)
      --redact-object-ids redact object IDs (default true)
      --redact-relations redact relations (default true)

```
<script async id="asciicast-mbCGGbHo1FpbJ0ClVXhMuO8h3" src="https://asciinema.org/a/mbCGGbHo1FpbJ0ClVXhMuO8h3.js"></script>
## How To Extract Information From A SpiceDB Backup

Once you have a SpiceDB backup file, you can inspect its contents using `zed` tool, and pipe its output into whatever shell workflows you may have built, as it's built with Unix program design principles in mind:

### How to extract the schema out of a SpiceDB backup

To print the schema of a SpiceDB backup, run the following command

```
zed backup parse-schema mybackup

```
<script async id="asciicast-p1SpFklRr1wtEgrwukhMW2AJm" src="https://asciinema.org/a/p1SpFklRr1wtEgrwukhMW2AJm.js"></script>
### How to extract the snapshot revision out of a SpiceDB backup

To print the snapshot revision of a SpiceDB backup, run the following command

```
zed backup parse-revision mybackup

```
<script async id="asciicast-h5grOeGumnHG58SfXd6pMNkDR" src="https://asciinema.org/a/h5grOeGumnHG58SfXd6pMNkDR.js"></script>
### How to extract all relationships out of a SpiceDB backup

To print all the relationships present in a SpiceDB backup, run the following command:

```
zed backup parse-relationships mybackup

```
<script async id="asciicast-KK9zydi76iFjOFwIw2Cv1jFWJ" src="https://asciinema.org/a/KK9zydi76iFjOFwIw2Cv1jFWJ.js"></script>
## Bonus Feature! ⭐ Context-Aware Shell Completion

This is a new backup-adjacent but cool feature that made it's way in the 0.16.0 release. `zed` already supports shell completion for the _static_ parts of the CLI, mainly commands and subcommands. But now it also supports shell completion of certain dynamic parts:

- `zed context use` will autocomplete the various context defined
- `zed permission` and `zed relationship` will autocomplete resource types, permission names of that resource, and subject types by querying the currently configured `zed` context.

This allows user more effectively construct commands based on the existing schema in your schema. No more back and forth between terminal and schema file!

<script async id="asciicast-avXvbXSN64Um0dq46oySolVZd" src="https://asciinema.org/a/avXvbXSN64Um0dq46oySolVZd.js"></script>

Got questions about `zed`? Reach out to us via [Discord](https://authzed.com/discord), [Twitter](https://twitter.com/authzed) or [Linkedin](https://www.linkedin.com/company/authzed/)!