VoluSnap — Cloud Volume Auto Snapshot Go Server
Tags: backup, cloudcomputing, go, opensource
Published: 2019-03-25 11:20:23

---
title: VoluSnap — Cloud Volume Auto Snapshot Go Server
published: true
tags: backup,cloud-computing,golang,open-source
canonical_url: https://medium.com/@didil/volusnap-cloud-volume-auto-snapshot-go-server-fb1a0e0cd104
---

### VoluSnap — Cloud Volume Auto Snapshot Go Server

Do you deploy your web apps to **cloud virtual machines** such as Digital Ocean Droplets or Scaleway Cloud Instances ?  
If your app saves any kind of data to disk and you’re worried about **data loss** due to software bugs, human errors or malicious actors (amongst other risks) then you need a **backup strategy**.

![](https://cdn-images-1.medium.com/max/1024/1*zea8_CAZQLFwUS-uG7KHXg.jpeg)<figcaption>Retro Data Backups :)</figcaption>

### Cloud Volume Backups

Most cloud providers provide a **_Hot Backup_** feature which takes a snapshot of your volume data without the need to stop your instances during the backup process (but there is a data loss risk involved, please read up on this topic and make an informed decision).

However, **automated recurring backups** are not always available, and sometimes the configuration options are **not granular enough**. At the time of writing this article Digital Ocean Backups are taken once a week, and Scaleway only offers manual backups.

Fortunately, most cloud providers provide **API access**. I’ve needed an automated backup solution for my personal cloud deployments for a while, so I decided to build an open source solution:

### VoluSnap: A self hosted solution built with Go

[Volusnap](https://github.com/didil/volusnap) is an API server built with Go. It allows triggering automated recurring snapshots of cloud provider volumes. Here is a simplified schema explaining how VoluSnap works:

![](https://cdn-images-1.medium.com/max/891/1*bxQvFXfltfySuMSshhCwkg.png)<figcaption>API Server + Rules Checker Goroutine</figcaption>

The basic workflow is :

- User signs up
- User logs in, generating a JWT token
- User adds a Cloud Account and relevant Cloud API access token
- User lists Cloud Volumes
- User creates Snapshot Rule(s) for the chosen Volume(s)
- Rules Checker Service checks the snapshot rules periodically, triggers snapshots via Cloud Provider API and saves snapshot Metadata to the Database

VoluSnap uses:

- [gorilla/mux](https://github.com/gorilla/mux) for routing
- [sqlboiler](https://github.com/volatiletech/sqlboiler) for data persistence to PostgreSQL
- [sql-migrate](https://github.com/rubenv/sql-migrate) for database migrations
- [viper](https://github.com/spf13/viper) for configuration
- [testify](https://github.com/stretchr/testify) for unit tests and mocks

Components:

- REST API
- SnapRules Checker Service

The REST API is documented in the [github repo](https://github.com/didil/volusnap#api). No client code is provided at the moment.

![](https://cdn-images-1.medium.com/max/907/1*rzQmA63fwMb-2P27E_itQw.png)<figcaption>VoluSnap API Docs</figcaption>

At the moment both the REST API and the SnapRules Checker components run in the same Go process. The SnapRules Checker Service runs in a goroutine at the moment but it would be doable to split it into its own process in the future if needed:

{% gist https://gist.github.com/didil/20319a7175de8f6bd52a932b86dfcbb2 %}

#### Adding Cloud Providers

VoluSnap currently supports [Digital Ocean](https://www.digitalocean.com/) and [Scaleway](https://scaleway.com/). I’ve tried to make adding other Cloud providers easy (Hint: PRs welcome ❤️). As you can see in the code below, pRegistry is a package level singleton service you can register provider services with. Your provider just has to implement the interface ProviderSvcer :

```
type ProviderSvcer interface {
    ListVolumes() ([]Volume, error)
    TakeSnapshot(snapRule \*models.SnapRule) (string, error)
}
```

{% gist https://gist.github.com/didil/b1f5ebf439bdc86ccfc2a1df9324436d %}

And don’t forget the corresponding unit tests

{% gist https://gist.github.com/didil/a8b770c7d1eabbad74c5cf56717b219d %}

### Conclusion

I hope that you’ll find VoluSnap useful for your Cloud Volume Backups. Definitely leave comments if you like/hate/have questions about the project or the code/testing patterns used !