CouchDB, The Open-Source Cloud Firestore Alternative?
Tags: javascript, react, offline, realtime
Published: 2019-09-25 14:29:47

---
title: CouchDB, The Open-Source Cloud Firestore Alternative?
canonical_url: https://marmelab.com/blog/2019/09/25/couchdb_pouchdb_serious_firebase_alternative.html
cover_image: https://marmelab.com/static/cover-e05bacd2b1b1a76380041b2c72519c9b.jpeg
published: true
description: How to develop an offline-first, real time React app using a NoSQL database, in practice.
tags: js, react, offline, realtime
---

_Note: This post was originally posted on [marmelab.com](https://marmelab.com/blog/2019/09/25/couchdb_pouchdb_serious_firebase_alternative.html)._

During one of our last customer projects, we used [**Firebase**](https://firebase.google.com/), a backend-as-a-service by Google, as our backend. Althought satisfied by this "all-included" suite as a whole, we remained disappointed by its proprietary aspect.

<img src="https://marmelab.com/static/0f8a741fb7fd02afba2b32f07a1362a4/f711f/firebaselogo.webp" alt="Firebase" />

That's why I took the initiative to **look for an open-source alternative to Firebase** that could cover all our needs without depending on a third party service.

The first step in this quest is to find a substitute to the [**Cloud Firestore**](https://firebase.google.com/docs/firestore) realtime NoSQL database for the web.

## What Do We Need?

Using Firestore rather than a more classic database is not trivial. It often results from the need to quickly develop an application with the following features:

- **Offline First**, client writes to local database that is synchronized with remote one
- **Realtime**, remote changes must be in sync with our local database

Some solutions exist to cover this need, most of them are based on **NoSQL databases** such as [MongoDB](https://www.mongodb.com/fr), [Cassandra](http://cassandra.apache.org/), [RethinkDB](https://www.rethinkdb.com/), [Gun](https://gun.eco) or others MongoDB based solutions like [Minimongo](https://docs.meteor.com/api/collections.html), [turtleDB](https://turtle-db.github.io/) or [tortoiseDB](https://github.com/turtle-DB/tortoiseDB).

In our case, we're going to give [CouchDB](http://couchdb.apache.org/) (and [PouchDB](https://pouchdb.com/) for the frontend) a try, because it's the more robust and best known solution from our point of view.

## CouchDB & PouchDB

CouchDB is an **open-source** / **cross-platform** document oriented database software. It is developed on the basis of the **concurrency-oriented** [Erlang](https://en.wikipedia.org/wiki/Erlang_(programming_language)) language, allowing it to benefit from an high scalability. It uses **JSON** to store its data, and an **HTTP API** to expose it.

CouchDB was born in 2005. Since 2008, CouchDB became an **Apache Software Foundation project**, which allows it to benefit from a lot of support and a large communauty.

Here are the main features of CouchDB:

- **Multi-Version** Concurrency Control (which lets you build offline-first solutions easily)
- Distributed Architecture with **Replication**
- Document Storage
- HTTP / REST API

Since **CouchDB runs on the server**, many client library allows to communicate with it thanks to the HTTP interface it offers.

The most known CouchDB client library for the web is called PouchDB. **PouchDB is an open-source Javascript database** that is designed to run within the browser. This way, it allows to **store data locally** offline, and sync it with the remote CouchDB server when the user comes back online.

## CouchDB & PouchDB in Practice

Enough introduction, let's get practical! In this section, I'll describe the development of a [ReactJS](https://reactjs.org/) application using CouchDB and PouchDB as database system, step by step. Meanwhile, I'll try, as much as I can, to compare the CouchDB implementation to the Firestore one.

Also, I'll present you some of my latest loves in terms on Javascript librairies: [Final-Form](https://github.com/final-form/final-form), [ElasticUI](https://elastic.github.io/eui/) and [Indicative](https://github.com/poppinss/indicative).

In this project, I'm going to create a beer registry, which allows users to keep track of their beer stocks.

### Project Setup

In order to keep this tutorial as simple as possible, I'll create a ReactJS application using [create-react-app](https://github.com/facebook/create-react-app).

```bash
create-react-app reactive-beers && cd reactive-beers

npm install -S pouchdb
```

The application skeleton looks like the following:

```bash
julien@julien-P553UA:~/Projets/marmelab/reactive-beers$ tree -L 1
.
├── node_modules
├── package.json
├── package-lock.json
├── public
├── README.md
└── src
```

Then, since I don't want to install CouchDB directly on my machine, I'll use [Docker](https://www.docker.com/). So, the first step is to configure a `docker-compose.yml` file and the associated `Makefile` to improve developer experience.

```yaml
// ./docker-compose.yml

version: "2.1"

services:
  couchdb:
    image: couchdb:2.3.0
    ports:
      - "5984:5984"

  node:
    image: node:10
    command: npm start
    working_dir: "/app"
    volumes:
      - ".:/app"
    ports:
      - "4242:3000"
    depends_on:
      - couchdb
```

```bash
# ./Makefile

USER_ID = $(shell id -u)
GROUP_ID = $(shell id -g)

export UID = $(USER_ID)
export GID = $(GROUP_ID)

DOCKER_COMPOSE_DEV = docker-compose -p reactive-beers

help: ## Display available commands
	@fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

install: ## Install docker stack
	$(DOCKER_COMPOSE_DEV) run --rm node bash -ci 'npm install'

start: ## Start all the stack
	$(DOCKER_COMPOSE_DEV) up -d

stop: ## Stop all the containers
	$(DOCKER_COMPOSE_DEV) down

log: ## Show logs
	$(DOCKER_COMPOSE_DEV) logs -f node
```

So, we're now ready to start our complete stack using `make install start`.

```bash
julien@julien-P553UA:~/Projets/marmelab/reactive-beers$ docker ps
CONTAINER ID        IMAGE            COMMAND                  CREATED       STATUS       PORTS                                        NAMES
6884f92c5341        node:10          "npm start"              3 hours ago   Up 3 hours   0.0.0.0:4242->3000/tcp                       reactive-beers_node_1
21897f166ce4        couchdb:2.3.0    "tini -- /docker-ent…"   3 hours ago   Up 3 hours   4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcp   reactive-beers_couchdb_1
```

Everything is launched. You may have noticed that the `5984` port is exposed in our `docker-compose.yml` file, it's the CouchDB api. Then, if you open `localhost:5984` in the browser, you'll see something similar to the following.

```json
{
    "couchdb": "Welcome",
    "version": "2.3.0",
    "git_sha": "07ea0c7",
    "uuid": "49f4e7520f0e110687dcbc8fbbb5409c",
    "features": ["pluggable-storage-engines", "scheduler"],
    "vendor": {
        "name": "The Apache Software Foundation"
    }
}
```

## Accessing The Document Store

OK, our server is up and running. But, is there an **interface to visualise / supervise** CouchDB just like Firestore does? The answer is YES! CouchDB already includes an administration interface called `Fauxton`. We can browse it at `http://localhost:5984/_utils/`.

<figure>
  <img src="https://marmelab.com/static/fdcdf33d6e0063a5389376d90d9ba863/5e3ff/firestore.webp" class="responsive" alt="Firestore interface" />
  <figcaption>Firestore Admin Interface</figcaption>
</figure>
<br />
<figure>
  <img src="https://marmelab.com/static/c378a34f9c75a0d001af9b8ae2a31db7/5e3ff/fauxton.webp" class="responsive" alt="Fauxton interface" />
  <figcaption>Fauxton Admin Interface</figcaption>
</figure>

The `Fauxton` interface allows to access databases, setup nodes and clusters, configure replication, setup permissions, etc. Although practical, **it is still preferable to automate these administration tasks with dedicated scripts**.

## React Kicks In

Now, we can start to develop our first PouchDB powered interface. Then, here are our main `App.js` entry point and the `Home.js` start screen.

```js
// ./src/App.js

import React from 'react';
import { Home } from './screens/Home';

const App = () => <Home />;

export default App;
```

The `App.js` file has no interest for the moment. It'll certainly become useful when we need to add more routes and screens in the future.

```js
// ./src/screens/Home.js

import React, { useState, useEffect } from 'react';
import { addBeer, getBeers, onBeersChange } from '../api/beers';

export const Home = () => {
  const [beers, setBeers] = useState([]);

  const refreshBeers = () => getBeers().then(setBeers);

  useEffect(() => {
    // We fetch beers the first time (at mounting)
    refreshBeers();

    // Each change in our beers database will call refreshBeers
    const observer = onBeersChange(refreshBeers);
    return () => {
        // Don't forget to unsubscribe our listener at unmounting
        observer.cancel();
    };
  }, []);

  return (
    <div>
      <button onClick={() => addBeer({ title: 'Beer X' })}>Add a beer</button>
      <ul>
        {/* beer._id is an unique id generated by CouchDB */}
        {beers.map(beer => <li key={beer._id}>{beer.title}</li>)}
      </ul>
    </div>
  );
};
```

## CouchDB Requires More Requests Than Firestore

As you see, in this example we use a combination of a listener (`onBeersChange`) and a a query (`getBeers`) to get the initial beers list and refresh it when a change is issued in the database.

This operation is not optimal compared to the one Firestore offers. Indeed, while **pouchdb is not able to return both changes and data** to us, Firestore is able to do so thanks to a `QuerySnapshot` system, thereby reducing server trips back and forth. See by yourself with the Firestore example below:

```js
  db.collection("anything")
    .onSnapshot(function(querySnapshot) {
        querySnapshot.forEach(function(doc) {
          // This forEach loop is executed at first execution
          // And executed each time the query result changes
        });
    });
```

So, if we had used Firestore instead, here's what it would have looked like:

```js
  //...

  const [beers, setBeers] = useState([]);

  useEffect(() => {
    const unsubscribe =  db.collection("beers")
      .onSnapshot(function(querySnapshot) {
          const snapBeers = [];
          querySnapshot.forEach(function(doc) {
              snapBeers.push(doc.data());
          });

          setBeers(snapBeers);
      });

    return () => {
        unsubscribe();
    };
  }, []);

  // ...
```

<div class="tips">

It's also possible to use `.map` on the `querySnapshot.docs` attribute to retrieve all the documents in a "non-imperative" way. Unfortunately, this functionality is not sufficiently covered by the official documentation.

</div>

## The Model

Just like in backend development, I like to separate the model logic from the view logic in my frontend apps. So here is the API file for our beers below:

```js
// ./src/api/beers.js

import PouchDB from 'pouchdb';

// We declare a PouchDB instance that is "remote only"
// There's no "offline" capability for the moment, everything is sync
export const beersDatabase = new PouchDB('http://localhost:5984/beers');

// If the beers database does not already exist
// => The database is automatically created when an object is added to it
export const addBeer = beer => beersDatabase.post(beer);

// Here, we list all the documents from our beers database
// A lot of options exists. Eg: we can paginate using "startKey", "endKey" or "limit"
export const getBeers = () =>
  beersDatabase
    .allDocs({
      include_docs: true,
      descending: true,
    })
    .then(doc => doc.rows.map(row => row.doc));

// We listen all the changes that happen since now
// We can also apply a "limit" option to this method
export const onBeersChange = callback => beersDatabase
    .changes({ since: 'now', live: true })
    .on('change', callback);
```

Here is the result of our first CouchDB application in action. As you can see, everything is in sync between multiple windows.

<img src="https://marmelab.com/sync-beers-b5961268f0dda773642fcd11173a5715.gif" class="responsive" alt="PouchDB sync" />

### Offline Sync

Sadly, our actual version only works when the internet access is up and running. In other cases, such as a bloated network or packet loss, beers will never (or slowwwwly...) be added in the beers list because of the "remote only" sync.

The right way to avoid this problem is to keep a **local first** approach. It means that we must achieve all our database operations on the local database, then synchronize it with the remote one when internet access comes back.

<img src="https://marmelab.com/static/f10e78636cdc43ba0a8ab24292528975/5443b/nointernet.webp" class="responsive" alt="Internet Offline" />

So, the first step is to declare a new PouchDB instance with a database name instead of a remote database url. This way, PouchDB automatically detects that we want to instantiate a local database.

```js
import PouchDB from 'pouchdb';

// Declare local database
const beersDatabase = new PouchDB('beers');

// Declare remote database
const remoteBeersDatabase = new PouchDB(`http://localhost:5984/beers`);

// Keep local and remote databases in sync
PouchDB.sync(beersDatabase, remoteBeersDatabase, {
  live: true, // replicate changes in live
  timeout: false, // disable timeout
  retry: true, // retry sync if fail
});
```

The `PouchDB.sync` instruction is the equivalent of a bidirectionnal `PouchDB.replicate` instruction between local and remote databases.

```js
PouchDB.replicate(beersDatabase, remoteBeersDatabase);
PouchDB.replicate(remoteBeersDatabase, beersDatabase);
```

By default, PouchDB uses [IndexedDB](https://developers.google.com/web/ilt/pwa/working-with-indexeddb) as local database (just like Firestore by the way). So, now that our setup is done, we can take a look at our local database using the Chrome console.

<img src="https://marmelab.com/static/f43455cb45fcdd00845ea719ce4ee1fd/5e3ff/indexeddbcapture.webp" class="responsive" alt="Indexed DB" />

As you can see, we find the complete list of beers we've created. Each one is uniquely identified by a `key` that is built from `_id` and `_rev` CouchDB attributes.

```js
{
  "_id": "0c2738a3-d363-405f-b9bb-0ab6f5ec9655",
  "_rev": "3-b90bd9d62fbe04e36fe262a267efbd42",
  "title": "Beer X"
}
```

Whereas the `_id` represents an unique document, the `_rev` represents the **revision** identifier of it. In fact, each modification of a document implies a new version of it which then makes it possible to manage conflicts.

<div class="tips">

Unlike CouchDB, Firestore documents do not have a **revision** id. So, the only way not to struggle with conflicts using Firestore is to use [**transactions**](https://firebase.google.com/docs/firestore/manage-data/transactions).

Moreover, since CouchDB records every submitted change, **it is possible to return back or resolve conflict in a second time**, which is essential in order not to risk losing data.

For more information on conflict management using PouchDB, check [the PouchDB Conflict documentation](https://pouchdb.com/guides/conflicts.html).

</div>

Now that we are able to communicate with both local and remote databases, we can focus on the business logic and on the user interface.  Moreover, it'll allow us to benefit from **optimistic rendering** while making our application **more flexible in addressing network issues**.

## Forms & Validation

In this section, we will implement a form to be able to add new beers. To do that, I'm going to use `final-form` (and `react-final-form`, an adapter for ReactJS).

``` bash
npm install -S final-form react-final-form
```

So, we can create a simple form to handle user input.

```js
// ./src/components/BeerForm.js

import React from 'react';
import { Form, Field } from 'react-final-form';

export const BeerForm = ({ onSubmit }) => (
  <Form
    validate={() => ({})}
    onSubmit={onSubmit}
    render={({
      handleSubmit,
      hasValidationErrors,
      pristine,
      invalid,
      submitErrors,
      submitting,
      form,
    }) => (
        <form onSubmit={handleSubmit}>
         <div>
            <label>Title</label>
            <Field name="title" component="input" />
          </div>
          <div>
            <label>Description</label>
            <Field
              name="description"
              component="textarea"
              rows={2}
              placeholder="Tape your description here..."
            />
          <div/>
          <button type="submit" disabled={pristine || hasValidationErrors || submitting}>
            Submit
          </button>
          {submitErrors && submitErrors.global && (
            <p>{submitErrors.global}</p>
          )}
        </form>
      )
    }
  />
);
```

Then, we can replace our action button by the form in our home screen.

```js
// ./src/screens/Home.js

import React, { useState, useEffect } from 'react';
import { addBeer, getBeers, onBeersChange } from '../api/beers';

export const Home = () => {
  const [beers, setBeers] = useState([]);

  /* ... */

  return (
    <div>
      <BeerForm onSubmit={beer => queries.addBeer(beer)} />
      <ul>
        {/* beer._id is an unique id generated by CouchDB */}
        {beers.map(beer => <li key={beer._id}>{beer.title}</li>)}
      </ul>
    </div>
  );
};
```

## Data Validation With Indicative

So, we have a form, but there's no data validation for the moment. Users can send anything they want at this time. That's why we're going to set up a data validator using `indicative`, a library that I've just discovered and that I want to give a try.

```bash
npm install -S indicative
```

The Indicative API is very simple. It consists of a `Validator` object that uses a set of validation rules and a `formatter`. Here is a usage example:

```js
import Validator from 'indicative/builds/validator';
import { Vanilla as VanillaFormatter } from 'indicative/builds/formatters';
import { required, email } from 'indicative/builds/validations';

const validator = Validator({ required, email }, VanillaFormatter);

const rules = {
  name: 'required',
  email: 'required|email',
};

const messages = {
  'required': '{{ field }} field is required', // This message works for all required rules
  'email.required': 'You must provide an email!', // This message is specific for required email
  'email.email': 'The email adress is invalid',
};

const values = {
  email: 'bad email',
};

// Validator.validate is async

validator
  .validate(values, rules, messages)
  .then(() => /* everything is ok! */)
  .catch((errors) => {
    /*
      [
        { field: 'name', message: 'name field is required!' },
        { field: 'email', message: 'The email adress is invalid' },
      ]
    */
  });
```

Here is our custom implementation for `BeerForm.js`.

```js
// ./src/components/BeerForm.js

import React from 'react';
import { Form, Field } from 'react-final-form';
import { Vanilla } from 'indicative/builds/formatters';
import Validator from 'indicative/builds/validator';
import { required } from 'indicative/builds/validations';

const validator = Validator({ required }, Vanilla);

const rules = {
  title: 'required',
  description: 'required',
};

const messages = {
  'title.required': 'Beer title is required',
  'description.required': 'Beer description is required',
};

const validate = async values =>
  validator
    .validate(values, rules, messages)
    .then(() => ({}))
    .catch(errors => {
      return errors.reduce((acc, error) => {
        acc[error.field] = error.message;
        return acc;
      }, {});
    });
```

Final Form needs an object as error model, so we format errors in the `catch` using a `reduce`. Alternatively, it would have been possible to use a [Custom Indicative Formatter](https://indicative.adonisjs.com/docs/formatters#_creating_your_own).

So, now we have our custom validation function, we can replace our empty validate function.

```diff
export const BeerForm = ({ onSubmit }) => (
  <Form
-  validate={() => ({})}
+  validate={validate}
```

And tadaaa! Our validated form is up and running and we are ready to play with it.

<img src="https://marmelab.com/finalformexample-01ec0a572c0ec1fa17489fca7b68df3e.gif" alt="Final Form Indicative" />

## Let's Make It Beautiful!

To summarize, we can display beers, we can add beers, everything works offline and is in sync with a remote server. But right now, it's not very aesthetic, and I wouldn't dare present it to my mother-in-law. So, how about making it a little prettier?

In this section, I'll use the [Elastic UI](https://elastic.github.io/eui/) framework (aka `eui`) that is in use at [Elastic](https://www.elastic.co/fr/), the company that develops ElasticSearch.

I think we all agree that we must remove this despicable list and replace it with a nice grid. Fortunately, Eui allows it easily.

<img src="https://marmelab.com/static/e8d11cc732ffe9172914a6b8b72c0d93/5e3ff/beergridfull.webp" alt="Beer Grid" />

As you can see, we took the opportunity to add editing and deleting beers right from the grid. We'll also put the form in a sliding panel from the right of the page. This way, we can directly add a beer from a "+" button in the navbar, or edit a beer directly from the grid, without changing page.

## Handling Image Attachments

I don't know about you, but seeing all these grey beer cans breaks my heart. So it's time to allow image upload in the form.

```js
// ./src/components/BeerForm.js

const handleIllustration = async files => {
  if (files.length === 0) {
    form.change('_image', undefined);
    return;
  }

  const file = files[0];

  form.change('_image', {
    data: file,
    type: file.type,
  });
};

<EuiFormRow label="Beer Illustration">
  <EuiFilePicker onChange={handleIllustration} />
</EuiFormRow>
```

This custom `_image` attribute that I just added to the beer object is then handled by our beer api, and considered as a [**PouchDB attachement**](https://pouchdb.com/guides/attachments.html).

```js
// ./src/api/queries.js

const saveBeer = async ({ _image, ...beer }) =>
  store
    .collection('beers')
    .post(beer)
    .then(
      ({ id, rev }) =>
        // if an "_image" attribute is present, we put an attachement to the document
        _image &&
        store
          .collection('beers')
          .putAttachment(id, 'image', rev, _image.data, _image.type)
    );

const getBeers = () =>
  store
    .collection('beers')
    .allDocs({
      include_docs: true,
      descending: true,
      attachments: true, // We include images in the output, so we can display them
    })
    .then(doc => doc.rows.map(row => row.doc));
};
```

In CouchDB, every file can directly be attached to its corresponding document as an `attachement`. This concept does not exist in Firestore. It is then preferable to use [Firebase Storage](https://firebase.google.com/docs/storage) (Google Cloud Storage) through its bucket system to store files and store paths in Firestore.

<img src="https://marmelab.com/beerregistry-24aa02cf340233c71fdb4429a3d8851e.gif" alt="Final Application" />

## Conclusion

The final result of my beer registry application is available on GitHub  at the following address: [github.com/marmelab/reactive-beers](https://github.com/marmelab/reactive-beers). Feel free to comment and improve!

While I was doubtful about the power of CouchDB at first, I was quickly conquered by its stability and the ease of use of its API.

Since I have not yet deployed this type of application in production, I am not in a position to comment on the ease of maintenance of this type of database. Nevertheless, I would rather recommend using **Firestore for POCs**, and a **third-party service like [Couchbase](https://www.couchbase.com/products/data-platform) or [IBM Cloudant](https://www.ibm.com/cloud/cloudant) for critical applications** in the first place.

Although this experience allowed me to balance the pros and cons of the main features of each database, it was not possible for me to go as far as I had expected.

Indeed, I didn't have time to cover many crucial points such as *document access security*, *rights management*, *server-side document validation*, *data pagination* or *deployment*. But no matter what, I am determined to write more articles on these topics.

So, Stay tuned!