internetarchive/openlibrary — Issue #99: Move away from CouchDB
Author: anandology
Created: 2011-12-13 06:32:43+00:00
State: open

CouchDB is not very stable. It has created us trouble couple of times. Also its been very inefficient and not very easy to replicate or compact as initially thought. We should move away from couch and find alternatives using existing components.

We use the following couchdb databases.

* works
* editions
* seeds
* admin
* celery

The first 3 are used for lists. They are used to answer the following queries.

* subjects and last-updated time for each seed
* find all editions of a list ordered by last-modified

The first query can be answered by solr if we start storing last-update time. The second can be answered by datastore.

The admin and celery databases can also be replaced by datastores.