koding/kite — pull request #65 by fatih (2014-10-19 15:40:48+00:00)
Title: kontrol: independent storage support, add PostgreSQL as additional storage.

This PR aims to decouple etcd dependency from Kontrol so that we can use any storage backend for kites. Removing the dependency means also some tradeoffs such as removing Watch functionality (will be implemented later again). This PR has the following changes:

* Better Storage support by providing a generic storage interface.
* A new `SetStorage()` method that needs to be set before we call the `Run()` method.
* Seperate `Set()` method into `Update()` and `Add()` methods for better granular control of the storage interface.
* Remove watcher functionality to make the storage interface simpler. Also the watcher functionality needed already some refactor. The watcher methods will be supported via a new `Watcher` interface which can be implemented later again with Etcd's own watch functionality or Postgres `NOTIFY` support.
* Add additional storage support via PostgreSQL. This is a huge change. The implementation details are:
* New constructor via the `NewPostgres()` function
* Data is stored in a table called `kites`, with the column names of `kite`, `url`, `id`, `created_at` and `updated_at`
* The `kite` column is defined as hierarchical trees via the `ltree` extension. It's similar to what etcd offering as "directory a like structure". `ltree` provides a performant and simple way to iterate over all nodes and set them individually.
* The `url` column is used to store the URL which the kite was registered
* The `id` column is added which acts as `PRIMARY KEY` so every unique kite can be added just once. This is import so we don't add the same kite over and over. It's the same the kites own `ID` which is in form of UUID V4. We don't have default's for this field because each kite does already have a unique ID.
* Both `created_at` and `updated_at` are using UTC timestamps
* Indexes on the `kite` column are added to further increase the performance
* Version constrains (such as `~> 1.1`) are fully supported.
* Custom functions are added to provide safety for `ltree` labels which can only be in the form of alphanumeric characters and underscores. So every field of a kite is checked and converted before we add a new row.
* Passes all tests.

It's still not finished. It needs testing. For example TTL of keys are missing. For this we are going to add a simple cron job or script that is going to check the `updated_at` column and cleaning up rows, or just a simple cleaner goroutine in the background.