abedra/accession — pull request #16 by ptaoussanis (2012-05-11 14:48:44+00:00)
Title: Complete refresh of Accession - connection pooling, performance improvements, etc.

Hi there,

Sorry about the monolithic pull request- please bear with me. I'll try keep my comments brief and break them into sections.

MOTIVATION:
* Redis is fantastic and keeps getting better, but the Clojure client story isn't great.
* clj-redis is based on Jedis which is a painful, unnecessary impedance mismatch for Clojure.
* redis-clojure is decent and production-ready, but is based on the old (pre 2.0) Redis protocol and has a lot of cruft.
* accession has wonderfully simple, Redis 2.0 protocol semantics but isn't production-ready (in particular, the lack of connection pooling is painful).
* There's too many clients! It'd be better if everyone could get behind one - and accession is IMO the best base.

WHAT I DID
* Added connection-pooling via Apache commons-pool, adapted from redis-clojure.
* Added other major features from redis-clojure (nice sort fn, bytes macro, automatic password authentication, etc.).
* Cleaned up the Pub/Sub API which was, IMO, overly complex. In particular, it was conflating Redis-side state (active subscriptions) with Clojure-side state (handlers).
* Changed some nomenclature/docstrings for clarity.
* Simplified the response parsing, speeding it up significantly. The multimethod was unnecessary.
* A number of tweaks & suggestions from other forks (e.g. breaking up the command-sending to allow sending of larger queries without OOM errors).
* Error responses now throw an exception as per the Redis client recommendation.
* Better handling of malformed data (e.g. request won't hang with empty query, invalid queries will generate an appropriate error response, etc.).
* Lua script commands.
* Converted all of the unit-tests and docs to reflect the new APIs.
* Probably a few other things I'm forgetting?

WHAT I KEPT
* All the Redis protocol stuff (save for one change: the query fn no longer provides the final CRLF).
* The basic calling style (with-connection )

CURRENT STATUS
* All the unit tests are passing.
* Connection pool appears to be working well but I haven't tested it formally.
* Lua commands completely untested.
* New sort fn should have some unit-tests.
* Initial performance tests appear promising: at least as fast as the best Clojure clients and not to far off the theoretical max for a JVM client.
* I only spent 2-3 days on this, and that included the time to understand the client- so there are likely still issues that aren't covered by the tests. I'm not using it in production myself either, so wouldn't feel safe to call it "production" ready until it's seen some action.

FUTURE WORK
* More testing would be #1. Assuming any remaining issues get ironed out, I don't see what would keep this from heading for a "1.0" release?
* I'm not sure what your feelings are on more hand-holding stuff like offering auto-coercion, but deep-freeze (https://github.com/halgari/deep-freeze) offers a fast serializer that'd be a great fit for Redis (it's Snappy-compressed and can read right from the Redis bytestrings for performance). It'll require some thought but might be nice to try offer a switch for users that want something to work simply out-the-box?

- Peter