groupme/faye-redis-node — pull request #1 by daveyeu (2014-10-17 14:06:56+00:00)
Title: Rewrite GC to handle Redis failures

The most important change here is that Redis failures are now expected, and the entire GC/destroy client process has been updated to handle them. Broadly, failures are tolerated by allowing the GC to be re-runnable for a failed client ID.

When GC starts up, we fan out and call `destroyClient(clientId)` for each candidate expired client ID. Each one of these calls can fail individually, and processing simply stops for that client ID. Since the removal of the client ID from the `/clients` sorted set is the last step, GC will simply pickup failed client IDs and re-run them in the future. As such, the lock around GC has also been removed, since failures are expected and multiple deletion runs for a single client ID are tolerable.

Other notable changes:

* The score is now assigned to a client ID in the initial `ZADD` call in `createClient`. Previously, the score was assigned to zero, and updated in the callback with `ping()`. This is susceptible to a race, however, where GC can run and pluck the client ID before its score is updated.

* Publishing no longer destroys clients. Previously, publish would check the expiry of every client ID, and call `destroyClient` if it detected an expired client. This could lead to races, so it's now the sole responsibility of GC to cleanup old clients.