HubSpot/java-memcached-client — pull request #5 by eabbott (2014-11-21 22:25:32+00:00)
Title: Synchronous incr/decr calls will now update the internal statistics

Synchronous incr/decr calls will now update the internal statistics spymemcached uses to determine a remote memcached node is timing out, and thus unavailable. This change is particularly aimed at JavaRateLimiter classes that do not use the regular memcached pool, but instead use the pool with FailureMode.Redistribute. With nothing else using the FailureMode.Redistribute pool, a timing out memcached node would never be pulled from the active duty and would not only time out continuously, but would never fulfill the FailureMode.Redistribute trait of failing over to an available node.

This failure condition is only for memcached nodes that continuously time out, such as our most common memcached failure situation in which an AWS instance just disappears into a black hole. Connections to that server continually time out until a developer forces the instance to reboot or replaces the instance.

To further clarify spymemcached driver behavior, the internal timeout statistics are a different mechanism of failure detection than the simple 'connection refused' situation such as a memcached process stopping/crashing. Situations in which the network layer knows the process is down (ie connection refused) currently act as desired.

@jdwyah @sit @axiak @HiJon89