openstack/swift — pull request #10 by linsomniac (2013-06-22 05:30:36+00:00)
Title: Stock library *DOES* persist existing keys.

Here is the relevant code from the stock module, it only relocates keys on servers which are down. It does not "invalidate a huge percentage of the cached items".

def _get_server(self, key):
if isinstance(key, tuple):
serverhash, key = key
else:
serverhash = serverHashFunction(key)

for i in range(Client._SERVER_RETRIES):
server = self.buckets[serverhash % len(self.buckets)]
if server.connect():
#print "(using server %s)" % server,
return server, key
serverhash = serverHashFunction(str(serverhash) + str(i))
return None, None

I didn't realize this either, and I maintain the python-memcache module (but I didn't write the code in question). Just figured we should clear up this statement to prevent others from thinking it's true.

Note that the replacement module I'm working on, python-memcached2, includes a Consistent Hashing implementation and one like the above. I have "issues" open to do connection pooling but haven't yet implemented it.

I appreciate these pointers, because I've been looking for feedback on things that the new module should do.

Note that the new module isn't ready for prime time, it's still in development mode.