Package: libmc
libmc is a memcached client library for Python without any other
dependencies in runtime. It’s mainly written in C++ and Cython. libmc
can be considered as a drop in replacement for libmemcached andpython-libmemcached.libmc is developing and maintaining by Douban Inc. Currently, It is
working in production environment, powering all web traffics in
douban.com. Realtimebenchmark
resultis
available on travis.Build and InstallationFor users:pip install libmcUsage:importlibmcmc=libmc.Client(['localhost:11211','localhost:11212'])mc.set('foo','bar')assertmc.get('foo')=='bar'Under the hoodUnder the hood, libmc consists of 2 parts: an internal fully-functional
memcached client implementation in C++ and a Cython wrapper around that
implementation. Dynamic memory allocation and memory-copy are slow, so
we tried our best to avoid them. Theset_multicommand is not
natively supported by thememcached
protocol.
Some techniques are applied to makeset_multicommand extremely fast
in libmc (compared to some other similiar libraries).Configurationimportlibmcfromlibmcimport(MC_HASH_MD5,MC_POLL_TIMEOUT,MC_CONNECT_TIMEOUT,MC_RETRY_TIMEOUT)mc=libmc.Client(['localhost:11211','localhost:11212','remote_host','remote_host mc.mike','remote_host:11213 mc.oscar'],do_split=True,comp_threshold=0,noreply=False,prefix=None,hash_fn=MC_HASH_MD5,failover=False)mc.config(MC_POLL_TIMEOUT,100)# 100 msmc.config(MC_CONNECT_TIMEOUT,300)# 300 msmc.config(MC_RETRY_TIMEOUT,5)# 5 sservers: is a list of memcached server addresses. Each address
can be in format ofhostname[:port][alias].portandaliasare optional. Ifportis not given, default port11211will
be used.aliaswill be used to compute server hash if given,
otherwise server hash will be computed based onhostandport(i.e.: Ifportis not given or it is equal to11211,hostwill be used to compute server hash. Ifportis not equal to11211,host:portwill be used).do_split: Memcached server will refuse to store value if size >=
1MB, ifdo_splitis enabled, large value (< 10 MB) will be
splitted into several blocks. If the value is too large (>= 10 MB),
it will not be stored. default:Truecomp_threshold: All kinds of values will be encoded into string
buffer. Ifbuffer length > comp_threshold > 0, it will be
compressed using zlib. Ifcomp_threshold = 0, string buffer will
never be compressed using zlib. default:0noreply: Whether to enable memcached’snoreplybehaviour.
default:Falseprefix: The key prefix. default:''hash_fn: hashing function for keys. possible values:MC_HASH_MD5MC_HASH_FNV1_32MC_HASH_FNV1A_32MC_HASH_CRC_32default:MC_HASH_MD5NOTE:fnv1_32, fnv1a_32, crc_32 implementations in libmc are
per each spec, but they’re not compatible with corresponding
implementions in libmemcached.failover: Whether to failover to next server when current server
is not available. default:FalseMC_POLL_TIMEOUTTimeout parameter used during set/get procedure.
(default:300ms)MC_CONNECT_TIMEOUTTimeout parameter used when connecting to
memcached server on initial phase. (default:100ms)MC_RETRY_TIMEOUTWhen a server is not available dur to server-end
error. libmc will try to establish the broken connection in everyMC_RETRY_TIMEOUTs until the connection is back to live.(default:5s)NOTE:The hashing algorithm for host mapping on continuum is always
md5.Contributing to libmcFeel free to send aPull Request. For feature requests or any
questions, please open anIssue.ForSECURITY DISCLOSURE, please disclose the information responsibly
by sending an email tosecurity@douban.comdirectly instead of creating
a GitHub issue.FAQDoes libmc support PHP?No. But if you like, you can write a wrapper for PHP based on the C++
implementation.Is Memcached binary protocol supported ?No. Only Memcached ASCII protocol is supported currently.Why reinventing the wheel?Before libmc, we’re usingpython-libmemcached,
which is a python extention forlibmemcached.
libmemcached is quite weird and buggy. After nearly one decade, there’re
still some unsolved bugs.Is libmc thread-safe ?libmc is a single-threaded memcached client. If you initialize a libmc
client in one thread but reuse that in another thread, a Python
ExceptionThreadUnsafewill raise in Python.Is libmc compatible with gevent?Yes, with the help ofgreenify,
libmc is friendly to gevent. Readtests/shabby/gevent_issue.pyfor
details.Notice:gevent.monkey.patch_all()will overridethreading.current_thread().identto Greenlet’s ID,
this will cause libmc to throw a ThreadUnSafe error
or run into dead lock, you should only patch the things
that you need, e.g.fromgeventimportmonkeymonkey.patch_socket()AcknowledgmentsThanks to@fahrenheit2539and
the llvm project for the standalone.SmallVectorimplementation.Thanks to@miloyipfor the high
performancei64toaimplementation.Thanks toIvan Novikovfor the
research inTHE NEW PAGE OF INJECTIONS BOOK: MEMCACHED
INJECTIONS.Thanks to the PolarSSL project for the md5 implementation.Thanks to@lericsonfor thebenchmark
script in
pylibmc.Thanks to the libmemcached project and some other projects possibly
not mentioned here.Contributors@mckelvin@zzl0@windreamer@lembacon@seansay@mosasiru@jumpeiManoWho is using豆瓣下厨房Some other projects on GitHubWant to add your company/organization name here?
Please feel free to send a PR!Documentationhttps://github.com/douban/libmc/wikiLICENSECopyright (c) 2014-2020, Douban Inc. All rights reserved.Licensed under a BSD license:https://github.com/douban/libmc/blob/master/LICENSE.txt