Saravanan666/pypi-org huggingface dataset:
Package: redispatcher
redispatcherDispatch and run distributed work asynchronously, brokered by RedisDocumentation:https://rafalstapinski.github.io/redispatcherSource Code:https://github.com/rafalstapinski/redispatcherWhat is redispatcherredispatcher allows you to dispatch work that needs to be done in a process separate from your main program loop. This is useful in cases when you need to process some long running work that doesn't necessarily need to be done synchronously within your code. A classic example of this is sending a welcome email to a user as they sign up for your service. It's not necessary to wait for the results of sending an email, and it may take a few seconds to do so. redispatcher lets you fire-and-forget this work (as a message put into a Redis server) and have it be executed in the background, asynchronously, in a separate process (or machine) entirely.redispatcher comes in two parts:A library that lets you define workers, define strongly typed messages sent to workers, and provides helper functions to facilitate dispatching that workA daemon that runs your workers in a pool, consistently listening for any incoming messages to be processedWhy use itThere are certainly other solutions for orchestrating distributed workers. redispatcher aims to be super lightweight, very fast and simple to set up (there are many free cloud-hosted Redis solutions available), and has robust type validation and intellisense support.FeaturesFull intellisense support across your code, despite a distributed workloadStrongly typed message contract between your publishing code and consumerMinimal boilerplate required to setup and start publishing compared than most alternativesMinimal performance overhead and completely non-blocking, thanks toasyncio(and works with alternatives likeuvloop)Dependenciesaioredisis used under the hood to publish message to and read messages from Redispydanticis used to to validate messages conform to your strongly typed contractsInstallationInstall withpoetry$poetryaddredispatcheror withpip$pipinstallredispatcherBasic UsageDefining your workerfromredispatcherimportBaseConsumerclassSendWelcomeEmail(BaseConsumer):QUEUE="send-welcome-email"classMessage(BaseConsumer.Message):email:strname:strasyncdefprocess_message(self,message:Message):# construct an email and send it to the `message.email` addressDispatching workfromclientsimportmy_aioredis_client@app.post("/register")asyncdefregister(...)...message=SendWelcomeEmail.Message(email=...,name=...,registered=True)awaitSendWelcomeEmail.dispatch(message,my_aioredis_client)...Running redispatcherfromredispatcherimportRedispatcher,RedispatcherConfig,ConsumerConfigconfig=RedispatcherConfig(redis_dsn="redis://localhost:6379/0",consumers=[ConsumerConfig(consumer_class=SendWelcomeEmail,count=2)])if__name__=="__main__":dispatcher=Redispatcher(config)dispatcher.start()Contributingredispatcheris already used in production, but is still in its infancy.If you find a bug,open an issuewith a detailed description and steps to reproduce.If you're looking for a feature,open an issuewith a detailed description and use case. Feel freeopen a pull requestif you want to contribure directly!
Package: redis-persistence
SetupYou can find the package,here.pip3 install redis-persistenceRequirementsPython 3.8Redis 3.5Importfromredispersistence.persistenceimportRedisPersistenceUsageYou need to use this persistence withpyton-telegram-botmodule.redis_instance=Redis(host='localhost',port=6379,db=0)persistence=RedisPersistence(redis_instance)updater=Updater(BOT_TOKEN,persistence=persistence)
Package: redisplus
使用手册快速开始importredisplus# 获取原生的Redis对象# redis = redisplus.getRedis(host="docker.kancy.top", password="root123")# 获取SimpleRedis对象，不支持的的value会自动实现序列化redis=redisplus.getSimpleRedis(host="docker.kancy.top",password="root123")# 设置属性redis.set("name","kancy")# 查询redis.get("name")# 删除redis.delete("name")# 设置过期时间redis.setex("name","kancy",0.01)# 设置，当key不存在时redis.setnx("name","kancy")
Package: redis-property
redis-propertyA decorator for caching properties in redis. Inspired bycached-propertyInstallationpip3installredis-propertyHow to use itSetup redis_propertyimportredis_propertyredis_property.configure("redis://:123456@example:6379/0",lambdao,f:f"{type(o).__name__}:{o.id}:{f.__name__}")use it like python propertyfromredis_propertyimportredis_propertyclassSomething:id=1@redis_property(10)# 24h by defaultdefname(self):return1something=Something()print(something.name)invalidating the cachedelsomething.namework with cached-propertyfromcached_propertyimportcached_propertyfromredis_propertyimportredis_propertyclassSomething:id=1@cached_property@redis_propertydefname(self):return1
Package: redis_protocol
===============================Redis Protocol===============================.. image:: https://badge.fury.io/py/redis_protocol.png:target: http://badge.fury.io/py/redis_protocol.. image:: https://travis-ci.org/youngking/redis_protocol.png?branch=master:target: https://travis-ci.org/youngking/redis_protocol.. image:: https://pypip.in/d/redis_protocol/badge.png:target: https://crate.io/packages/redis_protocol?version=latestRedis Protocol implemented by python* Free software: BSD license* Documentation: http://redis_protocol.rtfd.org.Usage--------This is the protocol implemented followed by `redis protocol specification <http://redis.io/topics/protocol>`_.I had used it in my `redis_proxy <https://github.com/youngking/redis_proxy>`_ project.::>>> from redis_protocol import decode, encode>>> encode("ping")  # encode a request... '*1\r\n$4\r\nping\r\n'>>> decode('*1\r\n$4\r\nping\r\n') # decode a request body... ["ping"]>>> decode("$6\r\nfoobar\r\n")  # decode a response... "foobar"parse redis protocol stream------parse redis protocol stream to redis commands，such as redis pipeline requests or raw responses.::>>> from redis_protocol import parse_stream>>> data = '*3\r\n$3\r\nSET\r\n$15\r\nmemtier-8232902\r\n$2\r\nxx\r\n' \'*3\r\n$3\r\nSET\r\n$15\r\nmemtier-8232902\r\n$2\r\nxx\r\n' \'*3\r\n$3\r\nSET\r\n$15\r\nmemtier-7630684\r\n$3\r\nAAA\r\n'>>> print(parse_stream(data))... ['SET memtier-8232902 xx', 'SET memtier-8232902 xx', 'SET memtier-7630684 AAA']History-------0.0.1 (2013-08-11)++++++++++++++++++* First release on PyPI.
Package: redis_proxy
redis_proxyA redis proxy that supprot read/write splitting. It works like haproxy,
but only used with redis.For now it’s a very rough project.The source is atredis proxy.Usageinstall:pip install redis_proxythen run:redis-proxy -c config.jsonConfig ExampleFor example:[
 {"listen": 6378,
  "master":{"host": "host01", "port": 6378},
  "slave": [{"host": "host02", "port": 6380}]
 },
 {"listen": 6377,
  "master":{"host": "host01", "port": 6379},
  "slave": [{"host": "host02", "port": 6380}, {"host": "host03", "port": 6380}]
 }
]The proxy will listen on port 6377 and 6378, and forward read to the slave, write to the master.
Package: redispubsub
redis-pub-subRedis RPC with PubSub simple lightweightGetting StartedInstall Redis rpcpipinstallredispubsubServer exampleimportosfromredisrpcimportRedisRPC# Add REDIS_URI application enviroment variableos.environ.setdefault("REDIS_URI","redis://localhost:6379/0")rpc=RedisRPC("channel_name")# rename what you want# event listsdefcalc_square(response):# `response` is a sender datapower_of_number=response**2returnpower_of_number# sent to clientdefcalc_cube(response):# `response` is a sender datacube_of_number=response**3returncube_of_number# sent to clientrpc.register(calc_square,"square")# event name default function namerpc.register(calc_cube,"cube")rpc.listen()Client interfaceimportosfromredisrpcimportRedisRPC# Add REDIS_URI application enviromentos.environ.setdefault("REDIS_URI","redis://localhost:6379/0")rpc=RedisRPC("channel_name")# channel name must be same as serversquare=rpc.send("square",5)# send data to spesific eventcube=rpc.send("cube",3)# response from server handlersprint(square)# 25print(cube)# 27ContributingFell free to open issue and send pull request.
Package: redis-pubsub-dict
A python class for using redis, or other key value stores, as a
dictionary and caching the values locally for read heavy workloads.
Heavily inspired bypylru.UsageThe idea is that Deleting or Updating keys in an instance ofPubSubRedisDictorPubSubCacheManagerwill update the matching
cached keys in all instances ofPubSubCacheManager.PubSubCacheManagerwill therefor maintain a cache of recently used
keys using anlruor just a straight updict(). This will reduce
the round trip latency and network overhead for any reads of the cached
keys.RedisDictandRedisDictshould work with instances ofredis.StrictRedisorrediscluster.StrictRedisCluster. Use theprefixfor managing redis key namespaces.RedisDictJust like a normal dictionary, but networked. Initialisation wont take a
dictionary or iterable for now as it need connection and namespace
information.rc=StrictRedisCluster(startup_nodes=[{"host":"redis","port":"6379"}])reddict=RedisDict(rc,'namespace')# you can setreddict[1]=1reddict[2]=[1,2,3]reddict['hello']='world'reddict[('complex',1)]={'I':{'Am':{'Quite':['a','complex',{'object':{}}]}}}# get somewhere elsereddict[1]reddict['1']# note its the same as reddict[1]reddict[('complex',1)]reddict["('complex', 1)"]# the key is str(('complex',1))# deletedelreddict[1]# .. ectPubSubRedisDictLikeRedisDictbut will publish key update and delete events to a<namespace>/[update|delete]channel.redpubsub=PubSubRedisDict(rc,'namespace')# ect as beforePubSubCacheManagerLikepylry.WriteThroughCacheManagerbut updates cache keys from
store when it receives a message from the<namespace>/[update|delete]channel.cache=pylru.lrucache(10)# maybe more than 10redstore=PubSubRedisDict(rc,'namespace')redcache=PubSubCacheManager(redstore,cache)# ect as before# see the cacheprintdict(redcache.cache)Further usesYou can hook upRedisDictorPubSubRedisDicttopylru.WriteBackCacheManagerto get a Redis backed dictionary which
only writes to Redis on ‘flush’ or when the item pops off thelrufor write intensive workloads. However a lot more work would need to be
done to add the pubsub mechanism as there difficult cases to consider,
such as what happens when the cache is dirty and we get notified that
the store key is updated?Limitationsall keys are strings.msgpackis used to marshal objects to redis, somsgpackobject limitations apply. Though you can monkey patch the modulesloadsanddumpsmethod if you like.publish will publish to all consuming dictionary instances, there is
no partitioning, so writes and updates are expensive. You could come
up with a partitioning strategy to improve this.The published items eventually end up in the watched cash. There may
be a time lag between a client publishing a change and the key
updating in another clients cache.Referencesredis-pyredis-py-clusterpylru
Package: redis-pubsub-dict-wo-cluster
A python class for using redis, or other key value stores, as a
dictionary and caching the values locally for read heavy workloads.
Heavily inspired bypylru.UsageThe idea is that Deleting or Updating keys in an instance ofPubSubRedisDictorPubSubCacheManagerwill update the matching
cached keys in all instances ofPubSubCacheManager.PubSubCacheManagerwill therefor maintain a cache of recently used
keys using anlruor just a straight updict(). This will reduce
the round trip latency and network overhead for any reads of the cached
keys.RedisDictandRedisDictshould work with instances ofredis.StrictRedisorrediscluster.StrictRedisCluster. Use theprefixfor managing redis key namespaces.RedisDictJust like a normal dictionary, but networked. Initialisation wont take a
dictionary or iterable for now as it need connection and namespace
information.rc=StrictRedisCluster(startup_nodes=[{"host":"redis","port":"6379"}])reddict=RedisDict(rc,'namespace')# you can setreddict[1]=1reddict[2]=[1,2,3]reddict['hello']='world'reddict[('complex',1)]={'I':{'Am':{'Quite':['a','complex',{'object':{}}]}}}# get somewhere elsereddict[1]reddict['1']# note its the same as reddict[1]reddict[('complex',1)]reddict["('complex', 1)"]# the key is str(('complex',1))# deletedelreddict[1]# .. ectPubSubRedisDictLikeRedisDictbut will publish key update and delete events to a<namespace>/[update|delete]channel.redpubsub=PubSubRedisDict(rc,'namespace')# ect as beforePubSubCacheManagerLikepylry.WriteThroughCacheManagerbut updates cache keys from
store when it receives a message from the<namespace>/[update|delete]channel.cache=pylru.lrucache(10)# maybe more than 10redstore=PubSubRedisDict(rc,'namespace')redcache=PubSubCacheManager(redstore,cache)# ect as before# see the cacheprintdict(redcache.cache)Further usesYou can hook upRedisDictorPubSubRedisDicttopylru.WriteBackCacheManagerto get a Redis backed dictionary which
only writes to Redis on ‘flush’ or when the item pops off thelrufor write intensive workloads. However a lot more work would need to be
done to add the pubsub mechanism as there difficult cases to consider,
such as what happens when the cache is dirty and we get notified that
the store key is updated?Limitationsall keys are strings.msgpackis used to marshal objects to redis, somsgpackobject limitations apply. Though you can monkey patch the modulesloadsanddumpsmethod if you like.publish will publish to all consuming dictionary instances, there is
no partitioning, so writes and updates are expensive. You could come
up with a partitioning strategy to improve this.The published items eventually end up in the watched cash. There may
be a time lag between a client publishing a change and the key
updating in another clients cache.Referencesredis-pyredis-py-clusterpylru
Package: redis-pubsub-helper
Blocking subscribe implementation of pyredis is annoying for some case.
It is controllable with some technique, but here is a quick approach using
a python thread.Note: This implementation is good only for few situation.Example>>> import redis
>>> redisc = redis.Redis()
>>> import redispubsub
>>> pubsub = redispubsub.RedisPubsub(redisc.pubsub())
>>> pubsub.subscribe('1') # You SHOULD subscribe before starting thread.
>>> pubsub.start() # Start the router
>>> pubsub.dequeue('1')
{'pattern': None, 'type': 'subscribe', 'channel': '1', 'data': 1L}
>>> redisc.publish('1', 'test')
>>> pubsub.dequeue('1') # None returned
>>> pubsub.dequeue('1')
{'pattern': None, 'type': 'message', 'channel': '1', 'data': 'test'}
>>> pubsub.dequeue('1') # None returned
Package: redis-query
redis-queryA query language for redis rejson.CREATECreates a documentof a choosen name and schema at a choosen path.CREATE DOCUMENT 'obj' ('{"obj":"value"}') AT PATH('ROOT')SELECTGets value from redis database by name.SELECT 'name' FROM PATH('ROOT')INSERTCREATE but only for alredy existion values.INSERT 'obj' ('{"obj":"value"}') INTO PATH('ROOT')PATHA Path inJSONPathsyntax. Type ROOT for root path.PATH('ROOT')
Package: redisqueue
# redisqueueVery basic implementation of a Redis queue for PythonHistory0.1.1 (2014-10-02)First release0.1.2 (2014-10-02)Changed AbstractTask uids to be shorter
(Easier to read for logging purposes)0.1.3 (2014-10-08)Added ability to specify Pickle protocol version0.1.4 (2014-10-11)Removed Pickle and reverted to using JSON intead due to compatibility
issues between Python 2 -> Python 3.0.1.5 (2014-10-23)Removed setup.py import of redisqueue. Causing failed installation if
redis is not already installed.
Package: redis_queue
redis_queue is useful for implementing a multi-producer,multi-consumer job queue. While it doesn’t have all the handy blocking and
locking features of Queue, it does have the advantages of being
multi-process safe and persistant.Example Usage:::>>> from redis import Redis
>>> from redis_queue import Queue
>>> redis = Redis(host='127.0.0.1', port=6379)
>>> queue = Queue(redis, 'test_queue')
>>> queue.append('one')
>>> queue.append('two')
>>> queue.append('three')
>>> queue.pop()
'three'
>>> queue.pop()
'two'
>>> queue.pop()
'one'
>>> queue.append('one')
>>> queue.append('two')
>>> queue.append('three')
>>> queue.popleft()
'one'
>>> queue.popleft()
'two'
>>> queue.popleft()
'three'
Package: redis-ratelimit
redis-ratelimitA fixed window rate limiter based on RedisInstallation$pipinstallredis-ratelimitExample UsageThe library itself is framework agnostic, but let's assume you want to use it withFlask:fromflaskimportFlask,jsonifyfromredis_ratelimitimportratelimitapp=Flask(__name__)@app.route('/')@ratelimit(rate='10/m',key='ccc')@ratelimit(rate='2/s',key='ccc')defindex():returnjsonify({'status':'OK'})This will allow a total of 10 requests in any given minute, but no faster than 2 requests a second.NotesRedis Rate Limiting Pattern #2LicenseMIT
Package: redis-rate-limiter
redis-rate-limiterInstallpipinstall-Uredis-rate-limiterUsefromredis_rate_limiter.configimportbasic_configfromredis_rate_limiter.rate_limiterimportRateLimiterbasic_config(redis_url='redis://localhost:6379/0')@RateLimiter(10,period=1)defgreet():print('Hello')for_inrange(100):greet()# Raise RateLimitExceeded after print('Hello') 10 times
Package: redis-redirect
(aio)REDIS wrapper to deal with cluster redirect exceptionsMOVEDLikeredis.exceptions.ResponseError: MOVED 4085 10.213.192.152:6379When Redis returns a MOVED exception, it indicates that the client should repeat its request for the specified host.This exception can occur in the following situations:Your Redis configuration is incorrect. In this case, you should fix your Redis settings,redis-redirectcannot help you in this scenario.You are connecting to the wrong host.
For instance, if you're using Amazon managed Redis (ElastiCache), Amazon provides a fixed
DNS name for the configuration node and an IP address for the work node. The IP address may change in the future.
In such cases, you should just use the DNS name,redis-redirectcannot help you in this scenario.You're using a multi-node Redis cluster.
In this case, you can use theredis-redirectto automatically switch between Redis nodes or shards.Theredis-redirectis designed to handleMOVEDexceptions seamlessly and transparently.Installationpip install redis-redirectUsageimport redis_redirect

    redis = redis_redirect.Redis(host='my-redis.com', port=6379, db=0)
    redis.set('foo', 'bar')How it worksTheredis-redirectis designed to transparently handle Redis server redirection exceptions.When a client sends a request to the Redis server, theredis-redirectchecks if
the server has returned aMOVEDexception.If aMOVEDexception is received, theredis-redirectupdates the address of the Redis server and resends
the request to the new address.After theredis-redirectupdates the Redis server address, it transparently forwards subsequent requests to
the new Redis server address.Coverage reportCodecovCoveralls
Package: redis-rs
Python wrapper for:redis-rs,redis_cluster_async,bb8,bb8-redis,bb8-redis-cluster,deadpool-redis-cluster,FeaturesAsync client for single and clusterSupport typingEncoding values from str, int, floatDecoding values to str, int, float, list, dictInstallpipinstallredis-rsUsingimportasyncioimportredis_rsasyncdefmain():asyncwithredis_rs.create_client("redis://redis-node001","redis://redis-node002",max_size=1,cluster=True,)asx:info=awaitx.execute("INFO","SERVER",encoding="info")print(info["redis_version"])# parse value as jsonprint(awaitx.get("foo",encoding="json"))print(awaitx.execute(b"HSET","fooh","a",b"asdfg"))print(awaitx.fetch_int("HSET","fooh","b",11234567890))print(awaitx.fetch_int("HGET","fooh","b"))print(awaitx.fetch_str("HGET","fooh","a"))print(awaitx.fetch_dict("HGETALL","fooh",encoding="utf-8"))print(awaitx.hgetall("fooh",encoding="utf-8"))print(awaitx.execute("CLUSTER","NODES"))print(awaitx.fetch_bytes("GET","foo"))print(awaitx.fetch_int("GET","foo"))print(awaitx.execute("HGETALL","fooh"))print(awaitx.execute("ZADD","fooz",1.5678,"b"))print(awaitx.fetch_scores("ZRANGE","fooz",0,-1,"WITHSCORES"))print(x.status())stream="redis-rs"print("x.xadd",awaitx.xadd(stream,"*",{"a":"1234","d":4567}))print("x.xadd",awaitx.xadd(stream,items={"a":"1234","d":4567}))print("x.xadd",awaitx.xadd(stream,{"a":"1234","d":4567}))print("x.xadd",awaitx.xadd(stream,"*","a","1234","d",4567))print("x.xadd",awaitx.xadd(stream,"a","1234","d",4567))print("xadd",awaitx.fetch_str("XADD",stream,"*","a","1234","d",4567))print("xread",awaitx.execute("XREAD","STREAMS",stream,0))print("xread",awaitx.fetch_dict("XREAD","STREAMS",stream,0,encoding="int"))print("x.xread",awaitx.xread({stream:0},encoding="int"))print("x.xread",awaitx.xread(stream,id=0,encoding="int"))print("x.xread",awaitx.xread(stream,stream))asyncio.run(main())Developmentcargofmtcargoclippymaturindevelopor use hatch envs:hatchrunfmthatchruncheckhatchrunbuild
Package: redisrwlock
Distributed reader-writer lock for python using redis as serverFeatures:Reader-writer lock (can have multiple readers or one exclusive writer)Stale locks collected (run as separate process,python3-mredisrwlock)Deadlock detectionNote: Deadlock detection and garbage/staleness collection is done in
client side, which can cause excessive I/O with redis server.  Tune
withretry_intervaland consider running the stale lock collection
appropriately for your purpose.Dependencies:python 3.5.2redis-py 2.10.5redis 3.2.6[test] Coverage.py 4.2Installpip install redisrwlockUsagesTry lock with timeout=0With timeout=0, RwlockClinet.lock acts as so called try_lock.fromredisrwlockimportRwlock,RwlockClientclient=RwlockClient()rwlock=client.lock('N1',Rwlock.READ,timeout=0)ifrwlock.status==Rwlock.OK:# Processings of resource named 'N1' with READ lock# ...client.unlock(rwlock)elifrwlock.status==Rwlock.FAIL:# Retry locking or quitWaiting until lock success or deadlockWith timout > 0, RwlockClient.lock waits until lock successfully or
deadlock detected and caller is chosen as victim.fromredisrwlockimportRwlock,RwlockClientclient=RwlockClient()rwlock=client.lock('N1',Rwlock.READ,timeout=Rwlock.FOREVER)ifrwlock.status==Rwlock.OK:# Processings of resource named 'N1' with READ lock# ...client.unlock(rwlock)elifrwlock.status==Rwlock.DEADLOCK:# 1. unlock if holding any other locks# 2. Retry locking or quitRemoving stale locksWhen a client exits without unlock, redis keys for the client’s locks
remain in server and block other clients from successful locking.redisrwlockrun in command line removes such garbage locks, waits
in server.python3 -m redisrwlockYou can repeat this gc periodically by specifying -r or –repeat option.TestsUnittestRunnig unittest in test directory:cd test
python3 -m unittest -qor in project top directory:python3 -m unittest discover test -qExamples below are assuming you run unittest in project top directory.Coveragecoverage erase
coverage run -a -m unittest discover test -q
coverage htmlAbove simple coverage run will report lower coverage than expected
because the tests use subprocess. Codes run by subprocess are not
covered in report by default.Subprocess coverageNeed some preperation:Editsitecustomize.py(under python intallation’ssite-packagesdirectory), add 2 linesimportcoveragecoverage.process_startup()Edit.coveragerc(default name of coverage.py’s config file)[run]branch=True[html]directory=htmlcovThen, run coverage with environment variableCOVERAGE_PROCESS_START={path/to/coveragerc}coverage erase
COVERAGE_PROCESS_START=.coveragerc coverage run -a -m unittest discover test -q
coverage htmlTODOsTODO: command line option to specify redis-serverTODO: high availability! redis sentinel or replication?
Package: redis-sacsc
redis-sacscRedis Server-Assisted Client-Side Caching in PythonWARNING!This feature is still in redis beta release, scheduled to be released in version 6FeaturesFully typed with annotations and checked with mypy,PEP561 compatibleRedis Server-Assisted Client-Side CachingInstallationpipinstallredis-sacscExampleShowcase how your project can be used:importredisfromredis_sacscimportManagerpool=redis.ConnectionPool.from_url("redis://127.0.0.1:6379")manager=Manager(pool,capacity=512)redis_conn=manager.get_connection()redis_conn.set('foo','bar')redis_conn.get('foo')# => 'bar'LicenseMIT