Saravanan666/pypi-org huggingface dataset:
Package: aioredis-rate-limiter
aioredis-rate-limiterRate limiter implements on RedisImplementation of distributed rate limiter with aioredis, an asyncio based redis client.Simple rate limiter based on Redis DB.The key features are:Concurrency workRate by limit requestsRate by time for requestsUsageWe want to limit requests from 5 pods to limited resource. Limits for resource: no more 10 request per 20 seconds.locker=AioRedisRateLimiter(redis,rate_limit=10,rate_key_ttl=20)importasyncioimportosfromaioredis.clientimportRedisfromaioredis_rate_limiterimportAioRedisRateLimiterclassExecutor:def__init__(self,name:str,locker:AioRedisRateLimiter,task_count:int=10):self._locker=lockerself._task_count=task_countself._name=nameasyncdefprocess(self):foriinrange(self._task_count):whileTrue:is_ok=awaitself._locker.acquire()ifis_ok:print(f'Executor{self._name}by{i+1}')breakelse:awaitasyncio.sleep(1)asyncdefmain():host=os.getenv('REDIS_HOST')db=os.getenv('REDIS_DB')redis=Redis.from_url(host,db=db,encoding="utf-8",decode_responses=True)locker=AioRedisRateLimiter(redis,rate_limit=10,rate_key_ttl=15)w1=Executor('first',locker,10)w2=Executor('helper',locker,8)w3=Executor('lazzy',locker,5)tasks=[w1.process(),w2.process(),w3.process()]awaitasyncio.gather(*tasks)if__name__=='__main__':asyncio.run(main())
Package: aioredis-rpc
aioredis-rpcA RPC interface usingaioredisandpydantic.Usagepipinstallaioredis-rpcpydanticis used to model complex objects which are transparently serialized
and packed into messages usingmsgpack.# Define Pydantic modelsclassFileData(BaseModel):filename:strdata:bytesDefine a class using the@endpointdecorator to specify which methods will be
accessible via rpc.fromredisrpcimportendpoint# Define an RPC classclassDropbox:files:Dict[str,FileData]max_files:intdef__init__(self,max_files:int=1000):self.files=dict()self.max_files=max_files@endpointasyncdefupload_file(self,file:FileData)->int:iflen(self.files)>=self.max_files:# Errors are propagated to the client-sideraiseException('too many files')self.files[file.name]=filereturnlen(file.data)@endpointdefget_file_names(self)->List[str]:returnlist(self.files.keys())@endpointasyncdefdownload_file(self,name:str)->FileData:returnself.files[name]Use thecreate_serverfunction to make an instance of your server-side rpc
class. The server instance will be assigned anrpcattribute to access server
functions likeconnectanddisconnect. Onceconnectis called methods
decorated with@endpointwill be invoked automatically by remote calls from
the client.NOTE: TheRpcProvider.connectmethod is non-blocking.server=create_server(Dropbox,max_files=2)# Returns once connected to redisawaitserver.rpc.connect(dsn="redis://localhost")# Wait foreverwhileTrue:awaitasyncio.sleep(1)Thecreate_clientfunction create a faux instance of the rpc class with only
the methods decorated by@endpointpresent. When these methods are called by
the client the function arguments are serialized and published to redis.NOTE: If there are no subscribers to the redis channel then the client will
throw aRpcNotConnectedError.client=create_client(Dropbox)awaitclient.rpc.connect(dsn="redis://localhost")Now that both ends are connected the@endpointdecorated methods may be called
like they are accessing the actual class passed tocreate_client.file1=FileData(name='file1',data=b'1234')size=awaitclient.upload_file(file1)
Package: aioredis-semaphore
===============
aioredis-semaphoreA distributed semaphore and mutex built on Redis.InstallationTo install aioredis-semaphore, simply::pip install aioredis-semaphoreExamples::# -*- coding:utf-8 -*-
import anyio
from aioredis import Redis
from anyio import create_task_group, run

from aioredis_semaphore import Semaphore

semaphore = Semaphore(Redis(), count=2, namespace="example")


async def task(i: int) -> None:
    async with semaphore:
        print("id: {}".format(i))
        print("sleep...")
        await anyio.sleep(2)


async def main() -> None:
    async with create_task_group() as tg:
        for i in range(5):
            tg.start_soon(task, i)


if __name__ == "__main__":
    run(main)
Package: aioredlock
The asyncioredlockalgorithm implementation.Redlock and asyncioThe redlock algorithm is a distributed lock implementation forRedis. There are many implementations of it in several languages. In this case, this is theasynciocompatible implementation for python 3.5+.UsagefromaioredlockimportAioredlock,LockError,Sentinel# Define a list of connections to your Redis instances:redis_instances=[('localhost',6379),{'host':'localhost','port':6379,'db':1},'redis://localhost:6379/2',Sentinel(('localhost',26379),master='leader',db=3),Sentinel('redis://localhost:26379/4?master=leader&encoding=utf-8'),Sentinel('rediss://:password@localhost:26379/5?master=leader&encoding=utf-8&ssl_cert_reqs=CERT_NONE'),]# Create a lock manager:lock_manager=Aioredlock(redis_instances)# Check wether a resourece acquired by any other redlock instance:assertnotawaitlock_manager.is_locked("resource_name")# Try to acquire the lock:try:lock=awaitlock_manager.lock("resource_name",lock_timeout=10)exceptLockError:print('Lock not acquired')raise# Now the lock is acquired:assertlock.validassertawaitlock_manager.is_locked("resource_name")# Extend lifetime of the lock:awaitlock_manager.extend(lock,lock_timeout=10)# Raises LockError if the lock manager can not extend the lock lifetime# on more then half of the Redis instances.# Release the lock:awaitlock_manager.unlock(lock)# Raises LockError if the lock manager can not release the lock# on more then half of redis instances.# The released lock become invalid:assertnotlock.validassertnotawaitlock_manager.is_locked("resource_name")# Or you can use the lock as async context manager:try:asyncwithawaitlock_manager.lock("resource_name")aslock:assertlock.validisTrue# Do your stuff having the lockawaitlock.extend()# alias for lock_manager.extend(lock)# Do more stuff having the lockassertlock.validisFalse# lock will be released by context managerexceptLockError:print('Lock not acquired')raise# Clear the connections with Redis:awaitlock_manager.destroy()How it worksThe Aioredlock constructor accepts the following optional parameters:redis_connections: A list of connections (dictionary of host and port and kwargs foraioredis.create_redis_pool(), or tuple(host, port), or string Redis URI) where the Redis instances are running.  The default value is[{'host':'localhost', 'port': 6379}].retry_count: An integer representing number of maximum allowed retries to acquire the lock. The default value is3times.retry_delay_minandretry_delay_max: Float values representing waiting time (in seconds) before the next retry attempt. The default values are0.1and0.3, respectively.In order to acquire the lock, thelockfunction should be called. If the lock operation is successful,lock.validwill be true, if the lock is not acquired then theLockErrorwill be raised.From that moment, the lock is valid until theunlockfunction is called or when thelock_timeoutis reached.Call theextendfunction to reset lifetime of the lock tolock_timeoutinterval.Use theis_lockedfunction to check if the resource is locked by other redlock instance.In order to clear all the connections with Redis, the lock_managerdestroymethod can be called.To-do
Package: aioredlock-neorisk
The asyncioredlockalgorithm implementation.Redlock and asyncioThe redlock algorithm is a distributed lock implementation forRedis. There are many implementations of it in several languages. In this case, this is theasynciocompatible implementation for python 3.5+.UsagefromaioredlockimportAioredlock,LockError# Define a list of connections to your Redis instances:redis_instances=[('localhost',6379),{'host':'localhost','port':6379,'db':1},'redis://localhost:6379/2',]# Create a lock manager:lock_manager=Aioredlock(redis_instances)# Check wether a resourece acquired by any other redlock instance:assertnotawaitlock_manager.is_locked("resource_name")# Try to acquire the lock:try:lock=awaitlock_manager.lock("resource_name",lock_timeout=10)exceptLockError:print('Lock not acquired')raise# Now the lock is acquired:assertlock.validassertawaitlock_manager.is_locked("resource_name")# Extend lifetime of the lock:awaitlock_manager.extend(lock,lock_timeout=10)# Raises LockError if the lock manager can not extend the lock lifetime# on more then half of the Redis instances.# Release the lock:awaitlock_manager.unlock(lock)# Raises LockError if the lock manager can not release the lock# on more then half of redis instances.# The released lock become invalid:assertnotlock.validassertnotawaitlock_manager.is_locked("resource_name")# Or you can use the lock as async context manager:try:asyncwithawaitlock_manager.lock("resource_name")aslock:assertlock.validisTrue# Do your stuff having the lockawaitlock.extend()# alias for lock_manager.extend(lock)# Do more stuff having the lockassertlock.validisFalse# lock will be released by context managerexceptLockError:print('Lock not acquired')raise# Clear the connections with Redis:awaitlock_manager.destroy()How it worksThe Aioredlock constructor accepts the following optional parameters:redis_connections: A list of connections (dictionary of host and port and kwargs foraioredis.create_redis_pool(), or tuple(host, port), or string Redis URI) where the Redis instances are running.  The default value is[{'host':'localhost', 'port': 6379}].retry_count: An integer representing number of maximum allowed retries to acquire the lock. The default value is3times.retry_delay_minandretry_delay_max: Float values representing waiting time (in seconds) before the next retry attempt. The default values are0.1and0.3, respectively.In order to acquire the lock, thelockfunction should be called. If the lock operation is successful,lock.validwill be true, if the lock is not acquired then theLockErrorwill be raised.From that moment, the lock is valid until theunlockfunction is called or when thelock_timeoutis reached.Call theextendfunction to reset lifetime of the lock tolock_timeoutinterval.Use theis_lockedfunction to check if the resource is locked by other redlock instance.In order to clear all the connections with Redis, the lock_managerdestroymethod can be called.To-do
Package: aioredlock-py
aioredlock-pySecure and efficient distributed locks (Redisson like) implemetation. Ensure efficient performance with biased locking's implementation, can load more than 1k/s of concurrent requests with default parameter settings.Requirementsaioredis>=2.0.0Installpip install aioredlock-pyFeatureEnsure reliability with context manager.Use lua scripts to ensure atomicity on lock release.Notification prompt you to cancel the following execution if acquisition failsReliable in high concurrency.Documentationhttps://aioredlock-py.readthedocs.ioBasic usageimportasyncioimportaioredisfromaioredlock_pyimportRedissonasyncdefsingle_thread(redis):for_inrange(10):asyncwithRedisson(redis,key="no1")aslock:ifnotlock:# If the lock still fails after several attempts, `__aenter__`# will return None to prompt you to cancel the following executionreturn'Do something, failed to acquire lock'# raise ...# else# Service logic protected by Redissonawaitredis.incr("foo")asyncdeftest_long_term_occupancy(redis):asyncwithRedisson(redis,key="no1",ex=10)aslock:ifnotlock:return;# Service logic protected by Redissonawaitredis.set("foo",0)# By default, a lock is automatically released if no action is# taken for 20 seconds after redisson holds it. Let's assume that# your service logic takes a long time (30s in this case) to process,# you don't need to worry about it causing chaos, because there's# background threads help you automatically renew legally held locks.awaitasyncio.sleep(30)awaitredis.incr("foo")asyncdefmain():redis=aioredis.from_url("redis://localhost")awaitredis.delete("redisson:no1")awaitredis.set("foo",0)awaitasyncio.gather(*(single_thread(redis)for_inrange(20)))assertint(awaitredis.get("foo"))==200# test_long_term_occupancy(redis)asyncio.run(main())
Package: aioredux
PythonicReduxPythonicReduxusing asyncio.aioreduxprovides a predictable state container with the following goal: “[Redux] helps
you write applications that behave consistently, run in different environments
…, and are easy to test” (from theReduxdocumentation).Free software: Mozilla Public LicenseThis package requires Python 3.4 or higherUsageimport asyncio
import aioredux

 async def go():
     initial_state = {
         'todos': (),
     }

     def add_todo(text):
         return {'type': 'ADD_TODO', 'text': text}

     def reducer(state, action):
         if action['type'] == 'ADD_TODO':
             todos = state['todos'] + (action['text'],)
             return {'todos': todos}
         return state

     store = yield from aioredux.create_store(reducer, initial_state)
     yield from store.dispatch(add_todo('todo text'))
     print(store.state['todos'])

 asyncio.get_event_loop().run_until_complete(go())Implementation notesdispatchis marked asasyncalthough in most cases it functions like
a plain Python function returning a Future. This is done to allow for cases
where dispatch performs a more complicated set of (async) actions.A Pythonic version ofredux-thunkis also included.
Package: aioregistry
AIORegistryaioregistryis a Python library and CLI tool for inspecting and copying container image data
from and between registries.This library primarily focuses on being a useful tool for dealing with container image
registries. It has very limited support for interpretation of the objects stored within.Library usageFind sub-manifest based on platform.asyncwithAsyncRegistryClient()asclient:manifest_ref=parse_image_name("alpine")manifest=awaitclient.manifest_download(manifest_ref)ifisinstance(manifest,ManifestListV2S2):forsub_manifestinmanifest.manifests:ifsub_manifest.platform.architecture=="amd64":manifest_ref.ref=sub_manifest.digestmanifest=awaitclient.manifest_download(manifest_ref)breakelse:raiseException("Found no matching platform")else:print("Not a manifest list")Download layers of an imageforlayerinmanifest.layers:assertlayer.media_type=="application/vnd.docker.image.rootfs.diff.tar.gzip"blob_ref=RegistryBlobRef(manifest_ref.registry,manifest_ref.repo,layer.digest)# For example we just download into memory. In practice don't do this.blob_data=io.BytesIO(b"".join([chunkasyncforchunkinclient.ref_content_stream(blob_ref)]))withtarfile.open(mode="r|*",fileobj=blob_data)astar:fortarinfointar.getmembers():print(tarinfo.name)CLI copy tool# By default it will pull credentials based on ~/.docker/config.jsonpython-maioregistryubuntu:18.04my.private.registry/my-repo:my-tag# Copy all tags matching regexpython-maioregistryubuntumy.private.registry/my-repo--tag-pattern'18\..*'
Package: aioreloader
Tool that reloads yourasyncio-based application automatically when you
modify the source code.Most of code has been borrowed fromTornadoreloader built mostly by@finiteloopand@bdarnell. Thanks!From 0.3.x version aioreloader natively supports-Xpython arguments. Which is recommended way for development debug mode in aiohttp.UsageHere’s an example of usage withaiohttpframework:app=aiohttp.web.Application()aioreloader.start()aiohttp.web.run_app(app)To add any file to watching list (which is not loaded as a python module):aioreloader.watch('/etc/app_config.yml')RequirementsPython - at least 3.5Installation$pipinstallaioreloader
Package: aiorest
JSON REST framework based on aiohttp (an asyncio (PEP 3156) http server).aiorest development has stoppedThe project always was in experimental status: we have tried to make the proof
of concept foraiohttphigh level server.Now the work is done, the most important parts transplanted toaiohttp.web:RequestandResponse.Someaiorestfeatures are not supported byaiohttp.webyet:
sessions, CORS and security.We are working hard on the issue by makingaiohttpextension
libraries for those ones.We will keepaiorestwork on top ofaiohttpnew versions for a
while.Please report about incompatibility bugs toaiorest github
issue tracker– we’ll fix those.Example usageSimple REST server can be run like this:import asyncio
import aiohttp
import aiorest


# define a simple request handler
# which accept no arguments
# and responds with json
def hello(request):
    return {'hello': 'world'}


loop = asyncio.get_event_loop()
server = aiorest.RESTServer(hostname='127.0.0.1',
                            loop=loop)

# configure routes
server.add_url('GET', '/hello', hello)
# create server
srv = loop.run_until_complete(loop.create_server(
    server.make_handler, '127.0.0.1', 8080))


@asyncio.coroutine
def query():
    resp = yield from aiohttp.request(
        'GET', 'http://127.0.0.1:8080/hello', loop=loop)
    data = yield from resp.read_and_close(decode=True)
    print(data)


loop.run_until_complete(query())
srv.close()
loop.run_until_complete(srv.wait_closed())
loop.close()this will print{'hello': 'world'}jsonSeeexamplesfor more.RequirementsPython 3.3asynciohttp://code.google.com/p/tulip/or Python 3.4+aiohttphttp://github.com/KeepSafe/aiohttpoptional moduleaiorest.redis_sessionrequires aioredishttps://github.com/aio-libs/aioredisLicenseaiorest is offered under the MIT license.CHANGES0.4.0 (2015-01-18)The aiorest library development has stopped, use aiohttp.web instead.Updateaiorestcode to be compatible withaiohttp 0.14 release.0.3.1 (2014-12-22)Fixed exceptions logging for unhandled errors0.3.0 (2014-12-17)Made aiorest compatible to aiohttp v0.120.2.5 (2014-10-30)Fix response.write_eof() to follow aiohttp changes0.2.4 (2014-09-12)Make loop keywork-only parameter in create_session_factory() function0.2.3 (2014-08-28)Redis session switched from asyncio_redis to aioredis0.2.2 (2014-08-15)Added Pyramid-like matchdict to request
(seehttps://github.com/aio-libs/aiorest/pull/18)Return “400 Bad Request” for incorrect JSON body in POST/PUT methodsREADME fixedCustom response status code
(seehttps://github.com/aio-libs/aiorest/pull/23)0.1.1 (2014-07-09)Switched to aiohttp v0.9.00.1.0 (2014-07-07)Basic REST API
Package: aio-rest
aio-rest – REST Helpers for Asyncio projects** WORK IN PROGRESS ** the package is not ready yetFeaturesSupport for Starlette, AIOHTTP frameworksContentsFeaturesRequirementsInstallationQuickStartBug trackerContributingLicenseRequirementspython >= 3.4Installationaio-restshould be installed using pip:pip install aio-restQuickStartTODOBug trackerIf you have any suggestions, bug reports or
annoyances please report them to the issue tracker
athttps://github.com/klen/aio-rest/issuesContributingDevelopment of the project happens at:https://github.com/klen/aio-restLicenseLicensed under aMIT license.
Package: aiorestapi
Rapid rest resources for aiohttpKey FeaturesSupports both client and server side of HTTP protocol.Getting startedaiorestapiallows you to quickly create a rest resource in a few steps. It automatically creates the resource routes on the collections or individual items; it's simply to specify the suffix '_collection' or '_item' on the methods.
The serialization / deserialization of results / requests occurs transparently using  python dictionaries.An example creating a simple rest resourcefromaiohttpimportwebfromaiorestapiimportRestView,routes@routes.resource("/views")classRestResource(RestView):# example call: GET to <server>/views?start=10asyncdefon_get_collection(self,start=0)->list:return[{"id":int(start)+1,"value":1},{"id":int(start)+2,"value":2},]# example call: GET <server>/views/80asyncdefon_get_item(self,id:str)->dict:returnself.key# example call: POST to <server>/viewsasyncdefon_post_collection(self,body:dict)->dict:returnbodyapp=web.Application()app.add_routes(routes)app['key']=[1,2,4,5]if__name__=='__main__':web.run_app(app)InstallationIt's very simple to install aiorestapi:pipinstallaiorestapiNotes@routes.resource decorator the decorator automatically adds the routes to '/myresources' and '/myresources/{id}'From the RestView object it is possible to access the aiohttp request with self.requestThe query parameters are automatically converted into parameters of the view method.If requests have a body it is necessary to specify in the method a parameter called 'body'If requests are to the single item it is necessary to specify a parameter called 'id'The items stored as 'app[]' are accessible into view as properties 'self.'To DoTests!!DocumentationConfigurable custom serializers/deserializersRequirementsPython >= 3.6aiohttpLicenseaiorestapiis offered under the Apache 2 license.Source codeThe latest developer version is available in a GitHub repository:https://github.com/aiselis/aiorestapi
Package: aioretry-decorator
aioretryHandy decorator to set retry policies for async callables with some useful featuresUsage examples@retry(
    tries=5,
    allowed_exceptions=(RuntimeError,),
    intervals=(5, 7, 10),
    fail_cb=make_request_callback,
)
async def make_request(address, client):

    ... http request code goes here...

    if response.status_code >= 300:
        raise RuntimeError()

def make_request_callback(address, client)
    ...5 retries will be performedwith 5, 7, 10, 10 and 10 seconds interval between retries.
I.e. ifintervalstuple length more thantriesnumber,
the last tuple interval will be used for the rest of the tries.make_request_callback()synchronous function will be called if all attempts are failedThe accepted exceptions tuple allows you to control when to retry.You can optionally pass a customlogger: logging.Loggerto the decorator
within alogger=parameter. Otherwiseretry_decoratorlogger will be created to log retries.Other possible ways to use this decorator:@retry(5, (MyCustomError,), (5, 7, 10), make_request_callback)

@retry(3, (MyCustomError,), (1,))

# this actually will either successfully return or fail on first exception occured
@retry(3)Look intotests.pyto see more on usage.InstallationAvailable as a package on pypi:pip install aioretry-decoratorOr install it directly from GitHub:pip install git+https://github.com/remort/aioretry.git#egg=aioretry_decorator
Package: aio-retrying
info:Simple retrying for asyncioInstallationpipinstallaio_retryingUsageimportasyncioimportaiohttpfromUtils.async_retryingimportretryclassAio:def__init__(self):self.session=aiohttp.ClientSession()asyncdef__aenter__(self):returnself@retry(attempts=3,delay=1,fallback="daoji")asyncdef__aexit__(self,exc_type,exc_val,exc_tb):awaitself.session.close()print("exit")asyncdefmain(self):foriinrange(3):assertself.session.closedisFalseawaitasyncio.sleep(1)print(f"task -{i}")asyncdefsecond_aio():aio=Aio()# task = asyncio.create_task(aio.main())# task.add_done_callback(lambda _: asyncio.create_task(aio.__aexit__(None, None, None)))ret=awaitaio.__aexit__(None,None,None)print(f"结果：{ret}")asyncdeffirst_aio():awaitsecond_aio()whileTrue:# print("first")awaitasyncio.sleep(3)defmain():asyncio.run(first_aio())if__name__=="__main__":main()Python 3.8+ is required