Package: quent
QuentYet Another Chain Interface.Installationpip install quentTable of ContentsIntroductionReal World ExampleDetails & ExamplesLiteral ValuesCustom Arguments (args,kwargs,Ellipsis)Flow ModifiersChain Template / ReuseChain NestingPipe SyntaxSafety CallbacksComparisonsIteratorsContextsAPICoreexcept,finallyConditionalsCascadeDirect Attribute AccessImportant NotesSuggestions and contributions are more than welcome.IntroductionQuent is anenhanced,chain interfaceimplementation for
Python, designed to handle coroutines transparently. The interface and usage of Quent remains exactly the same,
whether you feed it synchronous or asynchronous objects - it can handle almost any use case.Every documented API supports both regular functions and coroutines. It will work the exact same way as with a regular
function. Quent automatically awaits any coroutines, even a coroutine that the function passed to.foreach()may
return.Quent is written in C (using Cython) to minimize it's overhead as much as possible.As a basic example, take this function:asyncdefhandle_request(id):data=awaitfetch_data(id)data=validate_data(data)data=normalize_data(data)returnawaitsend_data(data)It uses intermediate variables that only serve to make to code more readable, as opposed to:asyncdefhandle_request(id):returnawaitsend_data(normalize_data(validate_data(awaitfetch_data(id))))With Quent, we can chain these operations:fromquentimportChaindefhandle_request(id):returnChain(fetch_data,id).then(validate_data).then(normalize_data).then(send_data).run()Upon evaluation (calling.run()), if an awaitable object is detected, Quent wraps it in a Task and returns it.
The task is automatically scheduled for execution and the chain evaluation continues within the task.
As Task objects need not beawait-ed in order to run, you may or may notawaitit, depending on your needs.BesidesChain, Quent provides theCascadeclass which implements thefluent interface.Quent aims to provide all the necessary tools to handle every use case.
See the full capabilities of Quent in theAPI Section.Real World ExampleThis snippet is taken from a thin Redis wrapper I wrote, which supports both the sync and async versions
ofrediswithout having a separate implementation for the async version.defflush(self)->Any|Coroutine:""" Execute the current pipeline and return the results, excludingthe results of inline 'expire' commands."""pipe=self.r.pipeline(transaction=self.transaction)# this applies a bunch of Redis operations onto the `pipe` object.self.apply_operations(pipe)return(Chain(pipe.execute,raise_on_error=True).then(self.remove_ignored_commands).finally_(pipe.reset,...).run())Once the chain runs, it will execute the pipeline commands, remove the unwanted results, and return the rest
of them. Finally, it will reset thepipeobject. Any function passed to.finally_()willalwaysbe invoked,
even if an exception has been raised during the execution of the chain. The purpose of the...here is explained
in theEllipsissection.pipe.executeandpipe.resetare both performing a network request to Redis, and in the case of anasyncRedis
object - are coroutines and would have to be awaited.
Notice that I return without an explicitawait- if the user of this wrapper has initialized the class with an
asyncRedisinstance, they will know that they need toawaitit. This allows me to focus on the actual logic,
without caring aboutsyncvsasync.Some would say that this pattern can cause unexpected behavior, since it isn't clear when it
will return a Task or not. I see it no differently than any undocumented code - with a proper
and clear documentation (be it an external documentation or just a simple docstring), there shouldn't be
any trulyunexpectedbehavior (barring any unknown bugs).Details & ExamplesLiteral ValuesYou don't have to pass a callable as a chain item - literal values works just as well.Chain(fetch_data,id).then(True).run()will executefetch_data(id), and then returnTrue.Custom ArgumentsYou may provideargsorkwargsto a chain item - by doing so, Quent assumes that the item is a callable
and will evaluate it with the provided arguments, instead of evaluating it with the current value.Chain(fetch_data,id).then(fetch_data,another_id,password=password).run()will executefetch_data(id), and thenfetch_data(another_id, password=password).EllipsisTheEllipsis/...is a special case - if the first argument formostfunctions that register a chain item
or a callback is...,
the item will be evaluated without any arguments.Chain(fetch_data,id).then(do_something,...).run()will executefetch_data(id), and thendo_something().Flow ModifiersWhile the default operation of a chain is to, well, chain operations (using.then()), there are cases where you may
want to break out of this flow. For this,Chainprovides the functions.root()and.ignore().
They both behave like.then(), but with a small difference:.root()evaluates the item using the root value, instead of the current value..ignore()evaluates the item with the current value but will not propagate its result forwards.There is also a.root_ignore()which is the combination of.root()and.ignore().Reusing A ChainYou may reuse a chain as many times as you wish.chain=Chain(fetch_data,id).then(validate_data).then(normalize_data).then(send_data)chain.run()chain.run()...There are many cases where you may need to apply the same sequence of operations but with different inputs. Take our
previous example:Chain(fetch_data,id).then(validate_data).then(normalize_data).then(send_data).run()Instead, we can create a template chain and reuse it, passing different values to.run():handle_data=Chain().then(validate_data).then(normalize_data).then(send_data)foridinlist_of_ids:handle_data.run(fetch_data,id)Re-using aChainobject will significantly reduce its overhead, as most of the performance hit is due
to the creation of a newChaininstance. Nonetheless, the performance hit is negligible and not worth to sacrifice
readability for. So unless it makes sense (or you need to really squeeze out performance),
it's better to create a newChaininstance.Nesting A ChainYou can nest aChainobject within anotherChainobject:Chain(fetch_data,id).then(Chain().then(validate_data).then(normalize_data)).then(send_data).run()A nested chain must always be a template chain (i.e. initialized without arguments).A nested chain will be evaluated with the current value of the parent chain passed to its.run()method.Pipe SyntaxPipe syntax is supported:fromquentimportChain,run(Chain(fetch_data)|process_data|normalize_data|send_data).run()Chain(fetch_data)|process_data|normalize_data|send_data|run()You can also usePipewith Quent:frompipeimportwhereChain(get_items).then(where(lambdaitem:item.is_valid()))Chain(get_items)|where(lambdaitem:item.is_valid())Safety CallbacksThe usage ofexceptandfinallyis supported:Chain(open('<file-path>')).then(do_something_with_file).finally_(lambdaf:f.close())Read more about it inCallbacks.ComparisonsMost basic comparison operations are supported:Chain(get_key).in_(list_of_keys)# == get_key() in list_of_keysSee the full list of operations inConditionals.IteratorsYou can easily iterate over the result of something:Chain(fetch_keys).foreach(do_something_with_key)The full details of.foreach()are explainedhere.ContextsYou can execute a function (or do quite anything you want) inside a context:Chain(get_lock,id).with_(fetch_data,id)The full details of.with_()are explainedhere.APIValue EvaluationMost of the methods in the following section receivesvalue,args, andkwargs. Unless explicitly told otherwise,
the evaluation ofvaluein all of those methods is roughly equivalent to:ifargs[0]isEllipsis:returnvalue()elifargsorkwargs:returnvalue(*args,**kwargs)elifcallable(value):returnvalue(current_value)else:returnvalueTheevaluate_valuefunction contains the full evaluation logic.Core__init__(value: Any = None, *args, **kwargs)Creates a new chain withvalueas the chain's root item.valuecan be anything - a literal value,
a function, a class, etc.
Ifargsorkwargsare provided,valueis assumed to be a callable and will be evaluated with those
arguments. Otherwise, a check is performed to determine whethervalueis a callable. If it is, it is
called without any arguments.Not passing a value will create a template chain (see:Reusing A Chain). You can still normally use
it, but then you must call.run()with a value (see the next section).A few examples:Chain(42)Chain(fn,True)Chain(cls,name='foo')Chain(lambdav:v*10,4.2)run(value: Any = None, *args, **kwargs) -> Any | asyncio.TaskEvaluates the chain and returns the result, or a Task if there are any coroutines in the chain.If the chain is a template chain (initialized without a value), you must call.run()with a value, which will act
as the root item of the chain.Conversely, if.run()is called with a value and the chain is a non-template chain, then an exception will be raised.
The only case where you can both create a template chain and run it without a value is for theCascadeclass,
which is documented below inCascade - Void Mode.Similarly to the examples above,Chain().run(42)Chain().run(fn,True)Chain().run(cls,name='foo')Chain().run(lambdav:v*10,2)then(value: Any, *args, **kwargs) -> ChainAddsvalueto the chain as a chain item.valuecan be anything - a literal value, a function, a class, etc.Returns theevaluationofvalue.This is the main and default way of adding items to the chain.(see:Ellipsisif you need to invokevaluewithout arguments)Chain(fn).then(False)Chain(42).then(verify_result)Chain('<uuid>').then(uuid.UUID)root(value: Any = None, *args, **kwargs) -> ChainLike.then(), but it first sets the root value as the current value, and then it evaluatesvalueby the defaultevaluation procedure.Calling.root()without a value simply returns the root value.Read more inFlow Modifiers.Chain(42).then(lambdav:v/10).root(lambdav:v==42)ignore(value: Any, *args, **kwargs) -> ChainLike.then(), but keeps the current value unchanged.In other words, this function does not affect the flow of the chain.Read more inFlow Modifiers.Chain(fetch_data,id).ignore(print).then(validate_data)root_ignore(value: Any, *args, **kwargs) -> ChainThe combination of.root()and.ignore().Chain(fetch_data,id).then(validate_data).root_ignore(print).then(normalize_data)attr(name: str) -> ChainLike.then(), but evaluates togetattr(current_value, name).classA:@propertydefa1(self):# I return something importantpassChain(A()).attr('a1')ChainAttr(A()).a1attr_fn(name: str, *args, **kwargs) -> ChainLike.attr(), but evaluates togetattr(current_value, name)(*args, **kwargs).classA:defa1(self,foo=None):# I do something importantpassChain(A()).attr_fn('a1',foo=1)ChainAttr(A()).a1(2)Foreachforeach(fn: Callable) -> ChainIterates over the current value and invokesfn(element)for each element. Returns a list
that is the result offn(element)for eachelement.If the iterator implements__aiter__,async for ...will be used.Example:Chain(list_of_ids).foreach(validate_id).run()will iterate overlist_of_ids, and return a list that is equivalent to[validate_id(id) for id in list_of_ids].foreach_do(fn: Callable) -> ChainLike.foreach(), but returns nothing. In other words, this is the combination of.foreach()and.ignore().Example:Chain(list_of_ids).foreach_do(Chain().then(fetch_data).then(validate_data).then(normalize_data).then(send_data)).run()will iterate overlist_of_ids, invoke the nested chain with each differentid, and then returnlist_of_ids.Withwith_(self, value: Any | Callable = None, *args, **kwargs) -> ChainExecuteswith current_value as ctxand evaluatesvalueinside the context block,withctxas the current value, and returns the result. Ifvalueis not provided, returnsctx.
This method follows thedefault evaluationprocedure, so passingargsorkwargsis perfectly valid.Depending onvalue(andargs/kwargs), this is roughly equivalent towithcurrent_valueasctx:returnvalue(ctx)If the context object implements__aenter__,async with ...will be used.Example:Chain(get_lock,id).with_(fetch_data,id).run()is roughly equivalent to:withget_lock(id)aslock:# `lock` is not used here since we passed a custom argument `id`.returnfetch_data(id)with_do(self, value: Any | Callable, *args, **kwargs) -> ChainLike.with_(), but returns nothing. In other words, this is the combination of.with_()and.ignore().Class MethodsChain.from_(*args) -> ChainCreates aChaintemplate, and registersargsas chain items.Chain.from_(validate_data,normalize_data,send_data).run(fetch_data,id)# is the same as doingChain().then(validate_data).then(normalize_data).then(send_data).run(fetch_data,id)Callbacksexcept_(fn: Callable | str, *args, **kwargs) -> ChainRegister a callback that will be called if an exception is raised anytime during the chain's
evaluation. The callback is evaluated with the root value, or withargsandkwargsif provided.Iffnis a string, then it is assumed to be an attribute method of the root value.Chain(fetch_data).then(validate_data).except_(discard_data)finally_(fn: Callable | str, *args, **kwargs) -> ChainRegister a callback that willalwaysbe called after the chain's evaluation. The callback is evaluated with
the root value, or withargsandkwargsif provided.Iffnis a string, then it is assumed to be an attribute method of the root value.Chain(get_id).then(aqcuire_lock).root(fetch_data).finally_(release_lock)Conditionalsif_(on_true: Any | Callable, *args, **kwargs) -> ChainEvaluates the truthiness of the current value (bool(current_value)).
Ifon_trueis provided and the result isTrue, evaluateson_trueand returns the result.
Ifon_trueis not provided, simply returns the truthiness result (bool).on_truemay be anything and follows the defaultevaluation procedureas described above.Chain(get_random_number).then(lambdan:n>5).if_(you_win,prize=1)else_(on_false: Any | Callable, *args, **kwargs) -> ChainIf a previous conditional result is falsy, evaluateson_falseand returns the result.on_falsemay be anything and follows the defaultevaluation procedureas described above.Can only be called immediately following a conditional.Chain(get_random_number).then(lambdan:n>5).if_(you_win,prize=1).else_(you_lose,cost=10)not_() -> Chainnot current_valueThis method currently does not support theon_trueargument since it looks confusing.
I might add it in the future.Chain(is_valid,'something').not_()eq(value: Any, on_true: Any | Callable = None, *args, **kwargs) -> Chaincurrent_value == valueChain(420).then(lambdav:v/10).eq(42)Chain(420).then(lambdav:v/10).eq(40).else_(on_fail)neq(value: Any, on_true: Any | Callable = None, *args, **kwargs) -> Chaincurrent_value != valueChain(420).then(lambdav:v/10).neq(40)is_(value: Any, on_true: Any | Callable = None, *args, **kwargs) -> Chaincurrent_value is valueChain(object()).is_(1)is_not(value: Any, on_true: Any | Callable = None, *args, **kwargs) -> Chaincurrent_value is not valueChain(object()).is_not(object())in_(value: Any, on_true: Any | Callable = None, *args, **kwargs) -> Chaincurrent_value in valueChain('sub').in_('subway')not_in(value: Any, on_true: Any | Callable = None, *args, **kwargs) -> Chaincurrent_value not in valueChain('bus').then(lambdas:s[::-1]).not_in('subway')CascadeAlthough considered unpythonic, in some cases thecascade designcan be very helpful. TheCascadeclass is identical toChain, except that during the chain's evaluation,
each chain item is evaluated using the root value as an argument
(or in other words, the current value is always the chain's root value).
The return value ofCascade.run()is always its root value.fromquentimportCascadefetched_data=(Cascade(fetch_data,id).then(send_data_to_backup).then(lambdadata:send_data(data,to_id=1)).then(print).run())will executefetch_data(id), thensend_data_to_backup(data), thensend_data(data, to_id=1),
and thenprint(data).You can also useCascadeto make existing classes behave the same way:fromquentimportCascadeAttrclassFoo:deffoo(self):...asyncdefbar(self):...defbaz(self):...asyncdefget_foo():f=Foo()f.foo()awaitf.bar()f.baz()returnfdefbetter_get_foo():returnCascadeAttr(Foo()).foo().bar().baz().run()Cascadeworks for any kind of object:fromquentimportCascadeAttrCascadeAttr([]).append(1).append(2).append(3).run()==[1,2,3]Cascade - Void ModeIn some cases it may be desired to run a bunch of independent operations. UsingCascade, one can
achieve this by simply not passing a root value to the constructor nor to.run(). All the chain items
will not receive any arguments (excluding explicitly providedargs/kwargs).await(Cascade().then(foo,False).then(bar).then(baz).run())will executefoo(False), thenbar(), thenbaz().A voidCascadewill always returnNone.Direct Attribute AccessBothChainandCascadecan support "direct" attribute access via theChainAttrandCascadeAttrclasses.
See theCascadesection above to see an example ofCascadeAttrusage. The same principle holds forChainAttr. Accessing attributes without using theAttrsubclass is possible using.attr()and.attr_fn().The reason I decided to separate this functionality from the main classes is due to the fact
that it requires overriding__getattr__, which drastically increases the overhead of both creating an instance and
accessing any properties / methods. And since I don't think this kind of usage will be common, I decided
to keep this functionality opt-in.Important NotesAsynchronousexceptandfinallycallbacksIf an except/finally callback is a coroutine function, and an exception is raisedbeforethe first coroutine of the chain has been evaluated, or if there aren't any coroutines in the chain - each
callback will be invoked inside a new Task, which you won't have access to.
This is due to the fact that we cannot return the new Task(s) from theexcept/finallyclauses.This shouldn't be an issue in most use cases, but important to be aware of.
ARuntimeWarningwill be emitted in such a case.As an example, suppose thatfetch_datais synchronous, andreport_usageis asynchronous.Chain(fetch_data).then(raise_exception,...).finally_(report_usage).run()will executefetch_data(), thenraise_exception(), and thenreport_usage(data). Butreport_usage(data)is a
coroutine, andfetch_dataandraise_exceptionsare not. Then Quent will wrapreport_usage(data)in a Task
and "forget" about it.If you must, you can "force" an async chain by giving it a dummy coroutine:asyncdeffn(v):returnvawaitChain(fetch_data).then(fn).then(raise_exception,...).finally_(report_usage).run()This will ensure thatreport_usage()will be awaited properly.