Package: scoped-functions
ScopedThe Python library to run code at function exit.ContentsBackgroundOverviewInstallationAPI ReferenceMotivation and backgroundGo to the topThis library is inspired by the common and natural C++ idiom calledResource Acquisition Is Initialization (RAII),
which supposes that the lifetime of an object is equal to its visibility in the scope.It is a composite idiom, relying on several core language properties, such as
strict object scoping rules, order of initialization, stack unwinding etc.
In Python, however, scoping rules for objects are not that strict. For instance,
the following code is totally valid, as conditional blocks and other controlling
structures do not introduce new scopes:deffoo():ifTrue:x=10print(x)Unlike C++, when local objects go out of scope in Python, they are
not guaranteed to be removed immediately or in the order of creation. Instead, the object lifetime
in Python is managed by its built-inGarbage Collector,
which relies on reference counting mechanism.
There is the__del__magic method, which can be used to display this behavior:classMyObj:def__init__(self,name,ref=None):self.name=nameself.ref=refdef__del__(self):print("deleted",self.name)deffoo():obj1=MyObj("1")obj2=MyObj("2",obj1)obj3=MyObj("3",obj2)obj1.ref=obj2# add a non-trivial reference cyclefoo()print("after return")The sample above produces the following output (Python 3.10.6, standard package on Ubuntu):deleted 3
after return
deleted 1
deleted 2meaning only the 3rd object was removed at function exit, while objects
1 and 2 were removed at the interpreter exit. The garbage collector is capable
of detecting simple reference cycles, but the time and order of object removal is
not specified. Additionally, garbage collection can be turned off.To allow scope-dependent object lifetime, Python offersContext managers, thewithstatement,
and the built-incontextliblibrary. There is also theExitStackutility that allows to write simpler code and control stack "unwinding".fromcontextlibimportExitStack,closingclassMyObj:def__init__(self,name,ref=None):self.name=nameself.ref=refdefclose(self):print("deleted",self.name)def__enter__(self):returnselfdef__exit__(self,*args,**kwargs):self.close()deffoo():withExitStack()ases:obj1=MyObj("1")# register an exit functiones.callback(obj1.close)obj2=MyObj("2",obj1)# register an exit context manageres.enter_context(closing(obj2))# acquire resource and register exit cmobj3=es.enter_context(MyObj("3",obj2))obj1.ref=obj2# add a non-trivial reference cycleprint("after es")foo()print("after return")Now, the example produces the desired output:deleted 3
deleted 2
deleted 1
after es
after returnThis approach follows Python design principles and idioms, such as
"Explicit is better than implicit". It allows both invasive and non-invasive uses
withContextManagerprotocol andExitStack.callback(). In practice, however,
the use of such constructs is often intermixed with try-except blocks,
which may reduce readability of the code (mostly because of the extra indentation levels added).
With some classes, almost every function needs to begin with exit stack initialization.
It often happens when there are some cleanup or rollback procedures supposed
to be executed on an error:importosimportos.pathimportshutilfromcontextlibimportExitStackclassSomeClass:...deffoo(self,dst_dir):dir_existed=os.path.isdir(dst_dir)ifnotdir_existed:os.makedirs(dst_dir)try:withopen("bar.txt","w")asinput_file:self._write_file_contents(input_file)exceptExceptionasexc:ifnotdir_existed:shutil.rmtree(dst_dir)If there are multiple resources to control in a single function, the code becomes a mess.
This code can be made a little bit more readable and maintainable withExitStack-
now we don't need to remember the original resource states, but we need to cleanup the stack
on success:classSomeClass:...deffoo_with_es(self,dst_dir):withExitStack()ases:ifnotos.path.isdir(dst_dir):os.makedirs(dst_dir)es.callback(shutil.rmtree,dst_dir)withopen("bar.txt","w")asinput_file:self._write_file_contents(input_file)es.pop_all()This library tries to go a step further and improve this situation a little bit more:fromscopedimportscoped,on_error_doclassSomeClass:...@scopeddeffoo_scoped(self,dst_dir):ifnotos.path.isdir(dst_dir):os.makedirs(dst_dir)on_error_do(shutil.rmtree,dst_dir)withopen("bar.txt","w")asinput_file:self._write_file_contents(input_file)OverviewGo to the topThe main interface of the library is the@scopedfunction decorator. It allows to use
helper functions such asscope_add(),on_error_do()andon_exit_do()inside the
function to define resource-managing variables and set up actions performed on error
and on exit.Thescope_add()function provides a readable way to declare variables,
that implement theContextManagerprotocol.Theon_error_do()andon_exit_do()functions provide a way to add custom callbacks
(including lambdas) to the list of the actions performed on error and on the function exitExample:importosimportos.pathimportshutilfromscopedimportscoped,on_error_do,scope_add@scopeddefwrite_directory(dst_dir):"""Creates a directory, if needed, and writes data inside.Cleans everything extra in the case of error."""ifnotos.path.isdir(dst_dir):os.makedirs(dst_dir)on_error_do(shutil.rmtree,dst_dir)db_connection=scope_add(open_db_conn())on_exit_do(extra_cleanup)withopen("bar.txt","w")asinput_file:_write_file_contents(input_file,db_connection)# Calls on the normal exit:## extra_cleanup()# open_db_conn().__enter__().__exit__()### Calls on an error:## extra_cleanup()# open_db_conn().__enter__().__exit__()# shutil.rmtree(dst_dir)Installationpipinstallscoped-functionsIf you want to install from the repository:pipinstall"git+https://github.com/zhiltsov-max/scoped"API ReferenceGo to the top@scoped(arg_name: str = None)A function decorator that allows to register context managers and exit callbacks
withscope_add(),on_error_do()andon_exit_do()inside the decorated function.Can be used 2 ways:Implicit:@scopeddeffoo():...Explicit: adds an additional kw-parameter with specified name to the function calls.
This can be useful if you want to be more explicit and if you want to use extra
functionality.@scoped(arg_name='scope')deffoo(*,scope:Scope):scope.add(...)Note that this decorator will not work with generators, because they implemented
differently from normal functions. Please use the "traditional" approach withScopeorExitStackinstead in these cases:@scopeddefgenerator():on_exit_do(print,"finished")yieldnext(gen())# error: no Scope objectdefgenerator2():withScope()asscope:scope.on_exit_do(print,"finished")yieldnext(gen())# okscope_add(cm: ContextManager[T]) -> TEnters the context manager and adds it to the exit stack. If called
multiple times, exit callbacks will be called on exit in the reversed order.Returns: cm.enter() resulton_error_do(callback, *args, ignore_errors: bool = False, kwargs=None) -> NoneRegisters a function to be called on scope exit because of an error. The primary use
is for error rollback functions. If called multiple times, callbacks will be called
on exit in the reversed order.Ifignore_errorsisTrue, the errors from this function call will be ignored.
Allows to pass function args with the*argsandkwargsparameters.defbar(*args,**kwargs):...@scopeddeffoo(x):on_error_do(bar,x,ignore_errors=True,kwargs={'y':42})# bar(x, y=42) will be called prior to foo() exit on errorraiseException("error")on_exit_do(callback, *args, ignore_errors: bool = False, kwargs=None) -> NoneRegisters a function to be called on scope exit. The callback is called unconditionally,
equivalently to thefinallyblock in thetry-exceptclause. If called
multiple times, callbacks will be called on exit in the reversed order.defbar(*args,**kwargs):...defbaz():...@scopeddeffoo(x,y):on_error_do(bar,x)on_exit_do(bar,y)baz()# Called on an error:## bar(y)# bar(x)### Called on the normal exit:## bar(y)