Package: wrapt-timeout-decorator
wrapt_timeout_decoratorVersion v1.5.1 as of 2024-02-28 seeChangelogThere are several timeout decorators available, but the one mentioned here
focuses on ensuring correctness when used with classes, methods, class methods,
static methods, etc. It also preserves traceback information for PyCharm debugging.The timeout can be dynamically adjusted, calculated from other parameters or methods accessible via an optional eval function.Two timeout strategies have been implemented:
one using “Signals” and the other using “Subprocess”.Signals StrategyThe “Signals” strategy (for POSIX Systems) is elegant and efficient,
but it has some important caveats which should be reviewed
in theConsiderations using Signalssection.Subprocess Strategy (the default)The utilization of subprocesses serves as the default approach for executing timeouts:Windows Compatibility:Given the absence of signal support,
subprocesses become the sole method for implementing timeouts on Windows,
automatically applied to accommodate the platform’s limitations.
On Windows the only available startmethod for subprocesses isspawnPOSIX Systems:On POSIX-compliant systems, signals cannot be employed within
subthreads, necessitating the use of subprocesses in these contexts as well.
On POSIX the available startmethods for subprocesses arefork,forkserver,spawnTo ensure compatibility and functionality across subprocesses,
it’s essential that as many object types as possible are pickleable.
To this end, thedilllibrary is preferred over Python’s standardpicklemodule,
andmultiprocessis chosen instead ofmultiprocessing.dillenhances the pickle module’s capabilities, extending support for
serialization and deserialization of a broader array of Python object types.Subprocess communication is facilitated throughmultiprocess.piperather thanqueue.
This choice not only boosts performance but also enhances compatibility,
potentially offering better support for environments like Amazon AWS.Subprocesses can be initiated using various methods,
including ‘fork’, ‘forkserver’, and ‘spawn’.
For detailed information on these methods and their implications,
please refer to SectionConsiderations using Subprocessesof this manual.automated tests, Github Actions, Documentation, Badges, etc. are managed withPizzaCutter(cookiecutter on steroids)Python version required: 3.8.0 or newertested on recent linux with python 3.8, 3.9, 3.10, 3.11, 3.12, pypy-3.9, pypy-3.10 - architectures: amd64100% code coverage, flake8 style checking ,mypy static type checking ,tested underLinux, macOS, Windows, automatic daily builds and monitoringTry it OnlineUsageUsage from CommandlineInstallation and UpgradeRequirementsAcknowledgementsContributeReport IssuesPull RequestCode of ConductLicenseChangelogTry it OnlineYou might try it right away in Jupyter Notebook by using the “launch binder” badge, or clickhereUsageBasic UsageGeneral Recommendationsuse with WindowsQuick Guide for the EagerIn-Depth Explanation for the CuriousWindows Compatibility IssueTiming ConsiderationsConsiderations using SignalsConsiderations using SubprocessesOverviewInitializationProcess Execution and CommunicationSubprocess Start MethodsChoosing the Right Start MethodSetting the Start MethodSpecial Considerations for Uvicorn, FastAPI, asyncioHandling Nested TimeoutsCustom Timeout ExceptionParametersOverride ParametersMultithreadingSubprocess Monitoringuse as function not as decoratorDynamic Timeout Value Adjustment with evalToolsdetect pickle errorsset_subprocess_starting_methodLogging Challenges with Subprocesseshard timeoutUnderstanding Timeout Durations Across PlatformsMYPY TestingBasic Usageimporttimefromwrapt_timeout_decoratorimport*@timeout(5)defmytest(message):# this example does NOT work on windows, please check the section# "use with Windows" in the README.rstprint(message)foriinrange(1,10):time.sleep(1)print('{}seconds have passed'.format(i))if__name__=='__main__':mytest('starting')General RecommendationsIt’s recommended to minimize the utilization of timeouts in your programming, reserving them for truly essential instances.Timers should be applied at an appropriate level of detail, tailored specifically to the needs of your application.
This precision aids in circumventing unwanted outcomes, such as the mishandling of exceptions by unrelated code sections
or complications with entities that cannot be pickled.Conversely, it’s prudent to refrain from embedding a Timeout Decorator within loops that execute multiple times.
Such an approach can induce notable delays, especially on Windows systems, owing to the additional burden of initiating subprocesses.Where possible, opt for the timeout features natively available in the functions and libraries at your disposal.
These inherent capabilities are often adequate for the majority of use cases.
The implementation of a Timeout Decorator is best reserved as a measure of last resort,
subsequent to the exhaustive consideration of alternative strategies.Additionally, be cognizant of the fact that the behavior and efficiency of subprocesses may vary significantly across platforms
(Windows versus Linux) and depending on the chosen method for subprocess initiation.
Refer to the documentation onSubprocess Start Methodsfor further details.BAD EXAMPLE (Pseudocode) - lets assume the write to the database fails sometimes for unknown reasons, and “hangs”# module file_analyzerimporttimefromwrapt_timeout_decoratorimport*defread_the_file(filename):...defanalyze_the_file(filename):...defwrite_to_database(file_content):...@timeout(5)# try to minimize the scope of the timeoutdefimport_file(filename):file_content=read_the_file(filename)structured_data=analyze_the_file(file_content)write_to_database(structured_data)BETTER EXAMPLE (Pseudocode)# module file_analyzerimporttimefromwrapt_timeout_decoratorimport*defread_the_file(filename):...defanalyze_the_file(filename):...@timeout(5)# better, because smaller scopedefwrite_to_database(file_content):...defimport_file(filename):file_content=read_the_file(filename)structured_data=analyze_the_file(file_content)write_to_database(structured_data)use with WindowsQuick Guide for the EagerTo bypass complexities, simply place the decorated function within a separate module, rather than in the main script.In-Depth Explanation for the CuriousOn Windows, due to the absence of native forking support, Python attempts to emulate a forking environment.
This emulation involves re-importing the main module under a different name, not as ‘__main__’.
This behavior is part of Python’s multiprocessing efforts to replicate the main process’s environment as closely as possible.
Consequently, it’s crucial to protect the entry point of your application with the well-known conditional statement
“if __name__ == ‘__main__’:”.importlib_foodefsome_module():lib_foo.function_foo()defmain():some_module()# here the subprocess stops loading, because __name__ is NOT '__main__'if__name__='__main__':main()Windows Compatibility IssueThe challenge arises from Windows OS’s lack of support for the “fork” process model, a limitation not present in Unix-based systems.Further details can be explored through these resources:[Stack Overflow discussion on multiprocessing and__name__ == ‘__main__’](https://stackoverflow.com/questions/45110287/workaround-for-using-name-main-in-python-multiprocessing)[Python’s multiprocessing documentation for Windows](https://docs.python.org/2/library/multiprocessing.html#windows)Due to this, whenmain.pyis re-imported under a name different from“__main__”, references within decorated classes
and functions become invalid. To circumvent this, it’s advisable to house decorated entities in a separate module.
Generally, and particularly on Windows, themain()function should be streamlined to act merely as an entry point,
with the substantive logic residing in modules.
Additionally, storing settings or configurations in a distinct file is beneficial for centralized access and to leverage features
like type hints and auto-completion in your preferred IDE.Thedillserializer, chosen for its broader compatibility, successfully serializes the__main__context,
enabling objects to be pickled to“__main__.lib_foo”,“__main__.some_module”,“__main__.main”, etc.
This overcomes the limitations faced when usingpickle, which cannot serialize various types including functions
with yields, nested functions, and more.Dillenhances functionality by enabling the saving/loading of Python sessions, extraction of source code, and interactive debugging of serialization errors.
However, it necessitates that decorated methods and classes not be defined in the__main__context but within a module.For more insights on serialization withpickleordill:
- [Stack Overflow discussion on serializing objects in__main__withpickleordill](https://stackoverflow.com/questions/45616584/serializing-an-object-in-main-with-pickle-or-dill)Timing ConsiderationsGiven the variable duration of the spawning process (due to re-importing modules),
thehard timeoutsection provides guidance on configuring the commencement of timeouts.An illustration highlights a scenario functional on Linux but problematic on Windows,
where the variable“name”and the function“sleep”are not recognized in the spawned process:main.py:fromtimeimportsleepfromwrapt_timeout_decoratorimport*name="my_var_name"@timeout(5,use_signals=False)defmytest():# this example does NOT work on windows, please check the example below !# You need to move this function into a module to be able to run it on windows.print("Start ",name)foriinrange(1,10):sleep(1)print("{}seconds have passed".format(i))returniif__name__=='__main__':mytest()here the same example, which will work on Windows:# my_program_main.py:importlib_testdefmain():lib_test.mytest()if__name__=='__main__':main()# conf_my_program.py:classConfMyProgram(object):def__init__(self):self.name:str='my_var_name'conf_my_program=ConfMyProgram()# lib_test.py:fromwrapt_timeout_decoratorimport*fromtimeimportsleepfromconf_my_programimportconf_my_program# use_signals = False is not really necessary here, it is set automatically under Windows# but You can force NOT to use Signals under Linux@timeout(5,use_signals=False)defmytest():print("Start ",conf_my_program.name)foriinrange(1,10):sleep(1)print("{}seconds have passed".format(i))returniConsiderations using SignalsABADGER1999 highlights in hisblog postthe
potential pitfalls of using signals alongside the TimeoutException.
This approach may not be advisable as the exception can be intercepted within the decorated function.While it’s possible to implement a custom Exception derived from the Base Exception Class,
this doesn’t guarantee the code will behave as anticipated.
For an illustrative example, you’re encouraged to conduct an experiment using aJupyter notebook.importtimefromwrapt_timeout_decoratorimport*# Considerations for Signal Usage - Handling TimeoutError# The TimeoutError triggered by a signal might be intercepted within the decorated function.# Utilizing a custom Exception, derived from the base Exception Class, is a possible workaround.# Within Python 3.7.1's standard library, there are over 300 instances where your custom timeout might be caught# if it's based on Exception. Should you base your exception on BaseException,# there still remain 231 potential catch points.# To ensure proper timeout management, it's advisable to set `use_signals=False`.# Consequently, `use_signals` defaults to `False` in this decorator to avoid these issues.@timeout(5,use_signals=True)defmytest(message):try:print(message)foriinrange(1,10):time.sleep(1)print('{}seconds have passed - lets assume we read a big file here'.format(i))# TimeoutError is a Subclass of OSError - therefore it is catched here !exceptOSError:foriinrange(1,10):time.sleep(1)print('Whats going on here ? - Ooops the Timeout Exception is catched by the OSError !{}'.format(i))exceptException:# even worse !passexcept:# the worst - and exists more then 300x in actual Python 3.7 stdlib Code !# so You never really can rely that You catch the TimeoutError when using Signals !passif__name__=='__main__':try:mytest('starting')print('no Timeout Occured')exceptTimeoutError():# this will never be printed because the decorated function catches implicitly the TimeoutError !print('Timeout Occured')Considerations using SubprocessesOverviewSubprocesses ares utilized by default to implement timeout functionality. This involves forking or spawning subprocesses, each with its own set of
considerations and caveats.InitializationWindows Considerations:On Windows, the spawn method can significantly slow down the process initiation.Main Context Protection:It is crucial to protect the__main__context for compatibility, especially on Windows. See the “Usage with Windows” section for more details.Pickle Requirements:Function codes and arguments must be pickleable. To accommodate a wider range of types,dillis used for serialization.Global Variables:Access to global variables from a child process might not reflect the parent process’s state at the time of the fork. Module-level constants are generally unaffected.Process Execution and CommunicationSubprocess Execution:Functions run in a separate subprocess, whether forked or spawned.Data Transmission:Parameters and results are communicated through pipes, withdillused for serialization.Timeout Management:Absent a result within the specified timeout, the subprocess is terminated usingSIGTERM. Ensuring subprocesses can terminate safely is essential; thus, disabling theSIGTERMhandler is not advisable.Subprocess Start MethodsWindows Limitation:Onlyspawnis available on Windows.Linux/Unix Options:Options includefork,forkserver, andspawn.Fork:Efficiently clones the parent process, including memory space, but may lead to issues with shared resources or in multi-threaded applications.Forkserver:Starts a server at program launch, creating new processes upon request for better isolation but at a slower pace due to the server communication requirement.Spawn:Initiates a fresh Python interpreter process, ensuring total independence at the cost of slower start-up due to the need for full initialization.Choosing the Right Start Methodforkoffers speed but can encounter issues with resource sharing or threading.forkserverenhances stability and isolation, ideal for applications requiring safety or managing unstable resources.spawnprovides the highest level of isolation, recommended for a clean start and avoiding shared state complications.Setting the Start MethodConfigure the start method withmultiprocessing.set_start_method(method, force=True). This should be done cautiously, ideally once, and within theif __name__ == '__main__'block to prevent unintended effects.
Since we usemultiprocessinstead ofmultiprocessing, we provide a method to set the starting method on both at the same time.
see :set_subprocess_starting_methodSpecial Considerations for Uvicorn, FastAPI, asyncioFor Uvicorn or FastAPI applications, a specific approach to theforkmethod is recommended to ensure proper signal handling and isolation, facilitated by thedec_mp_reset_signalsparameter.
This design aims to reset signal handlers and manage file descriptors in child processes effectively.
You can set that by passing the parameterdec_mp_reset_signals=Trueto the decorator.Handling Nested TimeoutsDue to Unix’s limitation of having just one ALARM signal per process, it’s necessary to setuse_signals=Falsefor nested timeouts
to function correctly. While the outermost decorator may utilize signals,
all inner decorators must haveuse_signalsset toFalse—which is the default setting.
For practical experimentation and to see this behavior in action,
you’re encouraged to use aJupyter notebook.# main.pyimportmylib# this example will work on Windows and Linux# since the decorated function is not in the __main__ scope but in another module !if__name__=='__main__':mylib.outer()# mylib.pyfromwrapt_timeout_decoratorimport*importtime# this example will work on Windows and Linux# since the decorated function is not in the __main__ scope but in another module !@timeout(1,use_signals=True)defouter():inner()@timeout(5)definner():time.sleep(3)print("Should never be printed if you call outer()")Custom Timeout ExceptionDefine a different exception to be raised upon timeout:importtimefromwrapt_timeout_decoratorimport*# this will throw StopIteration Error instead of TimeoutError@timeout(5,timeout_exception=StopIteration)defmytest(message):# this example does NOT work on windows, please check the section# "use with Windows" in the README.rstprint(message)foriinrange(1,10):time.sleep(1)print('{}seconds have passed'.format(i))if__name__=='__main__':mytest('starting')Parameters@timeout(dec_timeout,use_signals,timeout_exception,exception_message,dec_allow_eval,dec_hard_timeout,dec_mp_reset_signals)defdecorated_function(*args,**kwargs):# interesting things happens here ......dec_timeoutThis parameter sets the timeout duration. It accepts a float, integer, or a string
that can be evaluated to a number if dec_allow_eval is enabled.
By default, there’s no timeout (None). You can change the timeout dynamically
by passing a dec_timeout keyword argument to the decorated function.use_signalsThis boolean parameter controls whether to use UNIX signals for implementing timeouts.
It’s the most accurate method but comes with certain limitations,
such as being available only on Linux and macOS, and only in the main thread.
By default, signals are not used (False). It’s typically not necessary to modify
this setting manually, but you can override it by passing ‘use_signals=True’ to the decorated function.timeout_exceptionSpecifies the exception to raise when a timeout occurs.
by default, it’s set to TimeoutError
type: exception
default: TimeoutErrorexception_messageYou can customize the message of the timeout exception.
The default message includes the name of the function and the timeout duration.
This message gets formatted with the actual values when a timeout occurs.
type: str
default : ‘Function {function_name} timed out after {dec_timeout} seconds’ (will be formatted)dec_allow_evalWhen enabled (True), this boolean parameter allows the dec_timeout string to be evaluated dynamically.
It provides accessto the decorated function (wrapped),the instance it belongs to (instance),the positional arguments (args),and keyword arguments (kwargs).It’s disabled (False) by default for safety reasons but can be enabled by passing a dec_allow_eval
keyword argument to the decorated function.instance    Example: ‘instance.x’ - see example above or doku
args        Example: ‘args[0]’ - the timeout is the first argument in args
kwargs      Example: ‘kwargs[“max_time”] * 2’
type: bool
default: false
see section “Dynamic Timeout Value Adjustment with eval” in the manualdec_hard_timeoutThis boolean parameter is relevant when signals cannot be used,
necessitating the creation of a new process for the timeout mechanism.
Setting it to True means the timeout strictly applies to the execution time of the function,
potentially not allowing enough time for process creation.
With False, the process creation time is not included in the timeout, giving the actual function
the full duration to execute.
You can override this setting by passing a dec_hard_timeout keyword argument to the decorated function.
type: bool
default: false
can be overridden by passing the kwarg dec_hard_timeout to the decorated function*dec_mp_reset_signalsThis parameter is relevant when using the “fork” start method for multiprocessing.
Setting it to True accomplishes two primary objectives:Restores Default Signal Handlers in Child Processes:It ensures that child processes revert to the default signal handling behavior,
rather than inheriting signal handlers from the parent process.
This adjustment is crucial for applications utilizing frameworks like “unicorn” or “FastAPI”,
facilitating the use of the efficient “fork” method while maintaining correct signal handling.
For more context, refer to the Discussion on
FastAPI GitHub page:https://github.com/tiangolo/fastapi/discussions/7442Avoids Inheritance of the File Descriptor (fd) for Wakeup Signals:Typically, if the parent process utilizes a wakeup_fd, child processes inherit this descriptor.
Consequently, when a signal is sent to a child, it is also received by the parent process
via this shared socket, potentially leading to unintended termination or shutdown of the application.
By resetting signal handlers and not using the inherited fd, this parameter prevents such conflicts,
ensuring isolated and correct signal handling in child processes.Note: This parameter exclusively affects processes initiated with the “fork” method
and is not applicable to other multiprocessing start methods.For enhanced isolation of subprocesses, consider utilizing the “forkserver” or “spawn” start methods in multiprocessing.
These methods provide a greater degree of independence between the parent process and its children,
mitigating the risks associated with shared resources and ensuring a cleaner execution environment for each subprocess,
at the cost of slower startup times. This slowdown is due to the additional overhead involved in setting up a completely
new process environment for each child process, as opposed to directly duplicating the parent process’s environment,
which occurs with the “fork” method.that means the decorated_function must not use that kwarg itself, since this kwarg will be popped from the kwargsOverride Parametersdecorator parameters starting with dec_* and use_signals can be overridden by kwargs with the same name :importtimefromwrapt_timeout_decoratorimport*@timeout(dec_timeout=5,use_signals=False)defmytest(message):# this example does NOT work on windows, please check the section# "use with Windows" in the README.rstprint(message)foriinrange(1,10):time.sleep(1)print('{}seconds have passed'.format(i))if__name__=='__main__':mytest('starting',dec_timeout=12)# override the decorators setting. The kwarg dec_timeout will be not# passed to the decorated function.MultithreadingSignals will not work if your function is not executed in the main thread.use_signalsis therefore automatically disabled (if set) when the function is not running in the main thread.importtimefromwrapt_timeout_decoratorimport*@timeout(5,use_signals=False)defmytest(message):# this example does NOT work on windows, please check the section# "use with Windows" in the README.rstprint(message)foriinrange(1,10):time.sleep(1)print('{}seconds have passed'.format(i))if__name__=='__main__':mytest('starting')WarningMake sure that in case of subprocess strategy for timeout, your function does not return objects which cannot
be pickled, otherwise it will fail at marshalling it between master and child processes. To cover more cases,
we use multiprocess and dill instead of multiprocessing and pickle.Since Signals will not work on Windows, it is disabled by default, whatever You set.Subprocess Monitoringwhen using subprocesses, the subprocess is monitored if it is still alive.
if the subprocess was terminated or killed (for instance by OOMKiller),multiprocessing.context.ProcessErrorwill be raised.
By default the subprocess is monitored every 5 seconds, but can be set with parameterdec_poll_subprocess. polling can be turned off by setting to 0.0 secondsfromwrapt_timeout_decoratorimporttimeout@timeout(10,use_signals=False,timeout_exception=TimeoutError,dec_poll_subprocess=1)defslow_process()->None:# should have enough time to finish# but instead it gets terminated, and the# poll the subprocess every secondlogger.error(f"Slow process started at{get_str_time()}")time.sleep(5)logger.error(f"Slow process done at{get_str_time()}")deffake_oom_killer()->None:logger.error(f"Fake OOMKiller started at{get_str_time()}")time.sleep(2)# kill sibling slow_process# hacky way to find ittarget=psutil.Process().parent().children(recursive=True)[-1]target.kill()logger.error(f"Killed{target.pid}at{get_str_time()}")defstart_processes()->None:"""
    starts the 'fake_oom_killer' and 'slow_process' process -
    and kill 'slow_process' after two seconds

    >>> start_processes()
    Traceback (most recent call last):
        ...
    multiprocessing.context.ProcessError: Function slow_process was terminated or killed after ... seconds
    """process_oom_killer=multiprocessing.Process(target=fake_oom_killer,args=())process_oom_killer.start()slow_process()process_oom_killer.join()defget_str_time()->str:t=time.localtime()current_time=time.strftime("%H:%M:%S",t)returncurrent_timeif__name__=='__main__':start_processes()use as function not as decoratorYou can use the timout also as function, without using as decorator:importtimefromwrapt_timeout_decoratorimport*defmytest(message):print(message)foriinrange(1,10):time.sleep(1)print('{}seconds have passed'.format(i))if__name__=='__main__':timeout(dec_timeout=5)(mytest)('starting')Dynamic Timeout Value Adjustment with evalThe timeout value can be dynamically adjusted, calculated from other parameters or methods accessible via the eval function.
This capability is highly potent yet bears significant risks, especially when evaluating strings from UNTRUSTED sources.Caution!Utilizing eval with untrusted input is perilous.
For an in-depth understanding, refer tothis article by Ned Batchelder.When activated, thedec_timeoutfunction parameter,
or the value passed through thedec_timeoutkeyword argument (kwarg), will undergo evaluation if it’s a string type.Accessible objects within the eval context include:wrapped: Represents the decorated function and its attributes.instance: Accesses attributes of the class instance, e.g.,'instance.x'refers to an attributexof the instance.args: Refers to positional arguments, e.g.,'args[0]'might be used to indicate the first argument is the timeout.kwargs: Accesses keyword arguments, e.g.,'kwargs["max_time"]* 2'doubles the value ofmax_time.These elements underscore the feature’s versatility but also highlight its potential hazards.
By default,allow_evalis turned off to mitigate risks.
However, it can be enabled to address specific use cases without altering the timeout decorator’s core functionality.# this example does NOT work on windows, please check the section# "use with Windows" in the README.rstdefclassFunnyMemes(object):def__init__(self,x):self.x=x@timeout('instance.x',dec_allow_eval=True)defswallow(self):whileTrue:time.sleep(0.5)print('swallow')@timeout(1)defparrot(self):whileTrue:time.sleep(0.5)print('parrot')@timeout(dec_timeout='args[0] + kwargs.pop("more_time",0)',dec_allow_eval=True)defknight(self,base_delay):whileTrue:time.sleep(base_delay)print('knight')defmain():my_memes=FunnyMemes(2)my_memes.swallow()# this will time out after 2 secondsmy_memes.swallow(dec_timeout='instance.x * 2 + 1')# this will time out after 5 secondsmy_memes.parrot(dec_timeout='instance.x * 2 + 1',dec_allow_eval=True)# this will time out after 5 secondsmy_memes.knight(1,more_time=4)# this will time out after 5 secondsif__name__=='__main__':main()Toolsdetect pickle errorsKeep in mind that when employing subprocesses, both decorated functions and their return values must be pickleable.
To identify issues with pickling, you can utilize thedetect_unpickable_objectsfunction:fromwrapt_timeout_decoratorimport*detect_unpickable_objects(object_to_pickle,dill_trace=True)set_subprocess_starting_methodSet the start Method for Subprocesses. Since we use multiprocess,
we set the starting method for multiprocess and multiprocessing to the same value.
we did not test what would happen if we set that to different values.Windows Limitation: Onlyspawnis available on Windows.Linux/Unix Options: Options includefork,forkserver, andspawn.fork:Efficiently clones the parent process, including memory space,
but may lead to issues with shared resources or in multi-threaded applications.forkserver:Starts a server at program launch, creating new processes upon request
for better isolation but at a slower pace due to the server communication requirement.spawn:Initiates a fresh Python interpreter process, ensuring total independence
at the cost of slower start-up due to the need for full initialization.Choosing the Right Start Methodforkoffers speed but can encounter issues with resource sharing or threading.forkserverenhances stability and isolation, ideal for applications requiring safety or managing unstable resources.spawnprovides the highest level of isolation, recommended for a clean start and avoiding shared state complications.Setting the Start MethodConfigure the start method withset_subprocess_starting_method(method)This should be done cautiously, ideally once, and within theif __name__ == ‘__main__’block to prevent unintended effects.fromwrapt_timeout_decoratorimport*set_subprocess_starting_method("forkserver")Logging Challenges with SubprocessesWhensignals=Falseis set, implementing logging within a subprocess poses challenges.
A new process does not inherit the main process’s logger object, necessitating further development
for integration with the main process’s logger via mechanisms like sockets or queues.Utilizinglogger=logging.getLogger()within the wrapped function results in the instantiation of a new Logger Object.
Configuring this Logger, especially for file logging from concurrent processes, presents complications as direct file
logging from multiple processes is generally unsupported.
A potential solution involves employing a SocketHandler coupled with a Receiver Thread to facilitate logging.In the interim, it’s necessary to initialize a separate logger within the decorated function for logging purposes.
It’s crucial to remember that writing to the same logfile from multiple processes is not advisable.
While certain logging modules may offer solutions for concurrent logging, they require specific setup and configuration.hard timeoutWhen employing subprocesses (which is the default behavior), the timeout functionality is achieved by initiating
a new subprocess and terminating it once the specified timeout period elapses.
The process creation speed varies significantly between operating systems.
On Linux, thefork()method allows rapid creation of a new process.
In contrast, on Windows, thespawn()method can introduce a noticeable delay due to the necessity of reloading the main context,
with spawning a small module potentially taking upwards of 0.5 seconds.The timeout duration commences subsequent to the creation of the new process.
Consequently, the specified timeout reflects the period the decorated function is permitted to execute,
exclusive of the process setup time. This distinction is particularly vital for scenarios utilizing brief timeout intervals:@timeout(0.1)deftest():time.sleep(0.2)Understanding Timeout Durations Across PlatformsThe implementation of timeouts, yields different total timeout durations on Linux (fork, forkserver) compared to Windows (spawn).
On Linux, the timeout process may for instance complete in approximately 0.1 seconds with “fork”.
Conversely, on Windows, the total time to reach timeout could extend for instance to about 0.6 seconds,
comprising a 0.5-second delay to spawn a new process and then allowing 0.1 seconds for the functiontest()to execute.To enforce a decorated function to timeout strictly after the specified timeout period,
you may use thedec_hard_timeout=Trueparameter.With this setting, the targeted function will timeout precisely after the designated duration after start,
regardless of the process spawning time.
However, setting a very short timeout with this option may prevent the process from running at all,
resulting in an immediate timeout upon spawning.NoteThe term “precisely” should be interpreted with a degree of flexibility.
There remains a negligible delay in returning from the spawned process, making it imperative to approach very short timeouts with caution.MYPY Testingfor local MYPY Testing please make sure that the stub file “wrapt.pyi” is in in the MYPY Path (once!), in order to preserve the decorated function signature.Usage from CommandlineUsage: wrapt_timeout_decorator [OPTIONS] COMMAND [ARGS]...

  The better timout decorator

Options:
  --version                     Show the version and exit.
  --traceback / --no-traceback  return traceback information on cli
  -h, --help                    Show this message and exit.

Commands:
  info  get program informationsInstallation and UpgradeBefore You start, its highly recommended to update pip:python -m pip --upgrade pipto install the latest release from PyPi via pip (recommended):python -m pip install --upgrade wrapt_timeout_decoratorto install the latest release from PyPi via pip, including test dependencies:python -m pip install --upgrade wrapt_timeout_decorator[test]to install the latest version from github via pip:python -m pip install --upgrade git+https://github.com/bitranox/wrapt_timeout_decorator.gitinclude it into Your requirements.txt:# Insert following line in Your requirements.txt:
# for the latest Release on pypi:
wrapt_timeout_decorator

# for the latest development version :
wrapt_timeout_decorator @ git+https://github.com/bitranox/wrapt_timeout_decorator.git

# to install and upgrade all modules mentioned in requirements.txt:
python -m pip install --upgrade -r /<path>/requirements.txtto install the latest development version, including test dependencies from source code:# cd ~
$ git clone https://github.com/bitranox/wrapt_timeout_decorator.git
$ cd wrapt_timeout_decorator
python -m pip install -e .[test]via makefile:
makefiles are a very convenient way to install. Here we can do much more,
like installing virtual environments, clean caches and so on.# from Your shell's homedirectory:$gitclonehttps://github.com/bitranox/wrapt_timeout_decorator.git$cdwrapt_timeout_decorator# to run the tests:$maketest# to install the package$makeinstall# to clean the package$makeclean# uninstall the package$makeuninstallRequirementsfollowing modules will be automatically installed :## Project Requirementscli_exit_toolslib_detect_testenv# class decorators are failing on windows with dill 0.3.5, 0.3.5.1dill>0.3.0,!=0.3.5,!=0.3.5.1;sys_platform=="win32"dill;sys_platform!="win32"multiprocesspsutilwraptAcknowledgementsspecial thanks to “uncle bob” Robert C. Martin, especially for his books on “clean code” and “clean architecture”ContributeI would love for you to fork and send me pull request for this project.
-please ContributeLicenseThis software is licensed under theMIT license—Changelogv1.5.12024-02-28:overhaul documentationgithubactions/checkout@v4githubactions/setup-python@v5v1.5.02024-02-27:thanks toAlberto Ornaghi:parameterdec_mp_reset_signalsrestores the default behavior of signal handlers on multiprocessingforksuitible especially forFastAPIandUvicornv1.4.12024-01-10:thanks tofayak:omit mypy option –no-implicit-reexportexplicitly export methods in__init__.pyv1.4.02023-07-13:check for killed child processes (for instance by OOMKiller)change dill requirements for windowsrequire minimum python 3.8remove python 3.7 testsintroduce PEP517 packaging standardintroduce pyproject.toml build-systemremove mypy.iniremove pytest.iniremove setup.cfgremove setup.pyremove .bettercodehub.ymlremove .travis.ymlupdate black configclean ./tests/test_cli.pyadd codeql badgemove 3rd_party_stubs outside the src directory to./.3rd_party_stubsadd pypy 3.10 testsadd python 3.12-dev testsv1.3.12.22022-06-01: update to github actionscheckout@v3andsetup-python@v3v1.3.122022-05-23: update requirements.txtv1.3.112022-05-23:set dill version < 0.3.5 on windows, because decorating class methods fails with dill 0.3.5 upwardsupdate tests to the latest python versionsv1.3.102022-04-26: add tests for thread lockv1.3.92022-04-26: preserve Signature of the decoratorv1.3.82022-03-29: remedy mypy Untyped decorator makes function “cli_info” untypedv1.3.72022-03-28: extend time on test_timeout_decorator_arg - github macos seems to be slow, so sometimes that test failsv1.3.62022-03-25: fix github actions windows testv1.3.42022-03-23: extend time on test_timeout_ok_timeout_as_kwarg - github macos seems to be slow, so sometimes that test failsv1.3.32022-03-10: extend time on test_timeout_alternate_exception - github macos seems to be slow, so sometimes that test failsv1.3.22022-03-01: github actions pipeline, codestyle black, fix requirementsv1.3.12019-09-02: strict mypy static type checking, housekeepingv1.3.02019-05-03: pointing out caveats when using signals, the decorator defaults now to NOT using Signals !v1.2.92019-05-03: support nested decorators, mypy static type checkingv1.2.82019-04-23: import multiprocess as multiprocess, not as multiprocessing - that might brake other packagesv1.2.02019-04-09: initial PyPi releasev1.1.02019-04-03: added pickle analyze convenience functionv1.0.92019-03-27: added OsX and Windows tests, added parameter dec_hard_timeout for Windows, 100% Code Coveragev1.0.82019-02-26: complete refractoring and code cleaningv1.0.72019-02-25:  fix pickle detection, added some tests, codecov now correctly combining the coverage of all testsv1.0.62019-02-24: fix pickle detection when use_signals = False, drop Python2.6 support since wrapt dropped it.v1.0.52018-09-13: use multiprocessing.pipe instead of queue
If we are not able to use signals, we need to spawn a new process.
This was done in the past by pickling the target function and put it on a queue -
now this is done with a half-duplex pipe.it is fasterit probably can work on Amazon AWS, since there You must not use queuesv1.0.42017-12-02: automatic detection if we are in the main thread. Signals can only be used in the main thread. If the decorator is running in a subthread, we automatically disable signals.v1.0.32017-11-30: using dill and multiprocess to enhance windows functionalityv1.0.02017-11-10: Initial public release