Package: unasyncd
UnasyncdA tool to transform asynchronous Python code to synchronous Python code.Why?Unasyncd is largely inspired byunasync, and
a detailed discussion about this approach can be foundhere.Its purpose is to reduce to burden of having to maintain both a synchronous and an
asynchronous version of otherwise functionally identical code. The idea behind simply
"taking out the async" is that often, synchronous and asynchronous code only differ
slightly: A fewawaits,async defs,async withs, and a couple of different method
names. The unasync approach makes use of this by treating the asynchronous version as a
source of truth from wich the synchronous version is then generated.Why unasyncd?The originalunasyncworks by simply replacing
certain token, which is enough for most basic use cases, but can be somewhat restrictive
in the way the code can be written. More complex cases such as exclusion of functions /
classes or transformations (such asAsyncExitStacktoExitStackwich have not only
different names but also different method names that then need to be replaced only
within a certain scope) are not possible. This can lead to the introduction of shims,
introducing additional complexity.Unasyncd's goal is to impose as little restrictions as possible to the way the
asynchronous code can be written, as long as it maps to a functionally equivalent
synchronous version.To achieve this, unasyncd leverageslibcst, enabling a
more granular control and complex transformations.Unasyncd features:Transformation of arbitrary modules, not bound to any specific directory structure(Per-file) Exclusion of (nested) functions, classes and methodsOptional transformation of docstringsReplacements based on fully qualified names
(e.g.typing.AsyncGeneratoris different thanfoo.typing.AsyncGenerator)Transformation of constructs likeasyncio.TaskGroupto a thread based equivalentA full list of supported transformations is available below.Table of contentsUnasyncdWhy?Why unasyncd?Table of contentsWhat can be transformed?Asynchronous functionsawaitAsynchronous iterators, iterables and generatorsAsynchronous iterationAsynchronous context managerscontextlib.AsyncExitStackasyncio.TaskGroupanyio.create_task_groupasyncio.sleep/anyio.sleepanyio.PathType annotationsDocstringsUsageInstallationCLIAs a pre-commit hookConfigurationFileCLI optionsExclusionsExtending name replacementsHandling of importsIntegration with lintersLimitationsDisclaimerWhat can be transformed?Unasyncd supports a wide variety of transformation, ranging from simple name
replacements to more complex transformations such as task groups.Asynchronous functionsAsyncasyncdeffoo()->str:return"hello"Syncdeffoo()->str:return"hello"awaitAsyncawaitfoo()Syncfoo()Asynchronous iterators, iterables and generatorsAsyncfromtypingimportAsyncGeneratorasyncdeffoo()->AsyncGenerator[str,None]:yield"hello"SyncfromtypingimportGeneratordeffoo()->Generator[str,None,None]:yield"hello"AsyncfromtypingimportAsyncIteratorclassFoo:asyncdef__aiter__(self)->AsyncIterator[str]:...asyncdef__anext__(self)->str:raiseStopAsyncIterationSyncfromtypingimportIteratorclassFoo:def__next__(self)->str:raiseStopIterationdef__iter__(self)->Iterator[str]:...Asyncx=aiter(foo)Syncx=iter(foo)Asyncx=awaitanext(foo)Syncx=next(foo)Asynchronous iterationAsyncasyncforxinfoo():passSyncforxinfoo():passAsync[xasyncforxinfoo()]Sync[xforxinfoo()]Asynchronous context managersAsyncasyncwithfoo()assomething:passSyncwithfoo()assomething:passAsyncclassFoo:asyncdef__aenter__(self):...asyncdef__aexit__(self,exc_type,exc_val,exc_tb):...SyncclassFoo:def__enter__(self):...def__exit__(self,exc_type,exc_val,exc_tb):...AsyncfromcontextlibimportasynccontextmanagerfromtypingimportAsyncGenerator@asynccontextmanagerasyncdeffoo()->AsyncGenerator[str,None]:yield"hello"SyncfromcontextlibimportcontextmanagerfromtypingimportGenerator@contextmanagerdeffoo()->Generator[str,None,None]:yield"hello"contextlib.AsyncExitStackAsyncimportcontextlibasyncwithcontextlib.AsyncExitStack()asexit_stack:exit_stack.enter_context(context_manager_one())exit_stack.push(callback_one)exit_stack.callback(on_exit_one)awaitexit_stack.enter_async_context(context_manager_two())exit_stack.push_async_exit(on_exit_two)exit_stack.push_async_callback(callback_two)awaitexit_stack.aclose()Syncimportcontextlibwithcontextlib.ExitStack()asexit_stack:exit_stack.enter_context(context_manager_one())exit_stack.push(callback_one)exit_stack.callback(on_exit_one)exit_stack.enter_context(context_manager_two())exit_stack.push(on_exit_two)exit_stack.callback(callback_two)exit_stack.close()Seelimitationsasyncio.TaskGroupAsyncimportasyncioasyncwithasyncio.TaskGroup()astask_group:task_group.create_task(something(1,2,3,this="that"))Syncimportconcurrent.futureswithconcurrent.futures.ThreadPoolExecutor()asexecutor:executor.submit(something,1,2,3,this="that")Seelimitationsanyio.create_task_groupAsyncimportanyioasyncwithanyio.create_task_group()astask_group:task_group.start_soon(something,1,2,3)Syncimportconcurrent.futureswithconcurrent.futures.ThreadPoolExecutor()asexecutor:executor.submit(something,1,2,3)Seelimitationsasyncio.sleep/anyio.sleepCalls toasyncio.sleepandanyio.sleepwill be replaced with calls totime.sleep:Asyncimportasyncioawaitasyncio.sleep(1)Syncimporttimetime.sleep(1)If the call argument is0, the call will be replaced entirely:importasyncioawaitasyncio.sleep(0)anyio.PathAsyncimportanyioawaitanyio.Path().read_bytes()Syncimportpathlibpathlib.Path().read_bytes()Type annotationstyping.AsyncIterable[int]typing.Iterable[int]typing.AsyncIterator[int]typing.Iterator[int]typing.AsyncGenerator[int, str]typing.Generator[int, str, None]DocstringsSimply token replacement is available in docstrings:Asyncasyncdeffoo():"""This calls ``await bar()`` and ``asyncio.sleep``"""Syncdeffoo():"""This calls ``bar()`` and ``time.sleep``"""UsageInstallationpipinstallunasyncdCLIInvokingunasyncdwithout any parameters will apply the configuration from the config
file:unasyncdBut it's also possible to specify the files to be transformed directly:unasyncdasync_thing.py:aync_thing.pyThis will transformasync_thing.pyand write the result back intosync_thing.pyAs a pre-commit hookUnasyncd is available as a pre-commit hook:-repo:https://github.com/provinzkraut/unasyncdrev:v0.4.0hooks:-id:unasyncdConfigurationUnasyncd can be configured via apyproject.tomlfile, a dedicated.unasyncd.tomlfile or the command line interface.Fileconfig file keytypedefaultdescriptionfilestable-A table mapping source file names / directories to target file names / directoriesexcludearray-An array of names to exclude from transformationper_file_excludetable-A table mapping files names to an array of names to exclude from transformationadd_replacementstable-A table of additional name replacementsper_file_add_replacementstable-A table mapping file names to tables of additional replacementstransform_docstringsboolfalseEnable transformation of docstringsadd_editors_noteboolfalseAdd a note on top of the generated filesinfer_type_checking_importsbooltrueInfer if new imports should be added to an 'if TYPE_CHECKING' blockcachebooltrueCache transformation resultsforce_regenboolfalseAlways regenerate files, regardless if their content has changedruff_fixboolfalseRunruff --fixon the generated codeExample[tool.unasyncd]files={"async_thing.py"="sync_thing.py","foo.py"="bar.py"}exclude=["Something","SomethingElse.within"]per_file_exclude={"foo.py"=["special_foo"]}add_replacements={"my_async_func"="my_sync_func"}per_file_add_replacements={"async_thing.py"={"AsyncClass"="SyncClass"}}transform_docstrings=trueremove_unused_imports=falseno_cache=falseno_cache=falseforce_regen=falseCLI optionsFeature flags corresponding to configuration values. These will override the
configuration file valuesoptiondescription--cacheCache transformation results--no-cacheDon't cache transformation results--transform-docstringsEnable transformation of docstrings--no-transform-docstringsInverse of--transform-docstrings--infer-type-checking-importsInfer if new imports should be added to an 'if TYPE_CHECKING' block--no-infer-type-checking-importsInverse ofinfer-type-checking-imports--add-editors-noteAdd a note on top of each generated file--no-add-editors-noteInverse of--add-editors-note--ruff-fixRunruff --fixon the generated code--no-ruff-fixInverse of--ruff-fix--forceAlways regenerate files, regardless if their content has changed--no-forceInverse of--force--checkDon't write changes back to files--writeInverse of--checkAdditional CLI optionsoptiondescription--configAlternative configuration file--verboseIncrease verbosity of console output--quietSuppress all console outputExclusionsIt is possible to exclude specific functions classes and methods from the
transformation. This can be achieved by adding their fully qualified name
(relative to the transformed module) under theexcludekey:[tool.unasyncd]exclude=["Something","SomethingElse.within"]In this example, classes or functions with the nameSomething, and thewithinmethod of theSomethingElseclass will be skipped.The same option is available on a per-file basis, under theper_file_excludekey:[tool.unasyncd]per_file_exclude."module.py"=["Something","SomethingElse.within"]This sets the same exclusion rules as above, but only for the filemodule.py.Extending name replacementsAdditional name replacement rules can be defined by adding fully qualified names
(relative to the transformed module) and replacements under theadd_replacementskey:[tool.unasyncd]add_replacements={"some_module.some_name"="some_other_module.some_other_name"}The same option is available on a per-file basis, under theper_file_add_replacementskey:[tool.unasyncd]per_file_add_replacements."module.py"={"some_module.some_name"="some_other_module.some_other_name"}Handling of importsUnasyncd will add new imports when necessary and tries to be sensible about the way it
does. There are however no guarantees about import order or compatibility with e.g.
isort or black. It follows a few basic rules:Relativity of imports should be kept intact, e.g.typing.AsyncGeneratorwill be
replaced withtyping.Generatorandfrom typing import AsyncGeneratorwithfrom typing import GeneratorExisting imports will be updated if possible, for instancefrom time import timewould becomefrom time import time, sleepifsleephas been added by unasyncd
during the transformationNew imports are added before the first non-import block that's not a docstring or a
commentUnasyncd will not remove imports that have become unused as a result of the applied
transformations. This is because tracking of usages is a complex task and best left to
tools made specifically for this job likerufforautoflake.Integration with lintersUsing unasyncd in conjunction with linters offering autofixing behaviour can lead to an
edit-loop, where unasyncd generates a new file which the other tool then changes in a
non-AST-equivalent way - for example by removing an import that has become unused as a
result of the transformation applied by unasyncd -, in turn causing unasyncd to
regenerate the file the next time it is invoked, since the target file is no longer
AST-equivalent to what unasyncd thinks it should be.To alleviate this, unasyncd offers aruffintegration,
which can automatically runruff --fixon the generated code before writing it back.
It will use the existing ruff configuration for this to ensure the fixes applied to
adhere to the rules used throughout the project.If this option is used, the transformed code will never be altered by ruff, therefore
breaking the cycle.This option can be enabled with theruff_fix = truefeature flag, or by using the--ruff-fixCLI flag.Usage of this option requires an installation ofruff. If not independently installed,
it can be installed as an extra of unasyncd:pip install unasyncd[ruff].Why is only ruff supported?Ruff was chosen for its speed, having a negligible impact on the overall performance of
unasyncd, and because it can replace most of the common linters / tools with autofixing
capabilities, removing the need for separate integrations.LimitationsTransformations forcontextlib.AsyncContextManager,asyncio.TaskGroupandanyio.create_task_grouponly work when they're being called in awithstatement
directly. This is due to the fact that unasyncd does not track assignments or support
type inference. Support for these usages might be added in a future version.DisclaimerUnasyncd's output should not be blindly trusted. While it is unlikely that it will break
things the resulting code should always be tested. Unasyncd is not intended to be run at
build time, but integrated into a git workflow (e.g. withpre-commit).