Package: awaitwhat
Await, What?Tells you what waits for what in anasync/awaitprogram.Python 3.10.0a1It seems the API was changed in 3.10 and the C extension doesn't compile.
I'll investigate...AlpineYou'll needapk add build-base openssl-dev libffi-dev2019 Sprint SetupComms:https://gitter.im/awaitwhat/communityPython 3.9, Python 3.8 (preferred) or Python 3.7Your platform dev tools (compiler, etc).Ensure thatpythonis 3.9 or 3.8 or 3.7InstallpoetryInstallgraphvizClone this repositoryLook attestsLook atissues> python --versionPython 3.9.0b4  #🧡Python 3.8.4    #👌> dot -Vdot - graphviz version 2.40.1> curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python#add~/.poetry/bintoyourPATH> git clone git@github.com:dimaqq/awaitwhat.git> cd awaitwhat~/awaitwhat (dev|✔) > poetry shell    # creates a venv and drops you in it(awaitwhat-x-py3.9)~/awaitwhat (dev|✔) > poetry install  # installs projects dependencies in a venv(awaitwhat-x-py3.9)~/awaitwhat (dev|✔) > poetry build    # builds a C extension in this project(awaitwhat-x-py3.9)~/awaitwhat (dev|✔) > env PYTHONPATH=. python examples/test_shield.py | tee graph.dot(awaitwhat-x-py3.9)~/awaitwhat (dev|✔) > dot -Tsvg graph.dot -o graph.svg(awaitwhat-x-py3.9)~/awaitwhat (dev|✔) > open graph.svg  # or load it in a browserTL;DRSay you have this code:asyncdefjob():awaitfoo()asyncdeffoo():awaitbar()asyncdefbar():awaitbaz()asyncdefbaz():awaitleaf()asyncdefleaf():awaitasyncio.sleep(1)# imagine you don't know thisasyncdefwork():awaitasyncio.gather(...,job())Now that code is stuck and and you want to know why.Python built-inStackfor<Taskpendingcoro=<job()…>wait_for=<Futurependingcb=[<TaskWakeupMethWrapper…>()]>cb=[…]>(mostrecentcalllast):File"test/test_stack.py",line34,injobawaitfoo()This libraryStackfor<Taskpendingcoro=<job()…>wait_for=<Futurependingcb=[<TaskWakeupMethWrapper…>()]>cb=[…]>(mostrecentcalllast):File"test/test_stack.py",line34,injobawaitfoo()File"test/test_stack.py",line38,infooawaitbar()File"test/test_stack.py",line42,inbarawaitbaz()File"test/test_stack.py",line46,inbazawaitleaf()File"test/test_stack.py",line50,inleafawaitasyncio.sleep(1)File"/…/asyncio/tasks.py",line568,insleepreturnawaitfutureFile"<Sentinel>",line0,in<_asyncio.FutureIterobjectat0x7fb6981690d8>:…Dependency GraphReferenceshttps://mail.python.org/archives/list/async-sig@python.org/thread/6E2LRVLKYSMGEAZ7OYOYR3PMZUUYSS3K/Hi group,I'm recently debugging a long-running asyncio program that appears to get stuck about once a week.The tools I've discovered so far are:high level:asyncio.all_tasks()+asyncio.Task.get_stack()low level:loop._selector._fd_to_keyWhat's missing is the middle level, i.e. stack-like linkage of what is waiting for what. For a practical example, consider:asyncdefleaf():awaitsomesocket.recv()asyncdefbaz():awaitleaf()asyncdefbar():awaitbaz()asyncdeffoo():awaitbar()asyncdefjob():awaitfoo()asyncdefwork():awaitasyncio.gather(...,job())asyncdefmain():asyncio.run(work())The task stack will contain:main and body of work with line numberjob task with line number pointing to fooThe file descriptor mapping, socket fd,loop._recv()and aFuture.What's missing are connectionsfoo->bar->baz->leaf.
That is, I can't tell which task is waiting for what terminalFuture.Is this problem solved in some way that I'm not aware of?
Is there a library or external tool for this already?Perhaps, if I could get a list of all pending coroutines, I could figure out what's wrong.If no such API exists, I'm thinking of the following:asyncdeffoo():awaitbar()In[37]:dis.dis(foo)10LOAD_GLOBAL0(bar)2CALL_FUNCTION04GET_AWAITABLE6LOAD_CONST0(None)8YIELD_FROM10POP_TOP12LOAD_CONST0(None)14RETURN_VALUEStarting from a pending task, I'd get it's coroutine and:Get the coroutine frame, and if current instruction isYIELD_FROM, then the reference to the awaitable should be on the top of the stack.
If that reference points to a pending coroutine, I'd add that to the "forward trace" and repeat.At some point I'd reach an awaitable that's not a pending coroutine, which may be: anotherTask(I already got those), a low-levelFuture(can be looked up in event loop), anEvent(tough luck, shoulda logged allEvent's on creation) or a dozen other corner cases.What do y'all think of this approach?Thanks,
D.