Package: pystack
PyStackPrint the stack trace of a running Python process, or of a Python core dump.PyStack is a tool that uses forbidden magic to let you inspect the stack frames of a running Python
process or a Python core dump, helping you quickly and easily learn what it's doing (or what it was
doing when it crashed) without having to interpret nasty CPython internals.What PyStack can doPyStack has the following amazing features:💻 Works with both running processes and core dump files.🧵 Shows if each thread currently holds the Python GIL, is waiting to acquire it, or is currently
dropping it.🗑️ Shows if a thread is running a garbage collection cycle.🐍 Optionally shows native function calls, as well as Python ones. In this mode, PyStack prints
the native stack trace (C/C++/Rust function calls), except that the calls to Python callables are
replaced with frames showing the Python code being executed, instead of showing the internal C
code the interpreter used to make the call.🔍 Automatically demangles symbols shown in the native stack.📈 Includes calls to inlined functions in the native stack whenever enough debug information is
available.🔍 Optionally shows the values of local variables and function arguments in Python stack frames.🔒 Safe to use on running processes. PyStack does not modify any memory or execute any code in a
process that is running. It simply attaches just long enough to read some of the process's
memory.⚡ Optionally, it can perform a Python stack analysis without pausing the process at all. This
minimizes impact to the debugged process, at the cost of potentially failing due to data races.🚀 Super fast! It can analyze core files 10x faster than general-purpose tools like GDB.🎯 Even works with aggressively optimized Python interpreter binaries.🔍 Even works with Python interpreters' binaries that do not have symbols or debug information
(Python stack only).💥 Tolerates memory corruption well. Even if the process crashed due to memory corruption,
PyStack can usually reconstruct the stack.💼 Self-contained: it does not depend on external tools or programs other than the Python
interpreter used to run PyStack itself.What platforms are supported?At this time only Linux is supported.Building from sourceIf you wish to build PyStack from source, you need the following binary dependencies in your
system:libdwlibelfNote that sometimes both libraries are provided together as part of a distribution'selfutilspackage.Check your package manager on how to install these dependencies (e.g.,apt-get install libdw-dev libelf-devin Debian-based systems). Note that you may need to tell the
compiler where to find the header and library files of the dependencies for the build to succeed.
Ifpkg-configis available (e.g.apt-get install pkg-configon Debian-based systems), it will
automatically be used to locate the libraries and configure the correct build flags.
Check your distribution's documentation to determine the location of the header and library files
or for more detailed information. When building on Alpine Linux (or any other distribution that
doesn't use glibc) you'll need elfutils 0.188 or newer. You may need to build this from source if
your distribution's package manager doesn't have it.Once you have these binary dependencies installed, you can clone the repository and follow the
typical build process for Python libraries:gitclonegit@github.com:bloomberg/pystack.gitpystackcdpystack
python3-mvenv../pystack-env/# just an example, put this wherever you wantsource../pystack-env/bin/activate
python3-mpipinstall--upgradepip
python3-mpipinstall-e.
python3-mpipinstall-rrequirements-test.txt-rrequirements-extra.txtThis will install PyStack in the virtual environment in development mode (the-eof the lastpip installcommand), and then install the Python libraries needed to test it, lint it, and
generate its documentation.If you plan to contribute back, you should install the pre-commit hooks:pre-commitinstallThis will ensure that your contribution passes our linting checks.DocumentationYou can find the full documentationhere.UsagePyStack uses distinct subcommands for analyzing running processes and core dump files.usage:pystack[-h][-v][--no-color]{remote,core}...

GetPythonstacktraceofaremoteprocess

options:-h,--helpshowthishelpmessageandexit-v,--verbose--no-colorDeactivatecoloredoutput

commands:{remote,core}WhatshouldbeanalyzedbyPyStack(use<command>--helpforacommand-specifichelpsection).remoteAnalyzearemoteprocessgivenitsPIDcoreAnalyzeacoredumpfilegivenitslocationandtheexecutableAnalyzing running processesTheremotecommand is used to analyze the status of a running (remote) process. The analysis is
always done in a safe and non-intrusive way, as no code is loaded in the memory space of the
process under analysis and no memory is modified in the remote process. This makes analysis using
PyStack a great option even for those services and applications that are running in environments
where the running process must not be impacted in any way (other than being temporarily paused,
though--no-blockcan avoid even that). There are several options available:usage:pystackremote[-h][-v][--no-color][--no-block][--native][--native-all][--locals][--exhaustive]pid

positionalarguments:pidThePIDoftheremoteprocess

options:-h,--helpshowthishelpmessageandexit-v,--verbose--no-colorDeactivatecoloredoutput--no-blockdonotblocktheprocesswheninspectingitsmemory--nativeIncludethenative(C)framesintheresultingstacktrace--native-allIncludenative(C)framesfromthreadsnotregisteredwiththeinterpreter(implies--native)--localsShowlocalvariablesforeachframeinthestacktrace--exhaustiveUseallpossiblemethodstoobtainthePythonstackinfo(maybeslow)To use PyStack, you just need to provide the PID of the process:$pystackremote112Tracebackforthread112[](mostrecentcalllast):(Python)File"/test.py",line17,in<module>first_func()(Python)File"/test.py",line6,infirst_funcsecond_func()(Python)File"/test.py",line10,insecond_functhird_func()(Python)File"/test.py",line14,inthird_functime.sleep(1000)Analyzing core dumpsThecoresubcommand is used to analyze the status of a core dump file. Analyzing core files is
very similar to analyzing processes but there are some differences, as the core file does not
contain the totality of the memory that was valid when the program was live. In most cases, this
makes no difference, as PyStack will try to adapt automatically. However, in some cases, you will
need to specify extra command line options to help PyStack locate the information it needs. When
analyzing cores, there are several options available:usage:pystackcore[-h][-v][--no-color][--native][--native-all][--locals][--exhaustive][--lib-search-pathLIB_SEARCH_PATH|--lib-search-rootLIB_SEARCH_ROOT]core[executable]positionalarguments:coreThepathtothecorefileexecutable(Optional)Thepathtotheexecutableofthecorefile

options:-h,--helpshowthishelpmessageandexit-v,--verbose--no-colorDeactivatecoloredoutput--nativeIncludethenative(C)framesintheresultingstacktrace--native-allIncludenative(C)framesfromthreadsnotregisteredwiththeinterpreter(implies--native)--localsShowlocalvariablesforeachframeinthestacktrace--exhaustiveUseallpossiblemethodstoobtainthePythonstackinfo(maybeslow)--lib-search-pathLIB_SEARCH_PATHListofpathstosearchforsharedlibrariesloadedinthecore.Pathsmustbeseparatedbythe':'character--lib-search-rootLIB_SEARCH_ROOTRootdirectorytosearchrecursivelyforsharedlibrariesloadedintothecore.In most cases, you just need to provide the location of the core to use PyStack with core dump
files:$pystackcore./the_core_file
Usingexecutablefoundinthecorefile:/usr/bin/python3.8

Corefileinformation:
state:tzombie:Trueniceness:0pid:570ppid:1sid:1uid:0gid:0pgrp:570executable:python3.8arguments:python3.8

TheprocessdiedduereceivingsignalSIGSTOP
Tracebackforthread570[](mostrecentcalllast):(Python)File"/test.py",line19,in<module>first_func({1:None},[1,2,3])(Python)File"/test.py",line7,infirst_funcsecond_func(x,y)(Python)File"/test.py",line12,insecond_functhird_func(x,y)(Python)File"/test.py",line16,inthird_functime.sleep(1000)LicensePyStack is Apache-2.0 licensed, as found in theLICENSEfile.Code of ConductCode of ConductThis project has adopted a Code of Conduct. If you have any concerns about the Code, or behavior
that you have experienced in the project, please contact us atopensource@bloomberg.net.Security PolicySecurity PolicyIf you believe you have identified a security vulnerability in this project, please send an email
to the project team atopensource@bloomberg.net, detailing the suspected issue and any methods
you've found to reproduce it.Please do NOT open an issue in the GitHub repository, as we'd prefer to keep vulnerability reports
private until we've had an opportunity to review and address them.ContributingWe welcome your contributions to help us improve and extend this project!Below you will find some basic steps required to be able to contribute to the project. If you have
any questions about this process or any other aspect of contributing to a Bloomberg open source
project, feel free to send an email toopensource@bloomberg.netand we'll get your questions
answered as quickly as we can.Contribution LicensingSince this project is distributed under the terms of anopen source license,
contributions that you make are licensed under the same terms. For us to be able to accept your
contributions, we will need explicit confirmation from you that you are able and willing to provide
them under these terms, and the mechanism we use to do this is called a Developer's Certificate of
Origin(DCO). This is similar to the
process used by the Linux kernel, Samba, and many other major open source projects.To participate under these terms, all that you must do is include a line like the following as the
last line of the commit message for each commit in your contribution:Signed-Off-By: Random J. Developer <random@developer.example.org>The simplest way to accomplish this is to add-sor--signoffto yourgit commitcommand.You must use your real name (sorry, no pseudonyms, and no anonymous contributions).StepsCreate an Issue, select 'Feature Request', and explain the proposed change.Follow the guidelines in the issue template presented to you.Submit the Issue.Submit a Pull Request and link it to the Issue by including "#" in the Pull Request
summary.