Package: pyinstrument
pyinstrumentDocumentationPyinstrument is a Python profiler. A profiler is a tool to help you optimize
your code - make it faster. To get the biggest speed increase you shouldfocus on the slowest part of your program.
Pyinstrument helps you find it!☕️ Not sure where to start? Check out thisvideo tutorial from calmcode.io!Installationpip install pyinstrumentPyinstrument supports Python 3.7+.To run Pyinstrument from a git checkout, there's a build step.
Take a look atContributingfor more info.DocumentationTo learn how to use pyinstrument, or to check the reference, head to thedocumentation.Known issuesProfiling code inside a Docker container can cause some strange results,
because the gettimeofday syscall that pyinstrument uses is slow in that
environment. See#83When usingpyinstrument script.pywherescript.pycontains a class
serialized withpickle, you might encounter errors because the
serialisation machinery doesn't know where__main__is.See this issue
for workaroundsChangelogv4.6.226 January 2024Fixes a bug with the pstats renderer, where additional frames could be seen in the output. (#287)Addsshow_alloption toProfiler.output_htmlv4.6.18 November 2023Fixes a bug with unwanted variable expansion in the IPython magics%pyinstrument(#278)v4.6.012 October 2023Adds a feature-c, which allows profiling code directly from the command line, likepython -c. (#271)Adds a convenience methodProfiler.write_html, for writing HTML output to a file directly. (#266)v4.5.37 September 2023Fix a problem in the packaging process that prevented upload to PyPIv4.5.21 September 2023Show the program name in the header of the HTML output (#260)Improve program name capture through resilience to other programs modifying sys.argv (#258)Add support for Python 3.12 (#246)v4.5.122 July 2023Fix a bug that caused[X frames hidden]in the output when frames were deleted due to__tracebackhide__(#255)Fix a bug causing built-in code to display the filepathNonein the console output (#254)Some docs improvements (#251)v4.5.05 June 2023Adds a flat mode to the console renderer, which can be enabled by passing-p flaton the command line. This mode shows the heaviest frame as measured by self-time, which can be useful in some codebases. (#240)Adds the ability to savepstatsfiles. This is the file format used by cprofile in the stdlib. It's less detailed than pyinstrument profiles, but it's compatible with more tools. (#236)Fixes a detail of the--show-alloption - pyinstrument will no longer remove Python-internal frames when this option is supplied. (#239)Internally to the HTML renderer, it now uses Svelte to render the frontend, meaning profile HTML files bundle less javascript and so are smaller. (#222)v4.4.05 November 2022Adds the class name to methods in the console & HTML outputs (#203)Fix a bug that caused pyinstrument machinery to appear at the start of a profile (#215)Frames that set a__traceback_hide__local variable will now be removed from the output (#217)Jupyter/IPython magic now supports async/await, if you run with a--async_mode=enabledflag. (#212)Fix a crash when more than one root frame is captured in a thread - this can happen with gevent.A big refactor to the backend, allowing more than just static information to be captured. This currently is just powering the class name feature, but more is to come!v4.3.021 August 2022Adds buttons in the HTML output to switch between absolute and
proportional (percentage) time.Adds a command line flag--interval(seconds, default 0.001) to change the interval that
pyinstrument samples a program. This is useful for long-running programs,
where increasing the interval reduces the memory overhead.Includes wheels for CPython 3.11.v4.2.0Adds a command-line option-p--render-optionthat allows arbitrary
setting of render options. This lets you set options likefilter_thresholdfrom the command line, by doing something likepyinstrument -p processor_options.filter_threshold=0.Here's the help output for the option:-p RENDER_OPTION, --render-option=RENDER_OPTION
                    options to pass to the renderer, in the format
                    'flag_name' or 'option_name=option_value'. For
                    example, to set the option 'time', pass '-p
                    time=percent_of_total'. To pass multiple options, use
                    the -p option multiple times. You can set processor
                    options using dot-syntax, like '-p
                    processor_options.filter_threshold=0'. option_value is
                    parsed as a JSON value or a string.Adds the ability to view times in the console output as percentages,
rather than absolute times. Use the ConsoleRenderer optiontime='percent_of_total', or on the command line, use-p, likepyinstrument -p time=percent_of_total.Adds command line options for loading and saving pyinstrument sessions.
You can save the raw data for a pyinstrument session with-r session,
likepyinstrument -r session -o session.pyisession myscript.py. Loading
is via--load, e.g.pyinstrument --load session.pyisession.Command line output format is inferred from the-ooutput file
extension. So if you dopyinstrument -o profile.html myscript.py, you
don't need to supply-r html, pyinstrument will automatically use the
HTML renderer. Or if you dopyinstrument -o profile.pyisession myscript.py, it will save a raw
session object.Addsusage examples for FastAPI and pytestto the documentation.Fixes a bug causing NotImplementedError when usingasync_mode=strict.Adds support for Python 3.11v4.1.1Fixed an issue causing PYINSTRUMENT_PROFILE_DIR_RENDERER to output the
wrong file extension when used with the speedscope renderer.v4.1.0You can now use pyinstrument natively in an IPython notebook! Just use%load_ext pyinstrumentat the top of your notebook, and then%%pyinstrumentin the cell you want to profile.Added support for thespeedscopeformat.
This provides a way to view interactive flamecharts using pyinstrument. To
use, profile withpyinstrument -r speedscope, and upload to the
speedscope web app.You can now configure renderers for the Django middleware file output,
using thePYINSTRUMENT_PROFILE_DIR_RENDERERoption.Added wheels for Linux aarch64 (64-bit ARM).v4.0.4Fix a packaging issue where a package called 'test' was installed
alongside pyinstrumentUse more modern C APIs to resolve deprecation warnings on Python 3.10.Minor docs fixesv4.0.3CPython 3.10 supportImprove error messages when trying to use Profiler from multiple threadsFix crash when rendering sessions that contain a module in a FrameGroupv4.0.2Fix some packaging issuesv4.0.0Async support! Pyinstrument now detects when an async task hits an await,
and tracks time spent outside of the async context under this await.So, for example, here's a simple script with an async task that does a
sleep:importasynciofrompyinstrumentimportProfilerasyncdefmain():p=Profiler(async_mode='disabled')withp:print('Hello ...')awaitasyncio.sleep(1)print('... World!')p.print()asyncio.run(main())Before Pyinstrument 4.0.0, we'd see only time spent in the run loop, like
this:_     ._   __/__   _ _  _  _ _/_   Recorded: 18:33:03  Samples:  2
 /_//_/// /_\ / //_// / //_'/ //     Duration: 1.006     CPU time: 0.001
/   _/                      v3.4.2

Program: examples/async_example_simple.py

1.006 _run_once  asyncio/base_events.py:1784
└─ 1.005 select  selectors.py:553
      [3 frames hidden]  selectors, <built-in>
         1.005 kqueue.control  <built-in>:0Now, with pyinstrument 4.0.0, we get:_     ._   __/__   _ _  _  _ _/_   Recorded: 18:30:43  Samples:  2
 /_//_/// /_\ / //_// / //_'/ //     Duration: 1.007     CPU time: 0.001
/   _/                      v4.0.0

Program: examples/async_example_simple.py

1.006 main  async_example_simple.py:4
└─ 1.005 sleep  asyncio/tasks.py:641
      [2 frames hidden]  asyncio
         1.005 [await]For more information, check out theasync profiling documentationand
theProfiler.async_modeproperty.Pyinstrument has adocumentation site, including full Python API docs!v3.4.2Fix a bug that caused--show,--show-regex,--show-allto be ignored
on the command line.v3.4.1Under-the-hood modernisationv3.4.0Addedtimelineoption (boolean) to Profiler methodsoutput_html()andopen_in_browser().v3.3.0Fixed issue withpyinstrument -m module, where pyinstrument wouldn't find
modules in the current directory.Dropped support for Python 2.7 and 3.5. Old versions will remain available
on PyPI, and pip should choose the correct one automatically.v3.2.0Added the ability to track time in C functions. Minor note - Pyinstrument
will record time spent C functions as 'leaf' functions, due to a limitation
in how Python records frames.Python -> C -> Pythonis recorded asPython -> Python, butPython -> Python -> Cwill be attributed correctly.
(#103)v3.1.2Fix<__array_function__ internals>frames appearing as app code in reportsv3.1.1Added support for timeline mode on HTML and JSON renderersReleased as a tarball as well as a universal wheelv3.1.0Added PYINSTRUMENT_SHOW_CALLBACK option on the Django middleware to
add a condition to showing the profile (could be used to run pyinstrument
on a live server!)Fixed bug in the Django middleware where file would not be written because
of a unicode errorv3.0.3Fixed bug with the Django middleware on Windows where profiling would fail
because we were trying to put an illegal character '?' in the profile path.
(#66)v3.0.2Add--showand--show-regexoptions, to mark certain files to be
displayed. This helps to profile inside specific modules, while hiding
others. For example,pyinstrument --show '*/sympy/*' script.py.v3.0.1Fix #60: pass all arguments after -m module_name to the called moduleFix crash during HTML/JSON output when no frames were captured.v3.0.0Pyinstrument will now hide traces through libraries that you're using by default. So instead of showing you loads of frames going through the internals of something external e.g. urllib, it lets you focus on your code.BeforeAfterTo go back to the old behaviour, use--show-allon the command line.'Entry' frames of hidden groups are shown, so you know which call is the problemReally slow frames in the groups are shown too, e.g. the 'read' call on the socketApplication code is highlighted in the consoleAdditional metrics are shown at the top of the trace - timestamp, number of samples, duration, CPU timeHidden code is controlled by the--hideor--hide-regexoptions - matching on the path of the code files.--hide=EXPR           glob-style pattern matching the file paths whose
                        frames to hide. Defaults to '*/lib/*'.
  --hide-regex=REGEX    regex matching the file paths whose frames to hide.
                        Useful if --hide doesn't give enough control.Outputting a timeline is supported from the command line.-t, --timeline        render as a timeline - preserve ordering and don't
                        condense repeated callsBecause there are a few rendering options now, you can load a previous profiling session using--load-prev- pyinstrument keeps the last 10 sessions.Hidden groups can also call back into application code, that looks like this:(internal) When recording timelines, frame trees are completely linear now, allowing
for the creation of super-accurate frame charts.(internal) The HTML renderer has been rewritten as a Vue.js app. All the console improvements apply to the HTML output too, plus it's interactive.(internal) A lot of unit and integration tests added!Yikes! See #49 for the gory details. I hope you like it.v2.3.0Big refactor!Recordershave been removed. The frame recording is now internal to theProfilerobject.
This means the 'frame' objects are more general-purpose, which paves the way for...Processors! These are functions that mutate the tree to sculpt the output.
They are used by the renderers to filter the output to the correct form. Now, instead of
a time-aggregating recorder, the profiler just uses timeline-style recording (this is
lower-overhead anyway) and the aggregation is done as a processing step.The upshot of this is that it's now way easier to alter the tree to filter stuff out, and
do more advanced things like combining frames that we don't care about. More features to
come that use this in v3.0!Importlib frames are removed - you won't see them at all. Their children are retained, so
imports are just transparent.Django profile file name is now limited to a hundred of characters (#50)Fix bug with --html option (#53)Add--versioncommand line optionv2.2.1Fix crash when using on the command line.v2.2.0Added support for JSON output. Usepyinstrument --renderer=json scriptfile.py.PR@iddanhas put together aninteractive viewerusing the JSON output!When runningpyinstrument --htmland you don't pipe the output to a file, pyinstrument will write the console output to a temp file and open that in a browser.v2.1.0Added support for running modules with pyinstrument via the command line. The new syntax
is the-mflag e.g.pyinstrument -m module_name!PRv2.0.4Fix crashes due to multi-threaded use of pyinstrument. The fix is in the C extension,
over athttps://github.com/joerick/pyinstrument_cext/pull/3v2.0.3Pyinstrument can now be used in awithblock.For example:profiler = pyinstrument.Profiler()
with profiler:
    # do some work here...
print(profiler.output_text())Middleware fix for older versions of Djangov2.0.2Fix for max recursion error when used to profile programs with a lot of frames on the stack.v2.0.1Ensure license is included in the sdist.v2.0.0Pyinstrument uses a new profiling mode. Rather than using
signals, pyintrument uses a new statistical profiler built on
PyEval_SetProfile. This means no more main thread restriction, no more
IO errors when using Pyinstrument, and no need for a separate more
'setprofile' mode!Renderers. Users can customize Pyinstrument to use alternative renderers
with therendererargument onProfiler.output(), or using the--rendererargument on the command line.Recorders. To support other use cases of Pyinstrument (e.g. flame charts),
pyinstrument now has a 'timeline' recorder mode. This mode records captured
frames in a linear way, so the program execution can be viewed on a
timeline.v0.13pyinstrumentcommand. You can now profile python scripts from the shell
by running$ pyinstrument script.py. This is now equivalent topython -m pyinstrument. Thanks @asmeurer!v0.12Application code is highlighted in HTML traces to make it easier to spotAddedPYINSTRUMENT_PROFILE_DIRoption to the Django interface, which
will log profiles of all requests to a file the specified folder. Useful
for profiling API calls.AddedPYINSTRUMENT_USE_SIGNALoption to the Django interface, for use
when signal mode presents problems.ContributingTo setup a dev environment:virtualenv --python=python3 env
. env/bin/activate
pip install --upgrade pip
pip install -r requirements-dev.txt
pre-commit install --install-hooksTo get some sample output:pyinstrument examples/wikipedia_article_word_count.pyTo run the tests:pytestTo run linting checks locally:pre-commit run --all-filesSome of the pre-commit checks, likeisortorblack, will auto-fix
the problems they find. So if the above command returns an error, try
running it again, it might succeed the second time :)Running all the checks can be slow, so you can also run checks
individually, e.g., to format source code that failsisortorblackchecks:pre-commit run --all-files isort
pre-commit run --all-files blackTo diagnose whypyrightchecks are failing:pre-commit run --all-files pyrightThe HTML renderer Vue.js appThe HTML renderer works by embedding a JSON representation of the sample with
a Javascript 'bundle' inside an HTML file that can be viewed in any web
browser.To edit the html renderer style, do:cd html_renderer
npm ci
npm run serveWhen launched without a top-levelwindow.profileSessionobject, it will
fetch a sample profile so you can work with it.To compile the JS app and bundle it back into the pyinstrument python tool:bin/build_js_bundle.py [--force]