Package: codesnap
CodeSnapCodeSnap is a deterministic debugging/profiling tool that can trace and visualize python code. The major data CodeSnap displays is FEE(function entry/exit), or equivalently, the call stack.Unlike traditional flame graph, which is normally generated by sampling profiler, CodeSnap can display every function executed and the corresponding entry/exit time from the beginning of the program to the end, which is helpful for programmers to catch sporatic performance issues.With CodeSnap, the programmer can intuitively understand what their code is doing and how long each function takes.You can take a look at thedemoresult of an example program running recursive merge and quick sort algorithm.trace vieweris used to display the stand alone html data.CodeSnap also supports json output that complies with Chrome trace event format, which can be loaded usingperfettoRequirementsCodeSnap requires python 3.5+. No other package is needed. For now, CodeSnap binary on pip only supports CPython + Linux. However, in theory the source code can build on Windows/MacOS.InstallThe prefered way to install CodeSnap is via pippip install codesnapYou can also download the source code and build it yourself.UsageThere are a couple ways to use CodeSnapCommand LineThe easiest way to use CodeSnap it through command line. Assume you have a python script to profile and the normal way to run it is:python3 my_script.pyYou can simply use CodeSnap aspython3 -m codesnap my_script.pywhich will generate aresult.htmlfile in the directory you run this command. Open it in browser and there's your result.If your script needs arguments likepython3 my_script.py arg1 arg2Just feed it as it is to CodeSnappython3 -m codesnap my_script.py arg1 arg2You can also specify the tracer to be used in command line by passing --tracer argument. c tracer is the default value, you can use python tracer insteadpython3 -m codesnap --tracer c my_script.py
python3 -m codesnap --tracer python my_script.pyYou can specify the output file using -o or --output_file argument. The default output file is result.html. Two types of files are supported, html and json.python3 -m codesnap -o other_name.html my_script.py
python3 -m codesnap -o other_name.json my_script.pyInlineSometimes the command line may not work as you expected, or you do not want to profile the whole script. You can manually start/stop the profiling in your script as well.First of all, you need to importCodeSnapclass from the package, and make an object of it.fromcodesnapimportCodeSnapsnap=CodeSnap()If your code is executable byexecfunction, you can simply callsnap.run()snap.run("import random;random.randrange(10)")This will as well generate aresult.htmlfile in your current directory. You can pass other file path to the function if you do not like the nameresult.htmlsnap.run("import random; random.randrange(10)",output_file="better_name.html")When you need a more delicate profiler, you can manually enable/disable the profile usingstart()andstop()function.snap.start()# Something happens heresnap.stop()snap.save()# also takes output_file as an optional argumentWith this method, you can only record the part that you are interested in# Some code that I don't caresnap.start()# Some code I do caresnap.stop()# Some code that I want to skipsnap.start()# Important code againsnap.stop()snap.save()It is higly recommended thatstart()andstop()function should be in the same frame(same level on call stack). Problem might happen if the condition is not metDisplay ResultBy default, CodeSnap will generate a stand alone HTML file which you can simply open with Chrome(maybe Firefox?). The front-end uses trace-viewer to show all the data.However, you can generate json file as well, which complies to the chrome trace event format. You can load the json file onperfetto, which will replace the deprecated trace viewer in the future.At the moment, perfetto did not support locally stand alone HTML file generation, so I'm not able to switch completely to it. The good news is that once you load the perfetto page, you can use it even when you are offline.Choose TracerThe default tracer for current version is c tracer, which introduce a relatively small overhead(worst case 2-3x) but only works for CPython on Linux. However, if there's other reason that you would prefer a pure-python tracer, you can use python tracer usingtracerargument when you initializeCodeSnapobject.snap=CodeSnap(tracer="python")Cleanup of c TracerThe interface for c trace is almost exactly the same as python tracer, except for the fact that c tracer does not support command line run now. However, to achieve lower overhead, some optimization is applied to c tracer so it will withhold the memory it allocates for future use to reduce the time it callsmalloc(). If you want the c trace to free all the memory it allocates while collecting trace, usesnap.cleanup()PerformanceOverhead is a big consideration when people choose profilers. CodeSnap now has a similar overhead as native cProfiler. It works slightly worse in the worst case(Pure FEE) and better in easier case because even though it collects some extra information than cProfiler, the structure is lighter.Admittedly, CodeSnap is only focusing on FEE now, so cProfiler also gets other information that CodeSnap does not acquire.An example run for test_performance with Python 3.8 / Ubuntu 18.04.4 on Github VMfib       (10336, 10336): 0.000852800 vs 0.013735200(16.11)[py] vs 0.001585900(1.86)[c] vs 0.001628400(1.91)[cProfile]
hanoi     (8192, 8192): 0.000621400 vs 0.012924899(20.80)[py] vs 0.001801800(2.90)[c] vs 0.001292900(2.08)[cProfile]
qsort     (10586, 10676): 0.003457500 vs 0.042572898(12.31)[py] vs 0.005594100(1.62)[c] vs 0.007573200(2.19)[cProfile]
slow_fib  (1508, 1508): 0.033606299 vs 0.038840998(1.16)[py] vs 0.033270399(0.99)[c] vs 0.032577599(0.97)[cProfile]LimitationsCodeSnap usessys.setprofile()for its profiler capabilities, so it will conflict with other profiling tools which also use this function. Be aware of it when using CodeSnap.Bugs/RequirementsPlease send bug reports and feature requirements throughgithub issue tracker. CodeSnap is currently under development now and it's open to any constructive suggestions.LicenseCopyright Tian Gao, 2020.Distributed under the terms of theApache 2.0 license.