Package: pydoctrace
pydoctraceusespre-commit hooksandpre-commit.ci continuous integrationto enforce commit messages, code formatting and linting for quality and consistency sake.
See thecode conventionssection if you would like to contribute to the project.pydoctraceGenerate diagrams by tracing Python code execution.Here are the diagrams produced by 2 different implementations of thefactorialfunction (n * n-1 * n-2 * ... * 1):asequence diagramfor the recursive implementationacomponent diagramfor the looping implementationfrompydoctrace.doctraceimporttrace_to_sequence_puml@trace_to_sequence_pumldeffactorial_recursive(value:int)->int:ifvalue<=1:returnvaluereturnvalue*factorial_recursive(value-1)frompydoctrace.doctraceimporttrace_to_component_puml@trace_to_component_pumldeffactorial_reduce_multiply(value:int)->int:ifvalue<=1:returnvaluedefmultiply(agg:int,value:int)->int:returnagg*valuereturnreduce(multiply,range(1,value+1),1)tests.modules.factorial.factorial_recursive.pumltests.modules.factorial.factorial_reduce_multiply.pumlThesequence diagramgives more details about the execution but can easily grow large, thecomponent diagramis more concise and helps viewing the architecture behind the calls.InstallationUse your favorite dependency manager to installpydoctrace:# using pippipinstallpydoctrace# using poetrypoetryaddpydoctrace# using pipenvpipenvinstallpydoctraceUsageImport a decorator and apply it to the functions whom execution you want to trace:# in my_package/my_module.pyfrompydoctrace.doctraceimporttrace_to_component_puml,trace_to_sequence_pumldefmain(parameter):validate(parameter)do_something(parameter)@trace_to_component_pumldefvalidate(parameter):validate_is_not_none(parameter)validate_is_in_database(parameter)@trace_to_sequence_pumldefdo_something(parameter):stuff=load_stuff(parameter)save_work(process_job(stuff))Executingmain()will create 2 diagram files in the current working directory:validate-component.puml: a PlantUMLcomponentdiagram showing the interactions betweenvalidate()as the start, andvalidate_is_not_none()andvalidate_is_in_database()(and all their inner function calls, if any)do_something-sequence.puml: a PlantUMLsequencediagram showing the interactions betweendo_something()as the start, andload_stuff(),process_job()andsave_work()(and all their inner function calls, if any)You can then view these files either:using an extension for your IDE:jebbs.plantumlfor Codium / vsCode,7017-plantuml-integrationfor PyCharmonline atwww.plantuml.com/plantuml/uml/Customize the filenameTheexport_file_path_tplattribute of the decorators allows you to define another file path and name:# in my_package/my_module.pyfrompydoctrace.doctraceimporttrace_to_component_puml,trace_to_sequence_puml# will create a 'validate-component.puml' file in the 'doc/my_package-my_module' folder# (pydoctrace will create the subdirectories if necessary)@trace_to_component_puml(export_file_path_tpl='doc/my_package-my_module/validate-component.puml')defvalidate(parameter):...# will create a 'my_package.my_module/do_something-2024-02-27_19.16.09_473-sequence.puml' file@trace_to_sequence_puml(export_file_path_tpl='${function_module}/${function_name}-${datetime_millis}-sequence.puml')defdo_something(parameter):...As you can see, you can use the following template tags in the file name:${function_module}: which resolves todo_something.__module__,'my_package.my_module'in the example${function_name}: which resolves todo_something.__name__,'do_something'in the example${datetime_millis}: the datetime in ISO-ish format compatible with filenames (Windows does not support':'in filenames).
If the traced function is called several times during the execution, including${datetime_millis}in the filename template will generate different files that won't overwrite themselvesFilter what is tracedTo keep the generated diagrams useful and legible, you probably want to exclude some calls from the tracing (such as calls toprint(...)orjson.load(...)).pydoctracecomes with ready-to-usepresetsthat you can combine to filter out some calls you do not want to document:EXCLUDE_BUILTINS_PRESET: prevents from tracing calls tobuilt-in functions(print,len,all,any, etc.)EXCLUDE_STDLIB_PRESET: prevents from tracing calls to allthe standard library modules(csv,json,dataclass, etc.)EXCLUDE_TESTS_PRESET: prevents from tracing calls to functions belonging to thetests,_pytest,pytest,unittest,doctestmodules.
This one is particularly useful when you want to generate some documentation from an automated testsEXCLUDE_DEPTH_BELOW_5_PRESET: prevents from tracing any calls that involve a call stack of more than five levels after the traced functionBy default,EXCLUDE_STDLIB_PRESETandEXCLUDE_TESTS_PRESETare activated.
Specify something else at the decorator level to change the call filtering.frompydoctrace.doctraceimporttrace_to_component_pumlfrompydoctrace.callfilter.presetsimport(EXCLUDE_BUILTINS_PRESET,EXCLUDE_CALL_DEPTH_PRESET_FACTORY,EXCLUDE_DEPTH_BELOW_5_PRESET,EXCLUDE_STDLIB_PRESET,EXCLUDE_TESTS_PRESET,TRACE_ALL_PRESET,)# an empty presets list traces everything@trace_to_component_puml(filter_presets=[])defvalidate(parameter):...# or you can be explicitfrompydoctrace.callfilter.presetsimportTRACE_ALL_PRESET@trace_to_component_puml(filter_presets=[TRACE_ALL_PRESET])defvalidate(parameter):...# filter at a custom callstack levelfrompydoctrace.callfilter.presetsimportEXCLUDE_CALL_DEPTH_PRESET_FACTORYABOVE_3_PRESET=EXCLUDE_CALL_DEPTH_PRESET_FACTORY(3)@trace_to_component_puml(filter_presets=[ABOVE_3_PRESET])defcall():call_1()defcall_1():call_2()defcall_2():call_3()defcall_3():# will not be traced in the diagram...# documents the behavior of do_something without the noise of# the test framework nor the standard librayfrommy_package.my_moduleimportdo_somethingfrompydoctrace.callfilter.presetsimportEXCLUDE_STDLIB_PRESET,EXCLUDE_TESTS_PRESET@trace_to_component_puml(filter_presets=[EXCLUDE_STDLIB_PRESET,EXCLUDE_TESTS_PRESET])deftest_do_something():assertdo_something('param')isnotNone# for test-generated documentation, you can also use the decorator programmaticallydeftest_do_something():traceable_do_something=trace_to_component_puml(export_file_path_tpl='doc/${function_module}/${function_name}-component.puml',filter_presets=[EXCLUDE_STDLIB_PRESET,EXCLUDE_TESTS_PRESET,ABOVE_3_PRESET])(do_something)asserttraceable_do_something('param')isnotNonePurposes and mechanismsThe purpose ofpydoctraceis to document the execution of some code to illustrate the behavior and the structure of the code base.use one of the provided decorators (diagram type and format) to decorate the function whose execution you want to documentrun your code (with a unit test, for instance): when the execution hits the decorated function, the execution is traced and a diagram is drawn to show how the functions are called, how the values are returned and how the errors are handledpydoctraceis a pure Python tool relying on no other 3rd-party library to work.
The project uses only development libraries for testing and documentation purposes.Doc-tracingThis approach, which I called "doc-tracing" (tracing code execution for documentation purposes), is complementary of other approaches which generate documentation from static code analysis.Static code analysisreads the source code to detect and document data structures (classes, dataclasses, named tuples, enumeration, etc.), functions names and signatures (parameters name and types, returned value types).Useful as it is, static code analysis does not help much to understand how the code pieces work together; doc-tracing attempts to complement this approach by producing documentation while the code runs.
Some use cases:you start working on a legacy codebase: study the behavior of a particular function by temporarily tracing its executionsyou finished a user story, document its implementation by tracing the execution of someintegration tests("given ... when ... then ...") to illustrate how right cases and errors are handled.generally, the sequence diagrams illustrate how the modules and functions interacts; and as such,they help discussing architecturetracing code execution can also be useful when teaching computer programming to illustrate how algorithms workHow is the code execution traced?When a function decorated bypydoctraceis called:a context manager is createdduring which atracing functionis passed tosys.settrace, which is called when different events happen in the execution stack:when functions are calledwhen values are returnedwhen exceptions are raised or handledthe sequence diagram is drawn and exported in a stream fashion (when possible) so that its memory footprint is minimalonce the decorated function stops its execution, the tracing function is removed from the code execution⚠️Caveat:pydoctraceuses thesys.settraceAPI, which is meant to be used by debuggers.
Therefore, a warning is emitted whenpydoctraceis used in a debug mode (and does not trace the decorated function anymore).Tests# directly with poetrypoetryrunpytest-v# in an activated virtual environmentpytest-vCode coverage (withmissed branch statements):poetryrunpytest-v--cov=pydoctrace--cov-branch--cov-reportterm-missing--cov-fail-under82Changelog0.3.0: added configurable filters to exclude function calls from the tracing process (do not trace calls to the standard library modules and functions by default), templating for output file names.
Thank youPhilippe Tonyfor suggesting filtering out calls above a certain calls depth from the traced function.
Started integration tests0.2.0: PlantUML exporter for component diagrams, added unit tests0.1.2: added Github actions for the automated tests and interaction with pre-commit.ci for the code linting0.1.1: [deleted release: wrong contents] added Github actions for the automated tests and interaction with pre-commit.ci for the code linting0.1.0: ✨ first release, PlantUML exporter for sequence diagrams; diagram files are saved in the current working directoryLicenceUnless stated otherwise, all works are licensed under theMIT license, a copy of which is includedhere.ContributionsLuc Sorel-GiffoPull requestsPull-requests are welcome and will be processed on a best-effort basis.Pull requests must follow the guidelines enforced by thepre-commithooks:commit messages must follow the Angular conventions enforced by thecommitlinthookcode formatting must follow the conventions enforced by theisortandruff-formatterhookscode linting should not detect code smells in your contributions, this is checked by theruffhookWhen requesting a feature, please consider involving yourself in the review process once it is developed.Code conventionsThe code conventions are described and enforced bypre-commit hooksto maintain style and quality consistency across the code base.
The hooks are declared in the.pre-commit-config.yamlfile.When you contribute, set the git hooks (pre-commitandcommit-msgtypes) on your development environment:poetryrunpre-commitinstallBefore committing, you can check your changes manually with:# put all your changes in the git staging area (or add the changes manually and skip this)gitadd-A# run all hookspoetryrunpre-commitrun--all-files# run a specific hookpoetryrunpre-commitrunruff--all-filesCommit messagesPlease, follow theconventions of the Angular teamfor commit messages.
When merging your pull-request, the new version of the project will be derived from the messages.I use theredjue.git-commit-plugincodium/vsCode extension to help me write commit messages.Code formattingThis project usesisortandruffto format the code.
The guidelines are expressed in their respective sections in thepyproject.tomlfile.Best practicesThis project uses therufflinter, which is configured in its section in thepyproject.tomlfile.Similar tools and bibliographyhttps://stackoverflow.com/questions/45238329/it-is-possible-to-generate-sequence-diagram-from-python-codehttps://github.com/gaogaotiantian/viztracerhttps://9to5answer.com/how-to-generate-a-sequence-diagram-from-java-source-codehttps://medium.com/javarevisited/how-to-generate-sequence-diagrams-in-intellij-e2bb7cec2b0b