Package: py-import-tree
py-import-treeAnalyzing the tree of imports of running Python code.Uses a combination of AST (abstract syntax tree) and code execution (tracing), to give you useful insights into
your project.ExampleSample projectCreate a new directory, and create two files:simple.pywith contents:fromcollectionsimportdefaultdictdefcounts(arr):res=defaultdict(lambda:0)forelinarr:res[el]+=1returnresheavy.pywith contents:importtorchdeftorch_utils():print(torch.ones(10))defsomething_simple():print('Boiler')Now, let's analyze this project withpy_import_tree!py_import_tree usagefrompy_import_tree.import_trackerimportImportTracker# First time setup, this traverses imports found in code# And executes them to find out which additional packages they bring in.tracker=ImportTracker('py_import_tree_results')tracker.dump_for_directory('.')You should see output similar to:[0/2]: Dumping heavy.py...
Collecting import torch "import torch"
Collecting after import torch "import torch"
Exiting import torch "import torch"
[1/2]: Dumping simple.py...
Collecting from collections import defaultdict "from collections import defaultdict"
Collecting after from collections import defaultdict "from collections import defaultdict"
Exiting from collections import defaultdict "from collections import defaultdict"Next, we can load the results and inspect them (compute cohesion, etc.):frompy_import_tree.cohesionimportImportTreetree=ImportTree.from_dump('py_import_tree_results')cohesion=tree.cohesion()Notice that if you want to importsomething_simple, you will need to importtorch, despite the fact thattorchis
not used in thesomething_simplefunction.
However,torch_utilsandcountsfunction lead to imports that are exactly what they use.
So 2 out of the 3 function are with perfect cohesion, and 1 of them is with zero cohesion.cohesion.score0.6666666666666666We can also check per definition results:#dataframe with cohesion for every function and class.
cohesion.definitionspathdefinitionimportdependencydependency_weightdefinition_ideal_weightdefinition_actual_weightcohesion_score0heavy.pyFunctionDef:torch_utilsimport torchnan14521327861452497413145249741314heavy.pyFunctionDef:torch_utilsimport torchnan014524974131452497413178heavy.pyFunctionDef:torch_utilsimport torchtyping_extensions==3.7.4.383727145249741314524974131199heavy.pyFunctionDef:torch_utilsimport torchtqdm==4.59.0280900145249741314524974131306heavy.pyFunctionDef:something_simplenannan0014524974130307simple.pyFunctionDef:countsfrom collections import defaultdictnan0001You can also check how would the cohesion change if you move a function or a class to another file.
For example, if we move the other simple function into the file that importstorch, this would make
the cohesion even worse:tree.what_if_function_moves('simple.py','counts','heavy.py').cohesion().score0.3333333333333333However, if we move the function that usestorchinto a separate file, this would lead to perfect cohesion:tree.what_if_function_moves('heavy.py','torch_utils','new.py').cohesion().score1.0What if we move thesomething_simplefunction into thesimple.pyfile?tree.what_if_function_moves('heavy.py','something_simple','simple.py').cohesion().score1.0You can also use the resulting dataframe to analyze exact, locked versions for each function/class in your project:cohesion.definitions[['full_definition','dependency']].drop_duplicates()full_definitiondependency438heavy.py:FunctionDef:something_simplenan0heavy.py:FunctionDef:torch_utilstorch==1.10.1-py3.86heavy.py:FunctionDef:torch_utilsnan115heavy.py:FunctionDef:torch_utilstyping_extensions==3.7.4.3274heavy.py:FunctionDef:torch_utilstqdm==4.59.0439simple.py:FunctionDef:countsnanEnjoy!