Package: memory-graph
InstallationInstallmemory_graphusing pip:pip install memory-graphAdditionallyGraphvizneeds to be installed.Graph your MemoryDoes your Python code have a bug, is it behaving differently from what you expect? The problem could be a misunderstanding of the Python Data Model, and the first step to the solution could be drawing your data as a graph usingmemory_graph.show( your_data ), an example:importmemory_graphdata=[(1,2),[3,4],{5,6},{7:'seven',8:'eight'}]memory_graph.show(data,block=True)This shows a graph with the starting point of our 'data' drawn with thick lines, the program blocks until the ENTER key is pressed.Alternatively render the graph to an output file of our choosing (seeGraphviz Output Formats) using for example:memory_graph.render(data,"my_graph.pdf")memory_graph.render(data,"my_graph.png")memory_graph.render(data,"my_graph.gv")# Graphviz DOT filePython Data ModelThePython Data Modelmakes a distiction between immutable and mutable types:immutable: bool, int, float, complex, str, tuple, bytes, frozensetmutable: list, dict, set, class, ... (all other types)immutable typeIn the code below variableaandbboth reference the sameintvalue 10. Anintis an immutable type and therefore when we change variableaits value cannotbe mutated in place, and thus a copy is made andaandbreference a different value afterwards.importmemory_graphmemory_graph.rewrite_to_node.reduce_reference_children.remove("int")# shows references to 'int'a=10b=amemory_graph.render(locals(),'immutable1.png')a+=1memory_graph.render(locals(),'immutable2.png')mutable typeWith mutable types the result is different. In the code below variableaandbboth reference the samelistvalue [4, 3, 2]. Alistis a mutable type and therefore when we change variableaits valuecanbe mutated in place and thusaandbboth reference the same new value afterwards. The result is that changingaalso changesband vice versa. Sometimes we want this but other times we don't and then we will have to make a copy so thatbis independent froma.importmemory_grapha=[4,3,2]b=amemory_graph.render(locals(),'mutable1.png')a.append(1)memory_graph.render(locals(),'mutable2.png')Python makes this distiction between mutable and immutable types because a value of a mutable type generally could be large and therefore it would be slow to make a copy each time we change it. On the other hand, a value of a changable immutable type generally is small and therefore fast to copy.copyingPython offers three different "copy" options that we will demonstrate using a nested list:importmemory_graphimportcopya=[[1,2],['a','b']]# a nested list (a list containing other lists)# three different ways to make a "copy" of 'a':c1=ac2=copy.copy(a)# equivalent to:   a.copy() a[:] list(a)c3=copy.deepcopy(a)memory_graph.show(locals())c1is anassignment, all the data is shared.c2is ashallow copy, only the data referenced by the first reference is copied and the underlying data is sharedc3is adeep copy, all the data is copiedcustom copy methodWe can write our own custom copy function or method in case the three "copy" options don't do what we want. For example the copy() method of My_Class in the code below copies thenumbersbut shares thelettersbetween the two objects.importmemory_graphimportcopyclassMy_Class:def__init__(self):self.numbers=[1,2]self.letters=['a','b']defcopy(self):# custom copy method copies the numbers but shares the lettersc=copy.copy(self)c.numbers=copy.copy(self.numbers)returnca=My_Class()b=a.copy()memory_graph.show(locals())DebuggingOften it is useful to graph all the local variables using:memory_graph.show(locals(),block=True)So much so that functiond()is available as alias for this for easier debugging. Additionally it logs all local variables by printing them which helps comparing them over time. For example:frommemory_graphimportdmy_squares=[]my_squares_ref=my_squaresforiinrange(5):my_squares.append(i**2)d()my_squares_copy=my_squares.copy()d()in the end results in:my_squares: [0, 1, 4, 9, 16]
my_squares_ref: [0, 1, 4, 9, 16]
i: 4
my_squares_copy: [0, 1, 4, 9, 16]Functiond()has these default arguments:defd(data=None,log=True,graph=True,block=True):data: the data that is handled, defaults tolocals()log: if True the data is printedgraph: if True the data is visualized as a graphblock: if True the function blocks until the ENTER key is pressedTo print to a log file instead of standard output us:memory_graph.log_file=open("log_file.txt","w")Watchpoint in DebuggerAlternative you can also set for this expression as a 'watchpoint' in a debugger tool and open the "my_debug_graph.pdf" output file for a continuous visualization of all the local variables while debugging:memory_graph.render(locals(), "my_debug_graph.pdf")This avoids having to add any memory_graphshow()ord()calls to your code.Call StackFunctionmemory_graph.get_call_stack()returns the full call stack that holds for each called function all the local variables. This enables us to visualize the local variables of each of the called functions on the stack simultaneously. This helps to visualize if variables of different called functions share any data between them. Here for example we call functionadd_one()with argumentsa, b, cand add one to change each of them.importmemory_graphdefadd_one(a,b,c):a+=1b.append(1)c.append(1)memory_graph.show(memory_graph.get_call_stack())a=1b=[4,3,2]c=[4,3,2]add_one(a,b,c.copy())print(f"a:{a}b:{b}c:{c}")Asais of immutable type 'int' and as we call the function with a copy ofcthe visualization shows onlybis shared so onlybis changed in the calling stack frame as reflected in the printed output:a:1 b:[4, 3, 2, 1] c:[4, 3, 2]recursionThe call stack also helps to visualize how recursion works. Here we show each step of how recursivelyfactorial(3)is computed:importmemory_graphdeffactorial(n):ifn==0:return1memory_graph.show(memory_graph.get_call_stack(),block=True)result=n*factorial(n-1)memory_graph.show(memory_graph.get_call_stack(),block=True)returnresultfactorial(3)and the final result is: 1 x 2 x 3 = 6Visual Studio Code watchpointThememory_graph.get_call_stack()doesn't work well in a Visual Studio Code (VSCode) watchpoint context because its debugger introduces many stack frames that clutter the visualization. Use thememory_graph.get_call_stack_vscode()function in a VSCode watchpoint context to remove these frames. For example set:memory_graph.render(memory_graph.get_call_stack_vscode(), "call_stack_graph.pdf")as VSCode watchpoint and open the "call_stack_graph.pdf" file for a continuous visualization of the whole call stack while debugging.Datastructure ExamplesModule memory_graph can be very useful in a course about datastructures, some examples:Doubly Linked Listimportmemory_graphimportrandomrandom.seed(0)# use same random numbers each runclassNode:def__init__(self,value):self.prev=Noneself.value=valueself.next=NoneclassLinkedList:def__init__(self):self.head=Noneself.tail=Nonedefadd_front(self,value):new_node=Node(value)ifself.headisNone:self.head=new_nodeself.tail=new_nodeelse:new_node.next=self.headself.head.prev=new_nodeself.head=new_nodelinked_list=LinkedList()n=100foriinrange(n):new_value=random.randrange(n)linked_list.add_front(new_value)memory_graph.show(locals(),block=True)# <--- draw linked listBinary Treeimportmemory_graphimportrandomrandom.seed(0)# use same random numbers each runclassNode:def__init__(self,value):self.smaller=Noneself.value=valueself.larger=NoneclassBinTree:def__init__(self):self.root=Nonedefadd_recursive(self,new_value,node):memory_graph.show(locals(),block=True)# <--- draw tree when adding recursivelyifnew_value<node.value:ifnode.smallerisNone:node.smaller=Node(new_value)else:self.add_recursive(new_value,node.smaller)else:ifnode.largerisNone:node.larger=Node(new_value)else:self.add_recursive(new_value,node.larger)defadd(self,value):ifself.rootisNone:self.root=Node(value)else:self.add_recursive(value,self.root)tree=BinTree()n=100foriinrange(n):new_value=random.randrange(100)tree.add(new_value)memory_graph.show(locals(),block=True)# <--- draw tree after addingHash Setimportmemory_graphimportrandomrandom.seed(0)# use same random numbers each runclassHashSet:def__init__(self,capacity=20):self.buckets=[None]*capacitydefadd(self,value):index=hash(value)%len(self.buckets)ifself.buckets[index]isNone:self.buckets[index]=[value]else:self.buckets[index].append(value)defcontains(self,value):index=hash(value)%len(self.buckets)ifself.buckets[index]isNone:returnFalsereturnvalueinself.buckets[index]defremove(self,value):index=hash(value)%len(self.buckets)ifself.buckets[index]isnotNone:self.buckets[index].remove(value)hash_set=HashSet()n=100foriinrange(n):new_value=random.randrange(n)hash_set.add(new_value)memory_graph.show(locals(),block=True)# <--- draw hash setConfigDifferent aspects of memory_graph can be configured.Config Visualization, graphviz_nodesConfigure how the nodes of the graph are visualized with:memory_graph.graphviz_nodes.linear_layout_vertical: boolif False, linear node layout is horizontalmemory_graph.graphviz_nodes.linear_any_ref_layout_vertical: boolif False, linear node layout is horizontal if any of its elements is a refencememory_graph.graphviz_nodes.linear_all_ref_layout_vertical: boolif False, linear node layout is horizontal if all elements are referencememory_graph.graphviz_nodes.key_value_layout_vertical: boolif False, key_value node layout is horizontalmemory_graph.graphviz_nodes.key_value_any_ref_layout_vertical: boolif False, key_value node layout is horizontal if any of its elements is a refencememory_graph.graphviz_nodes.key_value_all_ref_layout_vertical: boolif False, key_value node layout is horizontal if all elements are referencememory_graph.graphviz_nodes.padding: intthe padding in nodesmemory_graph.graphviz_nodes.padding: intthe spacing in nodesmemory_graph.graphviz_nodes.join_references_count: intminimum number of reference we join togethermemory_graph.graphviz_nodes.join_circle_size: stringsize of the join circlememory_graph.graphviz_nodes.join_circle_minlen: stringextra space for references above a join circlememory_graph.graphviz_nodes.max_string_length: intmaximum string length where the string is cut offmemory_graph.graphviz_nodes.category_to_color_map: dictmapping van type/caterogries to node colorsmemory_graph.graphviz_nodes.uncategorized_color: dictcolor for unkown types/categoriesmemory_graph.graphviz_nodes.graph_attr: dictallows to set variousgraphviz graph attributesmemory_graph.graphviz_nodes.node_attr: dictallows to set variousgraphviz node attributesmemory_graph.graphviz_nodes.edge_attr: dictallows to set variousgraphviz edges attributesSee for color names:graphviz colorsTo configure more about the visualization use:digraph = memory_graph.create_graph( locals() )and see thegraphviz apito render it in many different ways.Config Graph Structure, rewrite_to_nodeConfigure the structure of the nodes in the graph with:memory_graph.rewrite_to_node.reduce_reference_parents: setthe node types/categories for which we remove the reference to childrenmemory_graph.rewrite_to_node.reduce_reference_children: boolthe node types/categories for which we remove the reference from parentsConfig Node Creation, rewriteConfigure what nodes are created based on reading the given data structure:memory_graph.rewrite.ignore_types: dictall types that we ignore, these will not be in the graphmemory_graph.rewrite.singular_types: setall types rewritten to node as singular values (bool, int, float, ...)memory_graph.rewrite.linear_types: setall types rewritten to node as linear values (tuple, list, set, ...)memory_graph.rewrite.dict_types: setall types rewritten to node as dictionary values (dict, mappingproxy)memory_graph.rewrite.dict_ignore_dunder_keys: booldetermines if we ignore dunder keys ('__example') in dict_typesmemory_graph.rewrite.custom_accessor_functions: dictcustom accessor functions to define how to read various data typesConfig ExamplesThis example shows a class with a class variable and has some recursive references.importmemory_graphmy_list=[10,20,10]classMy_Class:my_class_var=20# class variabledef__init__(self):self.var1="foo"self.var2="bar"self.var3=20obj1=My_Class()obj2=My_Class()data=[my_list,my_list,obj1,obj2]my_list.append(data)my_list.append(my_list)# recursive referencememory_graph.show(locals())With configuration:memory_graph.graphviz_nodes.linear_layout_vertical = False           # draw lists,tuples,sets,... horizontally
memory_graph.graphviz_nodes.category_to_color_map['list'] = 'yellow' # change color of 'list' type
memory_graph.graphviz_nodes.spacing = 15                             # more spacing in each node
memory_graph.graphviz_nodes.graph_attr['ranksep'] = '1.2'            # more vertical separation
memory_graph.graphviz_nodes.graph_attr['nodesep'] = '1.2'            # more horizontal separation
memory_graph.rewrite_to_node.reduce_reference_children.remove("int") # draw references to 'int' typethis example looks like:Custom Accessor FunctionsFor any type a custom accessor function can be introduced. For example Pandas DataFrames and Series are not visualized correctly by default. This can be fixed by adding custom accessor functions:importmemory_graphimportpandasaspddata={'Name':['Tom','Anna','Steve','Lisa'],'Age':[28,34,29,42],'Length':[1.70,1.66,1.82,1.73]}df=pd.DataFrame(data)memory_graph.rewrite.custom_accessor_functions[pd.DataFrame]=lambdad:list(d.items())memory_graph.rewrite.custom_accessor_functions[pd.Series]=lambdad:list(d.items())memory_graph.rewrite_to_node.reduce_reference_parents.add("DataFrame")memory_graph.rewrite_to_node.reduce_reference_parents.add("Series")memory_graph.graphviz_nodes.category_to_color_map['Series']='lightskyblue'memory_graph.show(locals())which results in:TroubleshootingWhen graph edges overlap it can be hard to distinguish them. Using an interactive graphviz viewer, such asxdot, on a '*.gv' DOT output file will help.AuthorBas TerwijnInspirationInspired byPython Tutor.