Package: visunn
Visunn: Aesthetic Visualizations of Neural Networks for Deep LearningVisunn is a visualization tool that leverages functional and modular visualizations to provide an visual understanding of neural network architectures. Currently,torch>=1.4.0backend is supported.SetupThe backend and user API is in Python (allpipdependencies can be found inrequirements.txt) and serves a frontend consisting of a fusion of React and Three.js (allnpmdependencies can be found invisunn/frontend/package.json).UsageThe following examples will use the modelmodels/ThreeLayerMLP.To pip installInstall python packagepipinstallvisunnInitializeVisuin Python scriptfromvisunnimportVisuvisu=Visu(model,dataloader,logdir='logs',name='ThreeLayerMLP')Launch web appvisu-llogs-nThreeLayerMLP-p5000To use sourceBuild frontend (requiresnpm)shbuild.shInitializeVisupythonsamples/train.py-llogs-nThreeLayerMLPLaunch web apppython samples/serve.py -l logs -n ThreeLayerMLP -p 5000MetricsBelow are time and space metrics for a few sample models, averaged over 5 runs. These can be obtained by runningpython samples/debug.py, which prints metrics of the 5 steps required to create a modularized topology (run on CIFAR-10):Step 1 is the largest bottlenecks in the algorithm.Convert model to protobuf (built in Pytorch function)Convert protobuf to dictPrune trivial nodesPrune trivial modulesBuild modularized topologyModelStep 1Step 2Step 3Step 4Step 5SpaceThreeLayerMLP0.042 s0.001 s0.000 s0.000 s0.000 s32.37 kbThreeLayerConvNet0.051 s0.002 s0.000 s0.000 s0.000 s35.33 kbresnet181.282 s0.016 s0.002 s0.001 s0.002 s335.40 kbresnet1526.819 s0.115 s0.065 s0.005 s0.013 s2.14 Mbdensenet1214.143 s0.103 s0.045 s0.008 s0.017 s1.94 Mbdensenet2017.528 s0.174 s0.125 s0.015 s0.034 s3.56 Mbgooglenet2.241 s0.046 s0.011 s0.001 s0.011 s941.52 kbshufflenet_v2_x2_02.289 s0.070 s0.023 s0.007 s0.010 s1.07 Mbmobilenet_v21.685 s0.045 s0.009 s0.003 s0.005 s674.06 kbresnext101_32x8d11.744 s0.085 s0.033 s0.004 s0.009 s1.35 Mbwide_resnet101_213.495 s0.002 s0.165 s0.007 s0.054 s1.10 Mbmnasnet1_31.562 s0.042 s0.008 s0.003 s0.005 s658.74 kbNotesThis section is dedicated to address nuances that come with the Pytorch backend.Recycled LayersThis problem is exemplified with the following following variation ofThreeLayerMLP:classModel(nn.Module):def__init__(self,num_classes=10,**kwargs):self.flatten=nn.Flatten(1,-1)self.linear1=nn.Linear(32*32*3,1024,bias=True)self.linear2=nn.Linear(1024,256,bias=True)self.linear3=nn.Linear(256,num_classes,bias=True)self.relu=nn.ReLU()defforward(self,x):x=self.flatten(x)x=self.linear1(x)x=self.relu(x)x=self.linear2(x)x=self.relu(x)x=self.linear3(x)returnxNote howself.relu = nn.ReLU()is reused multiple times. While this is not incorrect in any way. However, from a graph perspective, there is oneself.relu = ReLU()node, which has multiple input and output nodes. Rendering this topology declaration will show cycles in the graph due to the multiple passes through the recycled node. A potential solution would be to split all nodes that consist of >1 unconnected graph.Weights and BiasesWith howtorch>=1.4.0constructs the graph protobuf, it is not possible to acquire the_output_shapesattribute from the weights and biases nodes (recoverable for all other nodes). Those fields empty when aggregating attributes.