Package: ultimate-anatome
anatomeἈνατομή is a PyTorch library to analyze internal representation of neural networksThis project is under active development and the codebase is subject to change.Note the name of the pypi package isultimate-anatomebut the python import is done with
the original nameimport anatome.
Test via:python -c "import anatome.my_utils as my_utils;my_utils.hello()"
python -c "import anatome.my_utils as my_utils;my_utils.my_anatome_test()"Credit to original library:https://github.com/moskomule/anatomeand consider citing both of us! Citation at the end of readme.InstallationIf you are going to use a gpu the do this first before continuing
(or check the offical website:https://pytorch.org/get-started/locally/):pip3 install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.htmlOtherwise, just doing the follwoing should work.pip install ultimate-anatomeManual installation [for Development]To use code first get the code from this repo (e.g. fork it on github):git clone git@github.com:brando90/ultimate-anatome.gitThen install it in development mode in your python env with python >=3.9.
E.g. create your env with conda:conda create -n ultimate_anatome_env python=3.9
conda activate ultimate_anatome_envThen install code in edibable mode and all it's depedencies with pip in the currently activated conda environment:pip install -e ~/ultimate-anatomeAvailable ToolsRepresentation SimilarityTo measure the similarity of learned representation,anatome.SimilarityHookis a useful tool. Currently, the following
methods are implemented.Raghu et al. NIPS2017 SVCCAMarcos et al. NeurIPS2018 PWCCAKornblith et al. ICML2019 Linear CKADing et al. arXiv Orthogonal Procrustes distanceimporttorchfromtorchvision.modelsimportresnet18fromanatomeimportDistanceHookfromanatome.my_utilsimportremove_hookmodel=resnet18()hook1=DistanceHook(model,"layer3.0.conv1")hook2=DistanceHook(model,"layer3.0.conv2")model.eval()withtorch.no_grad():model(torch.randn(128,3,224,224))# downsampling to (size, size) may be helpfulhook1.distance(hook2,size=8)hook1.clear()hook2.clear()remove_hook(model,hook1)remove_hook(model,hook2)or to test do:fromanatome.my_utilsimportmy_anatome_testmy_anatome_test()Loss Landscape VisualizationLi et al. NeurIPS2018fromanatomeimportlandscape2dx,y,z=landscape2d(resnet18(),data,F.cross_entropy,x_range=(-1,1),y_range=(-1,1),step_size=0.1)imshow(z)Fourier AnalysisYin et al. NeurIPS 2019 etc.,fromanatomeimportfourier_mapmap=fourier_map(resnet18(),data,F.cross_entropy,norm=4)imshow(map)CitationIf you use this implementation in your research, please conser citing my version of anatome:@software{miranda2021ultimate_anatome,
    author={Brando Miranda},
    title={Ultimate Anatome, the ultimate PyTorch library to analyze internal representation of neural networks},
    url={https://github.com/brando90/ultimate-anatome},
    year={2021}
}and the original version:@software{hataya2020anatome,
    author={Ryuichiro Hataya},
    title={anatome, a PyTorch library to analyze internal representation of neural networks},
    url={https://github.com/moskomule/anatome},
    year={2020}
}Related libraries:The original anatome of coursehttps://github.com/moskomule/anatomehttps://github.com/AntixK/PyTorch-Model-CompareFrom the developers "A tiny package to compare two neural networks in PyTorch. There are many ways to compare two neural networks, but one robust and scalable way is using the Centered Kernel Alignment (CKA) metric, where the features of the networks are compared."Captum is trying to add support. Check it out here:https://github.com/pytorch/captum/issues/572#issuecomment-957306989The original svcca tutorials by Maithra et alhttps://github.com/google/svcca/tree/master/tutorialsif you know more create a pull request and add it here!