Package: awave
Adaptive waveletsWavelets which adapt given data (and optionally a pre-trained model). This yields models which are faster, more compressible, and more interpretable.QuickstartInstallation:pip install git+https://github.com/Yu-Group/adaptive-wavelets.gitor clone the repo and runpython setup.py installfrom the repo directoryThen, can use the core functions (see simplest example innotebooks/demo_simple_2d.ipynbornotebooks/demo_simple_1d.ipynb). Seethe docsfor more information on arguments for these functions.Given some dataX, you can run the following:fromawave.utils.miscimportget_wavefunfromawave.transform2dimportDWT2dwt=DWT2d(wave='db5',J=4)wt.fit(X=X,lr=1e-1,num_epochs=10)# this function alternatively accepts a dataloaderX_sparse=wt(X)# uses the learned adaptive waveletphi,psi,x=get_wavefun(wt)# can also inspect the learned adaptive waveletTo distill a pretrained model namedmodel, simply pass it as an additional argument to the fit function:wt.fit(X=X,pretrained_model=model,lr=1e-1,num_epochs=10,lamL1attr=5)# control how much to regularize the model's attributionsBackgroundOfficial code for using / reproducing AWD from the paper "Adaptive wavelet distillation from neural networks through interpretations" (ha et al. 2021).Recent deep-learning models have achieved impressive prediction performance, but often sacrifice interpretability and computational efficiency. Interpretability is crucial in many disciplines, such as science and medicine, where models must be carefully vetted or where interpretation is the goal itself. Moreover, interpretable models are concise and often yield computational efficiency. Here, we propose adaptive wavelet distillation (AWD), a method which aims to distill information from a trained neural network into a wavelet transform. Specifically, AWD penalizes feature attributions of a neural network in the wavelet domain to learn an effective multi-resolution wavelet transform. The resulting model is highly predictive, concise, computationally efficient, and has properties (such as a multi-scale structure) which make it easy to interpret. In close collaboration with domain experts, we showcase how AWD addresses challenges in two real-world settings: cosmological parameter inference and molecular-partner prediction. In both cases, AWD yields a scientifically interpretable and concise model which gives predictive performance better than state-of-the-art neural networks. Moreover, AWD identifies predictive features that are scientifically meaningful in the context of respective domains.Also provides an implementation for "Learning Sparse Wavelet Representations"(recoskie & mann, 2018)Related workTRIM (ICLR 2020 workshoppdf,github) - using simple reparameterizations, allows for calculating disentangled importances to transformations of the input (e.g. assigning importances to different frequencies)ACD (ICLR 2019pdf,github) - extends CD to CNNs / arbitrary DNNs, and aggregates explanations into a hierarchyCDEP (ICML 2020pdf,github) - penalizes CD / ACD scores during training to make models generalize betterDAC (arXiv 2019pdf,github) - finds disentangled interpretations for random forestsPDR framework (PNAS 2019pdf) - an overarching framewwork for guiding and framing interpretable machine learningIf this package is useful for you, please cite the following!@article{ha2021adaptive,title={Adaptivewaveletdistillationfromneuralnetworksthroughinterpretations},author={Ha,WooseokandSingh,ChandanandLanusse,FrancoisandSong,EliandDang,SongandHe,KangminandUpadhyayula,SrigokulandYu,Bin},journal={arXivpreprintarXiv:2107.09145},year={2021}}