Package: wavpool
WavPoolA network block with built in spacial and scale decomposition.Modern deep neural networks comprise many operational layers, such as dense or convolutional layers, which are often collected into blocks. In this work, we introduce a new, wavelet-transform-based network architecture that we call the multi-resolution perceptron: by adding a pooling layer, we create a new network block, the WavPool. The first step of the multi-resolution perceptron is transforming the data into its multi-resolution decomposition form by convolving the input data with filters of fixed coefficients but increasing size. Following image processing techniques, we are able to make scale and spatial information simultaneously accessible to the network without increasing the size of the data vector. WavPool outperforms a similar multilayer perceptron while using fewer parameters, and outperforms a comparable convolutional neural network by over 10% on accuracy on CIFAR-10.This codebase contains the experimental work supporting the paper. It is to be used additional material for replication.InstallationOur project can be installed with pipfrom pypiusing:pip install wavpoolThis project is build with pythonpoetry. And is our perfered method to install from source.Commands are as follows:pip install poetry
poetry shell
poetry init
poetry installTo install all the dependencies required for this project.We also supply distribution files (found in \dist), or you may use the provided pyproject.toml to install with your method of choice.ContentsData GeneratorsThe pytorch data generator objects for the experiments done in this paper.
Wrapped to work with the training framework, but functionally unmodified.
We include CIFAR-10 (cifar_generator.py), Fashion MNIST (fashion_mnist_generator.py), and MNIST (mnist_generator.py).TrainingTraining loops used in the experiments.finetune_networks.pygenerates a set of parameters optimial for a network/task combination.train_model.pyExecutes the training loop for a network/task/parameter combination.Modelswavpool.pyOur implimentation of the novel WavPool blockvanillaCNN.pyStandard two layer CNN containing 2D Convolutions, batch norms, and a dense outputvanillaMLP.pyStandard two hidden layer MLPwavelet_layer.pyTheMicroWavMLR analysis layerwavMLP.pySingleMicroWavlayer network with an additional dense layer and output. Not included in the paper.NotebooksVisualizations of experiments with plotting code for plots included in the paper, and code to produce weights.run_experiments.pyTakes a configuration and trains an model.
Current execution shows the optimization and subsquentical training and testing for a WavPool over CIFAR-10, Fashion MNIST and MNIST.AcknowledgementWe acknowledge the Deep Skies Lab as a community of multi-domain experts and collaborators who've facilitated an environment of open discussion, idea-generation, and collaboration. This community was important for the development of this project.
We thank Aleksandra Ciprijanovic, Andrew Hearin, and Shubhendu Trivedi for comments on the manuscript.
This manuscript has been authored by Fermi Research Alliance, LLC under Contract No.~DE-AC02-07CH11359 with the U.S.~Department of Energy, Office of Science, Office of High Energy Physics.FERMILAB-CONF-23-278-CSAIDCitationIf you use our work or our code, we request you cite the arxiv paper!@misc{mcdermott2023wavpool,
      title={WavPool: A New Block for Deep Neural Networks}, 
      author={Samuel D. McDermott and M. Voetberg and Brian Nord},
      year={2023},
      eprint={2306.08734},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}