Package: bristol
BristolParallel random matrix tools and random matrix theory deep learning applications.
Generate matrices from Circular Unitary Ensemble (CUE), Circular Ortogonal Ensemble (COE) and
Circular Symplectic Ensemble (CSE). Additional spectral analysis utilities are
also implemented, such as computation of spectral density and spectral ergodicity
for complexity of deep learning architectures.FeaturesGeneration of Circular Ensembles: CUE, COE and CSE.Random matrices: Reproducibility both in serial and parallel processing.Eigenvalue Spectra, spectral densitiy.Kullbach-Leibler divergence and spectral ergodicity measure functionality.Cascading Periodic Spectral Ergodicity (cPSE) : This is a complexity measure and could also detect when to stop addinig more layers.InstallationInstall with pip frompypi, as of 0.2.14 we prefer Python > 3.9pipinstallbristolRunning testsbashrun_tests.pyTo use the latest development versionpipinstall-upgradegit+https://github.com/msuzen/bristol.gitDocumentationComplexity of a deep learning model: cPSEVanilla caseIn the vanilla case a list of matrices that are representative of
ordered set of weight matrices can be used to compute cPSE over
layers. As an examples:frombristolimportcPSEimportnumpyasnpnp.random.seed(42)matrices=[np.random.normal(size=(64,64))for_inrange(10)](d_layers,cpse)=cPSE.cpse_measure_vanilla(matrices)Even for set of Gaussian matrices, d_layers decrease.
Note that different layer types should be converted to a matrix
format, i.e., CNNs to 2D matrices. See the main paper.When should I stop adding more layers in deep learning?d_layersis a decreasing vector, it will saturate at some point, that point is where
adding more layers won’t improve the performance. This is data, learning and architecture
independent measure.For torch modelsYou need to put your model as pretrained model format of PyTorch. An example for vgg,
and usecPSE.cpse_measurefunction simply:from bristol import cPSE
import torchvision.models as models
netname = 'vgg11'
pmodel = getattr(models, netname)(pretrained=True)
(d_layers, cpse) = cPSE.cpse_measure(pmodel)This would givecpsea single number expressing the complexity of your network andd_layersevolution ofperiodic spectral ergodicitywithing layers as a vector, order matters.Prototype notebooksBasics of circular ensemblesipynb.Computing spectral ergodicity for generated matricesipynb. This is to reproduce the main figure fromarXiv:1704.08693.The concept of cascading periodic ergodicity (cPSE)ipynbThis is only to reproduce paper's results fromarXiv:1911.07831.Empirical deviations of semicircle law in mixed-matrix ensembles,M. Suezen,hal-03464130|ipynbReproduces the work with the same title.ContactPlease create an issue for any type of questions or contactmsuzen.ReferencesBerry, M V & Pragya Shukla 2013, Hearing random matrices and random waves, New. J. Phys. 15 013026 (11pp)berry456Spectral Ergodicity in Deep Learning Architectures via Surrogate Random Matrices, Mehmet Süzen, Cornelius Weber, Joan J. Cerdà,arXiv:1704.08693Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural Networks and Neural Architecture Search, Mehmet Süzen, Cornelius Weber, Joan J. Cerdà,arXiv:1911.07831Empirical deviations of semicircle law in mixed-matrix ensembles,M. Suezen,hal-03464130|ipynbReproduces the work with the same title.CitationIf you use the ideas or tools from this package please do cite our manuscripts.@article{suezen2021a,
    title={Empirical deviations of semicircle law in mixed-matrix ensembles},
    author={Mehmet Süzen},
    year={2021},
    eprint={hal-03464130},
    url={https://hal.archives-ouvertes.fr/hal-03464130}
}@article{suezen2019a,
    title={Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural Networks and Neural Architecture Search},
    author={Mehmet Süzen and Joan J. Cerdà and Cornelius Weber},
    year={2019},
    eprint={1911.07831},
    archivePrefix={arXiv},
    primaryClass={stat.ML}
}@article{suezen2017a,
    title={Spectral Ergodicity in Deep Learning Architectures via Surrogate Random Matrices},
    author={Mehmet Süzen and Cornelius Weber and Joan J. Cerdà},
    year={2017},
    eprint={1704.08303},
    archivePrefix={arXiv},
    primaryClass={stat.ML}
}