Package: beyondml
BeyondML (formerly MANN)BeyondML is a Python package which enables creating sparse multitask artificial neural networks (MANNs) compatible withTensorFlowandPyTorch. This package contains custom layers and utilities to facilitate the training and optimization of models using the Reduction of Sub-Network Neuroplasticity (RSN2) training procedure developed byAI Squared, Inc.InstallationThis package is available throughPyPiand can be installed via the following command:pipinstallbeyondmlTo install the current version directly fromGitHubwithout cloning, run the following command:pipinstallgit+https://github.com/Beyond-ML-Labs/BeyondML.gitAlternatively, you can install the package by cloning the repository fromGitHubusing the following commands:# clone the repository and cd into itgitclonehttps://github.com/Beyond-ML-Labs/BeyondMLcdBeyondML# install the packagepipinstall.Mac M1 UsersFor those with a Mac with the M1 processor, this package can be installed, but the standard version of TensorFlow is not compatible with the M1 SOC. In order to install a compatible version of TensorFlow, please install theMiniforgeconda environment, which utilizes the conda-forge channel only. Once you are using Miniforge, using conda to install TensorFlow in that environment should install the correct version. After installing TensorFlow, the commandpip install beyondmlwill install the BeyondML package.ContributingFor those who are interested in contributing to this project, we first thank you for your interest! Please refer to the CONTRIBUTING.md file in this repository for information about best practices for how to contribute.Vulnerability reportingIn the event you notice a vulnerability within this project, please open aGitHub Issuedetailing the vulnerability to report it. In the event you would like to keep the report private, please emailmann@squared.ai.CapabilitiesTo view current capabilities within the BeyondML package, we welcome you to check theBeyondML documentation.Feature RoadmapLists of features slated for this project will be added here.ChangesBelow are a list of additional features, bug fixes, and other changes made for each version.MANNThe below version numbers and logged changes refer to the MANN package.Version 0.2.2Small documentation changesAddedquantize_modelfunctionAddedbuild_transformer_blockandbuild_token_position_embedding_blockfunctions for transformer functionalityRemoved unnecessary imports breaking imports in minimal environmentsVersion 0.2.3Per-task pruningFunctionality for this feature is implemented, but usage is expected to be incomplete. Note that task gradients have to be passed retrieved and passed to the function directly (helper function available), and that the model has to initially be compiled using a compatible loss function (recommended 'mse') to identify gradients.It has been found that this functionality is currently only supported for models with the following layers:MaskedConv2DMaskedDenseMultiMaskedDenseNote also that this functionality does not support cases where layers of an individual model are other TensorFlow models, but supporting this functionality is on the roadmap.Iterative training using per-task pruningFunctionality for this feature is implemented, but there are known bugs when trying to apply this methodology to models with theMultiMaskedConv2Dlayer presentVersion 0.3.0Support for PyTorch layersSupport for additional custom objects in thequantize_modelfunctionAdded tests to the package functionalityAdded auto-generated documentationBeyondMLThe below version numbers and changes refer to the BeyondML packageVersion 0.1.0Refactored existing MANN repository to rename to BeyondMLVersion 0.1.1Added theSparseDense,SparseConv,SparseMultiDense, andSparseMultiConvlayers tobeyondml.tflow.layers, giving users the functionality to utilize sparse tensors during
inferenceVersion 0.1.2Added theMaskedMultiHeadAttention,MaskedTransformerEncoderLayer, andMaskedTransformerDecoderLayerlayers tobeyondml.pt.layersto add pruning to the transformer architectureAddedMaskedConv3D,MultiMaskedConv3D,MultiConv3D,MultiMaxPool3D,SparseConv3D, andSparseMultiConv3Dlayers tobeyondml.tflow.layersAddedMaskedConv3D,MultiMaskedConv3D,MultiConv3D,MultiMaxPool3D,SparseConv3D,SparseMultiConv3D, andMultiMaxPool2Dlayers tobeyondml.pt.layersVersion 0.1.3Addedbeyondml.ptcompatibility with more native PyTorch functionality for using models on different devices and datatypesAddedtrain_modelfunction tobeyondml.tflow.utilsAddedMultitaskNormalizationlayer tobeyondml.tflow.layersandbeyondml.pt.layersVersion 0.1.4Updated documentation to use SphinxVersion 0.1.5Updated requirements to use newer version of TensorFlowFixed errors with changes to types ofinput_shapein TensorFlow Keras layersFixed errors resulting from model/configuration changes with TensorFlowVersion 0.1.6Fixed issues with converting between masked and unmasked models in TensorFlowVersion 0.1.7Updated Pytorch implementation of Transformer-based architectures