Package: mann-m1mac
MANN for M1 Mac UsersMANN, which stands for Multitask Artificial Neural Networks, is a Python package which enables creating sparse multitask models compatible withTensorFlow. This package contains custom layers and utilities to facilitate the training and optimization of models using the Reduction of Sub-Network Neuroplasticity (RSN2) training procedure developed byAI Squared, Inc.InstallationThis package is available throughPyPi, but because of the current experimental nature of TenroFlow on the M1 platform, there are additional steps required for installation. Firstly, the steps provided atthis linkmust be followed to install the TensorFlow metal plugin and version of TensorFlow supported on the M1 SOC. Once those steps are completed, the following command can be run to install this package:pipinstallmann-m1macPlease note that MacOS 12.0.1 or later is required.CapabilitiesWe strive to maintain complete feature parity between this version of the MANN package and the version available for all other platforms.The MANN package includes two subpackages, themann.utilspackage and themann.layerspackage. As the name implies, themann.utilspackage includes utilities which assist in model training. Themann.layerspackage includes custom Keras-compatible layers which can be used to train sparse multitask models.UtilsThemann.utilssubpackage has three main functions: themask_modelfunction, theget_custom_objectsfunction, and theconvert_modelfunction.mask_modelThemask_modelfunction is central to the RSN2 training procedure and enables masking/pruning a model so a large percentage of the weights are inactive.Inputs to themask_modelfunction are a TensorFlow model, a percentile in integer form, a method - either one of 'gradients' or 'magnitude', input data, and target data.get_custom_objectsTheget_custom_objectsfunction takes no parameters and returns a dictionary of all custom objects required to load a model trained using this package.remove_layer_masksTheremove_layer_masksfunction takes a trained model with masked layers and converts it to a model without masking layers.LayersThemann.layerssubpackage contains custom Keras-compatible layers which can be used to train sparse multitask models. The layers contained in this package are as follows:MaskedDenseThis layer is nearly identical to the Keras Dense layer, but it supports masking and pruning to reduce the number of active weights.MaskedConv2DThis layer is nearly identical to the Keras Conv2D layer, but it supports masking and pruning to reduce the number of active weights.MultiMaskedDenseThis layer supports isolating pathways within the network and dedicating them for individual tasks and performing fully-connected operations on the input data.MultiMaskedConv2DThis layer supports isolating pathways within the network and dedicating them for individual tasks and performing convolutional operations on the input data.MultiDenseThis layer supports multitask inference using a fully-connected architecture and is not designed for training. Once a model is trained with theMultiMaskedDenselayer, that layer can be converted into this layer for inference by using themann.utils.remove_layer_masksfunction.MultiConv2DThis layer supports multitask inference using a convolutional architecture and is not designed for training. Once a model is trained with theMultiMaskedConv2Dlayer, that layer can be converted to this layer for inference by using themann.utils.remove_layer_masksfunction.SelectorLayerThis layer selects which of the multiple inputs fed into it is returned as a result. This layer is designed to be used specifically with multitask layers.SumLayerThis layer returns the element-wise sum of all of the inputs.FilterLayerThis layer can be turned on or off, and indicates whether the single input passed to it should be output or if all zeros should be returned.MultiMaxPool2DThis layer implements Max Pool operations on multitask inputs.Additional Documentation and Training MaterialsAdditional documentation and training materials will be added to theAI Squared Websiteas we continue to develop this project and its capabilities.Feature RoadmapTransformersWe are in the process of adding theTransformer Layerinto this package. Creating these layers will enable the training of multitask compressed models specifically for Natural Language Processing (NLP). Stay tuned!