Package: nebulgym
Join the community|Contribute to the libraryHow nebulgym works•Benchmarks•Installation•Get started•Tutorials & examples•DocumentationWebsite|LinkedIn|TwitterNebulgymEasy-to-use library to accelerate AI training leveraging state-of-the-art optimization techniques 🤸‍♀️How nebulgym worksBenchmarksTutorials and examplesInstallation & get startedDocumentationJoin the community for AI accelerationHow nebulgym worksnebulgymgreatly reduces the training time of AI models without requiring any modification to the training setup.nebulgymoptimizes the full training computing stack, from efficient data loading, to faster forward and backward passes, to earlier convergence.No matter what model, framework, or training recipe you use, withnebulgymyou speed up training by simply adding nebulgym class decorators to your code. Decorators will make sure that you use your hardware's computing power to the fullest and achieve the shortest possible training time.Your code + @nebulgym_class_decorators = superfast training 🏃‍♀️So why nebulgym?🚀Superfast. The library speeds up training and thus the time to get to production, reduces computing costs and energy consumption.☘️Easy-to-use. Just addnebulgymclass decorators to your code and continue programming on your favorite training framework. nebulgym will let you achieve awesome training times.💥Training setup agnostic.nebulgymcan be coupled with any model, trainer, or other training technique to achieve a compound effect on training performance.🦾Framework agnostic. The library aims to support all frameworks (PyTorch, TensorFlow, Hugging Face, Jax, etc.) to allow any developer to continue working with the configuration they are used to. Currently, nebulgym supports PyTorch and we are working on expanding nebulgym capabilities.💻Deep learning model agnostic.nebulgymsupports all the most popular architectures such as transformers, LSTMs, CNNs and FCNs.🤖Hardware agnostic. The library aims to support any artificial intelligence hardware on the market, from general-purpose GPUs and CPUs to hardware accelerators such as FPGAs and ASICs. Currently, nebulgym has been tested on many CPUs and GPUs.Do you like the library? Leave a ⭐ if you enjoy the project and jointhe communitywhere we chat aboutnebulgymand AI acceleration.And happy training 🏋️Benchmarksnebulgymhas just been launched and has been tested on limited use cases. Early results are remarkably good, and it is expected thatnebulgymwill further reduce training time in future releases. At the same time, it is expected that nebulgym may fail in untested cases and provide different results, perhaps greater or worse than those shown below.We tested nebulgym on the custom model that you can find in theexample section. The test consists in a training over 10 epochs and a batch size of 8.Below are the training times in seconds beforenebulgymoptimization and after its acceleration, and the speedup, which is calculated as the response time of the unoptimized model divided by the response time of the accelerated model.Training time in secondsHardwareNot-optimizedAcceleratedSpeedupM1 Pro632.05347.521.8xIntel Xeon788.05381.012.1xAMD EPYC1547.351034.371.5xNVIDIA T4258.88127.322.0x_____________________________________________________________________Hardware setupM1 Pro: Apple M1 Pro 16GB of RAMIntel Xeon: EC2 Instance on AWS - t2.largeAMD EPYC: EC2 Instance on AWS - t4a.largeNVIDIA T4: EC2 instance on AWS - g4dn.xlargeHow doesnebulgymperform on your training setup? What do you think aboutnebulgymand what are ways to make it even better? Share your ideas and results with us in thecommunity chat.InstallationInstalling and using nebulgym is super easy! You can eitherinstallnebulgymfrom PyPI (withpip) orinstallnebulgymfrom source code.We strongly recommend that you installnebulgymin a new environment. You can create and manage your environment using Conda or another virtual environment management application. We tested our installation withvenvbyconda.Installation from PyPIpip install nebulgymSource code installationClone nebulgym repository to your local machine.git clone https://github.com/nebuly-ai/nebulgym.gitGo into the repo and run the setup.py file.cd nebulgym && python setup.py installGet startednebulgymaccelerates training by means of class decorators. A class decorator is a very elegant and non-intrusive method that allows nebulgym to tag your model (@accelerate_model) and your dataset (@accelerate_dataset) and add functionalities to their classes. When you run a training session,nebulgymwill greatly reduce the training time of your decorated model. As simple as that!You can find more information aboutnebulgymclass decorators, the parameters they can take as input, and othernebulgymclasses that can be used as an alternative to decorators in thedocumentation.How to use nebulgym class decoratorsPutnebulgymclass decorators right before defining your dataset and model classes.@accelerate_datasetmust be entered before the dataset definition. nebulgym will cache dataset samples in memory, so that reading these samples after the first time becomes much faster. Caching the dataset makes data loading faster and more efficient, solving what could become the main bottleneck of the whole training process.@accelerate_modelmust be entered before the model definition. nebulgym will accelerate both forward and backward propagations by reducing the number of computationally expensive propagation steps and making computations more efficient.nebulgym use caseHere we show an example of how you can easily usenebulgymclass decorators. To achieve awesome training speed, you can simply addnebulgymdecorators (@accelerate_modeland@accelerate_dataset) before defining your AI model and dataset.from typing import List, Callable
import torch
from torch.utils.data import Dataset

from nebulgym.decorators.torch_decorators import accelerate_model, accelerate_dataset

# Add nebulgym annotation before defining your model. 
#  This model takes as input an image of resolution 224x224
@accelerate_model()
class CustomModel(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self._avg_pool = torch.nn.AvgPool2d(4)
        self._linear = torch.nn.Linear(3136, 1024)
        self._relu = torch.nn.ReLU()
        self._linears = torch.nn.Sequential(
            torch.nn.BatchNorm1d(1024),
            torch.nn.Linear(1024, 2048),
            torch.nn.ReLU(),
            torch.nn.BatchNorm1d(2048),
            torch.nn.Linear(2048, 1024),
            torch.nn.ReLU(),
            torch.nn.BatchNorm1d(1024),
            torch.nn.Linear(1024, 512),
            torch.nn.ReLU(),
            torch.nn.BatchNorm1d(512),
            torch.nn.Linear(512, 256),
            torch.nn.ReLU(),
            torch.nn.Linear(256, 2),
        )

    def forward(self, x):
        x = self._avg_pool(x).mean(dim=-3).view(-1, 3136)
        x = self._relu(self._linear(x))
        return self._linears(x)


# Add nebulgym annotation before defining your dataset.
@accelerate_dataset()
class CustomDataset(Dataset):
    def __init__(self, img_paths: List[str], labelling_func: Callable, reading_func: Callable):
        self._images = img_paths
        self._labelling_func = labelling_func
        self._reading_func = reading_func

    def __getitem__(self, item):
        img_path = self._images[item]
        label = self._labelling_func(img_path)
        input_tensor = self._reading_func(img_path)
        return input_tensor, label

    def __len__(self):
        return len(self._images)And that's it. Now, as soon as you perform a training run,nebulgymwill optimize the full training computing stack, from efficient data loading, to faster forward and backward passes, to earlier convergence.Supported tech & roadmapnebulgymhas just been launched, and it is already capable of cutting training time in half. At the same time, it is expected thatnebulgymmay crash or fail in untested use cases. Moreover, the project is in its early stages and there is a lot of room for improvement fornebulgymto become a new paradigm for artificial intelligence training.nebulgymaims to support every framework, every model, every hardware, and make the most of your hardware and software capabilities to train your model in a fraction of the time required now. In addition,nebulgymwill always be extremely easy to use to empower any developer to build powerful AI applications.nebulgymalready embeds many great technologies. Below you can find a list of the features already implemented and those that will be implemented soon. More specific tasks can be found in theissue page.Any ideas about what could be implemented next? Would you like to contribute to this fantastic library? We welcome any ideas, questions, issues and pull requests! For more info go to theDocumentation.Supported frameworksPyTorchTensorFlow (open issue)Supported backendsPyTorch. Default compiler for models trained in PyTorch.Rammer. Compiler that can be used on Nvidia GPUs.ONNX Runtime. Training API leveraging on some techniques developed for inference optimization. It currently supports only Nvidia GPUs.Optimization techniques for data loadingCached datasets. Data loading is for some use cases slow and could become a major bottleneck of the whole training process. nebulgym provides cached dataset to speedup this process by caching data samples in memory, so that reading these samples after the first time becomes much faster.Model Optimization techniquesSparsified Back Propagation. As traditional neural network consumes a significant amount of computing resources during back propagation, leveraging a simple yet effective technique to alleviate this problem. In this technique, only a small subset of the full gradients are computed to update the model parameters and the model still achieves the same effect as the traditional CNN, or even better.Selective-Backprop. (open issue)Layer Replacement(open issue)ModelReshaper(open issue)Distributed training(open issue)Forward gradients(open issue)Library installation methodsFrom PyPISource codeBackend installation methodsFrom the backend source codeAutomatic installation with an auto-installer (open issue)LicenceThis project is released under theApache 2.0 licence.Join the community|Contribute to the libraryHow nebulgym works•Benchmarks•Installation•Get started•Tutorials & examples•DocumentationWebsite|LinkedIn|Twitter