Package: sconce
The advancement of deep learning has been primarily driven by the availability of large datasets and the computational power to train large models. The complexity increases with each day, and it is becoming increasingly difficult to train/deploy these models.However, the brain can learn from a few examples and is incredibly energy efficient (Psst.. that too sparsely).Humans tend to solve problems from their lens of perspective, and thus, we comprehend the universe through mathematical models. One such postulation is the concept of gradient descent or other optimization techniques we use to train our models. However, the brain does not use gradient descent to learn. How the brain learns and can solve complex problems is still a mystery.Until we hit the holy grail, we must make wise methods to achieve efficiency.The logical solution is to minimize the usage of CPM (Computation, Power, Memory) and thus achieve high throughputs and latency gains.Hence, this package aims to bridge this gap by compressing a model end-to-end and making it hardware-friendly with Minimal Human Intervention.AutoML at its Heart:Humans are lazy, at least I am; hence, we want to get things done with Minimal Human Intervention. Sconce was built on this ideology that anyone with nominal knowledge of DL should be able to use this package. Drop your model into the package, callsconce.compress, and let it do the magic for you.Compress the model throughPruning,Quantization, etc.Bring your own dataset and let theNeural Architecture Search (NAS)find the best model that fits your deployment constraints.LeverageSparsityin the model and deploy/infer usingSparse Engines.Accelerate Inferencing and Reduce Memory Footprint.In addition, this package also supportsSpiking Neural Networks(snnTorch)in this compression pipeline.If you like this project, please consider starring ⭐ this repo as it is the easiest and best way to support it.Let me know if you are using sconce in any interesting work, research or blogs, as we would love to hear more about it!If you have issues, comments, or are looking for advice on training spiking neural networks, you can open an issue, a discussion,
or chat in ourdiscordchannel.
|A Brief workflow is shown below:sconce Structuresconce contains the following components:ComponentDescriptionsconce.traina spiking neuron library like torch.nn, deeply integrated with autogradsconce.measure_latencyCompares the performance of two PyTorch models: an original dense model and a pruned and fine-tuned model. Prints a table of metrics including latency, MACs, and model size for both models and their reduction ratios.sconce.prune_modeCurrently supporting Gradual Magnitude Pruning(GMP), L1/L2 based Channel Wise Pruning(CWP), OBC, sparsegpt, etc…sconce.quantizeQuantize the computations of the model to make it more efficient for hardware Deployment/Inferences.sconce.compressAutomated compression pipeline encompassing of Pruning, Quantization, and Sparsification.sconceis designed to be intuitively used with PyTorch, compression for Linear, Convolutional and Attention blocks are supported.At present, I am working on adding support for more compression techniques and more models. kindly be patient for feature request/bug fixes.The package envisions to be a one stop solution for all your compression needs and deployed on resource constrained devices.
Provided that the network models and tensors are loaded onto CUDA, sconce takes advantage of GPU acceleration in the same way as PyTorch.sconce is a work in progress, and we welcome contributions from the community.RequirementsThe following packages need to be installed to use sconce:torch >= 1.1.0numpy >= 1.17torchprofilematplotlibsnntorchThey are automatically installed if sconce is installed using the pip command. Ensure the correct version of torch is installed for your system to enable CUDA compatibility.✌️InstallationRun the following to install:$python$pipinstallsconceTo install sconce from source instead:$ git clone https://github.com/satabios/sconce
$ cd sconce
$ python setup.py installAPI & ExamplesA complete API is availablehere. Examples, tutorials and Colab notebooks are provided.QuickstartHere are a few ways you can get started with sconce:Quickstart Notebook (Opens in Colab)The API ReferenceTutorialsQuickstart:Define Network:classNet(nn.Module):def__init__(self):super().__init__()self.conv1=nn.Conv2d(3,8,3)self.bn1=nn.BatchNorm2d(8)self.pool=nn.MaxPool2d(2,2)self.conv2=nn.Conv2d(8,16,3)self.bn2=nn.BatchNorm2d(16)self.fc1=nn.Linear(16*6*6,32)self.fc2=nn.Linear(32,10)defforward(self,x):x=self.pool(self.bn1(F.relu(self.conv1(x))))x=self.pool(self.bn2(F.relu(self.conv2(x))))x=torch.flatten(x,1)x=F.relu(self.fc1(x))x=self.fc2(x)returnxMake a Dict for Dataloaderimage_size=32transforms={"train":Compose([RandomCrop(image_size,padding=4),RandomHorizontalFlip(),ToTensor(),]),"test":ToTensor(),}dataset={}forsplitin["train","test"]:dataset[split]=CIFAR10(root="data/cifar10",train=(split=="train"),download=True,transform=transforms[split],)dataloader={}forsplitin['train','test']:dataloader[split]=DataLoader(dataset[split],batch_size=512,shuffle=(split=='train'),num_workers=0,pin_memory=True,)Define your Configurations:# Define all parametersfromsconceimportsconcesconces=sconce()sconces.model=Net()# Model Definitionsconces.criterion=nn.CrossEntropyLoss()# Losssconces.optimizer=optim.Adam(sconces.model.parameters(),lr=1e-4)sconces.scheduler=optim.lr_scheduler.CosineAnnealingLR(sconces.optimizer,T_max=200)sconces.dataloader=dataloadersconces.epochs=5#Number of time we iterate over the datasconces.device=torch.device("cuda"iftorch.cuda.is_available()else"cpu")sconces.experiment_name="vgg-gmp"# Define your experiment name heresconces.prune_mode="GMP"# Prune Mode: Currently supporting "GMP"(Supports Automated Pruning Ratio Detection), "CWP". Future supports for "OBC" and "sparseGPT"One Roof Solution:sconces.compress()Channel-WisePruning(CWP)==============================ComparisonTable==============================+---------------------+----------------+--------------+-----------------+||OriginalModel|PrunedModel|QuantizedModel|+---------------------+----------------+--------------+-----------------+|Latency(ms/sample)|6.4|4.7|1.9||Accuracy(%)|93.136|90.451|90.371||Params(M)|9.23|5.36|*||Size(MiB)|36.949|21.484|5.419||MAC(M)|606|406|*|+---------------------+----------------+--------------+-----------------+Granular-MagnitudePruning(GMP)==============================ComparisonTable==============================+---------------------+----------------+--------------+-----------------+||OriginalModel|PrunedModel|QuantizedModel|+---------------------+----------------+--------------+-----------------+|Latency(ms/sample)|6.2|6.3|2.2||Accuracy(%)|93.136|92.936|92.906||Params(M)|9.23|4.42|*||Size(MiB)|36.949|36.949|9.293||MAC(M)|606|606|*|+---------------------+----------------+--------------+-----------------+ContributingIf you’re ready to contribute to sconce, ping ondiscordchannel.Acknowledgmentssconce is solely being maintained bySathyaprakash Narayanan.Special Thanks:Prof. and Mentor Jason K. Eshraghianand his petsnnTorch(extensively inspired from snnTorch to build and document sconce)Prof. Song Hanfor his coursework MIT6.5940 and many other projects liketorchsparse.Neural Magic(Elias Frantar, Denis Kuznedelev, etc…)forOBCandsparseGPT.License & Copyrightsconce source code is published under the terms of the MIT License.
sconce’s documentation is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0).