Package: sc2bench
SC2 Benchmark: Supervised Compression for Split ComputingThis is the official repository ofsc2benchpackage and our TMLR paper,"SC2 Benchmark: Supervised Compression for Split Computing".As an intermediate option between local computing and edge computing (full offloading),split computinghas been
attracting considerable attention from the research communities.In split computing, we split a neural network model into two sequences so that some elementary feature transformations
are applied by the first sequence of the model on a weak mobile (local) device.
Then, intermediate, informative features are transmitted through a wireless communication channel to a powerful
edge server that processes the bulk part of the computation (the second sequence of the model).ImageNet (ILSVRC 2012): R-D (rate-distortion), ExR-D, and three-way tradeoffs for input compression and supervised compression with ResNet-50 as a reference modelInput compression is an approach to save transmitted data, but it leads to transmitting information irrelevant to
the supervised task.
To achieve better supervised rate-distortion tradeoff, we definesupervised compressionas
learning compressed representations for supervised downstream tasks such as classification, detection, or segmentation.
Specifically for split computing, we term the problem settingSC2(Supervised Compression for Split Computing).Note that the training process can be done offline (i.e., on a single device without splitting),
and it is different from "split learning".SC2 Metrics1. Encoder Size (to be minimized)Local processing cost should be minimized as local (mobile) devices usually have battery constraints and limited computing power.
As a simple proxy for the computing costs, we measure the number of encoder parameters and
define the encoder size as the total number of bits to represent the parameters of the encoder.2. Data Size (to be minimized)We want to penalize large data being transferred from the mobile device to the edge server while the BPP does not
penalize it when feeding higher resolution images to downstream models for achieving higher model accuracy.3. Model Accuracy (to be maximized)While minimizing the two metrics, we want to maximize model accuracy (minimize supervised distortion).
Example supervised distortions are accuracy, mean average precision (mAP), and mean intersection over union (mIoU) for
image classification, object detection, and semantic segmentation, respectively.Installationpipinstallsc2benchVirtual EnvironmentsFor pipenv users,pipenvinstall--python3.8# or create your own pipenv environmentpipenvinstallsc2benchFor conda users,condaenvcreate-fenvironment.yaml# or create your own conda environmentcondaactivatesc2-benchmark
pipinstallsc2benchDatasetsSee instructionshereCheckpointsYou can download our checkpoints including trained model weightshere.Unzip the downloaded zip files under./, then there will be./resource/ckpt/.Supervised CompressionCR + BQ:"Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks"End-to-End:"End-to-end Learning of Compressible Features"Entropic Student:"Supervised Compression for Resource-Constrained Edge Computing Systems"README.mdexplains how to train/test implemented supervised compression methods.Baselines: Input CompressionCodec-based input compression: JPEG, WebP, BPGNeural input compression: Factorized Prior, Scale Hyperprior, Mean-scale Hyperprior, and Joint Autoregressive Hierarchical PriorEachREADME.mdgives instructions to run the baseline experiments.Codec-based Feature Compression# JPEGpythonscript/task/image_classification.py-test_only--configconfigs/ilsvrc2012/feature_compression/jpeg-resnet50.yaml# WebPpythonscript/task/image_classification.py-test_only--configconfigs/ilsvrc2012/feature_compression/webp-resnet50.yamlCitation[Paper] [Preprint]@article{matsubara2023sc2,title={{SC2 Benchmark: Supervised Compression for Split Computing}},author={Matsubara, Yoshitomo and Yang, Ruihan and Levorato, Marco and Mandt, Stephan},journal={Transactions on Machine Learning Research},issn={2835-8856},year={2023},url={https://openreview.net/forum?id=p28wv4G65d}}NoteFor measuring data size per sample precisely, it is important to keep test batch size of 1 when testing.E.g., some baseline modules may expect larger batch size if you have multiple GPUs.Then, addCUDA_VISIBLE_DEVICES=0before your execution command (e.g.,sh,bash,python)
so that you can force the script to use one GPU (use GPU: 0 in this case).For instance, an input compression experiment using factorized prior (pretrained input compression model)
and ResNet-50 (pretrained classifier)CUDA_VISIBLE_DEVICES=0shscript/neural_input_compression/ilsvrc2012-image_classification.shfactorized_prior-resnet508Issues / Questions / RequestsThe documentation is work-in-progress. In the meantime, feel free to create an issue if you find a bug.If you have either a question or feature request, start a new discussionhere.ReferencesPyTorch (torchvision)PyTorch Image Models (timm)CompressAItorchdistillJohannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang and Nick Johnston."Variational image compression with a scale hyperprior"(ICLR 2018)David Minnen, Johannes Ballé and George D. Toderici."Joint Autoregressive and Hierarchical Priors for Learned Image Compression"(NeurIPS 2018)Yoshitomo Matsubara and Marco Levorato."Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks"(ICPR 2020)Saurabh Singh, Sami Abu-El-Haija, Nick Johnston, Johannes Ballé, Abhinav Shrivastava and George Toderici."End-to-end Learning of Compressible Features"(ICIP 2020)Yoshitomo Matsubara, Ruihan Yang, Marco Levorato and Stephan Mandt."Supervised Compression for Resource-Constrained Edge Computing Systems"(WACV 2022)