Learned Compression for Compressed Learning
Authors: name: Dan Jacobellis, name: Neeraja J. Yadwadkar

Modern sensors produce increasingly rich streams of high-resolution data. Due
to resource constraints, machine learning systems discard the vast majority of
this information via resolution reduction. Compressed-domain learning allows
models to operate on compact latent representations, allowing higher effective
resolution for the same budget. However, existing compression systems are not
ideal for compressed learning. Linear transform coding and end-to-end learned
compression systems reduce bitrate, but do not uniformly reduce dimensionality;
thus, they do not meaningfully increase efficiency. Generative autoencoders
reduce dimensionality, but their adversarial or perceptual objectives lead to
significant information loss. To address these limitations, we introduce WaLLoC
(Wavelet Learned Lossy Compression), a neural codec architecture that combines
linear transform coding with nonlinear dimensionality-reducing autoencoders.
WaLLoC sandwiches a shallow, asymmetric autoencoder and entropy bottleneck
between an invertible wavelet packet transform. Across several key metrics,
WaLLoC outperforms the autoencoders used in state-of-the-art latent diffusion
models. WaLLoC does not require perceptual or adversarial losses to represent
high-frequency detail, providing compatibility with modalities beyond RGB
images and stereo audio. WaLLoC's encoder consists almost entirely of linear
operations, making it exceptionally efficient and suitable for mobile
computing, remote sensing, and learning directly from compressed data. We
demonstrate WaLLoC's capability for compressed-domain learning across several
tasks, including image classification, colorization, document understanding,
and music source separation. Our code, experiments, and pre-trained audio and
image codecs are available at https://ut-sysml.org/walloc