Title: Bilateral Distribution Compression: Reducing Both Data Size and Dimensionality
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords: distribution compression, dimension reduction, joint distribution, manifold

Abstract: Existing distribution compression methods reduce dataset size by minimising the *Maximum Mean Discrepancy* (MMD) between original and compressed sets, but modern datasets are often large in both sample size and dimensionality. We propose *Bilateral Distribution Compression* (BDC), a two-stage framework that compresses along both axes while preserving the underlying distribution, with overall linear time and memory complexity in dataset size and dimension. Central to BDC is the *Decoded MMD* (DMMD), which quantifies the discrepancy between the original data and a compressed set decoded from a low-dimensional latent space. BDC proceeds by (i) learning a low-dimensional projection using the *Reconstruction MMD* (RMMD), and (ii) optimising a latent compressed set with the *Encoded MMD* (EMMD). We show that this procedure minimises the DMMD, guaranteeing that the compressed set faithfully represents the original distribution. Experiments show that across a variety of scenarios BDC can achieve comparable or superior performance to ambient-space compression at substantially lower cost.

TL;DR: We introduce a framework to compress datasets along both axes, i.e. in terms of numbers of observations and dimensions, while targeting distirbutional preservation in terms of MMD