Title: DRIFT: DATA REDUCTION VIA INFORMATIVE FEATURE TRANSFORMATION – GENERALIZATION BEGINS BEFORE DEEP LEARNING STARTS
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Feature representation, Neural network, Dimensionality reduction, Generalization gap.

Abstract: Modern deep learning models achieve remarkable optimization capabilities but remain highly dependent
on the quality and structure of input data. The primary challenge to robust generalization lies
in efficiently extracting minimal yet informative features that capture essential data patterns while
reducing noise and redundancy. We present DRIFT (Data Reduction via Informative Feature Transformation),
a novel, physics-inspired preprocessing method that leverages vibrational mode analysis
to derive resonant features from input data. By projecting data onto a low-dimensional basis formed
by spatial vibration mode shapes, DRIFT produces compact and interpretable representations that
significantly enhance training stability and generalization. Evaluations on MNIST and CIFAR100
datasets reveal that DRIFT enables neural networks to operate effectively with dramatically reduced
input dimensions, while maintaining or exceeding the classification accuracy of baseline pixel-based
models, Principal Component Analysis (PCA), and Discrete Cosine Transform (DCT). Across a
broad range of learning rates and feature dimensions, DRIFT consistently demonstrates superior
convergence behavior, sharper initial learning, and reduced sensitivity to hyperparameter settings.
Importantly, DRIFT mitigates overfitting, achieving smaller generalization gaps and smoother loss
trajectories than competing dimensionality reduction methods. Furthermore, reconstruction analyses
indicate that DRIFT provides stable and reliable reconstructions across varying mode counts,
exhibiting lower variability than PCA and DCT while retaining visually faithful representations even
at moderate dimensionality. To the best of our knowledge, this is the first effort to bridge physical
vibration modes with feature extraction for machine learning, offering a novel avenue for physics-informed
data representation.