Title: Expressive yet Efficient Feature Expansion with Adaptive Cross-Hadamard Products
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Efficient Vision Models, Hadamard Product, Neural Architecture Search, Differentiable Sampling

Abstract: Recent theoretical advances reveal that the Hadamard product induces nonlinear representations and implicit high-dimensional mappings for the field of deep learning, yet their practical deployment in efficient vision models remains underdeveloped. To address this gap, we introduce the Adaptive Cross-Hadamard (ACH) module, a novel operator that embeds learnability through differentiable discrete sampling and dynamic softsign normalization. This enables parameter-free feature reuse while stabilizing gradient propagation. Integrated into Hadaptive-Net (Hadamard Adaptive Network) via neural architecture search, our approach achieves unprecedented efficiency. Comprehensive experiments demonstrate state-of-the-art accuracy/speed trade-offs on image classification task, establishing Hadamard operations as fundamental building blocks for efficient vision models.

TL;DR: Hadaptive-Net, a new family of efficient vision models built upon a novel, learnable Hadamard product operator, which establishes a new state-of-the-art trade-off between accuracy and computational cost.