Title: Feature–Label Embedding Alignment for Backprop-Free tiny Networks
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Forward-Only Learning, Contextual Convolution, Embedded Deep Learning, Memory-Efficient Neural Networks, On-Device Learning

Abstract: Forward-only training methods, such as Forward-Forward (FF) learning, provide an alternative to backpropagation by eliminating the backward pass. This reduces memory usage and makes them well-suited for forward-propagation accelerators increasingly common in modern embedded devices. However, current forward-only methods struggle to scale to deep networks and challenging visual recognition tasks, resulting in a substantial performance gap compared to backpropagation.  
We address this limitation with the \textit{Feature-Label Embedding Alignment (FLEA)} block, a novel architectural component that allows FF networks to scale effectively while remaining forward-only. FLEA introduces \textit{layer-wise discriminative learning}, where each layer independently optimizes its parameters by aligning its feature embedding with the corresponding label embedding that maximizes class separability. This produces more discriminative representations in deeper layers and significantly narrows the performance gap with backpropagation.  
Experiments demonstrate that FLEA-equipped FF networks achieve competitive accuracy on complex visual benchmarks while retaining the memory efficiency and accelerator compatibility that make forward-only training appealing for resource-constrained systems.

TL;DR: We propose FLEA, a layer-wise embedding alignment block that lets forward-only networks scale to deeper models and achieve competitive accuracy while preserving their memory efficiency and hardware-friendly nature.