# Local Back-Propagation for Forward-Forward Networks: Independent Unsupervised Layer-Wise Training
## 1 Introduction  
Recently, large language models have demonstrated performance that even surpasses human capabilities [1]. Much of this success is due to the back-propagation algorithm (BP) [2, 3], supported by advancements in hardware technology. However, one criticism of BP is that it does not closely resemble the biological processes underlying brain function [4, 5, 6, 7]. Another drawback is that BP, by relying on backward passes, requires all layers of a model to remain continuously connected during training. These limitations have motivated researchers to explore alternative learning mechanisms that are more biologically grounded and resource-efficient [8, 9, 10]. Among such efforts, the Forward-Forward (FF) algorithm [10] represents a recent and influential attempt to train deep networks using only forward passes. FF avoids backward gradient propagation by comparing two forward activations (positive and negative) using a local loss. This design opens new possibilities for local, interpretable, and distributed learning. However, FF presents notable drawbacks: it requires specially constructed input variants (positive, negative and neutral), a non-standard loss function, and exhibits instability across datasets. These issues significantly limit its applicability in conventional machine learning workflows and hinder its scalability. Furthermore, its performance hasbeen shown to vary significantly across datasets and models, and its learning stability is limited. To address some of these limitations, subsequent studies have proposed several enhancements. For example, the Self-Contrastive FF [11] improves sample construction and stability by refining the positive/negative sample definitions. The Integrated FF [12] incorporates shallow gradient signals to stabilize training. The On-Tiny-Device FF [13] demonstrates FF's potential in constrained hardware settings by searching for FF-compatible initializations. However, these works still rely on the FF paradigm's core assumption of specialized inputs and retain its inherent training instability. Furthermore, they do not explore the broader class of unsupervised learning models that may better support local layer-wise learning. As shown in Figure 1, to overcome these gaps, we propose the Local Back-Propagation (LBP) algorithm, a forward-only training approach that replaces FF's layer with a compact unsupervised model such as an Auto-Encoder (AE), Denoising AE (DAE), Convolutional AE (CAE), or Generative Adversarial Network (GAN). Unlike FF, our method works with standard inputs and common reconstruction or classification losses, removing the dependency on handcrafted positive and negative examples. Each LBP layer, referred to as a 'cell' in this paper, is trained independently to reconstruct its input, and its latent output is passed forward to the next cell. This layer-wise unsupervised learning retains the local update advantage of FF while improving stability, expressivity, and compatibility with practical data pipelines. LBP is particularly well suited for federated and edge learning scenarios. Since each layer is trained independently, the entire model does not need to be loaded into memory  
Figure 1: (**a**) Original data used for the proposed Local Back-Propagation algorithm and the positive, negative, and neutral data used for the Forward-Forward algorithm. (**b**) Overview of Forward-Forward algorithm. (**c**) Overview of Local Back-Propagation algorithm. Forward-Forward and Local Back-Propagation models, and the data used in this models. during training. Layers can be trained sequentially using minimal memory, enabling the training of deep models on resource-constrained devices without requiring backward gradient computation or global synchronization. We evaluate the LBP framework across various unsupervised cell types and datasets (MNIST, CIFAR-10, and CIFAR-100), and we further assess its effectiveness in federated learning setups.