Title: Darwinian Optimization: Training Deep Networks with Natural Selection
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Deep network optimization, natural selection, sample weighting, image classification, emotion recognition

Abstract: In conventional deep learning training paradigms, all samples are usually subjected to uniform selective pressure, which fails to adequately account for variations in competitive intensity and diversity among them. This often leads to challenges such as class imbalance bias, insufficient learning of hard samples, and improper handling of noisy samples. Drawing inspiration from the principles of species competition and adaptation in natural ecosystems, we propose a bio-inspired optimization method for deep networks, termed Natural Selection (NS). NS introduces a competition mechanism by stitching and scaling a group of samples before forward prediction. Each sample is then assigned a natural selection score based on its prediction, reflecting its competitive status within the group. This score is further used to dynamically adjust the loss weight of each sample, thereby forming an optimization process that more closely mimics a Darwinian ecological equilibrium. Experimental results on 12 public datasets consistently demonstrate that NS improves performance without being tied to specific network architectures or task assumptions. This study offers a novel perspective on deep network optimization and holds instructive significance for broader applications. The code will be made publicly accessible.