Unlocking Emergent Modularity in Large Language Models (2024)

Abstract: Modular Neural Networks (MNNs) demonstrate various advantages over monolithic models.Existing MNNs are generally \textit{explicit} : their modular architectures are pre-defined, with individual modules expected to implement distinct functions.Recent works reveal that there exists \textit{implicit} modularity in standard pre-trained transformers, namely \textit{Emergent Modularity} .They indicate that such modular structures spontaneously exhibit during the early pre-training phase.Despite the benefits of modularity, most Language Models (LMs) are still treated as monolithic models in the pre-train and fine-tune paradigm, with their emergent modularity locked and underutilized.In this work, focusing on unlocking the emergent modularity in LMs, we showcase that standard LMs could be fine-tuned as their Mixture-of-Expert (MoEs) counterparts without introducing any extra parameters. Such MoEs are derived from emergent modularity and are referred to as Emergent MoEs (EMoE).Our experiments demonstrate that fine-tuning EMoE effectively improves downstream in-domain and out-of-domain generalization compared with vanilla fine-tuning.Our analysis and ablation studies further illustrate that it is robust to various configurations and can scale up to Large Language Models (i.e., Llama2-7B and Llama-30B). Code is available at https://github.com/qiuzh20/EMoE.

## introduction modularity attracts considerable attention from the artificial intelligence community @xcite . neural networks with modular designs, termed modular neural networks (mnns), have exhibited a wide range of advantages, including adaptation @xcite , data efficiency @xcite , and better generalization abilities @xcite ., * equal contribution † work done while interning at hkust. ‡ corresponding author 2022 ). typical mnns are usually explicitly modular. they have a pre-defined modular structure and are expected to achieve a divide-and-conquer solution for the given task. among various mnns, mixture-of-experts (moes) employ a conditional computation strategy where different submodulesso-called experts -are expected to be activated by different types of inputs. moes see substantial success in various domains @xcite @xcite @xcite in the era of large-scale transformers, and they are therefore a widespread modular neural architecture. apart from explicit mnns, some research finds that modular structures spontaneously emerge during training, not only in small-scale cnns or lstms @xcite , but also large-scale pre-trained transformer models. @xcite ; @xcite reveal notable sparse activation patterns within the feed-forward networks (ffns) in pre-trained transformer models. they find that in t5-base @xcite and vit-b16, only 3.0% and 6.3% neurons are activated during one forward process, respectively. furthermore, @xcite utilize handpicked semantic and knowledge-intensive tasks to probe the nature of neurons in ffns. they observe a strong correlation between neuron activation and specific tasks, further discovering clear function-based neuron grouping of the pretrained t5 model (neurons with similar functions are usually co-activated). they summarize such phenomenon as emergent modularity (em). though modularity emerges, pre-trained language models are generally treated as monolithic models in the standard pre-train and fine-tune paradigm. it is natural to ask whether their em and the potential improvements brought by em are locked in this process. in this paper, we advocate unlocking the em in the pre-trained language models could bring gen-eralization improvements for downstream tasks. specifically, we split certain ffns layers of the original model into moes layers. the moes is derived according to the em in that layer and can be regarded as the externalization of the em. hence, the obtained moes model is called emergent moes @xcite . we then fine-tune the obtained emoe model to investigate whether unlocking em encourages downstream task performance. we validate our empirical findings with various models, evaluation benchmarks, fine-tuning methods (parameter-efficient tuning and full finetuning). we find that fine-tuning emoe achieves stronger generalization performance than vanilla fine-tuning across various experimental settings, demonstrating that unlocking the em of lms boosts the models' downstream generalization abilities. we provide a comprehensive analysis for emoe: 1) we first validate that emoe indeed unlocks em in pre-trained language models by showcasing its task-specific expert choice. 2) we then reveal that emoe ameliorates the parameter updating during fine-tuning and can even be abandoned afterward. we want to highlight that this property further improves the practicality of emoe as the model architecture does not change before and after fine-tuning. meanwhile, our ablation studies show the emoe's robustness to various hyper-parameter configurations. 3) we also conclude that emoe could mask neurons with negative transfer effects. we hope our research discoveries can bring novel insights and serve as an example attempt towards further unlocking the em of llms. more precisely, for each column k :,i and row v i,: , eq.1 can be rewritten as: following @xcite ; @xcite , we regard columns in k as key vectors and rows in v as value vectors, the output of an ffns network can be viewed as a weighted sum of value vectors based on the activation scores σ(x • k). for the rest of this paper, we refer to one key-value memory pair using neuron and the co-activation of neurons using modularity. mixture-of-experts in transformers, moes is often implemented by replacing the original ffns with a group of parallel ffns and introducing a gating module. supposing there are n experts: {ffn n (•; k n , v n )| @xmath0, n ]}, the gating module g(• ; g, k), defined with its parameters g and an integer k, is to map input x to a score distribution of experts g(x; g, k) ∈ r n . typically, g is implemented with a simple linear layer followed by a softmax function and a top-k function. given @xmath1, the output @xmath2: when k for top-k is smaller than n , only a subgroup of experts is involved in the computation, termed sparse moes. ## ablation studies sparsity and modularity to provide further evidence that the emoe's improvements stem from leveraging modular features rather than just sparse activation or moes architecture, we compare the results of ( 1 ) key-vector-based clustering expert construction and (2) random construction. we employ the same setting of sec. 4.2. the relative changes in averaged outcomes compared to the vanilla fine-tuning are shown in tab. 4, while full results can be found in the appendix f.1, tab. 17. it's noteworthy that while cluster top-k exhibits a significant improvement over the standard, random top-k is conversely worse than vanilla fine-tuning. this suggests that random construction can negatively impact gating, and only moes structure itself can't bring improvement. moreover, when selecting weights with negative transfer under bottom-k selections, it's observed that cluster bottom-k also achieves lower results. in summary, clusteringbased methods can externalize implicit modu-larity within the pre-trained language model. within suitable frameworks like moes, such modularity can facilitate downstream tuning. ## related work works most related to our work is introducing modularity based on off-the-shelf pre-trained models. for example, gmoe @xcite and upcycling @xcite ) copy the ffns from a trained transformer model to form the moes architecture. their modular structure is introduced by replicating existing ffns modules, leaving em within pre-trained ffns unexplored. moefication @xcite and moebert @xcite explores the em within the model. they seek to improve the inference efficiency by decomposing the original ffns layer into sparse moes. they utilize the sparse expert activation to reduce inference overhead and do not touch on how em influences the model's performance in downstream fine-tuning. our method to split ffns is adopted from one simple method in the ablation studies of moefication paper @xcite : clustering-based expert construction and avg-k gating. we empirically find that such a simple method can validate the improvements brought by em and answer our research questions. we thus leave more elaborated methods for future works. please refer to appendix c for a detailed comparison of emoe with those related works. ## conclusion in this work, we validate that unlocking the em in standard lms improves downstream task id and ood performances. emoe can bring this benefit without adding any parameters, significant training costs, or any alterations to its deployment, which improves its practicability in the llms era. one possible reason is the modular structure can allevi-ate negative transfer effects presented in the lms. we hope our findings can deepen the understanding of neural networks' modularity, further helping the community develop more sophisticated modular neural architectures and utilize existing lms.