Global-Pruner: A Stable and Efficient Pruner for Retraining-Free Pruning of Encoder-Based Language Models (2024)

Abstract: Large language models (LLMs) have achieved significant success in complex tasks across various domains, but they come with high computational costs and inference latency issues. Pruning, as an effective method, can significantly reduce inference costs. However, current pruning algorithms for encoder-based language models often focus on locally optimal solutions, neglecting a comprehensive exploration of the global solution space. This oversight can lead to instability in the solution process, thereby affecting the overall performance of the model. To address these challenges, we propose a structured pruning algorithm named G-Pruner (Global Pruner), comprising two integral components: PPOM (Proximal Policy Optimization Mask) and CG²MT (Conjugate Gradient Squared Mask Tuning), utilizing a global optimization strategy. This strategy not only eliminates the need for retraining but also ensures the algorithm’s stability and adaptability to environmental changes, effectively addressing the issue of focusing solely on immediate optima while neglecting long-term effects. This method is evaluated on the GLUE and SQuAD benchmarks using BERTBASE and DistilBERT models. The experimental results indicate that without any retraining, G-Pruner achieves significant accuracy improvements on the SQuAD _{2.0} task with a FLOPs constraint of 60%, demonstrating a 6.02% increase in F1 score compared with baseline algorithms.

## introduction in recent years, transformer-based pre-trained language models (plms) @xcite ; @xcite ; @xcite ; @xcite ; @xcite have dominated the field of natural language processing (nlp) @xcite 2024 ) aiming to improve efficiency by eliminating redundant parts of the model. particularly, structured pruning has become a key technology for addressing size and speed issues in encoder-based language models, systematically removing redundancies without significantly impairing model performance. despite this, existing pruning methods still have limitations in practical applications. for example, @xcite avoided the high costs associated with retraining by employing three techniques: mask search, mask rearrangement, and mask tuning. however, this greedy-based pruning method has been proved to be effective only in the short term and faced challenges in finding global optima, particularly when applied to complex or dynamically changing tasks. moreover, the k-prune @xcite algorithm aimed to minimize pruning errors and enhance accuracy by preserving knowledge from pre-trained models. however, it did not fully consider the accuracy of weight selection and the long-term stability of the pruning strategy. similarly, the @xcite framework could quickly compress models and minimize performance loss by accurately assessing the importance of neurons in the short term. however, it overlooked the long-term stability and adaptability of the model to complex tasks, especially under high flops constraints. although these pruning methods can enhance the efficiency of models in the short term, they typically have a common drawback: they primarily focus on finding local optima and neglect the exploration of global optima. to address these issues, we introduce a new retraining-free pruning framework for transformer models-g-pruner, efficiently locating global optima quickly without retraining. this strategy integrates two advanced technologies: ppom and cg²mt. in the ppom phase, the algorithm first conducts a comprehensive mask search, then finetunes and optimizes the selected masks using the ppo (proximal policy optimization) technique from reinforcement learning. subsequently, in the cg²mt phase, we enhance the efficiency and stability of solving asymmetric matrix problems through an improved cgs (conjugate gradient squared ) solver. the primary contributions of this study include: ## background and baseline description the core of the pruning problem is to find the optimal methods for masking while considering sparsity constraints. this study focuses on the com-pression of encoder-based language models, notably bert base and distilbert. these encoderbased language models consist of two primary sublayer archetypes: multi-head attention (mha) and feed-forward network (ffn). in this section, we explain how to mask the attention heads and feed-forward networks. ## conclusion in this work, we introduce a structured pruning algorithm named g-pruner, which achieves highprecision pruning without the need to retrain transformer models. by incorporating two novel techniques, ppom and cg²mt, we effectively address the shortsightedness problem commonly encountered in traditional methods when assessing the importance of attention heads and feed-forward neural networks. simultaneously, our approach significantly optimizes the iterative process, reducing numerical instability during computation and achieving faster convergence. under the same flops constraints, g-pruner significantly outperforms all existing pruning techniques in pruning time without sacrificing model accuracy.