SPIN : Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification (2024)

Abstract: Among the many tasks that Large Language Models (LLMs) have revolutionized is text classification. Current text classification paradigms, however, rely solely on the output of the final layer in the LLM, with the rich information contained in internal neurons largely untapped. In this study, we present SPIN: a model-agnostic framework that sparsifies and integrates internal neurons of intermediate layers of LLMs for text classification. Specifically, SPIN sparsifies internal neurons by linear probing-based salient neuron selection layer by layer, avoiding noise from unrelated neurons and ensuring efficiency. The cross-layer salient neurons are then integrated to serve as multi-layered features for the classification head. Extensive experimental results show our proposed SPIN significantly improves text classification accuracy, efficiency, and interpretability.

## introduction large language models (llms) have achieved state-of-the-art performance in a wide spectrum of important tasks, including text classification such as sentiment analysis @xcite . although prompting methods @xcite have gained popularity in deploying llms for text classification, employing a classification head with these models remains a dominant paradigm, mainly due to its superior performance in specific tasks @xcite . this prevailing paradigm directly uses the terminal hidden states from models that are either pretrained on general tasks or fine-tuned for specific tasks. however, it is fundamentally limited in several ways. first, the implicit internal structures that contribute to llms' impressive performance is neglected, forgoing potential performance gains. second, achieving competitive performance often necessitates fine-tuning llms for the task at hand, which in turn can be computationally expensive. third, this approach inherently lacks interpretability, since it treats models as black boxes. as the demand grows for models that not only perform well but are also interpretable and cost-efficient to train and run, moving beyond the current paradigm is becoming increasingly important. we have reason to believe that delving into the internal of llms would bear fruit. as recent studies in ai interpretability @xcite @xcite have revealed, internal representations of artificial neural networks are remarkably adept at capturing essential features, yet the full potential of these insights in the realm of text classification awaits further exploration and demonstration. in this work, we introduce spin: a modelagnostic framework that sparsifies and integrates internal neurons of intermediate layers of llms for text classification. as shown in figure 1 , instead of relying solely on the final layer's hidden states, our method uses internal representations (feed-forward network activations and hidden states) as multilayered features to enhance the classification head. these raw internal representations require further processing before being utilized, as internal neurons do not contribute equally to predictions. irrelevant neurons that introduce noise and extraneous information can be counterproductive, potentially diluting the impact of crucial features. therefore, spin employs a linear probing based method to select salient neurons layer by layer, effectively sparsifying the internal representation. the selected neurons are then integrated across layer to serve as curated multi-layer features for text classification, ensuring that the textual features encompass a full spectrum from lower-level, simpler concepts to higher-level, more complex understandings across the hierarchical learning architecture of llms. spin presents multiple advantages that distinguish it in the realm of text classification with llms. primarily, its model-agnostic nature allows it to be employed upon various llms as a plug-andplay component. also, the use of curated internal representations as features enables spin to outperform conventional methods that rely solely on the terminal outputs. when applied on pretrained models, spin can achieve performance comparable to state-of-the-art baseline methods that involve fine-tuning llms, accomplished by significantly improved training efficiency. this is achieved by requiring only forward passes with llm weights untouched, and limiting trainable parameters to probing and the classification head, making spin a cost-effective alternative to fine-tuning for text classification. in terms of inference efficiency, spin enables early exiting, with up to 99% of the performance preserved from processing only 60% of llm layers, significantly speeding up the inference process. additionally, with its white-box approach of linear probing on internal neurons, spin enhances both intrinsic and post-hoc interpretability. our main contributions are summarized as: ## experiments we conduct extensive experiments to evaluate our proposed spin framework for text classification, focusing on three key dimensions: performance, efficiency, and interpretability. ## conclusion in this paper, we introduce spin, a novel modelagnostic plug-and-play framework designed for text classification tasks. our approach diverges from traditional paradigms that predominantly rely on the terminal hidden states of the final layer of llms by leveraging the untapped potential of internal neurons. our proposed framework sparsifies neurons from intermediate layers guided by linear probing-based selection, and integrates cross-layer salient neurons to provide rich and multi-layered features for text classification. we conduct comprehensive experiments and analysis, demonstrating that spin remarkably improves accuracy, efficiency, and interpretability for text classification.