Unveiling Multimodal Processing: Exploring Activation Patterns in Multimodal LLM s for Interpretability and Efficiency (2025)

Abstract: Recent Multimodal Large Language Models (MLLMs) have achieved remarkable advancements, yet their internal mechanisms for concurrently processing diverse modalities like text, image, and audio remain largely opaque. In this paper, we propose a methodology to convert dense MLLMs into fine-grained Mixture-of-Experts (MoE) architectures. This allows us to visually investigate their multimodal activation patterns through expert activation frequency heatmaps. Conducting comprehensive experiments on representative MLLMs, we analyze the similarities and differences in internal neuron activations when handling distinct modalities. Specifically, we examine the distribution of high-frequency activated experts, the distinct roles of high-frequency (e.g., fundamental logic) and low-frequency (e.g., domain-specific concepts) multimodal shared experts, and the prevalence and localization of modality-specific experts. Furthermore, we explore leveraging these discovered activation discrepancies to guide sparse activation and model pruning. Experimental results demonstrate that our approach substantially outperforms random expert pruning and can achieve comparable or even superior performance to the original unpruned models while utilizing significantly fewer active parameters. Our work not only sheds light on the multimodal processing mechanisms within MLLMs but also provides a practical pathway toward developing more interpretable and efficient multimodal systems.

## introduction large language models (llms) have demonstrated extraordinary capabilities in processing and generating human language, catalyzing a new era in natural language processing @xcite . the frontier has rapidly expanded towards multimodal llms (mllms), which integrate capabilities for understanding and reasoning * corresponding author. however, despite their impressive performance, a fundamental question remains largely unanswered: how do these mllms process and integrate information from disparate modalities within a unified architectural framework? while an llm's core is trained on text, its adaptation to handle visual or auditory signals introduces new complexities. previous studies in neuroscience suggest that while the multilingual llms exhibit considerable overlap in regions processing different languages, discernible specializations also exist @xcite . analogously, we hypothesize that mllms might develop both modality-agnostic (shared) computational pathways and modality-specific ones. uncovering these internal mechanisms is crucial not only for advancing our theoretical understanding but also for addressing practical challenges. mllms, inheriting the scale of their llm parents, are computationally intensive, making deployment and inference costly @xcite . a deeper insight into their internal workings could pave the way for more efficient model designs, targeted pruning strategies, and improved interpretability. currently, we lack an intuitive, fine-grained understanding of the internal neuron-level activity within mllms as they process different modalities-the system largely remains a "black box." this opacity hinders our ability to diagnose failures, improve robustness, and optimize performance efficiently. therefore, we aim to investigate the distinct and shared internal neuron activation patterns of mllms when confronted with inputs from various modalities. we term these phenomena "multimodal activation patterns." to demystify this black box and gain an intuitive understanding, we devise a methodology inspired by recent explorations in multilingual llms @xcite . our approach involves converting standard, dense mllms into fine-grained mixtureof-experts (moe) architectures @xcite . this transformation allows us to treat groups of neurons as "experts" and subsequently calculate their activation frequencies when processing data from different modalities (e.g., text-only, image-only, audio-only, or combined inputs). these frequencies are then visualized as heatmaps, offering a global view of expert utilization across layers and modalities. through comprehensive experiments on leading mllms, we analyze: (1) the distribution of highfrequency activated experts across different modalities. (2) the existence and functional roles of multimodal shared experts-those frequently activated regardless of input modality, potentially handling core reasoning or abstract concepts. we further differentiate these by activation frequency, hypothesizing that high-frequency shared experts manage fundamental cross-modal logic, while lowerfrequency shared experts might capture more nuanced, less common inter-modal abstractions. (3) the characteristics and distribution of modalityspecific experts-those predominantly activated by a single modality (e.g., visual experts, textual experts, auditory experts). furthermore, we explore their practical application in guiding sparse activation and model pruning. by identifying which experts are critical for specific modalities or general reasoning, we propose methods to selectively activate or retain only the most relevant experts during inference. this aims to significantly reduce computational load while preserving, or even in some cases enhancing, model performance on specific tasks. our preliminary findings indicate that this informed approach can substantially outperform random pruning baselines. in this work, we make the following contributions: our work builds upon several interconnected lines of research: understanding internal mechanisms of large language models, the development and analysis of mllms, and techniques for model compression and efficient inference, particularly through sparsity. the quest to understand the inner workings of llms has gained significant traction. much of this research has focused on monolingual or multilingual textual llms. for instance, studies have investigated how llms represent linguistic structures @xcite , store factual knowledge @xcite , and perform reasoning @xcite . more recently, inspired by neuroscience, researchers have begun to explore language-specific neurons or expert-like structures within llms @xcite . @xcite provided a direct inspiration by converting dense llms into moe architectures to study multilingual activation patterns, demonstrating the existence of language-specific and shared experts. for mllms, while their architectures combining vision/audio encoders with llms are well-documented @xcite , fine-grained analysis of how different modalities are processed and integrated at the neuron or expert level is still nascent. early work has explored cross-modal attention mechanisms @xcite or attempted to localize concepts across modalities @xcite , but a systematic, layerwise, and expert-level understanding of activation patterns across multiple modalities, akin to what we propose, remains less explored. our work extends the moe-based activation analysis from the multilingual to the multimodal domain, aiming to uncover similar notions of shared and modalityspecific computational units. moes @xcite have emerged as a promising approach to scale up model capacity while keeping computational costs manageable by sparsely activating only a subset of "experts" per input. while initially prominent in llms, their application and analysis in the multimodal context are growing. some mllms have started to incorporate moe layers explicitly in their design for efficiency @xcite . however, much of the existing mllm landscape still relies on dense architectures. @xcite demonstrated that pre-trained dense transformer ffn layers can be post-hoc converted into moe structures without significant performance loss, providing a powerful tool for analysis and potential efficiency gains. our methodology leverages this moefication concept, not primarily for pre-training efficient mllms, but as an analytical lens to decompose existing dense mllms and study their internal multimodal specialization. subsequently, the identifying frequently or infrequently activated experts per modality can inform strategies for sparse activation during inference or targeted pruning @xcite . unlike works that design moe mllms from scratch, we focus on understanding and re-purposing existing dense models. ## exploring multimodal activation patterns in mllms to investigate how mllms internally process and integrate information from diverse modalities, we propose a methodology centered around converting dense mllm architectures into fine-grained mixture-of-experts (moe) structures. this allows us to analyze the activation patterns of these "experts" in response to unimodal and multimodal inputs. ## conclusion in this paper, we systematically investigated the internal multimodal activation patterns of mllms by transforming dense architectures into fine-grained moe structures and visualizing expert activation frequencies. our analysis revealed that a small core of high-frequency shared experts, concentrated in deeper layers, appears crucial for fundamental cross-modal reasoning large contingent of lowfrequency shared experts, often in middle layers, likely handles more nuanced multimodal integration and abstract concept representation. we also characterized modality-specific experts, predominantly vision-focused and active in shallower layers, underscoring early-stage unimodal processing. crucially, we demonstrated that these identified activation patterns can effectively guide sparse activation and model pruning strategies. our functionbased and frequency-sorted pruning methods significantly outperformed random baselines and often matched or even surpassed the original model performance with substantially reduced computational costs. these findings not only offer novel insights into the "black box" of mllm multimodal processing but also provide a practical pathway towards developing more interpretable and efficient mllms.