AlterMOMA: Fusion Redundancy Pruning for Camera-LiDAR Fusion Models with Alternative Modality Masking

Authors: shiqi  sun
yantao  lu
ning  liu
bo  jiang
jinchao  chen
ying  zhang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-11 12:39:07

Abstract: Camera-LiDAR fusion models significantly enhance perception performance in autonomous driving. The fusion mechanism leverages the strengths of each modality while minimizing their weaknesses. Moreover, in practice, camera-LiDAR fusion models utilize pre-trained backbones for efficient training. However, we argue that directly loading single-modal pre-trained camera and LiDAR backbones into camera-LiDAR fusion models introduces similar feature redundancy across modalities due to the nature of the fusion mechanism. Unfortunately, existing pruning methods are developed explicitly for single-modal models, and thus, they struggle to effectively identify these specific redundant parameters in camera-LiDAR fusion models. In this paper, to address the issue above on camera-LiDAR fusion models, we propose a novelty pruning framework Alternative Modality Masking Pruning (AlterMOMA), which employs alternative masking on each modality and identifies the redundant parameters. Specifically, when one modality parameters are masked (deactivated), the absence of features from the masked backbone compels the model to reactivate previous redundant features of the other modality backbone. Therefore, these redundant features and relevant redundant parameters can be identified via the reactivation process. The redundant parameters can be pruned by our proposed importance score evaluation function, Alternative Evaluation (AlterEva), which is based on the observation of the loss changes when certain modality parameters are activated and deactivated. Extensive experiments on the nuScene and KITTI datasets encompassing diverse tasks, baseline models, and pruning algorithms showcase that AlterMOMA outperforms existing pruning methods, attaining state-of-the-art performance.

Main paper: 
 introduction: 
 Camera-LiDAR fusion models are prevalent in autonomous driving, effectively leveraging the sensor properties, including the accurate geometric data from LiDAR point clouds and the rich semantic context from camera images [1, 2] , providing a more comprehensive understanding of the environment [3, 4] . However, the exponential increase in parameter counts due to fusion architectures introduces significant computational costs, especially when deploying these systems on resource-Figure 1 : Motivating example of fusion-redundant features in the 3D object detection task. We employ backward propagation on camera-LiDAR fusion models with pre-trained backbones to observe the gradient difference (features utilization) between with camera backbone only and with both the camera and LiDAR backbone. Notably, certain pre-trained parameters in the camera backbone are redundant due to the amendment of LiDAR information. It reveals that similar feature extraction exists across modalities, which introduces additional redundancy when camera-LiDAR fusion models directly loads single-modal pre-trained backbones. constrained edge devices, which is a crucial challenge for autonomous driving [5] . Network pruning is one of the most attractive methods for addressing the challenge above of identifying and eliminating redundancy in models. Existing pruning algorithms target single-modal models [6, 7, 8, 9, 10, 11, 12] or multi-modal models that merge distinct types of data [13, 14] , such as visual and language inputs. However, it's important to note that directly applying these algorithms to camera-LiDAR fusion models can lead to significant performance degradation. The degradation can be reasoned for two main factors that existing pruning methods overlooked: 1) the key fusion mechanism specific to vision sensor inputs within models, and 2) the training scheme where models typically load single-modal pre-trained parameters onto each backbone [2, 15] . Specifically, since single-modality models lack the cross-modality fusion mechanism, existing pruning algorithms traditionally do not consider intermodality interactions. Furthermore, because the pre-trained backbones (image or LiDAR) are trained separately, they are not optimized jointly, exacerbating the redundancy in features extracted from each backbone. Though leveraging pre-trained backbone improves the training efficiency compared with models training from scratch, we argue that directly loading single-modal pre-trained camera and LiDAR backbones into camera-LiDAR fusion models introduces similar feature redundancy across modalities due to the nature of the fusion mechanism.In detail, since backbones are independently pre-trained on single-modal datasets, they extract features comprehensively, which leads to similar feature extraction across modalities. Meanwhile, the fusion mechanism selectively leverages reliable features while minimizing weaker ones across modalities to enhance model performance. This selective utilization upon similar feature extraction across modalities introduces the additional redundancy: Each backbone independently extracts similar features, which subsequent fusion modules will not potentially utilize. For instance, both camera and LiDAR backbones extract geometric features to predict depth during pre-training. However, geometric features extracted from the LiDAR backbone are considered more reliable during fusion because LiDAR input data contain more accurate geometric information than the cameras, e.g., object distance, due to the physical properties of sensors. Consequently, this leads to the redundancy of geometric features of the camera backbone. In summary, similar feature extraction across modalities, coupled with the following selective utilization in fusion modules, leads to two counterparts of similar features across modalities: those utilized by fusion modules in one modality (i.e., fusion-contributed), and those that are redundant in the other modality (i.e., fusion-redundant). We also illustrate the fusion-redundant features in Figure 1 .To address the above challenge, we propose a novel pruning framework AlterMOMA, specifically designed for camera-LiDAR fusion models to identify and prune fusion-redundant parameters. AlterMOMA employs alternative masking on each modality, followed by observing loss changes when certain modality parameters are activated and deactivated. These observations serve as important indications to identify fusion-redundant parameters, which are integral to our importance scores evaluation function, AlterEva. Specifically, the camera and LiDAR backbones are alternatively masked. During this process, the absence of fusion-contributed features and relevant parameters in the masked (deactivated) backbone compels the fusion modules to reactivate their fusion-redundant counterparts from the other backbone. Throughout this reactivation, changes in loss are observed as indicators for contributed and fusion-redundant parameters across modalities. These indicators are then combined in AlterEva to maximize the importance scores of contributed parameters while minimizing the scores of fusion-redundant parameters. Then, parameters with low importance scores will be pruned to reduce computational costs.To validate the effectiveness of our proposed framework, extensive experiments are conducted on several popular 3D perception datasets with camera and LiDAR sensor data, including nuScenes [16] and KITTI [17] . These datasets encompass a range of 3D autonomous driving tasks, including 3D object detection, tracking, and segmentation.The contributions of this paper are as follows: 1) We propose a pruning framework AlterMOMA to effectively compress camera-LiDAR fusion models 2) we propose an importance score evaluation function AlterEva, which identifies fusion-redundant features and their relevant parameters across modalities 3) we validate the effectiveness of the proposed AlterMOMA on nuScenes and KITTI for 3D detection and segmentation tasks. 
related work: 
 Camera-LiDAR Fusion. With the advancement of autonomous driving technology, the efficient fusion of diverse sensors, particularly cameras and LiDARs, has become crucial [18, 19] . Fusion architectures can be categorized into three types based on the stage of fusion within the learning framework: early fusion [20, 21] , late fusion [22] , and intermediate fusion [2, 3, 15] . Current stateof-the-art (SOTA) fusion models evolve primarily within intermediate fusion and combine low-level machine-learned features from each modality to yield unified detection results, thus significantly enhancing perception performance compared with early or late fusion. Specifically, camera-LiDAR fusion models focus on aligning the camera and LiDAR features through dimension projection at various levels, including point [23] , voxel [24] , and proposal [3] . Notably, the SOTA fusion paradigm aligns all data to the bird's eye view (BEV) [2, 4, 15, 25, 26] , has gained traction as an effective approach to maximize the utilization of heterogeneous data types.Network Pruning. Network pruning effectively compresses deep models by reducing redundant parameters and decreasing computational demands. Pruning algorithms have been well-explored for single-modal perception tasks [27, 28, 29, 30, 31, 32] , focusing on evaluating importance scores to identify and remove redundant parameters or channels. These scores are based on data attributes [7, 33] , weight norms [34, 35] , or feature map ranks [28] . However, single-modal pruning algorithms are not suited for the complexities of camera-LiDAR fusion models. While some multi-modal pruning algorithms exist [13, 14] , they are mainly designed for models combining different data types like language and vision. Therefore, there is a pressing need for pruning algorithms specifically devised for camera-LiDAR fusion models. From the perspective of granularity, pruning algorithms can be divided into two primary categories: 1) structured pruning, which entails removing entire channels or rows from parameter matrices, and 2) unstructured pruning, which focuses on eliminating individual parameters. For practical applications, we have adapted our method to support both types of pruning. 
preliminaries: 
 We firstly review some basic concepts including camera-LiDAR fusion models and pruning formulation. Camera-LiDAR fusion models consist of 1) a LiDAR feature extractor F l to extract features from point cloud inputs, 2) a camera feature extractor F c to extract features from image inputs, 3) the fusion module and following task heads F f to get the final task results. The parameters denote as θ = { θ l , θ c , θ f } for LiDAR backbone, camera backbone, and fusion and task heads, respectively. Take camera backbone for instance, θ c = {θ 1 c , θ 2 c , ..., θ Nc c } denotes all weights in the camera backbone, where N c represents the total number of parameters in camera backbone. Therefore, for the LiDAR input X l and camera input X c , the training process of models could be denoted asEQUATIONwhere Y denotes the ground truth, and L represents the task-specific loss functions.Alternatively mask one of backbones (LiDAR or Camera)AlterEva update importance scores with two indictors: DeCI and ReRIParameters with low importance scores will be pruned, then model are finetuned. Importance-based pruning typically involves using metrics to evaluate the importance scores of parameters or channels. Subsequently, optimization methods are employed to prune the parameters with lower importance scores, that are nonessential within the model. For the camera-LiDAR fusion models, the optimization process can be formulated as follows:arg maxEQUATIONwhere δ ij is an indicator which is 1 if θ j i will be kept or 0 if θ j i is to be pruned. S is designed to measure the importance scores for parameters, and k represents the kept parameter number, where k = (1 -ρ) • i∈{l,c,f } N i with the pruning ratio ρ.Similar feature extraction across modalities, coupled with the selective utilization of features in the following fusion modules introduce redundancy in camera-LiDAR fusion models. Therefore, similar features and their relevant parameters can be categorized into two counterparts across modalities: those that contribute to fusion and subsequent task heads (fusion-contributed), and those that are redundant (fusion-redundant). In this section, we propose the pruning framework AlterMOMA, which alternatively employs masking on camera and Lidar backbones to identify and remove the fusion-redundant parameters. AlterMOMA is developed based on a novel insight: "The absence of fusion-contributed features will compel fusion modules to 'reactivate' their fusion-redundant counterparts as supplementary, which, though less effective, are necessary to maintain functionality." For instance, if the LiDAR backbone is masked, the previously fusion-contributed geometric features it provided are absent. To fulfill the need for accurate position predictions, the model still needs to process geometric features. Consequently, the fusion module is compelled to utilize the geometric features from the unmasked camera backbone, which were previously fusion-redundant. We refer to this process as Redundancy Reactivation. By observing changes during this Redundancy Reactivation, fusion-redundant parameters can be identified. The overview of AlterMOMA is shown in Figure 2 , and the detailed steps are in Algorithm 1 of Appendix D. The key steps are introduced as follows:Modality Masking. Three binary masks are denoted as µ l , µ c , and µ f ∈ {0, 1}, correspond to the parameters applied separately on the LiDAR backbone, the camera backbone, and the fusion and tasks head. Our framework begins by masking either one of the camera backbones or the LiDAR backbone. Here we take masking the LiDAR backbone as the illustration. The masks are with µ l = 0, µ c = 1, µ f = 1. The camera backbone will be masked alternatively.Redundancy Reactivation. To allow masked models to reactivate fusion-redundant parameters, we train masked models with batches of data. Specifically, B batches of data D i , i ∈ {1, 2, ..., B} are sampled from the multi-modal dataset D.Importance Evaluation. After Redundancy Reactivation, the importance scores of parameters in the camera backbones are calculated with our proposed importance score evaluation function AlterEva detailed in Section 3.3. Since fusion modules need to consider the reactivation of both modalities, the importance scores of parameters in the fusion module and task heads will be updated once the importance scores of both the camera and Lidar backbones' parameters are calculated.Alternative Masking. After Importance Evaluation of camera modality, models will reload the initialized parameters, and then the other backbone will alternatively be masked, with µ l = 1, µ c = 0, µ f = 1. Then the step Redundancy Reactivation and Importance Evaluation will be processed again to update the importance scores of parameters in the LiDAR backbone and the fusion module.Pruning with AlterEva. After evaluating the importance scores using AlterEva, parameters with low importance scores are pruned with a global threshold determined by the pruning ratio. Once the pruning is finished, the model is fine-tuned with the task-specific loss, as indicated in Eqn. 1.In this section, we will detail the formulation of our proposed AlterEva, which consists of two distinct indicators to evaluate the parameter importance scores. As outlined in section 3.2, the importance scores are alternatively calculated with AlterEva in Importance Evaluation. Then, parameters with low importance scores are removed in the pruning process. The goal of AlterEva is to maximize the scores of parameters that contribute to task performance while minimizing the scores of fusion-redundant parameters. To achieve this, AlterEva incorporates two key indicators: 1) Deactivated Contribution Indicator (DeCI) evaluate the parameter contribution to the overall task performance of the fusion models, 2) Reactivated Redundancy Indicator (ReRI) identifies fusion-redundant parameters across both modalities. Since changes in loss can directly reflect the parameter contribution difference to task performance during alternative masking, both indicators are designed based on the observation of loss decrease or increase, when certain modality parameters are activated or deactivated. Specifically, take parameters in the camera backbone as an instance, DeCI observe the loss increases with masking camera backbone itself, while ReRI observe loss decrease with masking LiDAR backbone and reactivating camera backbone via Redundancy Reactivation. Formally, we formulate the loss for the fusion models with masks. With three binary masks and the dataset defined in Section 3.2, the loss is denoted as follows, by simplifying some of the extra notations used in Eqn. 1:EQUATIONFor brevity, we assume µ c = 1, µ l = 1, and µ f = 1, and we only specify in the formulation when a mask is zero. For example, L m (µ c = 0; D) indicates that µ c = 0, µ f = 1 and µ l = 1. Since the alternative masking is performed on both backbones, we illustrate our formulation by calculating two indicators for parameters in the camera backbone.Deactivated Contribution Indicator. If a parameter is important and contributes to task performance, deactivating this parameter will lead to task performance degradation, which will be reflected in an increase in loss. Therefore, to derive the contribution of the i-th parameter θ i c of the camera backbone, we observe the changes in loss when this parameter is deactivating via masking, denoted as follows:EQUATIONwhere µ i c represents the mask for θ i c , and Φθ i c denotes the indicator DeCI for θ i c . However, the total number of parameters is enormous, deactivating and evaluating each parameter independently are computationally intractable. Therefore, we design an alternative efficient method to approximate the evaluation in Eqn. 4 by leveraging the Taylor first-order expansion inspired by [36] . We first observe the loss changes |L m (; D) -L m (µ c = 0; D)| by deactivating the entire camera backbones. Then, the first-order approximation of evaluation in Eqn. 4 is calculated by expanding the loss change in each individual parameter θ i c with Taylor expansion, considering θ c = {θ 1 c , ..., θ Nc c }. This method allows us to estimate the contribution for each parameter, denoted as follows:EQUATIONWhen θ c is deactivating with µ c = 0, µ i c = 0 for i ∈ {1, ..., N c }, which means that the last term of Eqn. 5 is zero. Meanwhile, when considering importance scores on a global scale, the L m (; D) and L m (µ c = 0; D) can be treated as constant for all θ i c . Thus the first term and the third term can be disregarded. Therefore, the final indicator of each parameter's contribution, represented by our DeCI, can be expressed as follows:EQUATIONThis formulation enables tractable and efficient computation without Modality Masking of the camera backbone itself, achieved by performing a single backward propagation in the Importance Evaluation with initialized parameters.Reactivated Redundancy Indicator. As discussed in Section 3.2, the identification of fusionredundant parameters relies on our understanding of the fusion mechanism: when fusion-contributed features from the LiDAR backbone are absent due to masking, the previously fusion-redundant counterparts and their relevant parameters from the camera backbone will be reactivated during the Redundancy Reactivation. Therefore, to reactivate and identify fusion-redundant parameters in the camera backbone, the Modality Masking of the LiDAR backbone (µ l = 0) and Redundancy Reactivation are processed first. Throughout this process, the loss evolves from L m (µ l = 0; D 1 ) to L m (µ l = 0; D B ), and the parameters evolve from θ c,0 (i.e. θ c ) to θ c,B . Similar to the formulation of DeCI, we observe the decrease in loss during Redundancy Reactivation and refer to this observation as our ReRI, denoted as follows:EQUATIONSpecifically, this process is designed to identify parameters that contribute to the task performance of models with the masked LiDAR backbone, highlighting those that are fusion-redundant. Since we want to observe reactivation rather than parameters updating of this masked model across training batches, we apply the first-order Taylor expansion to the initial i-th parameters θ i c,0 , denoted as:EQUATIONTo derive the gradient on initial parameters θ i c,0 of the last term, we could use the chain rule and write out based on the gradient of the last step,EQUATIONAccording to the Proposition A in the Appendix A, this approximation is reached by dropping some small terms with sufficiently small learning rates [9] . Since θ i c is activating with µ i c = 1, and the L m (µ l = 0; D) and L m (µ c = 0; D B ) can be treated as constant for all θ i c , we could denote our final formulation by simplifying Eqn. 8, and denoted as follow:EQUATIONwhere θ i c is the θ i c,0 and Φθ i c represent the ReRI for θ i c . To the goal of parameters with significant contributions maintaining high importance scores while those identified as fusion-redundant are assigned lower scores, AlterEva calculate the final importance scores by subtracting ReRI from DeCI. Since DeCI of parameters in the camera backbone could be calculated without masking the camera backbone itself, DeCI and ReRI of camera parameters could be calculated in the same alternative masking stage (with LiDAR backbone masking), which simplifies the process of our framework AlterMOMA. Therefore, with the combination Eqn. 6 and Eqn. 10, the AlterEva of the camera backbones could be presented with the normalization:EQUATIONwhere α and β are the hyper parameters and S(θ i c ) represent the importance score evaluation function AlterEva for θ i c . Similarly, the AlterEva of parameters in the LiDAR backbones (i.e. θ l ) and in the fusion modules (i.e. θ f ) could be derived as:EQUATIONEQUATIONwhere Φθ i l and Φθ i f (µ c = 0) is calculated when camera backbone is masking, while Φθ i f (µ l = 0) is calculated with LiDAR backbone masking. AlterEva can efficiently calculate importance scores with backward propagation, enhancing the tractability of AlterMOMA. For brevity, we omit the derivations related to parameters in the LiDAR backbone and fusion modules, but additional details are available in Appendix B and Appendix C. 
baseline models and datasets: 
 To validate the efficacy of our proposed framework, empirical evaluations were conducted on several camera-LiDAR fusion models, including the two-stage detection models AVOD-FPN [3] , as well as the end-to-end architecture based on BEV space, such as BEVfusion-mit [15] and BEVfusion-pku [2] . For AVOD-FPN, the point cloud input is processed using a voxel grid representation, while all input views are extracted using a modified VGG-16 [37] . Notably, the experiment on the AVOD-FPN demonstrates the efficiency of AlterMOMA on two-stage models, although this isn't the SOTA fusion architecture for recent 3D perception tasks. Current camera-LiDAR fusion models are moving towards a unified architecture that extracts camera and LiDAR features within a BEV space. Thus, our primary results focus on BEV-based unified architectures, specifically BEVfusion-mit [15] and BEVfusion-pku [2] . We conducted tests using various backbones. For camera backbones, we included Swin-Transformer (Swin-T) [38] and ResNet [39] . For LiDAR backbones, we used SECOND [18] , VoxelNet [40] and PointPillars [41] .We perform our experiments for both 3D object detection and BEV segmentation tasks on the KITTI [17] and nuScenes [16] , which are challenging large-scale outdoor datasets devised for autonomous driving tasks. The KITTI dataset contains 14,999 samples in total, including 7,481 training samples and 7,518 testing samples, with a comprehensive total of 80,256 annotated objects. To adhere to standard practice, we split the training samples into a training set and a validation set in approximately a 1:1 ratio and followed the difficulty classifications proposed by KITTI, involving easy, medium, and hard. NuScenes is characterized by its comprehensive annotation scenes, encompassing tasks including 3D object detection, tracking, and BEV map segmentation. Within this dataset, each of the annotated 40,157 samples presents an assemblage of six monocular camera images, adept at capturing a panoramic 360-degree field of view. This dataset is further enriched with the inclusion of a 32-beam LiDAR scan, amplifying its utility and enabling multifaceted data-driven investigations.We conducted the 3D object detection and segmentation experiments with MMdetection3D [42] on NVIDIA RTX 3090 GPUs. To ensure fair comparisons, consistent configurations of hyperparameters were employed across different experimental groups. To train the 3D object detection baselines, we utilize Adam as the optimizer with a learning rate of 1e-4. We employ Cosine Annealing as the parameter scheduler and set the batch size to 2. For BEV segmentation tasks, we employ Adam as the optimizer with a learning rate of 1e-4. We utilize the one-cycle learning rate scheduler and set the batch size to 2. The hyperparameters α and β in Section 3.3 are both set with 1. The baseline pruning methods include IMP [43] , SynFlow [44] , SNIP [35] , and ProsPr [9] , with the hyperparameters specified in their papers respectively. To evaluate the efficiency of AlterMOMA with unstructured pruning, we conduct experiments across multiple fusion architectures and datasets for 3D object detection and semantic segmentation. Specifically, to evaluate the efficiency of AlterMOMA on two-stage fusion architectures, we applied AlterMOMA to AVOD-FPN [3] , using the KITTI dataset with the task of 3D detection. Besides, BEVfusion-mit [15] and BEVfusion-pku [2] , as two representative camera-LiDAR fusion models with unified BEV-based architectures, are applied with AlterMOMA using the nuScenes dataset to validate the efficiency on both 3D detection and semantic segmentation tasks. Additionally, to validate the robustness of AlterMOMA with various backbones, we conducted experiments with alternative images and point backbone, including ResNet [39] and PointPillars [41] .3D Object detection on nuScenes with BEV-based fusion Architectures. The experimental results are presented in Table 1 . Note that baseline models are BEVfusion-mit trained with SwinT and VoxelNet backbone. As reported in Table 1 , single-modal pruning methods, including IMP, SynFlow, SNIP, and ProsPr, experience significant declines in accuracy performance. Even the ProsPr, considered the best-performing method among these single-modal pruning techniques, demonstrates the mAP decrease of 3.5% in accuracy at the 80% pruning ratio and 9.2% at the 90% pruning ratio on BEVfusion-mit. Conversely, the incorporation of our AlterMOMA yielded promising results. For example, comparing with the baseline pruning method ProsPr, AlterMOMA boosts the mAP of BEVfusion-mit by 3.0% (64.3% → 67.3%), 3.6% (61.9% → 65.5%), and 4.9% (58.6% → 63.5%) for the three different pruning ratios. Similarly, AlterMOMA obtains much higher mAP and NDS than the other four pruning baselines with different pruning ratios on BEVFusion-mit and BEVFusion-pku.3D Object detection on KITTI with the two-stage fusion architecture. To validate the efficiency of AlterMOMA on the two-stage detection fusion architecture, we conduct experiments with various pruning ratios on KITTI with AVOD-FPN architecture as the baseline. The experimental results are presented in Table 2 . Specifically, Table 2 presents the results for the car class on the KITTI, detailing AP-3D and AP-BEV across various difficulty levels including easy, moderate, and hard.Existing pruning methods experience significant declines in performance on different metrics of different difficulties. Even the best-performing method among single-modal pruning methods, ProPr, shows a decrease in AP-3D of 3.5%, 2.6%, and 4.4% in the easy, moderate, and hard difficulty levels, respectively, at the 80% pruning ratio. Conversely, the AlterMOMA has yielded promising results. For instance, comparing with the pruning method ProsPr, AlterMOMA enhances both the AP-3D and AP-BEV on AVOP-FPN by 3.2% (74.2% → 77.4%) and 3.9% (81.2% → 85.3%) for easy difficulties at the 90% pruning ratio. Furthermore, AlterMOMA consistently outperforms the other four pruning baselines across various difficulties and pruning ratios on AVOD-FPN. The comprehensive experimental results on 3D object detection validate the effectiveness of our AlterMOMA across different camera-LiDAR fusion architectures.To validate the robustness of our work, we extend our performance evaluation of AlterMOMA to the semantic-centric BEV map segmentation task. Note that baseline models are BEVfusion-mit trained with SwinT and VoxelNet backbone. We use the nuScenes dataset and utilize the BEV map segmentation validation set. The pivotal evaluation metric for this task is the mean Intersection over Union (mIoU). with the experimental configuration detailed in the work by [15] , we perform our evaluation on the BEVfusion-mit, as shown in Table 3 . We observed that existing pruning methods still meet a significant accuracy drop by 8.6% (IMP), 5.3% (SynFlow), 5.9% (SNIP), and 4.1% (ProsPr) for the 80% pruning ratio. Alternatively, our proposed approach yields significant advancements in performance. Specifically, compared with the ProsPr, AlterMOMA achieves a substantial enhancement by 3.0% (57.7% → 60.7%), 3.0% (56.2% → 59.2%), and 3.6% (54.1% → 57.7%) for the three different pruning ratios. These results empirically prove the efficacy of our pruning algorithms when applied to the BEV map segmentation task.To comprehensively assess the efficacy of AlterMOMA, we conducted experiments with alternative images and point backbones which will influence fusion. Specifically, we replaced the original VoxelNet backbone with PointPillar and the SwinT backbone with ResNet101 on the architecture of BEVFusion-mit. As depicted in Table 4 , the results obtained from these experiments consistently demonstrate state-of-the-art performance with various pruning ratios. Particularly noteworthy is the achievement of substantial 1.6%, 3.1%, and 4.2% improvement compared to the ProsPr baseline under the pruning ratio of 80%, 85% and 90%. This consistent improvement is observed across different pruning ratios, affirming the effectiveness of AlterMOMA with different backbones employed. These outcomes robustly demonstrate the general applicability of AlterMOMA to various backbone architectures.To assess the efficacy of our proposed pruning approach in structure pruning, we conducted experiments with BEVfusion-mit models with ResNet101 as the camera backbone and SECOND as the LiDAR backbones. Specifically, We measured the performance of the pruned networks with a similar amount of FLOP reductions and reported the number of FLOPs('GFLOPs'). As depicted in Table 5 , the results obtained from these experiments consistently demonstrate state-of-the-art performance with various pruning sparsities. Our evaluations at 30% and 50% pruning sparsities reveal that Alter-MOMA not only maintains a competitive mAP and NDS but also achieves a substantial reduction in computational overhead. Notably, with the 30% pruning sparsities, AlterMOMA achieves a 0.7% on mAP and 0.5% on NDS compared with the unpruned baseline models, which reveals that removing similar feature redundancy improves the efficiency of models. Specifically, compared with the ProsPr baseline, AlterMOMA achieves a substantial enhancement by 1.1% (64.6% → 65.3%), and 2.0% (62.5% → 64.5%), for the two different pruning sparsities. 
discussion and conclusion: 
 Although our approach identifies similar feature redundancy in camera-LiDAR fusion models, it is limited to the perception field. Extending it to other multi-modal models, such as visionlanguage models, requires further research. Fusion modules across various modalities exhibit different functionalities. In multi-sensor fusion models (camera, LiDAR, and Radar), the focus is on supplementing and spatially aligning data by leveraging the sensors' physical properties, fusing low-level features. However, in models with disparate data types like vision and language, fusion modules focus on matching high-level semantic contexts. Therefore, AlterMOMA primarily addresses redundancy from supplementary functionality in multi-sensor fusion perception architectures.In this paper, we explore the computation reduction of camera-LiDAR fusion models. A pruning framework AlterMOMA is introduced to address redundancy in these models. AlterMOMA employs alternative masking on each modality and observes loss changes when certain modality parameters are activated and deactivated. These observations are integral to our importance scores evaluation function AlterEva. Through extensive evaluation, our proposed framework AlterMOMA achieves better performance, surpassing the baselines established by single-modal pruning methods.A Derivation of Eqn.9 in Section 3.3Proposition. For a camera-LiDAR fusion model with parameters θ c for the camera backbone and parameters θ l for the LiDAR backbone, we can mask one of the backbones using masks µ l = 0 for the LiDAR backbone and µ c = 0 for the camera backbone. Take the models with masking LiDAR backbone as instance. With a sufficiently small learning rate ϵ, the masked model is trained with batches of data D i , i ∈ {1, 2, ..., B} sampled from the dataset D. We assume the parameters update from θ c,0 to θ c,B , and the loss changes from L m (µ l = 0) to L m (µ l = 0; D B ). Then we could get the equation denoted as:EQUATIONProof. We can extend the left side of the equation using the chain rule, denoted as,EQUATIONDue to the updates with the learning rate ϵ, we can represent θ i c,j as follows:EQUATIONCombining with Eqn. 15 and Eqn. 16, we could derive as following:EQUATIONwhere I represents the identity matrix, and ∂ 2 represents the second-order derivative. By dropping the terms with the sufficiently small learning rate ϵ inspired by [9] , the approximation is as follows:EQUATIONAnd we finally prove thatEQUATIONQuestion: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: All the data used is from public available datasets (NuScenes and KITTI). Please refer to Section 4.1.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Please refer to Section 4.1 and Section 4.2.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: The training procedure for the proposed and baseline models spend considerable amount of time. It is infeasible to perform enough repeated experiments to calculate the statistical significance. Therefore, error bars are not reported.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: We discuss the development of our approach in further multi-modal models, such as vison-language models in Appendix K. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Due to the page limits, in Section 3.3, we only introduce the detailed formulation of the camera backbone. In this section, we will complete the details of the AlterEva of parameters from the LiDAR backbone, with two indicators DeCI and ReRI.Deactivated Contribution Indicator. Similar to the parameters of camera backbones, the loss changes when parameters are deactivating via masking is observed, denoted as follows,EQUATIONDue to the enormous number of parameters, we generalize this deactivating approach to encompass the entire LiDAR backbone. We then denote the new evaluation function as follows:EQUATIONThen, to accommodate the need for differentiation among various parameters, we apply the Taylor first-order expansion to Φθ l on each individual parameter θ i l in the camera backbone, considering θ l = {θ 1 l , ..., θ N l c }.EQUATIONSince µ i l = 0 when θ i l is deactivating, and constant loss values can be disregarded when considering importance scores on a global scale, the final indicator of each parameter's contribution, represented by our DeCI, can be expressed as follows:EQUATIONReactivated Redundancy Indicator. Similarly, aiming to reactivate fusion-redundant parameters in the camera backbone, the camera backbone will first be masked with µ c = 0 and then masked models are trained with sampled batches. Throughout this process, the loss evolves from L m (µ c = 0; D 1 ) to L m (µ c = 0; D B ), and the parameters evolve from θ l,0 (i.e. θ l ) to θ l,B . The loss changes and parameter differences are observed via our ReRI, denoted as follows: Then, we apply the first-order Taylor expansion to the initial i-th parameters θ i l,0 , denoted as:EQUATIONAccording to Proposition A, we eliminate the identical parts and apply the known value of µ c = 1, and the final formulation could be denoted as,EQUATIONTherefore, with the combination Eqn. 23 and Eqn. 26, the final importance scores evaluation function AlterMOMA of the LiDAR backbones could be presented with a normalization:EQUATION)C Detailed AlterEva for parameters in the Fusion Modules and Following Task HeadsSimilar with Appendix B, in this section, we will complete the details of the AlterEva of parameters from the fusion backbone, with two indicators DeCI and ReRI.Deactivated Contribution Indicator. Similar to the parameters of the camera and LiDAR backbones, the loss changes when parameters are deactivating via masking is observed and then expand with a Taylor first-order expansion, denoted as follows,EQUATIONThen, after simplifying the constant loss and masked (deactivated) terms, formulation DeCI can be expressed as follows:EQUATION)Reactivated Redundancy Indicator. Different from the above camera backbone formulation in Section 3.3 and LiDAR backbone formulation in Appendix B, the fusion backbone experience the both alternative masking stages of AlterEva. Therefore, the ReRI of parameters in fusion modules and following task heads are calculated twice, denoted as,EQUATIONEQUATIONSpecifically, when loss evolves from L m (µ c = 0; D 1 ) to L m (µ c = 0; D B ) with masking camera backbone, the parameters evolve from θ f,0,µc=0 (i.e. θ f )) to θ f,B,µc=0 . Meanwhile, when loss evolves from L m (µ l = 0; D 1 ) to L m (µ l = 0; D B ) with masking backbone, the parameters evolve from θ f,0,µ l =0 (i.e. θ f ) to θ f,B,µ l =0 . Then, we apply the first-order Taylor expansion to the initial i-th parameters θ i l,0 , denoted as:EQUATIONEQUATION)According to Proposition A, we eliminate the identical parts and apply the known value of µ c = 1, and the final formulation could be denoted as,EQUATIONEQUATION)Specifically, since the ReRI are calculated twice for parameters in fusion models, we using the hyperparameters (β/2) to control the normalization scale of AlterMOMA of fusion modules. Formally, with the combination Eqn. 29 and Eqn. 34, the final importance scores evaluation function AlterMOMA of the LiDAR backbones could be presented with a normalization:EQUATION)The overview of our framework AlterMOMA and the importance scores evaluation function AlterEva are respectively introduced in Section 3.2 and Section 3. alternative masking on each modality, followed by the observation of loss changes when certain modality parameters are activated and deactivated. These observations serve as important indications to identify fusion-redundant parameters, which are integral to our importance scores evaluation function, AlterEva. AlterMOMA begins with Modality Masking, where one of the backbones is initially masked. This step is followed by Redundancy Reactivation and Importance Evaluation, where the parameter importance scores are initially calculated with AlterEva including the computation of DeCI and ReRI. Afterward, the models undergo Reinitialization and Alternative Masking of the other backbone, leading to another round of Redundancy Reactivation and Importance Evaluation. When scores of all parameters in backbones are calculated fully with AlterEva, models are pruned to remove parameters with low importance scores and then finetuned.We have also extended AlterMOMA to accommodate structured pruning, where instead of pruning individual weights, entire channels (or columns in linear layers) are removed. Although this approach imposes more restrictions, it significantly enhances memory efficiency and reduces the computational costs associated with training and inference. Adapting AlterMOMA to structured pruning involves simply modifying the shape of the pruning mask µ and the parameter θ in formulation to represent each channel (or column of the weight matrix). For experimental results, please refer to Section 4.4.As detailed in Section 4.1, we conducted experiments using AVOD-FPN [3] on the KITTI dataset [17] to demonstrate the efficacy of our AlterMOMA on two-stage fusion architectures. Due to space constraints, we initially presented only a subset of our results on the KITTI dataset in Section 4.3, specifically excluding the results for the Pedestrian and Cyclist classes. In this section, we aim to provide a comprehensive view of our findings on KITTI. The complete experimental results are displayed in Table 6 . Specifically, Table 6 discloses results for all classes (Car, Pedestrian, and Cyclist) on the KITTI dataset, detailing AP-3D and AP-BEV accuracy across varying difficulty levels, including easy, moderate, and hard. As reported in Table 6 , single-modal pruning methods, including IMP, SynFlow, SNIP, and ProPr, experience significant declines in accuracy performance in all classes. Conversely, the incorporation of our AlterMOMA yielded promising results.While our primary focus has been on camera and LiDAR modalities, we recognize that testing on additional modalities would make our proposed method more convincing. Therefore, we conducted further evaluations on the camera-radar modality, as presented in Table 7 . This includes a comparison of pruning results on the 3D object detection task using BEVFusion-R, a camera-radar fusion model with ResNet and PointPillars as backbones. We list the mAP and NDS of models at 80% and 90% pruning ratios. Compared to the baseline pruning method ProsPr, AlterMOMA boosts the mAP of BEVFusion-R by 2.7% and 2.8% for the two different pruning ratios. The results demonstrate our method's superior performance and generality across multiple modalities, including camera-radar.To further validate the generalizability of our method across different tasks, we conducted additional evaluations on multi-object tracking (MOT), a critical task in autonomous driving, as shown in Table 8 . We performed tracking-by-detection evaluation on the nuScenes validation dataset, and list the AMOTA of models pruned at 80% and 90% pruning ratios. The baseline model was trained with SwinT and VoxelNet backbones. Compared to the baseline pruning method ProsPr, AlterMOMA boosts the AMOTA of BEVFusion-mit by 1.9% and 3.1% for the two pruning ratios. These results further demonstrate our method's superior performance and generality across multiple tasks, including tracking.In this section, we report the running time in milliseconds in Table 9 . We tested the inference time of models after pruning using the structure pruning settings of the Table 5 of the main paper. Here are the performance results for BEVFusion-mit models with SECOND and ResNet101 backbones on an single RTX 3090: the inference times are 124.04 ms for unpruned models, 106.39 ms for AlterMOMA-30%, and 87.46 ms for AlterMOMA-50%. These results highlight the improvements in inference speed achieved through our pruning techniques. The Figure 3 illustrates the features of different modalities at each stage of the entire pruning process of AlterMOMA, including LiDAR features, camera features and fused features in the states of original (before masking), reactivated (after reactivation), and pruned (after pruning with AlterMOMA). The fourth column provides enlarged views of crucial redundant parts of the camera features. Notably, due to masking one side of backbones during reactivation, there are no fused features within reactivated states. Specifically, despite the absence of distant objects, camera features still provide some redundant depth features, as shown in the fourth column. These redundant parts of the original camera features are retained in the subsequent original fused features after fusion. To address these redundant depth features, AlterMOMA reactivates these redundant parameters during the reactivation phase, as shown in the middle row images of the fourth column. These redundant depth features are then pruned from both fused features and camera features, as observed by comparing the enlarged fused features in the middle row images of the second column and the pruned camera backbone features in the third and fourth columns of the third row. As described in Section 3.3, the hyperparameters α and β determine the proportion of DeCI and ReRI within the importance score evaluation function AlterEva. To assess the impact of these indicators on overall pruning performance, we examine the relationship between mAP and the ratio β/α. Baseline models depicted in the left subfigures of Figure 4 are from BEVfusion-mit trained with SwinT and VoxelNet backbones, while the right subfigures represent BEVfusion-pku models trained with the same backbones. These experiments utilize the nuScenes dataset to evaluate the efficiency of 3D object detection tasks at pruning ratios of 80%, 85%, and 90%. The experimental results are presented in Figure 4 . Specifically, when β/α = 0 in the figure, indicating only relying on DeCI, there is a significant drop in mAP compared to our best-performing setup. This result underscore the critical role of addressing fusion-redundant parameters. As the ratio increases, indicating greater influence from ReRI, mAP increases, reflecting the beneficial impact of effectively identifying and pruning fusion-redundant parameters. However, as β surpasses a certain threshold, resulting in ReRI outweighing DeCI, mAP begins to decline again. It may be due to that our redundancy reactivation also reactivates some contributed parameters, which may be accidentally pruned with the excessive usage of ReRI. These findings highlight the selection of hyper parameters and the ablation study for DeCI and ReRI indicators in AlterMOMA. The results are presented in Figure 4 , which visually depicts these dynamics across different experimental settings.Although AlterMOMA explores similar feature extraction due to the fusion mechanism, it remains in the perception fields, especially camera-LiDAR fusion models. However, extending it to a wider range of multi-modal models, such as vision-language models, requires further refinement. As we hypothesize, fusion modules across various modalities and tasks exhibit different functionalities. In perception-only tasks involving multiple sensors (camera, LiDAR and radar), the fusion mechanism primarily focuses on supplementing and spatial aligning across modalities by fully leveraging the physical properties of different sensors since all inputs consist of vision-based data. Formally, lowlevel machine features are fused in the multi-sensor fusion mechanism. However, in the fusion mechanism devised for different types of data, such as vision and language, things differ. When input data is highly disparate, such as vision and language, fusion modules tend to focus more on matching, which means synchronizing the high-level semantic context between different input formats. Therefore, for the AlterMOMA framework, we concentrate on the redundancy stemming from supplementary functionality, which may only primarily exist in multi-sensor fusion architectures. 


Decision: he authors propose a novel pruning framework, AlterMOMA, which employs alternative masking to identify and prune redundant parameters in camera-LiDAR fusion models. The paper demonstrates the effectiveness of AlterMOMA through extensive experiments on the nuScenes and KITTI datasets, showing superior performance compared to existing pruning methods. The initial reviews consisted of two accepts and one borderline reject. The concerns raised were about the generalizability to additional modalities and tasks, as well as the running speed after pruning. In the rebuttal, the authors explained why camera-LiDAR is key for 3D object detection and demonstrated that the proposed method indeed speeds up the runtime after pruning. The final reviews resulted in two accepts and one weak accept. Therefore, the AC recommends acceptance.

Reviews: review: paper_summary: paper_summary: This paper introduces a novel and effective pruning framework (AlterMOMA) and outperforms existing pruning methods on multimodal 3D detection/segmentation tasks, based on a novel insight : "The absence of fusion-contributed features will compel fusion modules to ’reactivate’ their fusion-redundant counterparts as supplementary to maintain functionality".
strength_weakness: strength_weakness: - while AlterMOMA aims to reduce model complexity, the paper could clarify the computational overhead introduced by the pruning process itself, especially regarding the training time and resources required for reactivation and evaluation.
- it is interesting to use some more direct methods (e.g., feature visualization) to prove that the "reactivated" parameters are "redundant".
- some minor typing errors (e.g., line132, an incorrect quotation mark), which do not affect my rating
questions: questions: the paper presents a pruning strategy for multimodal strategy, which seems to work on any multimodal task. However, the authors only experimented on the Camera-LiDAR task. Did authors try more modal combinations (Radar, or even Language, etc.)? Is the approach applicable to other broader tasks?
limitations: limitations: the paper discussed the limitations in the Appendix.
paper_summary: paper_summary: This paper addresses the problem of feature redundancy in camera-LiDAR fusion models. The authors propose a novel pruning framework, AlterMOMA, which employs alternative masking to identify and prune redundant parameters in these models. The paper demonstrates the effectiveness of AlterMOMA through extensive experiments on the nuScenes and KITTI datasets, showing superior performance compared to existing pruning methods.
strength_weakness: strength_weakness: 1. While the method is tested on two camera-LiDAR datasets, the generalizability of the approach to other fusion tasks and modalities beyond camera-LiDAR is not verified, such as video, optical flow and audio in action recognition [A].
2. The running speed after pruning is not reported
3. In Table 5, the performance after pruning is even better than the original version. Are there any reasons and analysis?

[A] Munro et al., Multi-modal domain adaptation for fine-grained action recognition. In CVPR, 2020.
questions: questions: Can the proposed method be effective on other tasks like action recognition?
limitations: limitations: The paper mentions one limitation regarding its limited use in the perception field.
paper_summary: paper_summary: This paper introduces Alternative Modality Masking Pruning (AlterMOMA), a pruning framework utilizing alternative masking on each modality to pinpoint redundant parameters. It detects redundant features and relevant parameters through the reactivation process. These redundant parameters are then pruned using a proposed importance score evaluation function. Meanwhile, the observation of loss changes indicates which modality parameters are activated or deactivated.
strength_weakness: strength_weakness: The paper lacks an illustration of the costs associated with the proposed pruning method itself.
questions: questions: How many rounds does "Modal masking-Redundant activation-importance score evaluation" take?
limitations: limitations: Experiments indicate that the proposed Pruning method is effective. However, the effectiveness of other multi-modal fusion methods, such as VLM, still needs to be verified.

Comments: comment: Thank you for your insightful reviews and for maintaining your initial score. We will address your feedback and include the analysis of the computational overhead of the pruning process in the further camera-ready version. We greatly appreciate your valuable insights and the time you’ve dedicated to evaluating our work.
Thanks for the authors' rebuttals, the illustration is clear. I will keep my initial score of 7 (Accept).
Thank you for your thoughtful comments and for maintaining your initial score. We appreciate your recognition of the additional experiment results we included during the rebuttal period. We will ensure that these additional results are included in the supplementary material, and we will revise the tables in the manuscript to maintain a consistent font size for better readability, avoiding the use of \resizebox. We sincerely appreciate your feedback and are grateful for your time in evaluating our work.
The authors addressed most of my questions. It is surprising that the authors could add so many experiment results in the short rebuttal period. Based on this, I will maintain my initial score (7/Accept).

Meanwhile, I would like to suggest the authors (1) make sure these additional experiment results can be added to the supplementary material, and (2) make sure the tables in the manuscript have the same/similar font size for better view (try not use \resizebox).
Thank you for considering our rebuttal and increasing our score. We sincerely appreciate your feedback and are grateful for your time in evaluating our work.
Thank the authors for their rebuttal. Most of my concerns have been well-addressed and I thus increased my score to 6.