Title: Layer-Wise Modality Decomposition for Interpretable Multimodal Sensor Fusion
Venue: NeurIPS 2025
Accepted: yes
Keywords: Interpretable AI

Deep Taylor Decomposition

BEV Perception
Abstract: In autonomous driving, transparency in the decision-making of perception models is critical, as even a single misperception can be catastrophic. Yet with multi-sensor inputs, it is difficult to determine how each modality contributes to a prediction because sensor information becomes entangled within the fusion network. We introduce Layer-Wise Modality Decomposition (LMD), a post-hoc, model-agnostic interpretability method that disentangles modality-specific information across all layers of a pretrained fusion model. To our knowledge, LMD is the first approach to attribute the predictions of a perception model to individual input modalities in a sensor-fusion system for autonomous driving. We evaluate LMD on pretrained fusion models under camera–radar, camera–LiDAR, and camera–radar–LiDAR settings for autonomous driving. Its effectiveness is validated using structured perturbation-based metrics and modality-wise visual decompositions, demonstrating practical applicability to interpreting high-capacity multimodal architectures. Code is available at https://github.com/detxter-jvb/Layer-Wise-Modality-Decomposition.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Layer-Wise Modality Decomposition (LMD), a post-hoc interpretability method designed to disentangle modality-specific contributions in pretrained multimodal sensor fusion models for autonomous driving. LMD enables attribution of predictions to individual input modalities like camera and radar or camera and LIDAR across all layers without altering the model architecture.\n\nThe method locally linearises neural network operations to separate modality-specific features and bias, drawing inspiration from techniques like Layer-Wise Relevance Propagation (LRP) and Deep Taylor Decomposition (DTD). LMD’s effectiveness is evaluated through perturbation-based evaluations and visual decompositions by modality, showcasing its ability to clearly distinguish each modality’s role in a complex multimodal model.\n\nExperiments show LMD successfully identifies modality-specific influences, with radar-based predictions responding strongly to radar input changes while camera-based predictions remain stable, and vice versa. While focused on autonomous driving, the paper highlights LMD’s applicability beyond that application, offering a scalable solution for interpreting complex multimodal perception models without imposing constraints on the architecture.\n\nSTRENGTHS AND WEAKNESSES:\nClarity: The paper is well-structured, with clear sections that motivate the problem and introduce the proposed Layer-Wise Modality Decomposition (LMD) approach well. Technical concepts are explained with sufficient detail, supported by references to foundational works, and the contributions introduced are contextualised well relative to previous research. While code wasn't available for review, the promise of making code available for the model and experiments would further support the reproducibility, and enable researchers to more easily validate and build upon the method.\n\nExperiments and evaluation: the paper provides validation of their method through the perturbation-based metrics and modality-wise visual decompositions (as seen in the evaluation of camera-radar and camera-LIDAR setting). The perturbation-based metrics and visualizations are well designed to test the method’s core claim of modality separation, and support LMD’s claims. However, comparisons with more baselines and broader modality testing could enhance credibility. In general, the core set of experiments chosen were sensible but quite limited in scope. Broadening this could significantly improve the strength of the paper.\n\nSignificance and Impact: The paper addresses a critical gap in interpretable AI for multimodal perception models, particularly in autonomous driving, where transparency in decision-making is vital for safety and reliability. The model-agnostic nature and applicability to high-capacity architectures make it broadly relevant, with potential impact beyond autonomous driving. On the other hand, the lack of non-driving examples and real-world integration details limits its demonstrated impact. In addition, there are limited modalities explored: the evaluation focuses on camera-radar and camera-LIDAR settings, but the paper does not discuss whether other modality combinations were tested, limiting the generality of conclusions that can be drawn.\n\nQUESTIONS:\n- The paper does not discuss limitations or failure cases of LMD. This seems like a crucial missing element. Can you share scenarios where LMD might struggle (e.g., highly correlated modalities, extreme edge cases) and how you plan to address these in future work? \n- Have you considered benchmarking LMD against other post-hoc interpretability methods (e.g. Grad-CAM, SHAP) to further validate its advantages in multimodal settings? This would really help to contextualise the work. \n- The paper claims LMD’s applicability extends beyond autonomous driving (Page 1), and sensor fusion has much broader applicability beyond AV.  However, no specific examples or experiments in other domains are provided. Can you share potential applications in other fields (e.g., medical imaging, robotics) and any preliminary results or plans for such extensions? While it's not critical to address this, this could significantly expand the scope of the paper and contributions.\n\nTo summarise, in general the paper is lacking diversity in experiments performed, particularly in terms of exploring different models beyond SimpleBEV. If this were expanded to include other AV-related models and/or even better models in different domains, this could broaden the appeal and impact of this work, and I would raise my score.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Layer-Wise Modality Decomposition (LMD), a post-hoc interpretability method designed to attribute predictions of multimodal sensor fusion models in autonomous driving to individual input modalities. The paper introduces a method called LMD, designed for interpreting multimodal sensor fusion models in autonomous driving without altering the model architecture. The method is applied to camera-radar and camera-LiDAR fusion scenarios and assessed using structured perturbation-based metrics and visual decomposition approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1: The novel model-agnostic approach LMD is proposed to decompose contributions of individual modalities in pretrained fusion models. The research problem in this paper is important for numerous downstream applications.\n\n2. The linearization of activation and normalization layers is mathematically formulated. \n\n3. The proposed solution is broadly evaluated across across camera-radar and camera-LiDAR configurations to demonstrate the generality. \n solution is \n\n\nWeaknesses\n\n1. The paper relies on first-order Taylor expansions to linearize non-linear layers (e.g., activation and normalization layers), but it does not rigorously justify why this approximation is sufficient or how high-order interactions are handled. More details of the theoretical foundation for LMD should be provided.\n\n2. The decomposition process may introduce significant computational costs. The runtime and memory overhead analysis should be provided.\n\n3. The proposed solution is not evaluated under realistic conditions. However, it is important for performance evaluation under noisy or adversarial inputs to demonstrate the efficiency.\n\n4. The presentatoin of this paper should be improved to highlight the key contributions.\n\nQUESTIONS:\n1. The theoretical proof of LMD’s linearization is missing. This paper relies on first-order Taylor expansions to justify LMD’s decomposition, but it never formally proves that this approximation maintains the claimed properties across all layers. If LMD is to be trusted in safety-critical systems, the authors must provide formal guarantees—not just empirical observations. Otherwise, the entire theoretical foundation is not solid.\n2. The proposed LMD should be compared against existing well-known solutions like SHAP, LRP, fANOVA or Grad-CAM. \n3. The term \"model-agnostic\" should be clarified. More details should be provided to demonstrate its importance. In this paper, the proposed solution is only evaluated on camera-radar and camera-LiDAR models, but more experiments should be provided to demonstrate its generality. \n4. The legends of green markers in Figure 2 are missing, and more quantitative validation should be provided.\n5. There is no analysis of LMD’s behavior under adversarial or noisy inputs. Autonomous driving sensors frequently encounter noise, occlusions, and adversarial conditions. If radar data is corrupted, does LMD’s decomposition degrade gracefully, or does it produce nonsensical attributions?\n6. If camera and radar disagree (e.g., one detects an obstacle, the other doesn’t), how does LMD attribute the final prediction? Does it highlight the conflict, or does it silently favor one modality? \n7. The computational overhead of LMD should be discussed. The low latency is important for the safety-critical autonomous system. \n8. The authors point out that root point selection and high-order interactions in activation layers are unresolved, yet how to solve them in LMD is not pending.\n9. How to select the reference point for the Taylor expansion should be explained. \n10. Are the results statistically significant?\n\nLIMITATIONS:\nThe authors do not adequately address the limitations of their work. For example, they do not discuss the computational cost of LMD or how it scales with increasing model size or number of modalities.They do not analyze the sensitivity of LMD to hyperparameter choices or architectural variations. They fail to consider the potential misuse of the method in safety-critical systems where incorrect interpretations could lead to dangerous decisions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a method called Layer-Wise Modality Decomposition (LMD) to help understand how different sensors (like cameras, radar, and LiDAR) each contribute to a deep learning model’s prediction in autonomous driving. The method works after the model is trained and does not change the model itself. It breaks down the model’s output layer by layer, showing how much each sensor influenced the result. The authors test this method on real sensor fusion models and show that it can clearly separate the contributions from each sensor, helping make these systems more transparent and interpretable.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: (1) The paper presents a novel idea by extending Layer-Wise Relevance Propagation (LRP) and Deep Taylor Decomposition (DTD) to multimodal sensor fusion, enabling layer-wise decomposition by modality—a problem not well addressed in previous work. (2) The proposed method addresses an important gap in interpreting deep learning models used in safety-critical systems like autonomous driving. (3) LMD works post-hoc without modifying the original model, making it practical for real-world systems already in deployment. (4) The paper introduces structured perturbation-based metrics to measure modality separation, and tests the method across both radar-camera and LiDAR-camera setups.\n\nWeaknesses: (1) Major Concern – Please correct me if I’m wrong—I tried my best to understand this point. The LMD method builds upon Deep Taylor Decomposition (DTD) as its core mechanism for relevance propagation. However, as shown in recent rigorous analysis (e.g., Sixt et al., 2022) [1], DTD often reduces to a simple gradient × input formulation under common settings, such as ReLU activations and typical network architectures. This suggests that in practice, DTD (and methods based on it like LMD) may not offer significantly richer or more faithful attributions than simpler gradient-based methods.  (2) Minor Concern –The linearization for normalization layers (BatchNorm, LayerNorm) breaks the relevance-conservation principle that standard LRP relies on. The paper proposes work-arounds, e.g., the identity rule, the ratio rule etc, which seem to restore conservation empirically. However, these fixes are presented as engineering heuristics; the paper does not offer a formal proof.  (3) Major Concern –The experiments lack standard interpretability benchmarks, such as comparisons with SHAP [2] or Grad-CAM [3].\n\n[1] https://keonly.github.io/cards/Layer-wise-relevance-propagation\n[2] Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. NeurIPS.\n[3] Selvaraju et al. (2017). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. ICCV.\n\nQUESTIONS:\nI am leaning toward the negative side due to the major and minor concerns raised below. \n1. Major question: Can you include or at least discuss comparing LMD with standard interpretability methods such as SHAP, or Grad-CAM in your sensor fusion setting? If these methods are not directly applicable, could you clarify why and possibly provide qualitative or conceptual comparisons?\n2. Minor question: Can you provide a more detailed justification for your treatment of normalization layers (e.g., the use of Identity and Ratio rules)? These rules appear to be heuristic fixes to maintain LRP conservation principles—are there theoretical insights?\n3. (Optional) Can you report on the computational efficiency of your method, such as runtime, memory usage, or scalability to larger models or datasets? This would help assess the practicality of LMD in real-world deployment scenarios.\n\nLIMITATIONS:\nThe paper briefly acknowledges some limitations in the Discussion section, such as the lack of a thorough investigation into the choice of root points and high-order interactions between modalities. However, several important limitations remain insufficiently discussed:  The assumption of modality independence is strong and may not hold in real-world sensor fusion systems, but this is not explicitly addressed. The method’s feasibility for real-time or large-scale deployment is not discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a post-hoc interpretability method for perception models in autonomous driving, which process multimodal inputs (image, LiDAR or radar data). Their method locally linearizes learned operations to separate modality-specific information and attribute the impact on model outputs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The proposed method splits the processing of a multimodal model and generates predictions using each modality individually. This allows for comparing predictions based on fused features with predictions based on modality-specific features, which is a step toward interpreting the modality-specific contributions.\n- The authors further show that their method disentangles fused features by corrupting one input modality and measuring no impact on the predictions based on the other modality. They perform this experiment for a multimodal model that processes either image and LiDAR or image and radar data. This should be elaborated before publication.\n- The authors ablate different linearization approaches for batch, layer and instance norm layers. They measure the Pearson correlation and MSE between predictions from corrupted and non-corrupted forward passes.\n\nWeaknesses:\n- As a minor issue, while their method seems to be model-agnostic, the authors only evaluate it using one model. Experiments with further models would strengthen the paper. Nevertheless, this issue should at least be discussed in the final contribution.\n- The proposed method is a step toward interpreting the impact of all features per processed modality, but doesn’t quantify or interpret the impact of specific features per modality. Combining the proposed method with a feature-based method like SHAP [1] would significantly strengthen the contribution. \n- As a major shortcoming, the related work subsection on interpretation methods focuses on fairly old methods and doesn’t include recent mechanistic interpretability methods (see [3]), which are also used to analyze motion forecasting models for autonomous driving [4]. \n\n[1] Lundberg, S.M. and Lee, S.I., 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.\n[2] Bereska, L. and Gavves, S., Mechanistic Interpretability for AI Safety-A Review. Transactions on Machine Learning Research.\n[3] Tas, O.S. and Wagner, R., 2024. Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers. arXiv preprint arXiv:2406.11624.\n\nThe reported strengths make the paper generally interesting and publishable, assuming that the shortcomings will be resolved as outlined above.\n\nQUESTIONS:\n- As mentioned in the weaknesses, can you apply your method to another model and combine it with a feature-based method?\n- Can you briefly discuss how your method relates to methods for mechanistic interpretability?\n- Can you measure non-linear correlation between corrupted and non-corrupted forward passes, e.g. using the xicor metric [4]? This could allow for a more detailed comparison.\n[4] Chatterjee, S., 2019. A new coefficient of correlation. arXiv e-prints, page. arXiv preprint arXiv:1909.10140, 711.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Layer-Wise Modality Decomposition (LMD), a post-hoc interpretability method designed to disentangle and attribute the contributions of individual sensor modalities (such as camera, radar, and LiDAR) across all layers of multimodal perception models, particularly in autonomous driving. Motivated by the opacity of high-capacity fusion models, LMD locally linearizes neural network layers to decompose their outputs into modality-specific components without altering the original architecture.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Advancing interpretability in multimodal sensor fusion is highly valuable, particularly for safety-critical applications such as autonomous driving.\n\n2. The proposed method is well-grounded in theory, building on established frameworks like Layer-Wise Relevance Propagation and Deep Taylor Decomposition, and shows promising results in modality attribution.\n\nWeaknesses:\n\n1. The current evaluation and demonstrations appear limited to two-modality settings, without experiments or evidence showing its scalability to scenarios involving three or more modalities. While Appendix F is said to discuss generalization to >2 modalities, no experiments are shown for such settings. It’s unclear how well LMD scales or handles interference when more than two modalities are used.\n\n2. While the paper discusses bias components extensively, the semantic meaning of bias remains underexplored (Please see question 3 below).\n\n3. The paper doesn’t compare against other post-hoc multimodal interpretability methods, such as SHAP[1] or attention-based attribution techniques in transformer-based models [2], which are frequently adopted in multimodal fusion settings (e.g., [3])\n\n[1] A unified approach to interpreting model predictions, NeruIPS 2017\n\n[2] Transformer Interpretability Beyond Attention Visualization, CVPR 2021\n\n[3] TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving, TPAMI 2023.\n\nQUESTIONS:\n1. Have the authors tested LMD on settings with more than two modalities? While Appendix F theoretically extends the method to an arbitrary number of modalities, it remains unclear whether any practical challenges arise in such cases, for example, in the selection of reference points or maintaining modality separation when scaling beyond two modalities.\n\n2. Can the proposed method be readily applied to more complex fusion architectures, such as those incorporating cross-modal attention mechanisms — for example, the network used in TransFuser [1]?\n\n3. Figure 2 shows \"perceptual capability\" from the bias term, which is described as representing constant offsets. However, what exactly does this component capture, residual cross-modal interactions, constant activations, or something else? If it truly represents constant information, does the bias-based prediction remain stable or show minor variations when presented with similar inputs?\n\n[1] TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving, TPAMI 2023.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}