reviews: 
Comments: The paper introduces Neuron Path as a novel method for understanding Vision Transformers (ViT), focusing on the flow of information through the Feed-Forward Network (FFN) within the transformer encoder. Unlike methods that highlight input regions, Neuron Path examines influential neurons within the FFN across layers to reveal how information is processed. The authors propose the Joint Attribution Score (JAS), a gradient-based metric that measures the collective influence of neuron groups on the model’s output, along with a layer-by-layer selection method to identify impactful neurons, forming a path that traces critical information flow from input to output. Experiments show that Neuron Path effectively identifies neurons crucial for accurate predictions, with class-level analysis indicating that similar neurons activate for related classes, suggesting an internal organization of semantic information. Additionally, the authors find Neuron Path valuable for model compression, as retaining only essential neuron paths allows for nearly the same performance, revealing redundancy in Vision Transformers that could be leveraged for pruning and optimization.
Strengths: 1. The paper introduces the innovative concept of "Neuron Path," which identifies influential neuron paths specifically within the Feed-Forward Network (FFN) of vision transformers (ViTs). It also presents the Joint Attribution Score (JAS), a gradient-based metric that quantifies the collective influence of neuron groups across layers. Additionally, the Layer-Progressive Neuron Locating Algorithm is proposed to trace influential neuron paths efficiently.

2. The research methodology is robust, with comparisons against established baselines such as "Activation" and "Influence Pattern." The study includes comprehensive experiments, such as neuron intervention and model pruning, which provide strong empirical support for the effectiveness of the Neuron Path method.

3. The paper is well-structured and clearly explains its concepts, with well-defined terms, illustrative figures, and organized sections that enhance readability and understanding.

4. The study advances the interpretability of vision transformers by revealing insights into model behavior, potential for model compression, and implications for real-world applications. It fosters the development of more transparent and efficient vision models by uncovering redundancy that can be leveraged for pruning.
Weaknesses: 1. The current analysis is limited to neurons within the Feed-Forward Network (FFN) component of the Vision Transformer (ViT) encoder. Although FFNs are shown to play a critical role in encoding semantic concepts, excluding the self-attention mechanism from the Neuron Path analysis may lead to an incomplete understanding of information flow within the model, as it overlooks potential interactions between self-attention and FFN modules.

2. The study is focused solely on image classification, a discriminative task, leaving the applicability of Neuron Path to other vision tasks unexplored.

3. The paper does not explore theoretical connections between JAS and established interpretability frameworks, such as information theory or causal inference. Relating JAS to concepts like information bottlenecks or causal pathways could provide a stronger theoretical foundation, enhancing the rigor and interpretability of the findings.

4. While the paper suggests a potential link between neuron paths and semantic information, it does not examine this relationship in depth. Conducting a detailed analysis of how neuron paths correlate with human-interpretable concepts could significantly enhance the model’s interpretability.

5. The paper primarily focuses on identifying neuron paths and their impact on model performance but does not delve into the properties of these paths. Investigating the stability of neuron paths across varying inputs or their evolution during training could provide valuable insights into the model's robustness and behavior. However, due to the computational expense mentioned by the authors, such analyses may be challenging to conduct.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 2