From: NeurIPS.cc/2022/Workshop/NeurReps/Program_Chairs
title: Paper Decision
decision: Accept (Oral)

From: NeurIPS.cc/2022/Workshop/NeurReps/Paper67/Reviewer_M73N
title: Recommend to Accept
confidence: 3: The reviewer is fairly confident that the evaluation is correct
submission_track: Extended Abstract (4 Page)
summary: The authors propose an extension of Sheaf Neural Networks by considering attention weights that are multiplied to the morphisms going from vertices to neighboring vertices and benchmark their method on 4 datasets. GATs are a particular case for constant functors to $\mathbb{R}$.
strengths_and_weaknesses: Originality: The work is a straightforward extension combining GATs and Sheaf Neural Networks but writing it correctly can be tricky; this shows that authors did a very good job at making it clear. The result stays very interesting.

Quality: The equations are clear and sound and previous results Equations (1) and (2) are given clear references Barbero et al. 2022, Velickovic et al., 2017. The link between equation (5) of this paper  and (4) of Velickovic et al., 2017 could be optionally discussed.

Clarity: The paper is very clear and very easy to read. The second sentence of section 3 where $\Lambda$ relates to the attention function $a$ could be a bit confusing in the way it is stated as, if I understand well, $\Lambda$ and $a$ are the same. 


Significance : I believe the results to be of interest for the geometric deep learning community

questions: 
Could the authors motivate why they consider only O(n) and not any matrix?

Do the authors see other ways than the increasing layer analysis (Table 1) to asses oversmoothing and heterophily?
limitations: Addressed (Table 1)
soundness: 3: Good
presentation: 3: Good
contribution: 3: Good
overall_rating: 7: Technically solid paper, with moderate-to-high impact, and good-to-excellent evaluation and reproducibility.
relevance: 4: Highly relevant
recommended_decision: 3: Accept

From: NeurIPS.cc/2022/Workshop/NeurReps/Paper67/Reviewer_LLkx
title: Review of Sheaf attention networks
confidence: 3: The reviewer is fairly confident that the evaluation is correct
submission_track: Extended Abstract (4 Page)
summary: The manuscript introduces a novel approach for graph learning which combines sheaf diffusion on graphs and the attention mechanism. The latter is extended to be compatible with the sheaf
Two different formulations, a vanilla and a residual one,  of the attention mechanisms are proposed. The main purpose of the approach is to propose a graph learnin algorithm which is able to deal with oversmoothing and heterophily of graphs, which typically affects negatively the performance of most classical architectures on graphs. The approach is validated experimentally on different graph benchmars, including datasets which present high heterophily of the nodes features.





strengths_and_weaknesses: *Originality*

The method proposed is original as it combines two previous approaches (sheaf neural networks and graph attention networks) to obtain a novel approach to better tackle oversmoothing and graph heterophily.

*Quality*

To the best of my judgement, the paper is technically sound. The experimental section is quite rich although it misses comparisons with vanilla (i.e. without attention) sheaf neural networks (eg. Bodnar 2022). This makes more difficult to assess the contribution of the attention module on top of the existing neural sheaf diffusion, which is the core contribution of the work.

*Clarity*

The paper is clear and well written.

*Significance*
The approach proposed is of interest to tackle the problem of oversmoothing and  : to consolidate the contribution of the paper it would be interesting further to add experimental evidence of the benifits of the attention mechanism in the sheaf formalism and / or theorethical analysis (see limitations)  concerning the expressive power of the network.
questions: -From an experimental perspective (referring to table 1) how the performance of the proposed method compares to the vanilla sheaf neural networks
-Referring to table 1, why does the performance increase the number of layers not increase monotonically with respect to the accuracy? More generally it doesn't seem that the depth of the network is correlated with the accuracy score, even though the sheaf approach should counteract the effect of over smoothing. (as observed in GGCN (chien et al, 2020))


limitations: The main limitation of the work at its current state is that it is difficult to assess the contribution of the approach. In this regard the most straightforward way to tackle this, would be to include experimental results on the same datasets with sheaf neural networks Also of interest would be to characterize if the attention mechanism on top of the sheaf framework is able to enrich the expressivity of the network, allowing to compute a larger class of functions.

soundness: 3: Good
presentation: 4: Excellent
contribution: 3: Good
overall_rating: 6: Technically solid, moderate impact paper, with no major concerns with respect to evaluation and reproducibility.
relevance: 3: Solid fit
recommended_decision: 3: Accept

From: NeurIPS.cc/2022/Workshop/NeurReps/Paper67/Reviewer_L4nh
title: Official review of Paper 67 by Reviewer L4nh
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct
submission_track: Extended Abstract (4 Page)
summary: The authors propose a novel attention-based architecture derived from Sheaf Neural Networks. The model tackles such problems of graph attention networks as oversmoothing and poor performance on heterophilic graphs. The authors demonstrate that the proposed model outperforms its non-sheaf attention counterpart (GAT) on both synthetic and real-world data regarding classification accuracy while being resistant to oversmoothing.

strengths_and_weaknesses: **Strengths**
- incredibly well-written. 
- the theory is well delivered and easily digestible, which is appreciated as sheaf theory is (yet) not widespread in the community.
- the model is compared to an extensive set of baseline approaches on multiple datasets.
- the results are promising.

**Weaknesses**
- in Table 1, the values of standard deviation are pretty high everywhere. Thus, the current number of experiments might not be sufficient to draw conclusions as differences are not statistically significant in some cases. More runs for each experiment are desirable.
- the number of parameters and inference time are not provided.
- hardware information is not given. It would be helpful to put "out-of-memory" in context (Table 1).

questions: - how many runs did you have for each experimental setup (Table 1)? 
limitations: The authors did not provide any limitations. Perhaps, computational time and memory consumption would be helpful. 
soundness: 3: Good
presentation: 4: Excellent
contribution: 3: Good
overall_rating: 7: Technically solid paper, with moderate-to-high impact, and good-to-excellent evaluation and reproducibility.
relevance: 4: Highly relevant
recommended_decision: 3: Accept