Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes to leverage the public wisdom expressed in social media comments and replies to augment social text classification. To this end, this work proposes Hyphen, which consists of hyperbolic graph representation learning and a Fourier co-attention mechanism. Extensive experiments on 10 datasets demonstrate the effectiveness of Hyphen.
strengths_and_weaknesses: Strengths:
- proposes to leverage social media discourse for the task
- extensive experiments on 4 tasks, 10 datasets

Weaknesses:
- certain important experiment details are missing
- justification of the Hyperbolic Fourier Co-Attention for social text classification
- no societal impact discussions
questions: 1. I appreciate that the authors leverage 10 datasets for evaluation. Many of these datasets are well-explored in previous literature, and they often use different dataset splits. Maybe I missed it, but the authors did not seem to provide the train/validation/test split information of these 10 datasets. It would be helpful for readers to know if the authors followed the splits of any previous work or created splits in these experiments.

2. Did the authors implement "SOTA-1", "SOTA-2", and "SOTA-3" in Table 2 or directly quoted the reported results in these papers? It does not seem clear to me what the answer is. If the results are directly quoted, did the authors make sure that the train/validation/test split is identical?

3. The idea of using social discourse such as user comments or interactions on social media to augment social text classification is not new, for example, dEFEND. I hope the authors could better justify how introducing the maths of Hyperbolic Fourier Co-Attention actually help Hyphen achieve this goal.

4. While BERT and RoBERTa are indeed powerful models, the state-of-the-art of NLP has progressed from there and many improved LMs have emerged. In the spirit of [1], I suggest the authors check out newer LMs [2,3] as sota baselines against Hyphen.

5. I appreciate the authors' efforts in annotating the PolitiFact dataset for experiments in Section 6. Did the authors use crowdsourcing for these annotations? If so, please answer the questions in Checklist 5 accordingly.

[1] Bowman, Samuel. "The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail." Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

[2] Clark, Kevin, et al. "Electra: Pre-training text encoders as discriminators rather than generators." arXiv preprint arXiv:2003.10555 (2020).

[3] He, Pengcheng, et al. "Deberta: Decoding-enhanced bert with disentangled attention." arXiv preprint arXiv:2006.03654 (2020).
limitations: The authors discussed the limitations of this work in Section 7. However, the authors did not discuss the potential negative social impacts of this work. I look forward to discussing these issues with the authors and update my score accordingly.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes to apply hyperbolic fourier co-attention mechanism to aggregate public wisdom in social text classification. In social text classification tasks, e.g. fake news detection and hate speech detection, previous works have proved that the comments or reactions from other users on the internet could contribute a lot in boosting the performance of text classification models. This work proposes to use a co-attention mechanism to aggregate the public wisdom within the Abstract Meaning Representation (AMR) graph constructed from comments and reactions. More specifically, they replace the operations like addition and mutiplication in original attention and Fourier transforms with the corresponding definitions in hyperbolic geometry. The experiment results show that their proposed works outperform both task specific SOTA and generic SOTA nerual models.
strengths_and_weaknesses: Strengths:
1. The performance of the proposed model is impressive.
2. The idea to capture the hierarchy structure in the AMR graph is convincing.
3. I appreciate that the authors did so many experiments for the evaluation.

Weakness:
My main concern is about the comparison of task specific SOTA. The issue is that the data specific SOTAs are usually pre-BERT models. As a result, their backbones for capturing linguistic cues are generally weak. For example, the CSI model simply computes the mean of all word embeddings of the text for the detection. From the table 2, we can also see that those generic neural models almost defeat all those data specific SOTAs. As a result, it is hard to say that the proposed Fourier Hyperbolic Co-Attention mechanism can bring better boosts compared to the technologies applied in the data specific SOTAs. To address my concern, I expect the authors could either claim that those data specific SOTAs can not use BERT or provide some results of the data specific SOTAs with BERT backbone (not necessary on all SOTA or all datasets, since the time for rebuttal will be limited. Only several common SOTAs, like CSI, on several datasets would be helpful.)
questions: Based on my understanding, the reason why the Hyperbolic geomerty works is that it can capture hierachy structure more effectively. Would the authors like to do some visualization to the learnt embedding to verify that the model does learn a good embedding?
limitations: See the weakness. Also, I think that the idea of aggregating the public wisdom in social text classification is well known. The main contribution of this work is in designing neural architectures to capture such wisdom better.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper addresses the problem of social media classification. In detail, it targets fake news, hate speech, rumour, and sarcasm detection on ten datasets. The approach combines hyperbolic graph representation learning with Fourier co-attention. In detail, the authors encode the candidate with an attention-enhanced hyperbolic word encoder and use abstract meaning representations to learn hyperbolic graph representations of posts aligned with a candidate. During the evaluation, they compare their approach with generic neural baselines (i.e. Bert) and data-specific baselines - like CNN-based approaches. They found statistically significant improvements in the majority of the tasks. The final ablation study examinates the effect of incorporating social media posts and using the Fourier co-attention mechanism, 

strengths_and_weaknesses: Strengths:
- The proposed method shows constant improvement over different tasks, especially when the number of posts is large. Moreover, it shows that it can gain even from an extensive collection of posts. 
- Different aspects of this method are examined in detail during the ablation study and show their importance. 
- Apart from the pure performance evaluation, the authors include a concluding evaluation regarding the interpretability of single sentence importance from the candidates based on their approach. They found a clear improvement in the baseline.  

Weaknesses:
- The authors compared their approach to pre-trained language models - like Bert - and found overall tasks a constant performance improvement. From the paper, it is unclear how social media posts were presented to these models. In the case of concatenating the candidate and its posts, parts of these posts could be ignored by the models. Because they have a fixed token limitation and most datasets have many relevant social media posts.

questions: - How was a candidate and its post encoded?
- Does table 2 shows the average results over multiple runs using different seeds?

limitations: No limitations
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Social-text classification is challenged by the brevity and lack of context in individual posts. Previous work has tried to augment predictive models with external (to the post) sources of information such as signals derived from users or their networks. However, such signals may not always be readily available. In contrast, public discourse is usually available in the form of comments or replies to a given post. This paper, proposes Hyphen, a novel model that exploits public discourse to improve social-text classification. The model consists of three main modules: candidate post encoder, public discourse encoder and co-attention between the representations induced by the encoders. 

To encode the pubic discourse, individual comments are first represented as AMR graphs which are then merged into a single macro-AMR graph. The macro-AMR is then passed into a Hyperbolic Convolutional Neural Network to induce embedding representations.
The candidate post is encoded through a Hyperbolic Hierarchical Attention Network — individual word representations are transformed into sentence representations, using bi-directional Mobius-GRU layers with attention. Similarly, the final document representation is obtained  with a higher-level Mobius-GRU layer with attention that processes the sentence representations.
Both induced representations are then combined via an Hyperbolic Fourier Co-attention mechanism that produces affinity scores between representations. The representations are first transformed via a 2D Discrete Fourier Transform to extract the most salient (high “frequency”) features. Then an affinity matrix is learned such that it can produce attention weights between source words and user interactions, which are used to obtain the final weight aggregated representations for the source post and the public discourse signals. These attention weights can also be used to interpret the predictions: by ranking candidate sentences wrt to attention scores we can get a sense of which parts of the input most contribute to the predictions.

The evaluation performed on a variety of datasets and tasks such as detection of fake news, rumors, hate-speech and sarcasm, shows that the proposed model is effective. 
strengths_and_weaknesses: This paper addresses an important problem in social-text classification: how to augment predictive models with external information? The proposed approach seems to be effective at leveraging public discourse associated to a target post by exploiting the hierarchical nature of public discourse (in social-texts). Another benefit of this model is the Fourier co-attention module that allows models to focus on the most important parts of the input. Furthermore, this mechanism can afford some degree of interpretability by inspecting the attention weights attributed to each sentence on a given input. However, claims of model interpretability based on attention must be taken with a grain of salt as previous work has shown that “Attention is Not Explanation” (Jain and Wallace, 2019).

Overall the paper is well written and organized, and the main claims of the paper are supported by experimental evidence. However, the notation is a bit convoluted and thus hard to follow with the various superscripts and subscripts. Perhaps the authors can polish and simplify the notation on the final draft. There are also broken references (e.g. line 192 refers to Figure (e) which does not exist; line 272 refers a Table 5 that does not exist; line 261 discusses optimizing k and points to Eq. 5 but this equation has no k). 
questions: N/A
limitations: Yes, for the most part. However, given that this work entails collecting and analyzing personal data from social media, a discussion on potential negative societal impact is missing.
review_title: null

Meta-review: metareview: This paper proposed a discourse-aware hyperbolic spectral co-attention network for social text classification, via using public discourse and its hierarchy. Reviewers all agreed that this work presents an extensive amount of experiments/evaluations, with impressive performance gains and interpretability.  Thus, we recommend acceptance. 

confidence: Certain
Decision: Accept