Enhancing Argument Structure Extraction with Efficient Leverage of Contextual Information
Authors: Yun Luo
Zhen Yang
Fandong Meng
Yingjie Li
Jie Zhou
Yue Zhang
Venue: EMNLP 2023 Findings (2023)
Keywords: Argument Mining
Argument structure extraction
Discourse Structure of Arguments

Abstract: Argument structure extraction (ASE) aims to identify the discourse structure of arguments within documents. Previous research has demonstrated that contextual information is crucial for developing an effective ASE model. However, we observe that merely concatenating sentences in a contextual window does not fully utilize contextual information and can sometimes lead to excessive attention on less informative sentences. To tackle this challenge, we propose an Efficient Context-aware ASE model (ECASE) that fully exploits contextual information by enhancing modeling capacity and augmenting training data. Specifically, we introduce a sequence-attention module and distance-weighted similarity loss to aggregate contextual information and argumentative information. Additionally, we augment the training data by randomly masking discourse markers and sentences, which reduces the model's reliance on specific words or less informative sentences. Our experiments on five datasets from various domains demonstrate that our model achieves state-of-the-art performance. Furthermore, ablation studies confirm the effectiveness of each module in our model.

Enhancing Argument Structure Extraction with Efficient Leverage of Contextual Information
Yun Luo Zhen Yang 2, Fandong Meng ?, Yingjie Li ', Jie Zhou 2, Yue Zhang 1.3*
School of Engineering, Westlake University, Hangzhou, China.
2 Pattern Recognition Center, WeChat AI, Tencent Inc, Beijing, China.
Institute of Advanced Technology, Westlake Institute for Advaneed Study, Hangzhou, China. (luoyun, liyingjie, zhangyue?]weesttlake edu. cn (zieenyang, fandongmeng, wi withhoomzhou@@eenteern
Abstract
Peer Review
(1) that the problem studied in this paper interesting.
Argument structure extraction (ASE) aims to identify the discourse structure of arguments within documents. Previous research has demonstrated that contextual information is cru- cial for developing an effective ASE model. However, observe that merely concatenat- ing sentences in a contextual window does not fully utilize contextual information and can sometimes lead to excessive attention on less informative sentences. tackle this challenge, we propose Efficient Context-aware ASE model (ECASE) that fully exploits contextual information by enhancing modeling capacity and augmenting training data. Specifically, we introduce a requence-attentiin module and distance- tistance-weighted similarity loss aggregate contextual information and argumentative infor- mation. Additionally, we augment the training data by randomly masking discourse markers and sentences, which reduces the model's 's re- liance on specific words or less informative sen- tences. Our experiments five datasets from various domains demonstrate that our model achieves state-of-th of oh-the-art performance, Further- more, ablation studies confirm the effectiveness of each module in our model.
(2) However, after reading through the manuscript it is not clear to me what are the real contributions made this paper.
Suuoor
(3) failed find any rigorous results on genera- lization bounds.
(4) Therefore, I cannot recommend the acceptance this paper.
Figure An example of argument structure extraction in peer reviews. The argumentative discourse markers are marked in red.
ple where the second and third sentences describe the shortcomings of the reviewed paper and support the fourth conclusion sentence.
It has been verified that the contextual informa- tion of documents plays a crucial role in detecting argumentative relations (Nguyen and Litman, 2016; Opitz and Frank, 2019; Hua and Wang, 2022). Ac- cording to our preliminary experiments based on the Context-aware ASE model (CASE) (Hua and Wang, 2022), several findings motivate us to further explore the efficient use of contextual information. Firstly, based on our experimental results shown in Figure 2(a), we find that the performance of the model is initially improved with the increase of the context window, but then decreases as the context window continues to grow. These results indicate that contextual information can enhance the per- formance of the model, but the excessively long context may cause the model to overly focus on some irrelevant sentence information and behave relatively weak performance. Secondly, the per- formance of CASE decreases as the distances of argumentative pairs increase, shown in Figure 2(b), which suggests that detecting argumentative relations over long distances is still challenging for CASE. It also implies that there still room improvement by making full use of contextual information rather than simply concatenating the sentences in a context window. Thirdly, Figure
1 Introduction
Argument structure extraction (ASE) aims to iden- tify the argumentative discourse structure in docu- ments (Peldszus and Stede, 2013; Cabrio and Vil- lata, 2018; Lawrence and Reed, 2019; Li et al., 2020). As a basis of evidence-based reasoning ap- plications, ASE has attracted increasing attention from researchers in a wide spectrum of domains, such as legal documents (Palau and Moens, 2009; Lippi and Torroni, 2016; Poudyal et al., 2020), on- line posts (Cardie et al., 2008; Boltušić and Šna- jder, 2014; Park and Cardie, 2018; Hua and Wang. 2017), and scientific articles (Mayer et al., 2020; AI Khatib et al., 2021). Figure shows exam-
For details about the related work, we refer the readers to Appendix A.

---

AABIRCT
AMPERE
wo Dist
ww Disc
-00
75
H11 Score of Support
00
Macro-Fi
50
Matro-FI
74
40
20
60
AbaRRC
ECIRR
AMPERE
55
10
16
18
20
(0,4)
(4.8)
(8.12)
-12
AbsiRTT
ECIR
AMPERE
Context Length
Distance
Dataset
(8) Effect of Context Length
(b) Effect of Different Distance
(c) Effect of Discourse Marker
CASE model in three different datasets (AhstRCT, ECHR, and AMPERF). (a) shows the performance when the context length varies; (b) shows the macro-Fl scores with respect the distance between argumentative pairs; (c) refers to the model performance on the samples with/without discourse markers.
Figure 2: Analysis
2(c) shows that the model mostly achieves better performance in argumentative pairs where there are discourse markers, such as 'thus', and et. These specific words are regarded as significant sig- nals for identifying argumentative relations (Stab and Gurevych, 2014; Hua and Wang. 2022), but they may hinder the model from fully exploiting the contextual information between sentences.
documents, each consisting of several propositions {st]*- The task is to identify the existence oltack, support link from 8j to 8g. or no rel (no relation) between $j and 8g- We consider the task both in end-to-end setting (considering all proposition pairs) and head-given setting where the head propositions are given in advance. The framework of our model is illustrated in Figure 3.
Based on the above observations, we propose Efficient Context-aware ASE model (ECASE), which enhances the leverage of contextual informa- tion from two perspectives, i.e., modeling capacity and data augmentation. Regarding the modeling capacity, we adopt a sequence attention module on RoBERTa (Liu al., 2019) to aggregate the contex- tual information and optimize the attention distri- bution between sentences. We also use distance- weighted similarity loss to reinforce the representa- tion similarity in view of argumentative relations. In terms of data augmentation, the discourse markers and sentences in training to mitigate the model's dependence on specific words and less informative sentences. Hence, the model is encouraged to thoroughly comprehend the contextual information.
2.2 Modeling Capacity Enhancement
2.2.1 Encoding with Sentence-level Attention
Following Hua and Wang (2022), we build our model on top of RoBERTa (Liu et al., 2019), which a widely-used encoder-only pre-trained language model. Given a head proposition 8j, we concate- nate it with surrounding sentences, which provide the contextual information, including L proposi- tions before and after sj. Other propositions in the context regarded as tail propositions. [CLS] tokens are added separate the propositions and serve as their representations. The representations of head proposition 8) or other tail propositions can be obtained as follows:
randomly mask
HyJ Ro
(1)
Experiments on five datasets from different do- mains show that our model achieves state-of-the- art performance in both the givenn setting and the end-to-end setting of the ASE task. Ablation studies also demonstrate the effectiveness of each module in our model. The codes are released in http://gttubb.mmmLLxxxxxllee///f
To further extract the relation between sentences, we use sentence-level attention module to aggre- gate the contextual information. Specifically, we concatenate the sentence representations in the textual window and feed them into multi-head self-attention module (Vaswani et al., 2017) to ag- gregate the contextual information, which can be formulated as follows:
2 Method
(2) where each row of query, key, and value in the self-attention module corresponds to a sentence
11 Seq 1r,"),
2.1 Task Formulation
We formulate the ASE task as a classification task. Formally, let N be a dataset with N

---

Augmentation
Encoding
Classification
(1) think, the Massrbusllls
rule posiive step right
H
1J
Distacce-vcichiee Sumilrrit Loss
direction.
(2) However, for thos debiurs
L Lcn
only cannot all...
cuample, most disability musrince compamees.
Sequence-level Attention
Hj H
Hi 1
H
Discouse Maker & Context Masking
H
Hjiz
MLP
Massachuccts
RoBERTa Encoder
Support
rule is positive step in the right
P(S S
Anack
diccction
Navd
<Masks,
who truly cannat pay at all...
CLS
CLS
Cross Entropy Loss Lits
(3) <Mask>..
<Mack-
Head
Forward context propositions
Figure 3: The model framework of our proposed argument structure extraction model.
representation. The representations from the token level and sentence level are mixed in a simple way addition: H, = 3 + Hy..
AMPERE
Essays
AbstRCT
ECHR
CDCP
Prop.
10,386
12.373
5,693
6,331
4,932
Supp.
3,370
3,613
2.402
1,946
1,426
Att.
266
219
0
Then, we utilize a multi-layer perceptron for calculating the label distribution of the head-tail pairs (such as (j. k)), which can be shown as:
Head 2.268
1,707
1.138
1,037
Dise
14.58%
10.44%
13.94%
14.08%
12.30%
Table I: Dataset statistic. % Disc refers the ratio of propositions with discourse makers.
Ply 85,88 ) So fi Ha] W1) W2) (3)
we randomly mask discourse markers (with prob- ability Pw) to encourage the model to learn more contextual information in sentences and rely less on discourse markers. 18 discourse markers are lected from the PDTB manual (Prasad al., 2006) following Hua and Wang (2022) (Table 3). Second, since the long context could introduce noise into the sequences, the attention to the less informative sentences might have negative im- pact identifying specific head-tail pairs. There- fore, generate samples from original data by randomly (with probability p.) masking some sen- tences to enhance the comprehension of the contex- tual information and mitigate the excessive atten- tion on certain sentences.
where W1, W2 are trainable parameters.
2.2.2 Argumentative Relation Regularization We further propose similarity loss enhance the semantic information of the sentence embeddings (Reimers and Gurevych, 2019; Wieting and Gim- pel, 2018; Luo et al., 2022a), which is formulated as:
( exp( 44 (H, Hz) expl di ()sim(H., H2) ify +
ify =f
Com
(4)
Specifically, the loss aims to pull head-tail represen- tations with argumentative relations (y * 0) and push away them with no-rel links (y 0) through optimizing cosine similarity sim(-,) -). Considering that the representations encoded by PLMs have relatively weak similarity over long distances and vice versa, we use distance weight to reinforce the similarity differences. The weight is calculated by the exponent of the normalized distance, where dij is the distance between representations of sentence and 8j and d is the maximum token number.
2.4 Training
The overall training loss is calculated as follows: = Cats + XCoon where À the hyper-parameter to control the weight of representation similarity.
Experiments Setup
We carry out experiments on five ASE datasets from different domains, namely AMPERE, Essays, AbstRCT, ECHR, and CDCP, whose statistics are shown in Table (details in Appendix B). Our model is compared with several baseline models: 1) two classic lexical ASE models SVM-linear and SVM-RBE (Stab and Gurevych, 2017): 2) SEQ-
2.3 Data Augmentation
In this paper, propose both word-level and sentence-level data augmentation for the ASE task. First, our observations indicate that models tend to perform better on the head-tail pairs with discourse markers. Thus, in word-level data augmentation,

---

AMPERE
Essays
AbstRCT
ECHR
CDCP
Supp.
Atk.
Macro
Supp.
Atk.
Macro
Supp. Macro Supp.
Macro
Supp. Macro
SVM-Linear
24.82
28.69
21.18
29.01
SVM-RBF
26.38
31.68
21.36
30.34
SEQPAIR
17.34 7.40
26.42
31.42
24.32
30.37
17.32
32.34 23.11
33.23
14.16
28.44
Head Given
SEQCON-10
43.04
49.67
63.34
47.71
30.14
57.40
60.60
69.20
41.81
65.48
38.34
63.10
CASE-10
63.86
70.41
77.62
72.03
40.61
70.19
63.39
71.08
42.35
71.11
45.73
69.76
CASE-20
68.56 64.45
77.41
72.14
40.14
69.13
63.35
70.94
35.18
69.35
45.63
69.56
CASE-20"
77.64
71.30
70.82
70.37
ECASE-10 10
67.37
71.14
79.01
75.91
44,40
72.38
63.91
72.17
47.41
73.15
51.16
72.18
ECASE-20
66.12
71.10
78.46
74.21
42.30
72.36
64.42
72.10
41.55
70.46
51.85
72.36
End-to-Lnd
CASE-20
60.33
64.55
73.23
70.73
36.31
68.58
64.85
72.79
31.77
65.66
39.56
66.22
ECASE-20
60.82
57.56
72.82
73.74
37.54
69.51
63.58
73.59
39.63
69,49
46.87
70.16
Table 2: Results of model five different ASE datasets. '-' for not publishing, and ** refers the results reported in the original paper, considering pre- treitrainnng on unlabeled data for domain adaptation. '-10' and "-20" refer to the context length. 'Supp.' and 'Atk.' represent the F1 scores of 'Support' and 'Attack', respectively.
84
for ECASE-10 compared with ECASE-20, the gap between them is relatively smaller compared with CASE. Notably, the model performance of ECASE-20 (72.36%) in CDCD is even higher than that of ECASE-10 (72.18%) indicating that our model can efficiently extract contextual informa- tion while mitigating excessive attention to less informative sentences to some degree. Moreover, models with backward and forward context inputs, such as CASE and ECASE, outperform SEQPAIR and SEQCON-10, demonstrating the effectiveness of the input form and contextual information.
CACE CASE+Aug
EMC ECASE
Macro-Fi
76
72
AbatRCT
AMPERE
CDCP
BCHR
Essnys
Figure 4: Ablation study of the models with context length 20 in the head-gut given setting. 'Aug' and 'EMC' refer to data augmentation and the modules for enhanc- ing modeling capacity.
PAIR (Devlin et al., 2019) uses BERT to encode the head and tail in single sentences separately and concatenates their representations to predict the argumentative relation; 3) SEQCON, context- aware version of SEQPAIR, where the head and tail are encoded in the context of each other, and [CLS] representations of the head and tail are con- catenated for classification; 4) CASE (Hua and Wang, 2022) concatenates the head sentence with forward context propositions and backward context propositions in a window for encoding. In the ex- periments of ECASE and reproduced CASE, we keep the same training hyper- ppprmeparaeeter such as training steps (Appendix C).
In the end-to-end setting, ECASE-20s achieve stronger performance compared with CASE-20 in most datasets. The results also demonstrate that the efficient use of contextual information can boost the identification of argumentative structure in raw discourses. However, the performance of ECASE- 20s in the end-to-end setting is generally lower than in the head-given settings. This suggests that detect- ing argumentative relations becomes increasingly difficult in the end-to-end scenario due to the small ratio of support and attack labels. But the perfor- mance of ECASE-20 on AbstRCT is higher in such setting which indicates in the dataset the no-rel samples are significant for models to distinguish the argumentative relations and the argumentative pairs are expressed more evidently compared with no-rel sentence pairs."
4 Results
4.1
Overall Results
The main results are shown in Table 2. Specifically, in the head-given setting, ECASE- 10 achieves state- of of-the-art performance (79.01 1,.01,,77 77.38% 72.17% 73.15% and 72.16% macro-F1 scores) compared with other baselines. When increasing the con- text length of ECASE to 20, the model also per- forms better compared with CASE-20. Although shorter contexts still perform better in most datasets
4.2 Ablation Study
We also conduct ablation studies to highlight the ef- fectiveness of each module in our model. Figure 4 presents the results, indicating that the inclusion of
2 Further analysis of the context length, discourse makers, and case study of GPT3.5 can he found in Appendix D&E.

---

data augmentation (ugSE+Aug) and the modules for enhancing modeling capacity (CASE+EMC) can enhance the performance of CASEs. We also observe that compared with data augmentation, the refinement of the modeling capacity brings a more critical improvement. ECASEs achieve the most significant performance in most datasets indicating the effectiveness of our model.
Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Filip Boltušié and Jan Snajder. 2014. Back up your stance: Recognizing arguments in online discussions. In Proceedings of the First Workshop on Argumenta- lion Mining, pages 49-58.
Elena Cabrio and Serena Villata. 2018. Five years of argument mining: A data-driven analysis. In ICCAI, volume 18, pages 5427-5433.
5 Conclusion
Claire Cardie, Cynthia Farina, Matt Rawding, and Adil Aijaz. 2008. An eRulemaking corpus: Identifying substantive issues in public comments. In Proceed- ings of the Sixth International Conference on Lan- guage Resources and Evaluation (LREC'08). Mar- rakech, Morocco.
In this study, we proposed an efficient context- aware ASE model by enhancing modeling capacity and augmenting training data. Our experiments demonstrated the effectiveness of our model, and the ablation study proved the significance of each module. Our study highlighted the essential role of efficient use of contextual information.
Jacob Devlin, Ming-Wai wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Limitations
According to preliminary experiments, we find it difficult for both CASE and ECASE to identify the attack relations in AbstRCT dataset without using data for unsupervised training. The samples of altack are extremely scarce in the dataset. Thus we regard AbstRCT as a two-label task, where the relation attack is regarded as no - rel. And in this study, we only consider the encoder-only model RoBERTa as our backbone network, without the consideration of encoder-decoder or decoder-only models. In comparison with GPT-3.5. we do not test GPT3.5-turbo in full test data of ASE but show some randomly selected samples to demonstrate its performance because of the complex output form as well as the time and expense costs. But the results have shown the feature of the output from GPT3.5-turbo in ASE tasks.
Xinyu Hua and Lu Wang. 2017. Understanding and detecting supporting arguments of diverse types. In Proceedings of the 55th Anaual Meeting of the As- sociation for Computational Linguistics (Volume 2: Short Papers), pages 203 208, Vancouver, Canada. Association for Computational Linguistics.
Xinyu Hua and Lu Wang. 2022. Efficient argument structure extraction with transfer learning and active learning. In Findings of the Association for Compu- tational Linguistics: ACL 2022, pages 423-437.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
John Lawrence and Chris Reed. 2019. Argument min- ing: A survey. Computational Linguistics, 45(4))765- 818.
Jialu Li, Esin Durmus, and Claire Cardie. 2020. Explor- ing the role of argument structure in online debate persuasion. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pages 8905-8911 8912.
References
Khalid Al Khatib, Tirthankar Ghosal, Yufang Hou, Anita de Waard, and Dayne Freitag. 2021. Argument mining for scholarly document processing: Taking stock and looking ahead. In Proceedings of the Sec- ond Workshop on Scholarty Document Processing, pages 56-65.
Marco Lippi and Paolo Torroni. 2016. Argumentation mining: State of the art and emerging trends. ACM Trans. Internet Technol. 16(2).
Bing Liu. 2022. Sentiment analysis and opinion mining. Springer Nature.
Jianzhu Bao, Yuhang He, Yang Sun, Bin Liang, Ji- achen Du, Bing Qin. Min Yang, and Ruifeng Xu. 2022. generative model for end-to-end argument mining with reconstructed positional encoding and constrained pointer mechanism. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pages 10437-10449, Abu
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint 1907. 1692..

---

Yun Luo. Fang Guo, Zihan Liu, and Yue Zhang. 2022a, Mere contrastive learning for cross- doss-- senti- ment analysis. In Proceedings of the 29th Inter- national Conference on Computational Linguistics, pages 7099-7111, Gyeongju, Republic of Korea. In- ternational Committee on Computational Linguistics.
Andreas Peldszus and Manfred Stede. 2015. Joint pre- diction in -ty-s discourse parsing for argumen- tation mining. In Proceedings of the 2015 Confer- ence on Empirical Methods Natural Language Processing, pages 938-948, Lisbon, Portugal. Asso- ciation for Computational Linguisties.
Yun Luo, Zihan Liu, Stan Z Li, and Yue Zhang. 2023, Improving (dis) agreement detection with inductive social relation information from comment-reply inter- actions. In Proceedings of the ACM Web Conference 2023, pages 1584-1593.
Prakash Poudyal, Jaromir Savelka, Aagje leven, Marie Francine Moens. Teresa Goncalves, and Paulo Quaresma. 2020. ECHR: Legal corpus for argument mining. In Proceedings the 7th Workshop on Argu- ment Mining, pages 67-75, Online. Association for Computational Linguisties.
Yun Luo, Zihan Liu, Yuefeng Shi. Stan Z. Li, and Yue Zhang. 2022b. Exploiting sentiment and common sense for zero-shot stance detection. In Proceed- ings of the 29th International Conference on Com- putational Linguistics. pages 7112-7123, Gyeongju, Republic of Korea. International Committee on Com- putational I. Linguisties.
Rashmi Prasad, Fleni Miltsakaki, Nikhil Dinesh, Alan Lee, Aravind Joshi, and Bonnie L Webber. 2006. The penn discourse treebank 1.0 annotation manual.
Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJCNP)), pages 3982-3992.
Yubo Ma, Yixin Cao, YongChing Hong, and Aixin Sun. 2023. Large language model is not good few-shot information extractor, but good reranker for hard samples! arXiv preprint arXivv2303.0853 085559
Christian Stab and Iryna Gurevych. 2014. Identifying argumentative discourse structures in persuasive es- says. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 46-56, Doha, Qatar, Association for Computational Linguistics.
Tobias Mayer, Elena Cabrio, and Serena Villata. 2020. Transformer-base argument mining for healthcare applications. In ECAl 2020. pages 2108-21000011 2115.215..110 IOS
Press.
Huy Nguyen and Diane Litman. 2016. Cantext-aware argumentative relation mining. In Proceedings of the 54th Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages 1127-1137, Berlin, Germany. Association for Com- putational Linguistics.
Christian Stab and Iryna Gurevych. 2017. Parsing Ar- gumentation Structures in Persuasive Essays. Com- putaaiiona Linguistics, 43(3):619-65)
Yang Sun, Bin Liang, Jianzhu Bao, Min Yang, and Ruifeng Xu. 2022. Probing structural knowledge from tre-trained language model for argumentation relation classification. In Findings of the Associa- tion for Computational Linguisties: EMNLP 2022. pages 3605-3615, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Vlad Niculae, Joonsuk Park, and Claire Cardie. 2017. Argument mining with structured SVMs and RNNs. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics ( (Volume
1: Long Popers), pages 985-995, Vancouver, Canada. Association for Computational Linguistics.
Juri Opitz and Anette Frank. 2019. Dissecting content
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need, Advances in neural information processing systems, 30.
and context in argumentative relation analysis. In Proceedings of the 6th Workshop on Argument Min- ing, pages 25-34, Florence, Italy. Association for Computational Linguistics.
Raquel Mochales Palau and Parie-Francine Moens. 2009. Argumentation mining: the detection, classifi- cation and structure of arguments in text. In Proceed- ings of the 12th international conference on artificial intelligence and law, pages 1877
John Wieting and Kevin Gimpel. 2018. Paranmt-50m: Pushing the limits of paraphrastic sentence embed- dings with millions of machine translations. In Pro- ceedings of the 56th Annual Meeting of the Associa- tion for Computational I Linguistics (Volume I: Long Papers), pages 451 162-
Joonsuk Park and Claire Cardie. 2018. A corpus of eRulemaking user comments for measuring evalua- bility of arguments. In Proceedings of the Fleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. Eu- ropean Language Resources Association (ELRA).
Yuxiao Ye and Simone Teufel. 2021. End-to-end ar- gument mining as biaffine dependency parsing. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Lin- guistics: Main Volume, pages 669-678. Online. As- sociation for Computational Linguisties.
Andreas Peldszus and Manfred Stede. 2013. From argu- ment diagrams to argumentation mining in texts: survey. Int. J. Cogn. Inform. Nat. Intell., 7(1):1311.

---

A Related Work
Review2 (Hua and Wang, 2022). Each review in the dataset is annotated with segmented propositions and the corresponding types such as evaluation, re- quest, fact, and so on. The relations of support of attack are further labeled among the propositions. We use 300 reviews for training, 20 for validation, and 80 for testing.
Argument structure extraction has attracted increas- ing attention in recent years, which is a subtask of opinion mining (Lippi and Torroni, 2016; Cardie et al., 2008; Lawrence and Reed, 2019). The task aims to identify the structure relations of the ar- gument propositions. It differs from sentiment anal- ysis (Liu, 2022) and stance detection (Luo al., 2023, 2022b) since argument mining plays role expressing and promoting an opinion, i.e. deter- mining why individuals hold the opinions but not what opinion they hold (Lawrence and Reed, 2019). It can be divided into two subtasks as premise de- tection and relation classification, where the former aims to identify the targeted propositions (head) and the latter requires classifying the relations of other propositions (tail) the head. Research in different levels proposed for analyzing the dis- course structure such as segment level (Sun et al., 2022; Bao et al., 2022; Ye and Teufel, 2021) and proposition level (Hua and Wang, 2017, 2022).
Essays. The dataset Essays contains 402 essays collected by Stab and Gurevych (2017) from 4. The propositions are annotated the sub-sentence level with corresponding types such as premise, claim, or major claim. The relations (support or attack) are annotated from a premise to a claim or to another premise. The dataset is split into 228, 40, and 80 for training. validation, and testing. Biomedical Paper Abstract. The dataset Ab- stRTT contains 700 paper abstracts from PubMed (Mayer et al., 2020). Note that the dataset contains fewer propositions compared with the above ones, with only 70 attack links. For simplicity, we re- gard attack as no-rel, and only make classification on support and no-rel since the ratio of allack is significantly low.
Early methods model the task based on discourse parsing (Peldszus and Stede, 2013, 2015; Stab and Gurevych, 2017) and some methods use statistical and dumman-mace features for classification (Stab and Gurevych, 2014; Niculae et al., 2017). In re- cent years, ASE models based on pre-trained lan- guage models are proposed and achieve great per- formance (Sun et al., 2022; Bao et al., 2022; Hua and Wang, 2022). For example, Sun et al. (2022) use BERT (Devlin et al., 2019) as the backbone net- work and use probing for further extraction of the semantic information in the language model. Hua and Wang (2017) propose context-aware model based on RoBERTa (Liu al., 2019) encoding the head propositions in a window for the ASE task. Our model built on Hua and Wang (2022), and further considers how to efficiently leverage the contextual information.
Legal Documents. The dataset ECHR (Poudyal et al., 2020) contains 42 documents from the Euro- pean Court of Human Rights, which annotated the links from premises to conclusions.
Online User Comments. Park and Cardic (2018) (CDCP) contains annotated comments related Consumer Debt Collection Practices, where labels of supporting relations are given.
C Implementation Details
We perform experiments using the official pre- trained RoBERTa (Liu et al., 2019) (roberta-base) provided by Huggingface $. The models are trained on GPU (Tesla V100) using the Adam optimizer (Kingma and Ba, 2014). The learning rate is le- 5, and the scheduler is set constant. We train our model in 15 epochs. We hyper-tune ECASE with X E {0.1, 0.05, 0.01, 0.001), and find that the experimental results are relatively stable, indicat- ing that the model is less sensitive, and we take 0.01 for our experiments. The probability of ps and Pw are 0.15 and 0.5, respectively. We evaluate the model performance based on maero-
B Datasets
Following Hua and Wang (2022), we adopt the datasets across the domains of peer review, essays, biomedical paper abstracts, online user discussions, and legal documents. The statisties of the datasets are shown Table 3, and the details are described as follows:
F1 scores. The reported results are averaged with 5 different random seeds. The reproduced exper-
hhups://openeeview.nee//
Peer Reviews. The dataset AMPERE consists of 400 ICLR 2018 paper reviews collected from Open-
hhtp://csssyssfrru.com
hhups:/hhuggingfacc.oo

---

hecause though hence
therefore
however nonetheless for this reason specifically
although
nevertheless consequently particularly but
ECASE-AMPERE
due to
CAPERE
particular
fact
Score of Support
60
ECASE-ECHR
actually
CASE-ECHR
Table 3: Discourse markers of PDTB manual.
40
= wo Dist w Disc
20
30
10
MMagg-FF
(4,8)
(8,12)
12
Distance
Figure Results of CASE and ECASE with different distances of the head-tail pairs.
CASP-FCHR
ECAST-FCHR
CASE-AMPERE RCASE-AMPERE
Figure 5: Results CASE and ECASE w/o dis- course markers.
does not support the conclusion.
Meanwhile, in some situations with complex logic, both GPT3.5 and ECASE can not give cor- rect answers. For example, in case 5 GPT3.5 replies as: "The argumentative relation of other sentences to the first sentence is attack. The second sentence is attacking the first sentence by saying that texts do work for consumers, but they have to pay for it. The third sentence further attacks the first sentence by stating that charging consumers for texts against the law and rightly so.'
iments of CASE are based on the released codes and follow the same setting as our model.
D Case Study with GPT3.5
Recently, LLMs in generative schema become dom- inant in the area of natural language processing such as ChatGPT and Claude ? But it has been verified that in-context learning based on LLMs performs weaker compared with fine-tuned small language models, especially in tasks with complex outputs (Ma et al., 2023). Here we also show some cases tested GPT3.5-turbo in the head-give given setting (Table 4). Note that in the zero-shot ting, the performance of GPT3.5 is not excellent in solving the task of ASE and the results indicate that developing small models with supervised data for the specific task of ASE is still significant and valuable.
E
Further Analysis
Here we also compare our model with CASE cor- responding to the experiments in the introduction (Figures 5 and 6). As can boost the performance both the pairs with without discourse marker, and the sentence pairs with different distances of head-tail pairs. The re- sults demonstrate that our model makes more ef- ficient use of contextual information and achieves stronger performance in ASE tasks.
observe,
model
The instruction fed into GPT is that 'Given the text (document), what is the argumentative relation of other sentences to the sentence (head), support, against or no relation? And show which sentence." We observe that the model tends to give a consistent prediction of tail sentences to head sentences. For example, in case 3, GPT3.5 replies as: 'The first, second, and third sentences support the argument made in the fourth sentence. They state that the problem studied in the paper is interesting but that the manuscript does make clear contributions or provide rigorous results on generalization bounds. The fourth sentence concludes that, based on these shortcomings, the paper cannot be recommended for acceptance.' But the first sentence obviously
hitmps//oppmai..oo/chhae hhnps//wwwaattnnpp..ciiieemtttoe

---

Document
Relation.
GPT3.5
ECASE
The paper studies the problem DNN loss function design reducing intra- class variance the output feature space. The key contribution proposing an isotropic variant of the softmax loss that balance the accuracy classification and compactness individual class. The proposed loss has been compared extensively against a number closely related approaches in methodology. Numerical results on benchmark datasets show some im- provement of the propused loss over softmax loss and center loss (Wen et al., 2016), when upplied to distance-bused classifiers such as k-NN k-means. Pros: idea of isotropic normalization class is well motivated. The paper mostly clearly organized and presented. Numerical study shows some promise the proposed method. Cons: The novelty of method mostly incremental given the prior work (Wen et al.,
(7,4,Supp)
(7,4,Supp)
enhancing compactness
2016) which has provided different isotropic variant softmax loss.
The paper is not anonymized. In page 2, the first line, the authors revealed [15) is self-citation. And [15] not anonumized in the reference list.
(2.1.Supp)
(2,1,Supp)
(2,1
(3,1.Supp)
(3.1,Supp)
(3,1.Supp)
It is clear that the problem studied in this paper is interesting. However, after reading through the manuscript, it is not clear to what are the real contributions made in this paper. I also failed to find rigorous results generalization bounds. In this way, I cannot recommend the acceptance of this paper.
(4.2.Supp)
(4.1.Supp)
(4,2,Supp)
(4.3.Supp)
(4.2.Supp)
(4,3.Sup))
(4.3,Supp)
Why should the consumer pay filing fee all if the collector is fault?
(1,2.Supp)
(1.3,Supp)
(1,2.Supp)
That could be hardship many people. The collection agencies need follow the rules of doing their validation correctly. This would not be issue.
Texts not work consumers. Consumers must pay for texts. this is already against the will rightly so.
(I,2,Supp)
(1,2,Atk)
no-rel
(1.3.Supp)
(1,3.Atk)
would like to have some protection from the calls I have received over several years from debt collection services looking for a woman who does not live at this address and has never lived at this address. keep getting reassurances that my number will be removed, but the calls continue.
(1,2,Supp)
(1,2,Supp)
no-rel
Table 4: Case Study of ASE, In the column of Relation., the first term is head, the second term tail, and the third term the argumentative relation from tails to heads. The cases are tested head-given setting.


Reviews:
[{"review_id": "mF65cDKuFz", "reviewer": "Reviewer_LkxQ", "summary": "", "questions": "- (A)  It is quite puzzling that both CASE+Aug and CASE+EMC perform better than CASE on AbstRCT, but combining Aug with EMC slightly hurts the performance. Is it the masking from Aug that is making EMC less effective, or is it the additional loss term from EMC that is making training on the augmented data more difficult? Which one of Aug and EMC is reducing the gain from the other?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the careful comments and suggestions provided by the reviewer. \n\n**Question**: Why ECASE seems to perform slightly worse than CASE+Aug and CASE+EMC on the AbstRCT dataset.\n\n**Answer**: For the reviewer's concern, given the significant differences in language expressions across these domains, the effects of the EMC and Aug modules also vary. When these modules are combined, the information complexity increases, potentially necessitating additional steps to achieve optimal performance. We have kept the training hyper-parameters consistent across all datasets to maintain fairness. Our findings indicate that extending the training duration improves the performance of ECASE in Abst_RCT compared to CASE-EMC and CASE-Aug. We will incorporate this observation into the description in our forthcoming version.\n\nWe acknowledge the typographical errors and will rectify them in the next version. We will also modify the descriptions as you suggest.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The core idea behind the development of this method is clear and well-motivated to improve context-aware ASE such that the model learns from informative but distant context without being negatively impacted by the length of context.\n\n- The method is well-designed and clearly explained, and it can accommodate custom changes or additions to its various components for future developments.\n\n- The authors have appropriately introduced a weighted regularization term in the training loss to take advantage of similarities between representations of propositions with argumentative relation. \n\n- The experiments are thorough with sufficient comparison with benchmark models some of which are specifically designed for context-aware ASE. \n\n- The experiments show the success of ECASE in achieving its intended goals. In particular, the ablation study shows that both model capacity enhancement and data augmentation are improving the performance.", "reasons_to_reject": "- While not a weakness of the method itself, why ECASE seems to perform slightly worse than CASE+Aug and CASE+EMC on the AbstRCT dataset (Figure 4) is not addressed. Perhaps a discussion of specific features of this dataset that the authors think could cause this could help.", "questions_to_authors": "- (A)  It is quite puzzling that both CASE+Aug and CASE+EMC perform better than CASE on AbstRCT, but combining Aug with EMC slightly hurts the performance. Is it the masking from Aug that is making EMC less effective, or is it the additional loss term from EMC that is making training on the augmented data more difficult? Which one of Aug and EMC is reducing the gain from the other?", "paper_topic_and_main_contributions": "The paper proposes an efficient context-aware framework for argument structure extraction (ASE). The method, ECASE, aims to enhance the model capacity by learning the context more efficiently than context windows, while taking advantage of a data augmentation that helps the model learn from more informative sentences. The model capacity is enhanced via an attention module. The data augmentation is implemented through randomly masking discourse markers and sentences. The experiments show that ECASE outperforms context-aware baselines on a suite of benchmark datasets, and the ablation study confirms the effectiveness of both model capacity enhancement and data augmentation towards the success of ECASE.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- It seems like the first legend in Figure 4 should be “CASE” (instead of “CACE”).\n\n- Please spell out \"PLM\" before using the abbreviated form (line 158). \n\n- Although clear to familiar readers, it would be stylistically better to briefly mention what $\\mathcal{L}_{cls}$ (line 188) is in the text. It is only specified in Figure 3.\n\n- Experiments with different head to tail distances are reported in Appendix E, but since the introduction specifically motivates the research to learn efficiently from longer context, this deserves at least a reference to the appendix in the main text."}}, {"review_id": "ZxIx5JZNL9", "reviewer": "Reviewer_d9xz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the careful comments and suggestions provided by the reviewer. \n\n**Question 1**: The results of the ablation studies are not quite clear. \n\n**Answer**: We will improve the visualization of the results. Given the significant differences in language expressions across these domains, the effects of the EMC and Aug modules also vary. But generally, ECASE achieves stronger performance compared with CASE-EMC and CASE-Aug, and a significant improvement compared with the baseline CASE. \n\n**Question 2**: All the techniques seem marginal and irrelevant.\n\n**Answer**: In this study, our primary focus is on the efficient way to utilize the contextual information inherent in the data. We approach this problem from various perspectives, including data analysis and modeling capacity, the latter of which encompasses both model structure and training objectives. For modeling capacity, we introduce a sequence-attention module and distance-weighted similarity loss to aggregate contextual information and argumentative information. For data augmentation, we augment the training data by randomly masking discourse markers and sentences, which reduces the model's reliance on specific words or less informative sentences.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The techniques are presented clearly and contribute to the performance. \n- The presented method ECASE outperforms the baselines in most of the settings.", "reasons_to_reject": "- The results of the ablation studies are not quite clear. The authors could at least plot more horizontal lines or display the numbers. In the current plot, the improvement of ECASE over CASE+EMC seems marginal on AbstRCT, AMPERE, and Essays.\n- All the techniques seem marginal and irrelevant. This paper can have a better story if the techniques of concatenating [CLS] tokens of in-context sentences and data augmentation can be more connected.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on argument structure extraction, which is formulated as a 3-way classification task. To solve this task, this paper presents several techniques including: \n  1. sequence attention module on RoBERTa (concat the [CLS] tokens of all sentences in the context window).\n  2. distance-weighted similarity loss.\n  3. data augmentation with randomly masked discourse markers and randomly masked random sentences.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wVyUGwaWqD", "reviewer": "Reviewer_xDHq", "summary": "", "questions": "\n\nThis papaer can be improved according to the following suggestions.\n1. In general, the model explanation is relatively clear, but the model design is simple, the focus of work is not prominent enough, and the sequence-attention and discourse marker are outdated.\nThe discourse marker was similar to Hua and Wang (2022).\nXinyu Hua and Lu Wang. 2022. Efficient argument structure extraction with transfer learning and active learning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 423–437.\n2. There are few comparison methods in the experimental part, which can not effectively explain the superiority of the model performance.\n3. The experimental results are slightly different from the paper \"Efficient Argument Structure Extraction with Transfer Learning and Active Learning\", and the author needs to give corresponding explanations.\n4. There are some errors, such as CACE in the legend in Figure 4, which is not mentioned in the paper.\n5. In the ablation experiment, the corresponding experiment for enhancing modeling capacity (CASE+EMC) is given. The author should compare the experimental results of removing sentence-level attention module with argumentative relation regularization.\n", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the careful comments and suggestions provided by the reviewer.\n\n**Question 1**: The model design is simple, the focus of work is not prominent enough.\n\n**Answer**: In this study, we identify several issues with ASE models, including the impact of discourse markers and the distances between sentences. Based on our analysis, our goal is to propose simple yet effective solutions to these problems. Our focus is not on introducing highly novel techniques. Regarding discourse markers, they serve as crucial signals for detecting argumentative relations, a factor also emphasized in previous studies' annotations. However, we aim to reduce the models' dependence on these discourse markers and instead encourage a deeper understanding of the contextual information within the content. We believe this approach significantly diverges from previous studies. We will emphasize these differences in our forthcoming version. \n\n**Question 2**: There are few comparison methods in the experimental part.\n\n**Answer**: Our study is mainly compared with Hua et al (2022), which is shown to be SOTA in this specific task, and we also consider some other baselines, such as SVM-Linear, SVM-RBF, SEQPAIR, and SEQCON, which are the mainstream methods in this field. \n\n**Question 3**: The experimental results are slightly different from the paper \"Efficient Argument Structure Extraction with Transfer Learning and Active Learning.\n\n**Answer**: We reproduce the experimental results based on the codes of Efficient Argument Structure Extraction with Transfer Learning and Active Learning (Hua et al. (2022)) released. When carrying out experiments, there are several differences in our experimental setting. First, we do not adopt data for self-supervised training of the models to adapt to different domains as Hua et al. (2022), since it increases the complexity of the model. Second, we find that when training the model in more epochs (compared with their original work), the performance is enhanced in short context windows. Thus, we report both the results of CASE they reported and those we reproduced in the manuscript. \n\n**Question 4**: There are some errors, such as CACE in the legend in Figure 4, which is not mentioned in the article.\n\n**Answer**: Sorry for the typos. It should be CASE, and we will modify this issue.\n\n**Question 5**: The author should compare the experimental results of removing sentence-level attention module with argumentative relation regularization.\n\n**Answer**: In this version, we carry out the ablation study with respect to EMC and Aug (according to the motivation of our work) to show the effectiveness of our model. We will add the results of the ablation study you mentioned in the next version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper proposes an Efficient Context-aware ASE model (ECASE) that fully exploits contextual information. Specifically, introduce a sequence-attention module, a distance-weighted similarity loss, and a discourse marker.", "reasons_to_reject": "\n\n\n1. In general, the model explanation is relatively clear, but the model design is simple, the focus of work is not prominent enough, and the sequence-attention and discourse marker are outdated.\nThe discourse marker was similar to Hua and Wang (2022). \nXinyu Hua and Lu Wang. 2022. Efficient argument structure extraction with transfer learning and active learning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 423–437.\n2. There are few comparison methods in the experimental part, which can not effectively explain the superiority of the model performance.\n3. The experimental results are slightly different from the paper \"Efficient Argument Structure Extraction with Transfer Learning and Active Learning\", and the author needs to give corresponding explanations.\n4. There are some errors, such as CACE in the legend in Figure 4, which is not mentioned in the article.\n5. In the ablation experiment, the corresponding experiment for enhancing modeling capacity (CASE+EMC) is given. The author should compare the experimental results of removing sentence-level attention module with argumentative relation regularization.\n", "questions_to_authors": "\n\nThis papaer can be improved according to the following suggestions.\n1. In general, the model explanation is relatively clear, but the model design is simple, the focus of work is not prominent enough, and the sequence-attention and discourse marker are outdated.\nThe discourse marker was similar to Hua and Wang (2022).\nXinyu Hua and Lu Wang. 2022. Efficient argument structure extraction with transfer learning and active learning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 423–437.\n2. There are few comparison methods in the experimental part, which can not effectively explain the superiority of the model performance.\n3. The experimental results are slightly different from the paper \"Efficient Argument Structure Extraction with Transfer Learning and Active Learning\", and the author needs to give corresponding explanations.\n4. There are some errors, such as CACE in the legend in Figure 4, which is not mentioned in the paper.\n5. In the ablation experiment, the corresponding experiment for enhancing modeling capacity (CASE+EMC) is given. The author should compare the experimental results of removing sentence-level attention module with argumentative relation regularization.\n", "paper_topic_and_main_contributions": "The topic of this paper is about approaches for data- and compute efficiency.\nThe main contributions of this paper are as follows: \n(1)This paper proposes an Efficient Context-aware ASE model (ECASE) that fully exploits contextual information. Specifically, introduce a sequence-attention module, a distance-weighted similarity loss, and a discourse marker.\n(2)The comparative experiments on five datasets from various domains demonstrate that the proposed model achieves state-of-the-art performance.\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a context-aware framework for argument structure extraction (ASE). The method, ECASE, consists of a sequence attention module, a distance-weighted similarity loss, and a discourse marker masking technique. The paper evaluates ECASE on five benchmark datasets and shows that it outperforms context-aware baselines on most of them.

The reviewers have mixed opinions about the paper. Reviewer 1 is very positive and praises the paper for its clear motivation, well-designed method, and thorough experiments ablation study. Reviewer 2 is ambivalent and gives two reasons to accept the paper (clear presentation and superior performance) and two reasons to reject the paper (unclear ablation results and marginal techniques). The reviewer was not convinced by the authors' response. Reviewer 3 is borderline and gives only one reason to accept the paper (exploiting contextual information) and several reasons to reject the paper (outdated model design, lack of comparison methods, inconsistency with a previous paper, and missing ablation experiment). This reviewer didn't change their opinion after reading the authors' response to these concerns. 

Based on these reviews and other comments, I think the paper has some merits in proposing an efficient context-aware ASE model that achieves state-of-the-art results on some datasets, but it also has some weaknesses in its comparison with older work in terms of model design, ablation analysis, and presentation quality. The authors should address the reviewers’ comments and questions in their revision and clarify their contributions and limitations.
Decision: Accept-Findings