Using Subtext to Enhance Generative IDRR
Authors: Zhipang Wang, Yu Hong, Weihao Sun, Guodong Zhou
Accepted at: acl25

Using Subtext to Enhance Generative IDRR

Zhipang Wang, Yu Hong*, Weihao Sun, Guodong Zhou
School of Computer Science and Technology, Soochow University, Suzhou, China
{zhipangwang, tianxianer, whs16}@gmail.com; gdzhou@suda.edu.cn

Abstract

Implicit Discourse Relation Recognition (abbr., IDRR) is a NLP task of classifying argument pairs into different types of semantic relations. Arguments contain subtexts, some of which are beneficial to the perception of semantic relations. However, subtexts are connotative. The neural IDRR model fails to be aware of them without being given pertinent prompts. In this paper, we leverage LLaMA to generate subtexts for argument pairs, and verify the effectiveness of subtext-based IDRR. We construct an IDRR baseline using the decoder-only backbone LLaMA, and enhance it with subtext-aware relation reasoning. A confidence-diagnosed dual-channel network is used for collaboration between in-subtext and out-of-subtext IDRR. We experiment on PDTB-2.0 and PDTB-3.0 for both the main-level and secondary-level relation taxonomies. The test results show that our approach yields substantial improvements compared to the baseline, and achieves higher \( F_1 \)-scores on both benchmarks than the previous decoder-only IDRR models. We make the source codes and data publicly available.¹

1 Introduction

IDRR determines the semantic relation between arguments when the in-between connective is absent (Prasad et al., 2008). For example, it outputs the relation “Concession” for the arguments Arg1 and Arg2 in 1), where the possible connective “however” is not given in the source text:

1) Arg1: The new rate will be payable Feb. 15.
   Arg2: A record date hasn’t been set.
   Relation: Concession

Encoder-only language models such as RoBERTa (Long and Webber, 2022; Wu et al., 2023; Cai et al., 2024) and XLNet (Jiang et al., 2024) have been used for IDRR, where multi-class relation classification is conducted by linear layers with Softmax. Meanwhile, both T5 (Jiang et al., 2021; Chan et al., 2023) and the decoder-only Large Language Mdoels (LLMs) like GPT-3.5 (Chan et al., 2024) and GPT-4 (Yung et al., 2024) have also been verified for IDRR, where relations are properly generated conditioned on prompts and/or CoT. Significant improvements are reported in these arts.

Subtext hasn’t been considered in the study of IDRR. Though, it is potentially useful for enhancing the IDRR models. A subtext is characterized by the metaphorical meaning hidden in the arguments. For example, the subtext of the two arguments in 1), most probably, is “the rate should be recorded earlier though it hasn’t been”. Such a subtext is more explicit or even straight in revealing the Concessive relation. Accordingly, we suggest that subtext can be used as a crucial evidence for enhancing the perception of implicit relation.

In this paper, we explore the method of applying subtexts, and systematically investigate the effectiveness upon LLM-based generative IDRR. The effort we made is to provide a preliminary study and stimulate innovative researches in subtext-based IDRR enhancement. Specifically, our contributions are summarized as follows:

• We first propose a Subtext-based Confidence-diagnosed Dual-channel Network (SCDN) for IDRR. In SCDN, subtext is generated by LLM. Confidence comparison is conducted to reconcile in-subtext and out-of-subtext IDRR.

• We verify the effectiveness of SCDN on the benchmarks PDTB-2.0 and 3.0 (Webber et al., 2019). We report varied influences cased by the settings of prompting, confidence diagnosis, subtext generation and augmentation.

¹Corresponding author
¹https://github.com/ZpWang-AI/IDRR_Subtext.
2 Approach

Figure 1 shows the architecture of SCDN, which is constructed with three LLMs \( \mathcal{M}_\alpha, \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \). \( \mathcal{M}_\alpha \) serves to generate the subtext for the given arguments. \( \mathcal{M}_\beta \) takes the arguments as input and uses them as the only reliance for relation reasoning. \( \mathcal{M}_\lambda \) combines the generated subtext and arguments, and infers the relation according to all of them. A probabilistic diagnosis model (diagnoser) is used to reconcile the decisions from \( \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \) based on confidence estimation.

2.1 Subtext and Relation Generators

Our baseline is the generator \( \mathcal{M}_\beta \) which performs out-of-subtext IDRR. We prompt it by Question Answering (QA). Given the arguments \( \hat{A} \) and \( \tilde{A} \), we combine them with a question \( Q_\beta \) of “what is the relation between arguments”: \( I_\beta = \{[\hat{A}, \tilde{A}]; Q_\beta\} \). We feed \( I_\beta \) into \( \mathcal{M}_\beta \) to generate a relation label.

To fulfill the in-subtext IDRR, we construct a subtext generator \( \mathcal{M}_\alpha \). Its input is formed by the arguments and a prompting question \( Q_\alpha \) of “what is the implicit meaning”: \( I_\alpha = \{[A, \tilde{A}]; Q_\alpha\} \). There isn’t any constraint applied to subtext generation (i.e., \( \mathcal{M}_\alpha(I_\alpha) \)) such as the length of subtext. Further, we build the generator \( \mathcal{M}_\lambda \) to perform in-subtext IDRR. It uses both subtext and arguments as input, and combines them with a multi-choice question \( Q_\lambda \): \( I_\lambda = \{[\hat{A}, \tilde{A}]; \{S\}; Q_\lambda\} \). The question \( Q_\lambda \) is designed as “what is the relation between arguments given subtext”, which allows \( \mathcal{M}_\lambda \) to generate a relation label in the manner of multi-choice QA (Yung et al., 2024) as follows.

2) \( Q_\lambda \): What is the relation of \( \hat{A} \) and \( \tilde{A} \) given \( S \)?
A. Contingency   B. Expansion
C. Temporality   D. Comparison

In our experiments, we uniformly use LLaMA3-8B-Instruct (Dubey et al., 2024) to construct the generators \( \mathcal{M}_\alpha, \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \). Due to the zero-resource situation that there isn’t any ground-truth subtext provided in PDTB-2.0 and 3.0, we train the subtext generator \( \mathcal{M}_\alpha \) by teacher-student knowledge distillation (Hu et al., 2023). GPT-3.5-turbo (Brown et al., 2020) is used as the teacher.

2.2 Confidence Diagnoser

It is unavoidable that the subtext-based generator encounters two problems, including 1) arguments inherently don’t contain a subtext, and 2) the generated subtext is unqualified. To relieve the problems, we use a diagnoser to reconcile \( \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \), where \( \mathcal{M}_\beta \) conducts out-of-subtext IDRR, while \( \mathcal{M}_\lambda \) additionally uses subtext for in-subtext IDRR.

Assume \( \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \) output the relations \( R_\beta \) and \( R_\lambda \) respectively. The diagnoser first verifies the reliability of \( R_\lambda \). Confidence score \( \mathcal{C} \) is measured for verification. \( \mathcal{C} \) is an average logistic probability over all the tokens output by \( \mathcal{M}_\beta \). Each \( c_i \in \mathcal{C} \) is the non-normalized probability estimated by the logistic function in the final layer of LLaMA3:

\[
\mathcal{C} = \frac{1}{\sum_{t_i \in R_\beta} log_{\mathcal{M}_\beta}(t_i)}
\]

On this basis, the diagnoser verifies whether \( \mathcal{C} \) is larger than the type-specific threshold \( \theta \). If larger, the diagnoser determines that \( R_\lambda \) is reliable for output, otherwise the prediction \( R_\beta \) of \( \mathcal{M}_\beta \) is adopted. In our experiments, we provide an exclusive threshold for each relation type in the taxonomy of PDTB. They are obtained by empirical observation upon the IDRR performance obtained on the training set. More details of threshold settings and performance curves can be found in Appendix A.

3 Experiments

3.1 Dataset and Evaluation Metrics

We experiment on two versions of discourse relation analysis datasets, including PDTB-2.0 (Prasad et al., 2008) and 3.0 (Webber et al., 2019). We follow the previous work to use sections 0-22 for IDRR, where sections 2-20 are used for training, and sections 0-1 are used as the development set, while 21-22 for testing. Appendix B shows the data statistics in all the datasets.

Multi-class Macro-F1 (\( F_1 \)) and accuracy rate (\( Acc \)) are used as the evaluation metrics.

3.2 Implementation Details

We use AdamW optimizer (Loshchilov and Hutter, 2019) to optimize LLaMA3. For subtext generation \( \mathcal{M}_\alpha \), the learning rate is set to 1e-4. A 5-epoch training process is conducted. For the relation generators \( \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \), the learning rates are uniformly set to 5e-5, and the best checkpoint is reached based on \( F_1 \) within 10 epochs. Both
<table>
  <tr>
    <th>Method</th>
    <th>Backbone Model</th>
    <th>Parameters</th>
    <th colspan="2">PDTB2</th>
    <th colspan="2">PDTB3</th>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th></th>
    <th>F<sub>1</sub></th>
    <th>Acc</th>
    <th>F<sub>1</sub></th>
    <th>Acc</th>
  </tr>
  <tr>
    <td>ChatGPT (Chan et al., 2024)</td>
    <td>GPT-3.5-turbo</td>
    <td>-</td>
    <td>36.11</td>
    <td>44.18</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>PIDRA (Yung et al., 2024)</td>
    <td>GPT-4</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>47.53</td>
    <td>52.84</td>
  </tr>
  <tr>
    <td>FCL (Long and Webber, 2022)</td>
    <td>RoBERTa-base</td>
    <td>125M</td>
    <td>69.60</td>
    <td>72.18</td>
    <td>70.05</td>
    <td>75.31</td>
  </tr>
  <tr>
    <td>CP-KD (Wu et al., 2023)</td>
    <td>RoBERTa-base</td>
    <td>125M</td>
    <td>68.86</td>
    <td>75.43</td>
    <td>72.07</td>
    <td>77.00</td>
  </tr>
  <tr>
    <td>CP-KD (Wu et al., 2023)</td>
    <td>RoBERTa-large</td>
    <td>355M</td>
    <td><b>71.88</b></td>
    <td><b>76.77</b></td>
    <td><b>75.52</b></td>
    <td><b>78.56</b></td>
  </tr>
  <tr>
    <td>SCIDER (Cai et al., 2024)</td>
    <td>RoBERTa-base</td>
    <td>125M</td>
    <td>67.00</td>
    <td>72.11</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>OTMT (Jiang et al., 2024)</td>
    <td>XLNet-large</td>
    <td>355M</td>
    <td>64.46</td>
    <td>72.34</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>CG-T5 (Jiang et al., 2021)</td>
    <td>T5-base</td>
    <td>223M</td>
    <td>57.18</td>
    <td>65.54</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>DiscoPrompt (Chan et al., 2023)</td>
    <td>T5-base</td>
    <td>223M</td>
    <td>65.79</td>
    <td>71.70</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>DiscoPrompt (Chan et al., 2023)</td>
    <td>T5-large</td>
    <td>738M</td>
    <td>70.84</td>
    <td>75.65</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>IICOT (Lu et al., 2023)</td>
    <td>Flan-T5-base</td>
    <td>248M</td>
    <td>65.26</td>
    <td>71.13</td>
    <td>69.79</td>
    <td>73.98</td>
  </tr>
  <tr>
    <td>IICOT (Lu et al., 2023)</td>
    <td>Flan-T5-large</td>
    <td>783M</td>
    <td>69.23</td>
    <td>76.04</td>
    <td>73.06</td>
    <td><b>77.46</b></td>
  </tr>
  <tr>
    <td>Baseline</td>
    <td>Llama3-8B-Instruct</td>
    <td>8.03B</td>
    <td>66.72</td>
    <td>73.90</td>
    <td>70.71</td>
    <td>75.31</td>
  </tr>
  <tr>
    <td>SCDN (ours)</td>
    <td>Llama3-8B-Instruct</td>
    <td>8.03B</td>
    <td><b>71.14</b></td>
    <td><b>78.20</b></td>
    <td><b>73.33</b></td>
    <td><b>76.93</b></td>
  </tr>
</table>

Table 1: Performance on PDTB 2.0/3.0. Encoder-only PLMs, decoder-only LLMs and T5-based encoder-decoder models are considered. The best results are separately marked in bold for encoder-only and decoder-only models.

<table>
  <tr>
    <th>Model</th>
    <th colspan="2">PDTB2</th>
    <th colspan="2">PDTB3</th>
  </tr>
  <tr>
    <th></th>
    <th>F<sub>1</sub></th>
    <th>Acc</th>
    <th>F<sub>1</sub></th>
    <th>Acc</th>
  </tr>
  <tr>
    <td>Out-of-subtext</td>
    <td>66.72</td>
    <td>73.90</td>
    <td>70.71</td>
    <td>75.31</td>
  </tr>
  <tr>
    <td>In-subtext</td>
    <td>70.56</td>
    <td>77.82</td>
    <td>72.79</td>
    <td>76.32</td>
  </tr>
  <tr>
    <td>SCDN</td>
    <td><b>71.14</b></td>
    <td><b>78.20</b></td>
    <td><b>73.33</b></td>
    <td><b>76.93</b></td>
  </tr>
</table>

Table 2: Test results in ablation study.

employ a weight decay of 1e-2, a batch size of 1, and gradient accumulation over 8 steps. We don’t extensively tune the hyperparameters.

All experiments are performed on a NVIDIA A100 GPU. Our model implementations are based on PyTorch 2 and the Transformers library3.

3.3 Main Results
We compare SCDN to the recently-proposed advanced models, including 1) decoder-only ChatGPT (Chan et al., 2024) and PIDRA (Yung et al., 2024), 2) encoder-only FCL (Long and Webber, 2022), CP-KD (Wu et al., 2023), SCIDER (Cai et al., 2024), and OTMT (Jiang et al., 2024), as well as 3) T5-based CG-T5 (Jiang et al., 2021), DiscoPrompt (Chan et al., 2023), and IICOT (Lu et al., 2023). The decoder-only models take full advantage of prompt engineering for IDR. The encoder-only models are effective in representation learning for relation understanding. T5-based models combine the advantages. More contributions of these arts are summarized in Appendix C.

Table 1 shows the comparison results on the test set, where the 4-way relation classification performance on the main relation taxonomy is reported. It can be observed that our SCDN achieves higher F1-score than both the decoder-only and T5-based IDR models. However, it still fails to outperform the encoder-only models. This is partially attributed to hallucination of LLaMA3 and off-topic results it generated.

Besides, we conduct experiments for the 2nd-level taxonomy, where each main relation type is divided into fine-grained relation senses. For example, the relation type “Contingency” contains the senses of “Conditionality” and “Causality”. In this experiment, SCDN shows promising performance, which is reported in Appendix D due to page limit.

3.4 Ablation Study
To provide a direct insight into the influence of subtexts, we conduct an ablation study. Three IDR models are considered in it, including 1) out-of-subtext generator \( \mathcal{M}_\beta \) which is separately fine-tuned without using subtexts, 2) in-subtext generator \( \mathcal{M}_\lambda \) which additionally uses the generated subtexts during fine-tuning, and 3) SCDN which uses both \( \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \), and reconciles them with the confidence-based diagnoser.

Table 2 shows the test results for the main relation taxonomy. It proves that the utilization of
<table>
  <tr>
    <th>Subtext Generation Model</th>
    <th>\( F_1 \)</th>
    <th>\( Acc \)</th>
  </tr>
  <tr>
    <td>GPT-3.5-turbo</td>
    <td>71.55</td>
    <td>75.98</td>
  </tr>
  <tr>
    <td>LLaMA3 w/o Distill</td>
    <td>71.07</td>
    <td>75.37</td>
  </tr>
  <tr>
    <td>LLaMA3 w/ Distill (Partial)</td>
    <td>71.66</td>
    <td>76.19</td>
  </tr>
  <tr>
    <td>LLaMA3 w/ Distill (Whole)</td>
    <td><b>72.79</b></td>
    <td><b>76.32</b></td>
  </tr>
</table>

Table 3: Contributions from different subtext generators.

subtexts yields various levels of improvements. Appendix E provides a case study to show the effect.

3.5 Comparison among Subtext Generators

The qualified subtexts are crucial for SCDN. We investigate the subtexts generated by different LLMs, and verify their effects by our in-subtext model. There are three types of LLMs considered, including 1) GPT-3.5-turbo, 2) LLaMA3-8B-Instruct, 3) LLaMA3 which is strengthened by teacher-student knowledge distillation. During distillation, the subtexts generated by GPT-3.5-turbo for all training data are specified as Guidance Data from teacher. We use two different-sized guidance data: Partial and Whole. In the “Partial” case, we only adopt the guidance data which enables the subtext-based relation generator \( \mathcal{M}_\lambda \) to output correct results. In the “Whole” case, all the guidance data is used.

Table 3 shows the performance of in-subtext IDRR models on the test set of PDTB 3.0, where different subtext generators are used. It can be observed that, compared to LLaMA3 (w/o distillation), GPT-3.5 enables the in-subtext model to perform better. Furthermore, no matter whether “Partial” or “Whole” guidance data is used, knowledge distillation causes improvements, and the latter case improves the in-subtext model more substantially. It is surprising that distillation allows the weaker LLaMA3 to be more contributive than its teacher GPT-3.5. The possible reason is because that LLaMA3 takes the advantage of itself when absorbing beneficial experience from GPT-3.5.

3.6 Prompts for Subtext Generation

The reliability of subtexts relies heavily on the design methods of prompts. For example, if we didn’t remind LLMs of the ultimate purpose (i.e., being applied for IDRR), they fail to provide reliable subtexts. In our experiments, we evaluate different prompts as follows:

• P₁: It contains \( Q_1 \) and \( Q_2 \) (Section 2.1) that ask for subtext generation and IDRR in turn.

<table>
  <tr>
    <th>Prompt</th>
    <th>LLM</th>
    <th>\( F_1 \)</th>
    <th>\( Acc \)</th>
  </tr>
  <tr>
    <td>P₁</td>
    <td>GPT-3.5-turbo</td>
    <td>35.88</td>
    <td>41.79</td>
  </tr>
  <tr>
    <td>P₂</td>
    <td>GPT-3.5-turbo</td>
    <td>38.30</td>
    <td>42.76</td>
  </tr>
  <tr>
    <td>P₃</td>
    <td>GPT-3.5-turbo</td>
    <td><b>39.24</b></td>
    <td><b>43.72</b></td>
  </tr>
  <tr>
    <td>P₁</td>
    <td>GPT-4-turbo</td>
    <td>44.84</td>
    <td>50.07</td>
  </tr>
  <tr>
    <td>P₂</td>
    <td>GPT-4-turbo</td>
    <td><b>46.29</b></td>
    <td><b>52.21</b></td>
  </tr>
</table>

Table 4: Reliability of Prompts (on PDTB 3.0).

• P₂: It replaces the key words in \( P_1 \) with their different synonyms, e.g., “subtext” is replaced with “<i>implicit meaning</i>”.

• P₃: It expands P₂ by adding a prefix to \( Q_1 \), where the prefix is an additional question about “<i>whether there truly is a subtext in the considered argument</i>”. This prompt helps to avoid a forcible subtext generation.

Table 4 shows the IDRR performance on the development set when the above prompts are separately used, where GPT-3.5 and 4.0 are considered during the validation process. It can be observed that both synonym replacement and non-forcible subtext generation yield improvements. Accordingly, P₂ has been introduced into our SCDN for optimization. However, P₃ fails to be used in SCDN as it actually causes performance degradation. For example, by P₃, the in-subtext “Single” generator \( \mathcal{M}_\beta \) obtains a \( F_1 \)-score of 71.7% on the test set of PDTB 3.0, causing a performance reduction of about 1.1% (compared to the “Single” case in Table 2). This implies that LLaMA3 in \( \mathcal{M}_\beta \) isn’t able to effectively perform reasoning with a relatively-complex Chain-of-Thought (CoT). Besides, P₃ also causes severe performance reduction when it is used in SCDN. This is because that a limited number of subtexts are generated by GPT-3.5 due to the constraint from P₃, and thus GPT-to-LLaMA distillation falls into the low-resource scenario.

4 Conclusion

In this paper, we verify that the utilization of subtexts helps to strengthen the LLMs-based generative IDRR. Experiments demonstrate that reconciliation of in-subtext and out-of-subtext IDRR is effective. We also exhibit that distilling a lightweight LLM-based subtext generator is contributive, when the prompt doesn’t raise a complex CoT.

In the future, we will investigate the default subtext which isn’t implied in the given argument. On
the contrary, it derives from common-sense knowledge. Accordingly, we will convert binary arguments analysis to triplet, where the default subtext is regarded as the third non-negligible argument.

5 Limitations

This study proves the effectiveness of utilizing subtexts for enhancing IDRR. Nevertheless, a deeper analysis upon subtexts is still required. Our findings reveal that, in some cases, the shareable subtext is implied in one argument but irrelevant to the other, where the irrelevant argument appears as the noise during detecting the subtext. In some other cases, the default subtext occurs, which is not implied in any argument but derives from the common-sense knowledge. However, the subtext generation method in this paper cannot deal with these two problems. In the future, we will firstly study the common sense based default subtext generation. On this basis, we will convert the conventional binary arguments analysis to triplet, where the default subtext is used as a supplementary argument. This work will encounter the issues of 1) how to determine whether a pair of arguments are relevant to some default subtexts, and what they are, 2) how to detect and generate default subtexts conditioned on common-sense knowledge, and 3) how to reconcile the utilization of a triple of arguments and assign proper attention to them during relation discrimination.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under No.62376182.

References

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.

Mingyang Cai, Zhen Yang, and Ping Jian. 2024. Improving implicit discourse relation recognition with semantics confrontation. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 8828–8839, Torino, Italia. ELRA and ICCL.

Chunkit Chan, Cheng Jiayang, Weiqi Wang, Yuxin Jiang, Tianqing Fang, Xin Liu, and Yangqiu Song. 2024. Exploring the potential of ChatGPT on sentence level relations: A focus on temporal, causal, and discourse relations. In Findings of the Association for Computational Linguistics: EACL 2024, pages 684–721, St. Julian’s, Malta. Association for Computational Linguistics.

Chunkit Chan, Xin Liu, Jiayang Cheng, Zihan Li, Yangqiu Song, Ginny Wong, and Simon See. 2023. DiscoPrompt: Path prediction prompt tuning for implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: ACL 2023, pages 35–57, Toronto, Canada. Association for Computational Linguistics.

Zujun Dou, Yu Hong, Yu Sun, and Guodong Zhou. 2021. CVAE-based re-anchoring for implicit discourse relation classification. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1275–1283, Punta Cana, Dominican Republic. Association for Computational Linguistics.

Abhimanyu Dubey, Abhinav Jauhari, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, and Angela Fan, et al. 2024. The Llama 3 herd of models. Preprint, arXiv:2407.21783.

Chengming Hu, Xuan Li, Dan Liu, Haolun Wu, Xi Chen, Ju Wang, and Xue Liu. 2023. Teacher-student architecture for knowledge distillation: A survey. Preprint, arXiv:2308.04268.

Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics, 3:329–344.

Congcong Jiang, Tiejun Qian, and Bing Liu. 2024. One general teacher for multi-data multi-task: A new knowledge distillation framework for discourse relation analysis. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:239–249.

Feng Jiang, Yaxin Fan, Xiaomin Chu, Peifeng Li, and Qiaoming Zhu. 2021. Not just classification: Recognizing implicit discourse relation on joint modeling of classification and generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2418–2431, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Yuxin Jiang, Linhan Zhang, and Wei Wang. 2023. Global and local hierarchy-aware contrastive framework for implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8048–8064, Toronto, Canada. Association for Computational Linguistics.
Wei Liu and Michael Strube. 2023. Annotation-inspired implicit discourse relation classification with auxiliary discourse connective generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15696–15712, Toronto, Canada. Association for Computational Linguistics.

Wei Liu, Stephen Wan, and Michael Strube. 2024. What causes the failure of explicit to implicit discourse relation recognition? Preprint, arXiv:2404.00999.

Wanqiu Long and Bonnie Webber. 2022. Facilitating contrastive learning of discourse relational senses by exploiting the hierarchy of sense relations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10704–10716, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. Preprint, arXiv:1711.05101.

Yuxiang Lu, Yu Hong, Zhipang Wang, and Guodong Zhou. 2023. Enhancing reasoning capabilities by instruction learning and chain-of-thoughts for implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5634–5640, Singapore. Association for Computational Linguistics.

Kazumasa Omura, Fei Cheng, and Sadao Kurohashi. 2024. An empirical study of synthetic data generation for implicit discourse relation recognition. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1073–1085, Torino, Italia. ELRA and ICCL.

Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Mitsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Association (ELRA).

Chenxu Wang, Ping Jian, and Mu Huang. 2023. Prompt-based logical semantics enhancement for implicit discourse relation recognition. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 687–699, Singapore. Association for Computational Linguistics.

Bonnie Webber, Rashmi Prasad, Alan Lee, and Aravind Joshi. 2019. The Penn Discourse TreeBank 3.0 annotation manual. Philadelphia, University of Pennsylvania, 35:108.

Changxing Wu, Liwen Cao, Yubin Ge, Yang Liu, Min Zhang, and Jinsong Su. 2022. A label dependence-aware sequence generation model for multi-level implicit discourse relation recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):11486–11494.

Hongyi Wu, Hao Zhou, Man Lan, Yuanbin Wu, and Yadong Zhang. 2023. Connective prediction for implicit discourse relation recognition via knowledge distillation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5908–5923, Toronto, Canada. Association for Computational Linguistics.

Wei Xiang, Bang Wang, Lu Dai, and Yijun Mo. 2022a. Encoding and fusing semantic connection and linguistic evidence for implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3247–3257, Dublin, Ireland. Association for Computational Linguistics.

Wei Xiang, Zhenglin Wang, Lu Dai, and Bang Wang. 2022b. ConnPrompt: Connective-cloze prompt learning for implicit discourse relation recognition. In Proceedings of the 29th International Conference on Computational Linguistics, pages 902–911, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.

Jing Xu, Ruifang He, Haodong Zhao, Huijie Wang, and Lei Zeng. 2023. Dual hierarchical contrastive learning for multi-level implicit discourse relation recognition. In Natural Language Processing and Chinese Computing, volume 14303, pages 55–66. Springer Nature Switzerland, Cham.

Frances Yung, Mansoor Ahmad, Merel Scholman, and Vera Demberg. 2024. Prompting implicit discourse relation annotation. Preprint, arXiv:2402.04918.

Lei Zeng, Ruifang He, Haowen Sun, Jing Xu, Chang Liu, and Bo Wang. 2024. Global and local hierarchical prompt tuning framework for multi-level implicit discourse relation recognition. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 7760–7773, Torino, Italia. ELRA and ICCL.

Haodong Zhao, Ruifang He, Mengnan Xiao, and Jing Xu. 2023. Infusing hierarchical guidance into prompt tuning: A parameter-efficient framework for multi-level implicit discourse relation recognition. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6477–6492, Toronto, Canada. Association for Computational Linguistics.

Hao Zhou, Man Lan, Yuanbin Wu, Yuefeng Chen, and Meirong Ma. 2022. Prompt-based connective prediction method for fine-grained implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3848–3858, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
<table>
  <tr>
    <th>Relation</th>
    <th>PDTB2-Top</th>
    <th>PDTB3-Top</th>
  </tr>
  <tr>
    <td>Comparison</td>
    <td>31.06</td>
    <td>29.97</td>
  </tr>
  <tr>
    <td>Contingency</td>
    <td>35.21</td>
    <td>33.45</td>
  </tr>
  <tr>
    <td>Expansion</td>
    <td>31.22</td>
    <td>29.83</td>
  </tr>
  <tr>
    <td>Temporal</td>
    <td>31.11</td>
    <td>28.53</td>
  </tr>
</table>

Table 5: Optimal thresholds for the main relation types.

A  Threshold for Confidence Diagnosis

We set up a threshold for each relation type in the taxonomies of IDRR. For example, there are 4 thresholds provided for the four main IDRR relation types (i.e., Expansion, Temporality, Contingency, and Comparison). The adoption of different thresholds is because that the varying token lengths of relation labels cause unbalanced ranges of average confidence scores.

Let us consider the relation type \( T \) as an example. To seek for the optimal threshold \( \theta_T \) for \( T \), we empirically observe the \( T \)-oriented IDRR performance curve obtained when different optional values are used as thresholds. Within this empirical observation, the IDRR generator \( \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \) is used to predict relations and confidence scores for all the instances that hold a relation of \( T \) in the training set. And the accuracy \( Acc_T \) is used as the performance metric, which is calculated as follows:

\[
Acc_T(\hat{\theta}) = \frac{n}{|D_T|}
\]

where, \( \hat{\theta} \) is an optional threshold which is sequentially sampled from the range of confidence scores in the training set. \( n \) is the number of argument pairs which are given a positive relation prediction by \( \mathcal{M}_\beta \) and \( \mathcal{M}_\lambda \), and \( |D_T| \) is the number of all argument pairs which hold the relation \( T \).

We adopt the optimal threshold \( \theta \) by maximum likelihood estimation on all optional thresholds:

\[
\theta_T = \arg\max_{\hat{\theta}_T \in \hat{\theta}_{all}}(Acc_T(\hat{\theta}_T))
\]

Figure 2 shows the curves of \( Acc_T \) changing with different thresholds in PDTB 3.0. We also present the specific values of the finally adopted thresholds \( \theta \) in Table 5, 6 and 7, where Table 5 provides the thresholds for the main relation types (labeled as “Top”), while Table 6 and 7 give the thresholds for the relation senses in the secondary-level taxonomies (labeled as “Second”).

![Four line plots showing the relationship between threshold of confidence score and acc for Comparison, Contingency, Expansion, and Temporal](page_1012_186_377_1202.png)

Figure 2: \( Acc_T \) on the training dataset with varying thresholds. The final selected thresholds are marked with red dots, which corresponds to the highest \( Acc_T \).
<table>
  <tr>
    <th>Relation</th>
    <th>PDTB2-Sec</th>
  </tr>
  <tr>
    <td>Comparison.Concession</td>
    <td>28.10</td>
  </tr>
  <tr>
    <td>Comparison.Contrast</td>
    <td>29.10</td>
  </tr>
  <tr>
    <td>Contingency.Cause</td>
    <td>30.79</td>
  </tr>
  <tr>
    <td>Contingency.Pragmatic cause</td>
    <td>29.05</td>
  </tr>
  <tr>
    <td>Expansion.Alternative</td>
    <td>31.68</td>
  </tr>
  <tr>
    <td>Expansion.Conjunction</td>
    <td>28.02</td>
  </tr>
  <tr>
    <td>Expansion.Instantiation</td>
    <td>30.80</td>
  </tr>
  <tr>
    <td>Expansion.List</td>
    <td>28.48</td>
  </tr>
  <tr>
    <td>Expansion.Restatement</td>
    <td>27.56</td>
  </tr>
  <tr>
    <td>Temporal.Asynchronous</td>
    <td>28.12</td>
  </tr>
  <tr>
    <td>Temporal.Synchrony</td>
    <td>28.25</td>
  </tr>
</table>
Table 6: Thresholds for all relation senses of PDTB 2.0.

<table>
  <tr>
    <th>Relation</th>
    <th>PDTB3-Sec</th>
  </tr>
  <tr>
    <td>Comparison.Concession</td>
    <td>28.01</td>
  </tr>
  <tr>
    <td>Comparison.Contrast</td>
    <td>30.75</td>
  </tr>
  <tr>
    <td>Contingency.Cause</td>
    <td>32.01</td>
  </tr>
  <tr>
    <td>Contingency.Cause+Belief</td>
    <td>29.41</td>
  </tr>
  <tr>
    <td>Contingency.Condition</td>
    <td>30.89</td>
  </tr>
  <tr>
    <td>Contingency.Purpose</td>
    <td>30.31</td>
  </tr>
  <tr>
    <td>Expansion.Conjunction</td>
    <td>27.16</td>
  </tr>
  <tr>
    <td>Expansion.Equivalence</td>
    <td>28.31</td>
  </tr>
  <tr>
    <td>Expansion.Instantiation</td>
    <td>30.05</td>
  </tr>
  <tr>
    <td>Expansion.Level-of-detail</td>
    <td>28.44</td>
  </tr>
  <tr>
    <td>Expansion.Manner</td>
    <td>29.08</td>
  </tr>
  <tr>
    <td>Expansion.Substitution</td>
    <td>27.96</td>
  </tr>
  <tr>
    <td>Temporal.Asynchronous</td>
    <td>28.03</td>
  </tr>
  <tr>
    <td>Temporal.Synchronous</td>
    <td>29.65</td>
  </tr>
</table>
Table 7: Thresholds for all relation senses of PDTB 3.0.

<table>
  <tr>
    <th>Relation</th>
    <th>Train</th>
    <th>Dev</th>
    <th>Test</th>
  </tr>
  <tr>
    <td>Comparison</td>
    <td>1,894</td>
    <td>191</td>
    <td>146</td>
  </tr>
  <tr>
    <td>Contingency</td>
    <td>3,281</td>
    <td>287</td>
    <td>276</td>
  </tr>
  <tr>
    <td>Expansion</td>
    <td>6,792</td>
    <td>651</td>
    <td>556</td>
  </tr>
  <tr>
    <td>Temporal</td>
    <td>665</td>
    <td>54</td>
    <td>68</td>
  </tr>
  <tr>
    <td>Total</td>
    <td>12,632</td>
    <td>1,183</td>
    <td>1,046</td>
  </tr>
</table>
Table 8: Data statistics of PDTB 2.0.

<table>
  <tr>
    <th>Relation</th>
    <th>Train</th>
    <th>Dev</th>
    <th>Test</th>
  </tr>
  <tr>
    <td>Comparison</td>
    <td>1,830</td>
    <td>190</td>
    <td>154</td>
  </tr>
  <tr>
    <td>Contingency</td>
    <td>5,896</td>
    <td>579</td>
    <td>529</td>
  </tr>
  <tr>
    <td>Expansion</td>
    <td>7,941</td>
    <td>748</td>
    <td>643</td>
  </tr>
  <tr>
    <td>Temporal</td>
    <td>1,418</td>
    <td>136</td>
    <td>148</td>
  </tr>
  <tr>
    <td>Total</td>
    <td>17,085</td>
    <td>1,653</td>
    <td>1,474</td>
  </tr>
</table>
Table 9: Data statistics of PDTB 3.0.

B Statistics of PDTB datasets

We use the benchmark datasets PDTB-2.0 and 3.0 in our experiments, and follow the common practice (Ji and Eisenstein, 2015) to divide each of them into training (Train), validation (Dev) and test sets. The statistics in the datasets are shown in Table 8 and Table 9.

C Related Work

Recent research has demonstrated that PLMs outperform traditional machine learning methods for the IDR task. Consequently, many studies have explored incorporating novel modules into the encoder-only transformer architecture to obtain better representations and extract more comprehensive features from the input. The new added modules includes Conditional Variational AutoEncoder (Dou et al., 2021), Graph Convolutional Network (Wu et al., 2022), Gated Recurrent Unit (Wu et al., 2022), and attention mechanism (Wu et al., 2022; Xiang et al., 2022a; Jiang et al., 2023).

On the other hand, some studies applied new training strategies like contractive learning (Long and Webber, 2022; Jiang et al., 2023; Xu et al., 2023; Zeng et al., 2024), knowledge distillation (Wu et al., 2023; Jiang et al., 2024), and extra pre-training (Wang et al., 2023).

Notably, Zhou et al. (2022) proposed a prompt-based approach that involves connective prediction and answer mapping. Their work paved the way for better leveraging of connectives, like enhancing the input with predicted connectives (Liu and Strube, 2023; Liu et al., 2024), or mapping the answers by connectives directly (Xiang et al., 2022b; Zhou et al., 2022; Wu et al., 2023; Wang et al., 2023; Zeng et al., 2024). Additionally, several studies investigated the potential of multi-level hierarchical information for IDR. These works explored modeling the relationships between labels and fusing the global and local information within the multi-level hierarchical structure(Jiang et al., 2023; Xu et al., 2023; Zhao et al., 2023).

However, the potential of generative models and LLMs has been relatively underexplored for IDR(Chan et al., 2024; Omura et al., 2024; Yung et al., 2024). Therefore, our work aims to address this gap by investigating how to effectively utilize LLMs’ reasoning capabilities and incorporate additional relevant information into the input.
<table>
  <tr>
    <th>Method</th>
    <th>Backbone Model</th>
    <th colspan="2">PDTB2-Sec</th>
    <th colspan="2">PDTB3-Sec</th>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th>F<sub>1</sub></th>
    <th>Acc</th>
    <th>F<sub>1</sub></th>
    <th>Acc</th>
  </tr>
  <tr>
    <td>ChatGPT (Chan et al., 2024)</td>
    <td>GPT-3.5-turbo</td>
    <td>9.27</td>
    <td>15.59</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>PIDRA (Yung et al., 2024)</td>
    <td>GPT4</td>
    <td>-</td>
    <td>-</td>
    <td>25.77</td>
    <td>36.98</td>
  </tr>
  <tr>
    <td>FCL (Long and Webber, 2022)</td>
    <td>RoBERTa-base</td>
    <td>49.66</td>
    <td>61.69</td>
    <td>57.62</td>
    <td>64.68</td>
  </tr>
  <tr>
    <td>CP-KD (Wu et al., 2023)</td>
    <td>RoBERTa-base</td>
    <td>44.77</td>
    <td>64.00</td>
    <td>50.12</td>
    <td>66.21</td>
  </tr>
  <tr>
    <td>CP-KD (Wu et al., 2023)</td>
    <td>RoBERTa-large</td>
    <td>47.78</td>
    <td>66.41</td>
    <td>52.16</td>
    <td>67.84</td>
  </tr>
  <tr>
    <td>SCIDER (Cai et al., 2024)</td>
    <td>RoBERTa-base</td>
    <td>-</td>
    <td>59.62</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>OTMT (Jiang et al., 2024)</td>
    <td>XLNet-large</td>
    <td>-</td>
    <td>61.06</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>CG-T5 (Jiang et al., 2021)</td>
    <td>T5-base</td>
    <td>37.76</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>DiscoPrompt (Chan et al., 2023)</td>
    <td>T5-base</td>
    <td>43.68</td>
    <td>61.02</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>DiscoPrompt (Chan et al., 2023)</td>
    <td>T5-large</td>
    <td>49.03</td>
    <td>64.58</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>SCDN (ours)</td>
    <td>LLaMA3-8B-Instruct</td>
    <td>46.38</td>
    <td>62.46</td>
    <td>55.04</td>
    <td>64.35</td>
  </tr>
</table>

Table 10: Performance of the secondary level classification on PDTB 2.0/3.0.

D Performance on Relation Senses

Besides of the relation types in the main-level taxonomy of PDTB, we additionally evaluate our models upon the secondary-level taxonomy. The latter taxonomy consists of the fine-grained relation senses. Table 10 shows the Macro F1-scores and accuracies of our models, as well as the previous work that reported the performance on the secondary-level taxonomy.

It can be found that our SCDN achieves promising performance, outperforming the T5-base based generative models. Nevertheless, SCDN has an obvious performance gap compared to the T5-large based DiscoPrompt (Chan et al., 2023). DiscoPrompt is a strong IDRR model which learns from the reliance between relations and implicit connectives during training. The implicit connectives are informative when being used as guidance during training, which allows T5-large to infer relations from additional perspectives. By contrast, we didn’t use implicit connectives as guidance when fine-tuning SCDN. Besides, as shown in Table 11 and 12, some relation senses in the secondary-level taxonomy are given much less available training data than others in PDTB 2.0 and 3.0. More seriously, some arguments of such relation senses fail to be given a subtext by our subtext generator. This results in the insufficient training towards these relation senses when we fine-tune our in-subtext IDRR model and SCDN. And this causes severe performance degradation.

<table>
  <tr>
    <th>Relation</th>
    <th>Number</th>
  </tr>
  <tr>
    <td>Comparison.Concession</td>
    <td>180</td>
  </tr>
  <tr>
    <td>Comparison.Contrast</td>
    <td>1,566</td>
  </tr>
  <tr>
    <td>Contingency.Cause</td>
    <td>3,227</td>
  </tr>
  <tr>
    <td>Contingency.Pragmatic cause</td>
    <td>51</td>
  </tr>
  <tr>
    <td>Expansion.Alternative</td>
    <td>146</td>
  </tr>
  <tr>
    <td>Expansion.Conjunction</td>
    <td>2,805</td>
  </tr>
  <tr>
    <td>Expansion.Instantiation</td>
    <td>1,061</td>
  </tr>
  <tr>
    <td>Expansion.List</td>
    <td>330</td>
  </tr>
  <tr>
    <td>Expansion.Restatement</td>
    <td>2,376</td>
  </tr>
  <tr>
    <td>Temporal.Asynchronous</td>
    <td>517</td>
  </tr>
  <tr>
    <td>Temporal.Synchrony</td>
    <td>147</td>
  </tr>
  <tr>
    <th>Total</th>
    <th>12,406</th>
  </tr>
</table>

Table 11: Counts of secondary level relation types on the training set of PDTB 2.0.

E Case Study

To qualitatively assess the impact of subtexts on the reasoning ability of LLaMA3 for IDRR, we manually examine the test data as well as predictions of the Out-of-subtext and In-subtext models. The out-of-subtext IDRR model \( \mathcal{M}_\beta \) predicts relations conditioned only on arguments, while the in-subtext IDRR model \( \mathcal{M}_\lambda \) combines subtext and arguments, and uses them as clues for relation reasoning. The following analysis showcases the benefits associated with the use of subtexts:

Example 1
Argument 1: I personally don’t enjoy seeing players who I remember vividly from their playing days running about and being gallant about their deficiencies.
<table>
  <tr>
    <th>Relation</th>
    <th>Number</th>
  </tr>
  <tr>
    <td>Comparison.Concession</td>
    <td>1,170</td>
  </tr>
  <tr>
    <td>Comparison.Contrast</td>
    <td>639</td>
  </tr>
  <tr>
    <td>Contingency.Cause</td>
    <td>4,483</td>
  </tr>
  <tr>
    <td>Contingency.Cause+Belief</td>
    <td>157</td>
  </tr>
  <tr>
    <td>Contingency.Condition</td>
    <td>154</td>
  </tr>
  <tr>
    <td>Contingency.Purpose</td>
    <td>1,102</td>
  </tr>
  <tr>
    <td>Expansion.Conjunction</td>
    <td>3,584</td>
  </tr>
  <tr>
    <td>Expansion.Equivalence</td>
    <td>252</td>
  </tr>
  <tr>
    <td>Expansion.Instantiation</td>
    <td>1,117</td>
  </tr>
  <tr>
    <td>Expansion.Level-of-detail</td>
    <td>2,493</td>
  </tr>
  <tr>
    <td>Expansion.Manner</td>
    <td>191</td>
  </tr>
  <tr>
    <td>Expansion.Substitution</td>
    <td>278</td>
  </tr>
  <tr>
    <td>Temporal.Asynchronous</td>
    <td>985</td>
  </tr>
  <tr>
    <td>Temporal.Synchronous</td>
    <td>433</td>
  </tr>
  <tr>
    <td>Total</td>
    <td>17,038</td>
  </tr>
</table>

Table 12: Counts of secondary level relation types on the training set of PDTB 3.0.

Argument 2: People should be allowed to remember players as they were.
Subtext: The implicit meaning between the arguments is that people should be allowed to remember and cherish the memories of players from their playing days without seeing them participate in activities that may diminish their legacy or reputation. It suggests a desire to preserve the image and impact of these players in the eyes of the public.
Ground-truth: Expansion
Ouf-of-subtext model: Comparison
In-subtext model: Expansion

Example 2
Argument 1: For a family with taxable income of $50,000 to pay the same 28% incremental tax rate as a family with taxable income of $250,000.
Argument 2: The 33% bracket should apply to all income over the applicable level, not just the 5% rate adjustment amount.
Subtext: The implicit meaning between these arguments is that both are advocating for a more progressive tax system where higher-income individuals pay a higher percentage of their income in taxes. Argument 1 suggests that the current tax system is unfair because a family with a much lower income is paying the same percentage of taxes as a family with a much higher income. Argument 2 suggests that the tax brackets should be adjusted to ensure that higher-income individuals pay a higher tax rate on all of their income, not just the amount above a certain threshold. Both arguments imply that the current tax system is not equitable and needs to be restructured to address income inequality.
Ground-truth: Comparison
Ouf-of-subtext model: Expansion
In-subtext model: Comparison

Example 3
Argument 1: He believes in what he plays, and he plays superbly.
Argument 2: His recent appearance at the Metropolitan Museum, dubbed "A Musical Odyssey," was a case in point.
Subtext: The implicit meaning between these arguments is that the musician's belief in his music is evident in his exceptional performance at the Metropolitan Museum, showcasing his talent and skill as a musician.
Ground-truth: Expansion.Instantiation
Ouf-of-subtext model: Expansion.Restatement
In-subtext model: Expansion.Instantiation

Reviewer paper summaries: The paper leverage LLaMA to generate subtexts for argument pairs, and verify the effectiveness of subtext-based IDRR. We construct an IDRR baseline using the decoder-only backbone LLaMA, and enhance it with subtext-aware relation reasoning. A confidence diagnosed dual-channel network is used for collaboration between in-subtext and out-of-subtext IDRR.

In this paper, the authors propose a framework that outputs the subtext behind the Arg1 and Arg2 sentences of IDRR to LLM and uses that subtext to infer discourse relations.
They realize this framework by combining three models.
The proposed method overcomes the previous research using RoBERTa and T5.
Summary of strengths: 1.This paper is the first to explore the application of "subtext" in the task of Implicit Discourse Relation Recognition (IDRR), proposing a novel research direction.
2.The proposed SCDN (Subtext-based Confidence-diagnosed Dual-channel Network) architecture is clearly designed, incorporating three key components: subtext generation, relation reasoning, and confidence diagnosis.
3.The paper conducts comprehensive experiments on two mainstream datasets, PDTB-2.0 and PDTB-3.0, comparing the proposed method with various baseline models, thereby demonstrating its effectiveness.

- The method of generating subtext and using it to predict IDRR is interesting.
- It has been demonstrated that subtext is effective to some extent for IDRR.
Summary of weaknesses: 1.As noted in the "Limitations" section, the current subtext generation methods cannot handle certain complex situations, such as when shared subtext is only implied in one argument and irrelevant to another.
2.The use of three LLMs increases computational and resource demands, which may limit practical applications.

- The model size of the previous studies being compared is small. While it is important to include the scores of the previous studies, it would also be good to include the scores of models of a similar size. Alternatively, a fair comparison would be to simply include the scores when using only the Llama-8B model.

- The proposed method uses a total of three models, but the score increase is not large. Compared to the conventional IDRR method, the Llama-8B model is a large model. However, the score is almost the same as the RoBERTa-large-based model. Can it really be claimed that this method is effective?
Comments, suggestions and typos: 1.This paper presents a novel and promising research direction，will future research directions focus on improving the quality of subtext?
2.Processing with three large language models (Mα, Mβ, Mλ) requires high computational and resource demands, which may limit practical applications. Is it possible to integrate them into an expert model for operation?

Q. There have been many proposals for methods of performing IDRR using connectives as a clue. The reason why connectives are used is that EDRR achieves a high level of performance[1]. Is the subtext proposed this time more effective than connectives? Do you have any results of comparisons?

[1] A Regularization Approach for Incorporating Event Knowledge and Coreference Relations into Neural Discourse Parsing
Overall assessment scores: 3.5

2.5
Metareview score: 4
Metareview: This paper focuses on the task of Implicit Discourse Relation Recognition (IDRR) and explores the utilization of subtexts to improve the performance of generative IDRR. I like the idea to use subtext behind arguments for IDRR, it is interesting and worth investigating. Althought I agree with some reviewers that the experimental parts can be further improved, it does not affect the value of this paper.
Summary of reasons to publish: - A very interesting idea to solve IDRR with subtext.
- The results are promissing.
- The paper is well-written and easy to follow.