GLGR: Question-aware Global-to-Local Graph Reasoning for Multi-party Dialogue Reading Comprehension
Authors: Yanling Li
Bowei Zou
Yifan Fan
Xibo Li
AiTi Aw
Yu Hong
Venue: EMNLP 2023 Findings (2023)
Keywords: Multi-party dialogue reading comprehension
Global-to-local graph reasoning

Abstract: Graph reasoning contributes to the integration of discretely-distributed attentive information (clues) for Multi-party Dialogue Reading Comprehension (MDRC). This is attributed primarily to multi-hop reasoning over global conversational structures. However, existing approaches barely apply questions for anti-noise graph reasoning. More seriously, the local semantic structures in utterances are neglected, although they are beneficial for bridging across semantically-related clues. In this paper, we propose a question-aware global-to-local graph reasoning approach. It expands the canonical Interlocutor-Utterance graph by introducing a question node, enabling comprehensive global graph reasoning. More importantly, it constructs a semantic-role graph for each utterance, and accordingly performs local graph reasoning conditioned on the semantic relations. We design a two-stage encoder network to implement the progressive reasoning from the global graph to local. The experiments on the benchmark datasets Molweni and FriendsQA show that our approach yields significant improvements, compared to BERT and ELECTRA baselines. It achieves 73.6\% and 77.2\% F$1$-scores on Molweni and FriendsQA, respectively, outperforming state-of-the-art methods that employ different pretrained language models as backbones.

GLGR: Question-aware Global-to-Local Graph Reasoning for Multi-party Dialogue Reading Comprehension
Yanling Li', Bowei Zou', Yifan Fan', Xibo Li', Ai Ti Aw?, Yu Hong
'Scoool of Computer Science and Technology. Soochow University, Suzhou, China Instiiute for Infocomm Research, A SSSAR, Singapore
(1i4861988, yifanfannlp, cipolee20000210, Jlanxianer)@ggaall. {zou_bowei, aaiti]@i2r www.aa.ssse.... edu. sg
Abstract
U/ Nbx000%; How do find the address of usb device! dinner
Aowddooigen: taking
wine
dine it tell you.
U/ Llkwidooigen: What sont of devisee
Graph reasoning contributes to the integration of suucretll-diisribucee attentive information (clues) for Multi-party Dialogue Reading Com- prehension (MDRC). This is attributed primar- ily to multi-hop reasoning over global con- versational structures. However, existing ap- proaches barely apply questions for anti-noise graph reasoning. More seriously, the local se- mantic structures in utterances are neglected, although they are beneficial for bridging across semantically-relatee clues. In this paper, propose a question-awer awarre global-to-local graph reasoning approach. It expands the canonical Interlocutor- Utturnece graph by introducing question node, enabling comprehensive global graph reasoning. More importantly, it con- structs semantic-role graph for each utterance, and accordingly performs local graph reasoning conditioned on the semantic relations. We de- sign a two-stage encoder network to implement the progressive reasoning from the global graph to local. The experiments the benchmark datasets Molweni and FriendsQA show that our approach yields significant improvements, compared to BERT and ELECTRA baselines. It achieves 73.66 6% and 77.2% Fl-scores on Mol- weni and FriendsQA, respectively, outperform- ing state-of -oe--h----thaa--- the-art methods that employ differ- ent pretrained language models as hackhones.
1/2 Baha: Can'tl jest copy over the os and leave the data files untouched??
Nbxt00 if
upgrade.
Nuked: Should just restart afler installing!
By Llowidoxigen: I'd do dall restart so that re-loads the modales.
[n]
Q: What sut of devise shx000 aduc??
Hasellne prodictos: Only you rpgrode
Qu: What does tubo say in addioon
Buseline prediction: Curlt luxt copy over
ospying the openning sydem??
leeve the don files untooched.
(B)
Now tie
Ry
Ny Nutid
Uueming
Repiick-ty
AROO
ARDS
Denob aeppere
Piii
(atarkoataa GIL alge
Figure I: (a) A dialogue from Molweni (Li et al., 2020). (b) Two
pairs of the dialogue. The correct answer to each question is highlighted in the dialogue with the same color as the question.
(c) The QIUG of the dia- logue, where edges between utterances indicate the discourse relationship hetween them (replying and replied-by). (d) The LSRG subgraph of U4 and Us in the dialogue.
Multi-hop reasoning suffers from discontinu- ous utterances and disordered conversations (see Figure 1-a,b).
Recently, a variety of graph- multi-hop rea- soning (abbr., graph reasoning) approaches have been proposed to tackle MDRC (Li et al., 2021; Ma et al., 2021, 2022). Graph reasoning is generally effective for bridging across the clues hidden in the discontinuous utterances, with less interference of redundant and distracting information occurring in the disordered conversations. The effectiveness is attributed primarily to the perception of global in- teractive relations of mnerlocutor-utterane graphs. However, existing approaches encounter two bot- tlenecks. First, the struction methods (Li et al., 2021; Ma et al., 2021) fail to model the bi-directional interactions between
1 Introduction
MDRC is a special Machine Reading Comprehen- sion (MRC) task. It involves answering questions conditioned on the utterances of multiple interlocu- tors (Yang and Choi, 2019; Li et al., 2020). MDRC presents unique challenges due to two aspects:
MDRC relies heavily on multi-hop reasoning, where the necessary clues for reasoning are discretely distributed across utterances.
graph con-
*Corresponding author.

---

question and utterances, As result, is prone to involving question-unrelated information during reasoning. Second, the inner token-level semantic relations in every utterance are omitted, making difficult to perceive the exact and unabridged clues occurring in the local contexts.
answer
Pointer Network
OP
CM
O
O
Oe
O2
0
O'
O'
O
0
6
Ó
O
0
0
A
Fg
Ar
11.
Ag
R
Graph Transfomeer (Local graph reasoning)
TTo address the issues, we propose Global-to- Local Graph Reasoning approach (GLGR) with Pre-trained Language Models (PLMs) (Kenton and Toutanova, 2019; Clark et al., 2020) as back- bones. encodes two heterogeneous graphs, including Question-aware Aiterlocuto--ttterame Graph (QIUG) and Local Semantic Role Graph (LSRG). QIUG connects the question with all ut- terances in the canonical Interlocutor- Utteracce graph (Figure 1-c). It depicts the global interac- tive relations. By contrast, LSRG interconnects fine-grained nodes (tokens, phrases and entities) in an utterance in terms of their semantic roles, where semantic role labeling (Shi and Lin, 2019) is used. It signals the local semantic relations. To enable connectivity between LSRGs different utterances, we employ coreference resolution (Lee et al., 2018) and synonym identification to identify shareable nodes (Figure 1-d).
ARGi
Each
a
Ô
O
Oe
Node Initialization (Attention Pooling)
O'
6
A
1g
11
12
Graph Attention (Global graph reasoning)
Sade rrog erooding
DIJ3
D
O2
Node Initializatiion (Attention/Meen Pooling!
A
Rg
A
Ag
Ag
H
develop two-stage en- coder network for progressive graph reasoning. is conducted by successively encoding QTUG and LSRG, where attention modeling is used. The atten- tive information squeezed from QIUG and LSRG is respectively used to emphasize the global and local clues for answer prediction. Accordingly, the representation of the input is updated step-by-step during the progressive reasoning process. Residual network is used for information integration.
Methodologically,
PLM-based Context Encoder
Q
Si
0;
S:
0:
S,
F2
Figure 2: The main urchitecture of the two-stage encoder for Global-to-I.no Graph Reasoning approach (GI.GR).
ments when different PLMs are adopted as backbones (BERT and ELECTRA).
The rest of the paper is organized as follows. Section 2 presents the details of GLGR. We discuss the experimental results in Section 3, and overview the related work in Section 4. We conclude the paper in Section 5.
We carry GLGR within an extractive MRC framework, where a pointer network (Vinyals et al., 2015) is used to extract answers from utterances. The experiments on Molweni (Li et al., 2020) and FriendsQA (Yang and Choi, 2019) demonstrate three contributions of this study, including:
2 Approach
The separate use QIUG and LSRG for graph reasoning yields substantial improve- ments, compared to PLM-based baselines.
The overall architecture of GLGR-based MDRC model is shown in Figure 2. First of all, we utilize PLM to initialize the representation of the question, interlocutors and utterances. On this basis, the first- stage graph reasoning is conducted over QIUG, where Graph Attention network (GAT) (Veličković et al., 2018) is used for encoding. Subsequently, we carry out the second-stage graph reasoning over LSRG, where graph transformer layers (Zhu et al., 2019) are used for encoding. Finally, we concate-
Global-to-local progressive reasoning on both graphs (i.e., GLGR) yields further improve- ments, allowing MDRC performance to reach 73.6% and 77.2% F1-scores, as well as 59.2% and 59.8% EM-scores.
GLGR is stable. It obtains general improve-

---

nate the initial hidden states and their updated ver- sions obtained by GLGR, and feed them into the pointer network for answer prediction. In the following subsections, we detail computa- tional details after presenting the task definition.
2.3.1 Global Graph Reasoning on QIUG
QIUG- QIUG an expanded version of the canon- ical interlocutor-uttrrane graph (Li et al., 2021; Ma et al., 2021) due to the involvement of question- oriented relations, as shown in Figure 1-(c).
Specifically, QIUG comprises one question node, Nu utterance nodes and N, interlocutor nodes. We connect the nodes using the following scheme:
2.1 Task Definition
Formally, the task is defined as follows. Given a multi-party dialogue D = {U1, .,.,, 3 with n utterances and a question Q, MDRC aims to extract the answer A of Q from D. When the question is unanswerable, A is assigned tag "Impossible". Note that each utterance Ug comprises the name of an interlocutor St who issues the utterance, well as the concrete content of the utterance.
Each question node is linked to all utterance nodes. Bidirectional connection is used, in the directions of "querying" and "queried-by".
Each interlocutor node is connected to all the utterance nodes she/he issued. Bidirectional connection is used, in the directions of "issu- ing" and "issued-by".
2.2 Preliminary Representation
We follow Li and Zhao (2021)'s study encode the question Q and dialogue D using PLM, so as to obtain token-level hidden states of all tokens in Q and D. Specifically, we concatenate the question Q and dialogue D to form the input sequence X, and feed X into PLM to compute H:
Utterance nodes are connected to each other in terms of Conversational Discourse Struc- tures (CDS) (Liu and Chen, 2021; Yu et al., 2022). CDS is publicly available in Mol- weni (Li et al., 2020), though undisclosed in FriendsQA (Yang and Choi, 2019). Therefore, we apply Liu and Chen (2021)'s open-source CDS parser to tread with FriendsQA. Bidirec- tional connection is used, i.c., in the directions of "replying" and "replied-by".
X ([CLS), Q [SEP), U1, [SEP). .... Un, [SEP]) (I)
H- (X,
where, [CLS] and [SEP] denote special tokens. The hidden states H C Rixdd serve as the uni- versal representation of X, where
the maxi- mum length of X, and d is the hidden size. The symbol 0 denotes all the trainable parameters of PLM. In our experiments, we consider three differ- ent PLMs backbones in total, including BERT- base-uncased (BERT BERT-largc-uncascl (BERT large) (Kenton and Toutanova, 2019) and ELECTRA (Clark al., 2020).
Consequently, QIUG contains six types of inter- active relations T=(querying, queried-by, issuing, issued-by, replying, replied-by). Each directed edge in QIUG solely signals one type of relation.
Node Representation- For interlocutor node Sü we consider the tokens of her/his name and look up their hidden states in the universal representa- tion H. We aggregate the hidden states by mean pooling (Gholamalinezhad and Khosravi, 2020). The resultant embedding O C Rllxd is used as the node representation of S,.
To facilitate understanding, we clearly define different levels of hidden states as follows:
II refers the hidden states of all tokens in X, i.e., the universal representation of X. is the hidden state of a token x (x E X).
For an utterance node U, aggregate the hid- den states of all tokens in U,. Attention pooling (Santos et al., 2016) is used for aggregation. The re- sultant embedding Oy E BIxd is used as the node representation of U,- We obtain the representation O of the question node Q in a similar way.
O is the hidden state of a specific node, known as node representation. Specifically, 09, O² and O denote the representations of question node, interlocutor node and utterance node.
2.3 Global-to-Local Graph Reasoning
Multi-hop Reasoning- Multi-hop hop reasoning is used to discover and package co-attentive informa- tion across nodes and along edges. Methodologi- cally, updates the hidden states of all tokens in a node using the attentive information Ô the neigh- boring nodes. Formally, the hidden state of each
We carry out Global-to-Local Graph Reasoning (GLGR) to update the hidden states H/ of the input sequence X. GL.GR fulfilled by two-stage pro- gressive encoding over two heterogeneous graphs, including QIUG and LSRG.

---

token is updated by: h=th:O), i.e. concatenating h e IIxd and Ô 1
reasoning. However, such local semantic correla- tions actually contribute to the reasoning process. such as the predicate-time and predicate-negation relations, as well as coreferential relations. There- fore, we construct the semantic-role graph LSRG, and use it to strengthen local graph reasoning.
We utilize Llyyer GAT (Veličković et al., 2018) networks to compute Ô where aetention-weighted information fusion is used:
6, EoSeR, a 2f
LSRG- LSRG is an undirected graph which comprises semantic-role subgraphs of all utterances in D. To obtain the subgraph of utterance, we leverage AllennIp-SRL parser (Shi and Lin, 2019) to extract the predicate- arguument structures in the utterance, and regard predicates and arguments as the fine- finegrrined nodes. Each predicate node is con- nected to the associated argument nodes with undi- rected role-specific edges (e.g., "ARGI-V"). Both the directly arguments and indirectly- associated are considered for constructing the sub- graph, as shown in Figure 1-(d).
where, Eg comprises a set of neighboring nodes of the i-th node O,. W is a trainable scalar parameter, and the superscript (!) signals the computation at the 1-the layer of GAT. Besides, Oijj is a node- level attention score of O, (O; & E,) relative to Oj (O3 E F.). The resultant 0. is used to update the hidden states of all tokens in the i-th node.
divide-and-conque Attention Modeling Dif- ferent interactive relations have distinctive impacts on attention modeling. For example, the "queried- by" relation (i.e., an edge directed from utterance to question) most likely portends the payment of more attention to the possible answer in the utterance. By contrast, the "replying" and "replied-by" rela- tions (i.e., edges between utterance nodes) induce the payment of more attention to the complemen- tary clues in the utterances. order to distinguish between the impacts, we separately compute node- level attention scores a for different types of edges in QIUG. Given two nodes O₂ and Oj with a f-type edge, the attention score oij is computed as:
Given the semantic-role subgraphs of all utter- ances, we form LSRG using the following scheme:
We combine the subgraphs containing similar fine-grained nodes. It is fulfilled by connect- ing the similar nodes. A pair of nodes is deter- mined to be similar if their inner tokens have an overlap rate more than 0.5.
Interlocutor name is regarded as special fine-grained node. We connect it to the fine- grained nodes in the utterances she/he issued.
exp of 1,
0. Eo,eE,, exp (f( " W()
(3)
We combine subgraphs containing coreferen- tial nodes. Lee al. (2018)'s coreference resolution toolkit is used.
where [:] is the concatenation operation, while f (-) denotes the LeakyRelu (Maas et al., 2013) activa- tion function. Esi is the set of all neighboring nodes that are connected to O with t-type edge. wilh E 122x is trainable parameter. Prediction and Training- Using multi-hop rea- soning, reproduce the universal representation: N=1H:O.11 We feed E into two-layer pointer network to predict the answer, determining the start and end positions of the answer in X. Note that the hidden state of [CLS] in H is used to predict the "Impossible" tag, i.e., a tag signaling unanswerable question. During training, we use the cross-entropy loss function to optimize the model.
Fine-grained Node Representation- The fine- grained nodes generally contain varied number of tokens (e.g., "can not" and "the data files"). To obtain identically-sized representations of them, we aggregate the hidden states of all tokens in each fine-grained node. Attention pooling (Santos et al., 2016) is used for aggregation.
In our experiments, there are two kinds of token- level hidden states considered for fine-grained node representation and reasoning LSRG, including the initial case h obtained by PLM, as well the refined case h (h E H) by global graph reasoning. When h is used, we perform local graph reasoning independently, without the collaboration of global graph reasoning. It is carried out for MDRC in an ablation study. When h is used, we perform global-to-local graph reasoning. It is conducted in
2.3.2 Local Graph Reasoning on LSRG
Global graph reasoning is grounded on the global relations among question, interlocutor and utter- ance nodes, as well as their indecomposable node representations. It barely uses the inner token-level semantic relations in every utterance for multi-hop
http:///dleaallllmoooooooooceeee

---

the comparison test. We concentrate on h in the following discussion.
Molweni
FriendsQA
Datasets
Train
Dev
Test
Train
Dev
Test
Questions
24.682
2,513
2,871
8,535
1,010
1,065
Multi-hop Reasoning on LSRG- It updates hid- den states of all tokens in each fine-grained node O₁ where the attentive information Ô of its neigh- boring nodes in LSRG is used for updating. For- mally, the hidden state of each token is updated by: h=[h;O]. We use Lz-layer graph transformer (Zhu et al., 2019) compute Ô E IR Ixd follows:
Utterances
77,374
7,823
2,513
21,607
2,847
2,336
Dialogues
8,771
883
100
977
122
123
Table 1: Statistics in Molweni and FriendsQA.
We feed H into the two-layer pointer network for predicting answers. Cross-entropy loss is used to optimize the model during training.
0.4h== 3 W + + (
(4)
Experimentation
where, E, a of neighboring fine tee-grained nodes of the i-th node O in LSRG. W. Raxd and W, E mad are trainable parameters. addition, Ti I the learnable embedding of the role- specific edge between Og and its j-th neighboring node Oj. Before training, the edges holding the same semantic-role relation are uniformly assigned randomly-initialized embedding. Besides, Bijj is scalar, denoting the attention score between O and Oj. It is computed follows:
3.1 Datasets and Evaluation
We experiment on two benchmark datasets, includ- ing Molweni (Li et al., 2020) and FriendsQA (Yang and Choi, 2019). Molweni is an MDRC dataset manufactured using Ubuntu Chat Corpus (Lowe el al., 2015). The dialogues in Molweni are accom- panied with ground-truth CDS, as well as either answerable or unanswerable questions. Friend- sQA is another MDRC dataset whose dialogues are excerpted from the TV show Friends. It is characterized by colloquial conversations. CDS is undisclosed in FriendsQA. We use Liu and Chen (2021)'s CDS parser to pretreat the dataset.
exp (0)
(5)
D CaP
(5k)
(1)
10 = of w! (o wi + +r@w!!) Ma
(6)
We follow the common practice (Li et al., 2020; Yang and Choi, 2019) to split Molweni and Friend- sQA into the training, validation and test sets. The data statistics about the sets are shown in Table 1. We use Fl-score and EM-score (Li and Zhao, 2021; Ma et al., 2021) as the evaluation metrics.
where, " wh and W are trainable parame- ters of dimensions RRdxd M denotes the number of neighboring fine-grained nodes in E,. Question-aware Reasoning- Obviously, the attentive information Ö, is indepen- dent of the question. To fulfill question-awar rea- soning, we impose the drecttio--orcntte attention upon Ó, Formally, it is updated by: O < Ó. where the attention score is computed as follows:
3.2 Backbone and Hyperparameter Settings
To verify the stability GLGR, we construct three GLGRR-baeed models using different PLMs as back- bones, including BERRT-base-uncased (BERT base), HERT-large-uncasee (BERT (Kenton and Toutanova, 2019) and ELECTRA (Clark et al., 2020). The hyperparameters
Tt Sigmoid (O',o,,/ vd)
(7)
where, 09 is the representation of the question node, and W, E Rdxd a trainable parameter. Prediction and Training- To fulfill global-to- local reasoning (GLGR), we reproduce the uni- versal representation: #= ; It is actually equivalent to the operation of concatenating the initial representation H, QIUG-based co-attentive information Ô4t and LSRG-based co-attentive in- formation Ö i.e., A= O
set follows. All the models are implemented using Trans- formers Library (Li and Choi, 2020). AdamW op- timizer (Loshchilov and Hutter, 2015 8) is used for training. Towards the experiments on Molweni, we set the batch size to 16 for BERT and 12 for ELEC- TRA. The learning rates are set to 1.8e-5, 5.2e- 5 and le-5 for BERT BERT large and ELEC- TRA, respectively. For FriendsQA, set the batch sizes 16, 8 and 12 for BERT base. BERT and ELECTRA, respectively. The learning rates are set to 1.8e-5 for BERT base and Ic-5 for both BERT iunge and ELECTRA backbone. In addition,
In the scenario of independent local graph rea- soning, the QIUG-based co-attentive information is omitted. Accordingly, the universal representa- tion is calculated as: #=[H:O42) O1,2, where ÓL2 is computed using h instead of h.

---

Molweni FriendsQA
a speaker masking module, which is able highlight co-attentive information within ut- terances of the same interlocutor, as well as that among different interlocutors.
Model
EM
EM
Fl
BERT base (Kenton and Toutanova, 2019)
Baseline
47.6
61.6 42.7 60.0
DADgraph (Li et al., 2021)
46.5
61.5
ULM-UOP (Li and Choi, 2020)
46.8 63.1
BiDeN (Li et al., 2022)
48.1
63.2
BiDeN (Li et al., 2022) incorporates latent in- formation of utterances using different tempo- ral structures (e.g., future-to-currenl, current- to-current and past-to- coctret interactions).
SKIDB (Li and Zhao, 2021)
49.2
64.0 46.9
63.9
ESA (Ma et al., 2021)
49.7
64,4 47.0
63.0
GI.GR! (Ours)
48.2 64.4 47,0
64.3
BERT large (Kenton and Toutanova, 2019)
Bascline
50.4
65.8 46.0 63.3
ESA (Ma et 2021)
52.9 66.9 49.0 64.0
3.4
Main Results
CLGR (Ours)
53.7 67.5 49.8
67.1
ELECTRA (Clark et al., 2020)
Table 2 shows the test results of our GLGR models and the compared models, where different back- bones (BERT base. BERTiarge and ELECTRA) are considered for the general comparison purpose.
Baseline
56.8
70.6 57.0
74.8
SKIDB (Li and Zhao, 2021)
58.0
72.9 55.8
72.3
ESA (Ma et al., 2021)
58.6
72.2
58.7
75.4
GLGR! (Ours)
59.2 73.6 59.8
77.2
Table Results on the test sets of Molweni and FriendsQA. The superscript denotes a statistically significant improve- ment -scor (p < 0.05) compared the baseline.
It can be observed that our GL.GR model yields significant improvements, compared to the PLM baselines. The most sigmificant performance gains are 4.3% F1-score and 4.3% EM-score, which are obtained on FriendsQA compared to the lite BERT base (110M parameters). When compared to the larger BERTlarge (340M parameters) and ELECTRA (335M parameters), GL.RR is able achieve the improvements of no less than 1.7% Fl Fi-score and 2.4% EM-score. In addition, GLGR outperforms most of the state-of-the-ar MDRC models. The only exception is that GLGR ob- tains a comparable performance relative to ESA when BERT base is used as the backbone. By con- trast, GLGR substantially outperforms ESA when BERT large. and ELECTRA are used.
The numbers of network layers of GAT and graph transformer are set in the same way: L1=L==2.
3.3 Compared MDRC Models
Following Ma et al. (2021)'s study, we consider the standard span-based MRC model (Kenton and Toutanova, 2019) as the baseline. We compare with a variety of state-of-thh-ar MDRC models:
DADgraph (Li et al., 2021) constructs a CDS- based dialogue graph. It enables the graph reasoning over conversational dependency fea- tures and interlocutor nodes. Graph Convolu- tional Network (GCN) is used for reasoning.
The test results reveal distinctive advantages of the state-of-the MDRC models. DADgraph is a vest cest-pocket model due to the involvement of a sole interlocutor- aware CDS-base based graph. It of- fers the basic performance of graph reasoning for MDRC. ESA is grounded on multiple graphs, and it separately analyzes co-attentive information for subdivided groups of utterances. Multi Multi-graph -raph rea- soning and coarse-to-fine attention perception al- low ESA to be a competitive MDRC model. By contrast, ULM-UOP doesn't 't rely heavily on con- versational structures. Instead, it leverages larger dataset and diverse tasks for fine-tuning, and thus enhances the ability of BERT in understanding domain-specifer languages at the level of semantics. It can be observed that ULM-UOP achieves similar performance compared to ESA. SKIDB success- fully leverages multi-task learning, and applies interesting and effective self-supervised learning tasks. Similarly, it enhances PLMs in encoding domain-specific languages, which is not limited to
ULM-UOP (Li and Choi, 2020) fine-tunes BERT larger number of FriendsQA tran- scripts (known as Character Mining dataset (Yang and Choi, 2019)) before task-specific training for MDRC. Two self-supervised tasks are used for fine-tuning, including utterance- oriented masked language modeling and ut- terance order prediction. In addition, BERT is trained to prediet both answers and sources (i.e., IDs of utterances containing answers).
SKIDB (Li and Zhao, 2021) uses a multi-task learning strategy to enhance MDRC model. Self-supervised interlocutor prediction and key-utterance prediction tasks are used within the multi-task framework.
ESA (Ma et al., 2021) uses GCN encode the interlocutor graph and CDS-based utter- ance graph. In addition, it is equipped with

---

Table 3: Graph ablation study, where ELECTRA (Clark et al., Performancs, EM scores (%)
Model
Molweni
FriendsQA
97.5
Performance, F1-scores (%)
05.5
F1
Baseline
Basellne
GLGR
59,2
73.6 59.8
77.2
GLGR
w/o QIUG
58.1
72.3
58.3
76.4
w/o LSRG
$7.7
72.6
58.9
76.0
w/o QIUG& LSRG (baseline) 56.8
70.6
57.0 74.8
51.
78.5
2020) is considered as the backbone this study.
$4.5
15.5
$2.9
Molweni
FriendsQA
10<19 20"29
>=30
Model
EM
F1
EM
Fl
QIUG
57.7
72.6
58,9
76.0
Figure 3: Performance of GLGR (ELECTRA) and baseline on the FriendsQA with different utterance numbers.
w/o interloctuor
57.0
71.3
57.7
75.6
w/o question
57.2
71.0
57.5
75.6
w/o utterance structure
57.3
71.1
57.8
75.2
can be found that the performance degradation rates in the three ablation scenarios are similar. This demonstrates that all the conversational structural factors are crucial for multi-hop reasoning. For LSRG, we implement relation ablation by unbind- ing the co-referential fine-grained nodes, omitting semantic-role relations, and removing question- aware reasoning, respectively. The way to omit semantic-role relations is accomplished by full con- nection. It can be observed that ablating semantic- role relations causes relatively larger performance degradation rates.
LSRG
58.1
72.3
58.3
76.4
w/o coreference
57.2
71.5
57.8
75.6
semantic role relation
56.6
71.5
57.8
75.3
w/o question-awan aware reasoning
56.8
71.8
58.1
75.4
Table 4: Relation ablation study, whre ELECTRA (Clark
et al., 2020) is considered as the backbone in this study.
BERT. It can be found that SKIDB obtains compa- rable performance on Molweni compared to ESA. Our GLGR model combines the above advan- tages by external conversational structure analysis and internal semantic role analysis. On this basis, GLGR integrates global co-attentive information with local for graph reasoning. It can be observed that GLGR shows superior performance, although it is trained without using additional data.
3.6 The Impact of Utterance Number
We follow the common practice (Li and Zhao, 2021; Li al., 2022) to verify the impacts of ut- terance numbers. The FriendsQA test set is used in the experiments. is divided into four subsets. including the subsets of dialogues containing no more than utterances, 2) 10~19 utterances. 3) 20~~9 utterances and 4) no less than 30 utterances. The GLGR is re-evaluated over the subsets. We illustrate the performance in Figure 3.
3.5 Ablation Study
We conduct the ablation study from two aspects. First of all, we progressively ablate global and local graph reasoning, where QIUG and LSRG are omit- ted accordingly. Second, respectively ablate different classes of edges from the two graphs, i.e.. disabling the structural factors during multi-hop reasoning. We refer the former "Graph ablation" while the latter the "Relation ablation".
It can be found that the performance decreases for the dialogues containing larger number of ut- terances, matter whether the baseline GLGR is used. In other words, both the models are dis- tracted by plenty of noises in these cases. Nev- ertheless, we observe that GLGR is able to stem the tide of performance degradation to some extent. Therefore, we suggest that GLGR contributes to the anti-noise multi-hop reasoning, although it fails to solve the problem completely.
Graph ablation The negative effects of graph ablation is shown in Table 3. It can be easily found that similar performance degradation occurs when QIUG and LSRG are independently pruned. This implies that local reasoning on LSRG is effectively equivalent to global reasoning on QIUG, to some extent. When graph reasoning is thoroughly dis- abled (i.e., pruning both QIUG and LSRG), the performance degrades severely.
3.7 Reliability of Two-stage Reasoning
There are alternative versions of GLGR: one changes the reason sequence of QIUG and LSRG (local-to-global graph reasoning). The other per- forms the single-phase reasoning over a holistic graph that interconnects QIUG and LSRG. In this
Relation ablation- The negative effects of rela- tion ablation is shown in Table 4. For QIUG, we condense the graph structure by respectively dis- abling interlocutor, question and utterance nodes.

---

Molweni
FriendsQA
3.8
Case Study
Model
EM
GLGR is characterized as the exploration of both
GLGR (Two-stage)
59.2
73.6
59.8
77.2
GLGR (Two-stage rev.)
58.3
72.7
59.1
76.2
global and local clues for reasoning. It is imple- mented by highlighting co-attentive information in coarse-grained and fine- ineeggained nodes.
GLGR (Single-phase)
58.0
72.0
58.5
75.4
Table 5: Comparing single-phuse GLGR and two two-stage GI,GR, here "rev." indicates the local-to-ge local-to-global reasoning version.
Figure 4 shows case study for GLGR-based MDRC, where the heat maps exhibit the attention distribution on both utterance nodes and token- level nodes. There are three noteworthy phenom- ena occurring the heat maps. First, GL.GR as- signs higher attention scores to two utterance nodes, which contain the answer and closely-related clues, respectively. Second, both the answer and clues are assigned higher attention scores, compared to other token-level nodes. Finally, the answer and clues emerge from different utterance nodes.
Qkivnoolf
Us Qkstorwelf Urw do get 1TL EPPATI manager
How
to different von sry, folden?
is jear: Pudeably ned is odit the icon theme.
U/
ad
B1 CO-L-T Help ur instaling lovid caseel
FIL ERNTH crange
can not.
U/
sue
Ug C-O-T::: am iillowing the directions but
ciffrcnt
ell can not
G, COL. Should my the other program??
U. joun: thes deperdency ayyway
Un C-0---T: can not mppayer syraptic.
fix
gild anower
que
Q: How does Qual-pooll use the different loos?
Haseline: unnnovembble
GUGBR icon theme.
This is not an isolated case, and the phenomena stand for the crucial impacts GLGR on MDRC.
Figure 4: A case study for the ELECTRA LECTRA-based GLGR
Related Work
subsection, we intend to compare them to the global-to-local two-stage GLGR.
4.1 Multi-party Dialogue Reading Comprehension
For the single-phase reasoning version, we com- bine QIUG with LSRG by two steps, including 1) connecting noun phrases in the question node to the similar fine- -raiind nodes in utterances, and 2) connecting utterance nodes to the entities occur- ring in them. On this basis, we encode the resul- tant holistic graph using GAT, which serves as the single-phase GLGR. It is equipped with ELECTRA and the pointer network.
A variety of graph-based approaches have been studied for MDRC. They successfully incorporate conversational structural features into the dialogue modeling process.
Ma et al. (2021) construct a provenance- graph to enhance the co-attention encoding of dis- continuous utterances of the same interlocutor. Li et al. (2021) and Ma al. (2022) apply CDS bring the mutual dependency features of utterances into the graph reasoning process, where GCN (Kipf and Welling. 2017) is used for encoding. Recently, Li et al. (2022) propose a mack-and-forth compre- hension strategy. It decouples the past and future dialogues, and models interactive relations in terms of conversational temporality. Li et al. (2023) add the coreference-awee aware attention modeling in PLM to strengthen the multi-hop reasoning ability. Another research branch focuses on the study of language understanding for dialogues, where self-supervised learning is used for the general-to- specific modeling and transfer of the pretrained models. Li and Choi (2020) transfer BERT to the task-specific data, where two self-supervised tasks used for fine-tuning, including Utterance-level Masked Language Modeling (ULM) and Utterance- Order Prediction (UOP). During the transfer, the larger-sized dataset of FriendsQA transcripts is used. Similarly, Li and Zhao (2021) transfer PLMs to dialogues using a multi-task learning framework,
The comparison results (single-phase GLGR ver- sus two two-stage GLGRs) are shown in Table 5, where ELECTRA is used as the backbone. It is evident that the single-stage GLGR obtains infe- rior performance. It is most probably because that the perception of co-attentive information among fine-grained nodes in LSRG suffers from the inter- ference of irrelevant coarse-graing nodes in QIUG, This drawback raises problem of combining het- erogeneous graphs for multi-hop reasoning. Be- sides, we observe that the global-to-local reasoning method exhibits better performance compared to local-to- bocal-to-globul global graph reasoning. We attribute it to the initial local graph reasoning in the local-to- global reasoning, which ineffectively integrates the distant and important context information while focusing on local semantic information. This leads to suboptimal multi-hop reasoning and highlights the importance of the graph reasoning order in han- dling complex information dependencies.

---

including the tasks of interlocutor prediction and key utterance prediction.
GLGR relies on annotated conversation structures. co-reference, and SRL information. This depen- dency necessitates a complex data preprocessing process and makes the model susceptible to the quality and accuracy of the annotations. Therefore, it is important to ensure the accuracy and robust- ness of the annotations used in the model train- ing and evaluation process. Second, GLGR may encounter challenges in handling longer dialogue contexts. The performance may exhibit instability when confronted with extended and more intricate conversations. Addressing this limitation requires further investigation of the stability and consistency in a real application scenario.
4.2 Semantic Role Labeling
In this study, we use Semantic Role Labeling (SRL) to build LSRG for local graph reasoning. To fa- cilitate the understanding of SRL, we present the related work as follows.
SRL is shallow semantic parsing task that aims to recognize the predicate-rguument struc- ture of each sentence. Recently, advances in SRL have been largely driven by the development of neural networks, especially the Pre-trained Lan- guage Models (PLMs) such as BERT (Kenton and Toutanova, 2019). Shi and Lin (2019) propose BERT-based model that incorporates syntactic in- formation for SRL. Larionov al. (2019) design the first pipeline model for SRL of Russian texts. has been proven that SRL is beneficial for MRC by providing rich semantic information for answer understanding and matching. Zheng and Kordjamshidi (2020) introduce an SRL-based graph reasoning network to the task of multi-hop question answering. They demonstrate that the fine- grained semantics of an SRL graph contribute to the discovery of an interpretable reasoning path for answer prediction.
A Acknowledgements
The research is supported by National Key R&D Program of China (0020YFB1313600 Na- tional Science Foundation of China (62376182, 62076174) and Institute of Infocomm Research of A *STAR (CR-2021-001).
References
Kevin Clark, Minh-Thang Luong, Quoe Le, and Christopher D Manning. 2020. ELECTRA: Pre- training text encoders as discriminators rather than generators. Proceedings of the ICLR 2020, pages 1-18.
5 Conclusion
Hossein Gholamalinezhad and Hossein Khosravi. 2020. Pooling methods in deep neural networks, a review. arXiv preprint vrXiv:2009.07444
We propose global-to-local graph reasoning ap- proach towards MDRC. It explores attentive clues for reasoning in both coarse-grained graph QIUG and fine aie-ngrined graph LSRG. Experimental re- sults show that the proposed approach outperforms the PLMs baselines and state-of-the-arr models. The codes available at https: / //gi thub. com/ YanLingLi-AI/C A
Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the 17th NAACL, volume 1, pages 4171-4186
Thomas N Kipf and Max Welling. 2017. Semi- supervised classification with graph convolutional networks. In Proceedings of the ICLR 2017, pages 1-14.
The main contribution of this study to jointly use global conversational structures and local se- mantic structures during encoding. However, it can be only implemented by two-stage reasoning due to the bottleneck of in-coordinate interaction between heterogencous graphs. To overcome the issuc, will use the pretrained Heterogeneous Graph Trans- formers (HGT) for encoding in the future. Besides, the graph- graph-structure based pretraining tasks will be designed for task- speciffii transfer learning.
Daniil Larionov, Artem Shelmanov, Elena Chistova, and Ivan Smirnov. 2019. Semantic role labeling with pretrained language models for known and unknown predicates. In Proceedings of the RANLP 2079, pages 619-628.
Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to- fine inference. In Proceedings of the 16th NAACL, volume 2, pages 687-692.
Limitations
Changmao Li and Jinho Choi. 2020. Transformers to learn hierarchical contexts in multiparty dialogue for span- question answering. Proceedings of the 58th ACL, pages 5719-5
While GLGR demonstrates several strengths, it also has limitations that should be considered. First,

---

Jiaqi Li, Ming Liu, Min- Yen Kan, Zihao Zheng, Zekun Wang. Wengiang Lei, Ting Liu, and Bing Qin. 2020. Molweni: A challenge multiparty dialogues-based machine reading comprehension dataset with dis- course structure. In Proceedings of the 28th COL- ING, pages 2642-2652.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the ICLR 2018, pages 1-12.
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. Advances in neural infor- mation processing systems, 28.
Jiaqi Li, Ming Liu, Zihao Zheng, Heng Zhang. Bing Qin, Min- Yen Kan, and Ting Liu. 2021. DADgraph: A discourse-awe aware dialogue graph neural network for multiparty dialogue machine reading comprchension. In Proceedings of the IJCN 2021, pages 1-8.
Zhengzhe Yang and Jinho D Choi. 2019. FriendsQA: Open- domain question answering on tv show tran- scripts. In Proceedings of the 20th SIGDIAL, pages 188-197.
Yanling Li, Bowei 7ou, Yifan Fan, Mengxing Dong, and Yu Hong. 2023. Coreference-aware double-channel attention network for multi-party dialogue reading comprehension. In Proceedings of the IJCNN 2023, pages 1-8.
Nan Yu, Guohong Fu, and Min Zhang. 2022. Speaker- aware discourse parsing on multi-party dialogues. Procediiggs of the 29th COLING, pages 5372-5382.
Chen Zheng and Parisa Kordjamshidi. 2020. Srigm:: Semantic role labeling graph reasoning network. In Proceedings of EMNLP 2020, pages 8881-8891.
Yiyang Li and Hai Zhao. 2021. Self-and pseudo-self- supervised prediction of speaker and key-utterance for multi party dialogue reading comprehension. In Findings of the EMNLP 2021, pages 0053-2063.
Jie Zhu, Junhui Li, Muhua Zhu, Longhua Qian, Min Zhang, and Guodong Zhou. 2019. Modeling graph structure in transformer for better amr-to-text gener- ation. In Proceedings of the EMNLP-IJCNIPP 2019, pages 5459-5468.
Yiyang Li, Hai Zhao, and Zhuosheng Zhang. 2022. Back the future: Bidirectional information decou- pling network for multi-turn dialogue modeling. In Proceedings of the EMNLP 2022, pages 2761-2774.
Zhengyuan Liu and Nancy Chen. 2021. Improving multi-party dialogue discourse parsing via domain integration. In Proceedings of the 2nd CODI, pages 122-127.
Ily Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. In Proceedings of the ICLR 2018, pages! 1-19.
Ryan Lowe, Nissan Pow, Iulian Vlad Serban, and Joelle Pineau. 2015. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn turn dia- logue systems. In Proceedings of the 16th SIGDIAL, pages 285-294.
Xinbei Ma. Zhuosheng Zhang, and Hai Zhao. 2021. Enhanced speaker-aware multi-party multi- turn dialogue comprehension. arXiv preprint 1XXiv2109.040065
Xinbei Ma, Zhuosheng Zhang, and Hai Zhao. 2022. Structural characterization for dialogue disentangle- ment. Proceedings of the 60th ACL, volume pages 285-297.
Andrew I. Maas, Awni Y Hanmun, Andrew Y Ng, et al. 2013. Rectifier nonlinearities improve neural net- work acoustic models. In Proc. ieml, pages 1-6.
Cicero dos Santos, Ming Tan, Bing Xiang, and Bowen Zhou. 2016. Attentive pooling networks. arXiy preprint
Peng Shi and Jimmy Lin. 2019. Simple bert models for relation extraction and semantic role labeling. arXiv preprint 1xXiv11944.05255


Reviews:
[{"review_id": "wlhdTKaz8c", "reviewer": "Reviewer_F64H", "summary": "", "questions": "A: Have you conducted an ablation study on Question-aware Reasoning (line 341) to verify its effectiveness?\n\nB: Have you considered reversing the order of graph encoding, i.e., local-to-global? If so, how would it potentially impact the results?\n\nC: Given that longer dialogues can introduce more complex relationships between utterances and entities, have you explored any denoising methods to address this issue?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere gratitude for dedicating your valuable time to reviewing our paper and providing us with invaluable insights and suggestions. Your feedback is greatly appreciated and has undoubtedly contributed to the refinement of our work. We have meticulously considered each of your suggestions and comments. We answer the concerns as follows:\n\n----------Response to Comments----------\n\n**Comment #1:** The proposed graph construction heavily relies on external tools, such as co-reference, which may impact the stability of the approach.\n\n**Response to #1:** We acknowledge your concern regarding the dependence of the LSRG's construction on other NLP technologies. These technologies are mentioned in Lines 276 and 296 of the paper. Let us briefly introduce their performance first. The performance of the coreference resolution toolkit on the Conll-2012 test set stands at the precision of 0.79, recall of 0.78 and F1-score of 0.79. Detailed information can be found by accessing the hyperlink of “https://demo.allennlp.org/coreference-resolution”. The semantic role label toolkit achieves an F1-score of 0.86 on the Ontonotes-5.0 test set. Detailed information can be found by accessing the hyperlink of “https://demo.allennlp.org/semantic-role-labeling”. It can be observed that, the performance of these techniques is not flawless. Furthermore, these models are not explicitly tailored to accommodate multi-party dialogue contexts. They are intended for broader domains. Nevertheless, the extracted co-reference and semantic role information remain effective within our approach, as evidenced in Table 4 (between Lines 485-486). The incorporation of both coreference and semantic-role structural information demonstrably enhances model performance. Therefore, the co-reference and semantic role information are pragmatically applicable to our task in terms of model stability, and their influence on the GLGR’s stability remains well-manageable.\n\n\n**Comment #2:** As dialogues become longer, the relationships between utterances and entities can become complex and introduce additional noise, thereby limiting the applicability and generalization ability of the approach.\n\n**Response to #2:** We acknowledge your observation regarding the escalating complexity of relationships in longer dialogues, which can indeed impact model performance. This trend is also illustrated in Figure 3, and it is explained in Lines 517-518. Nevertheless, it is worth highlighting that Figure 3 also reveals a compelling trend, i.e., GLGR effectively mitigates performance decline compared to the baseline models as dialogues are lengthened. In particular, in conversations with over 30 utterances, GLGR's performance remains more stable compared to the baseline models (i.e., the baseline models experience more significant degradation). This demonstrates the relative stability of GLGR within extensive conversational contexts, and suggests its potential for enhancing adaptability in real-world application scenarios.\n\n\n**Comment #3:** The effect of Question-aware Reasoning (line 341) has not been sufficiently validated.\n\n**Response to #3:** The opinion is highly valuable. Our exploration of Question-aware Reasoning primarily centers on QIUG, as we consider that it holds significant pertinence within the QIUG reasoning. Our ablation experiments are outlined in Table 4, spanning Lines 485-486 of the paper. To further delve into the analysis of Question-aware Reasoning within LSRG (as indicated in Line 341), we plan to introduce an additional ablation experiment in subsequent sections. This analysis aims to provide a thorough understanding of the impact of this aspect on GLGR's performance. \n\n\n----------Response to Questions----------\n\n**Question #1:** Have you conducted an ablation study on Question-aware Reasoning (line 341) to verify its effectiveness?\n\n**Response to #1:** Our investigation into Question-aware Reasoning currently centers on the QIUG reasoning component, supported by specific ablation experiments detailed in Table 4 (Lines 485-486). Concerning the effect of Question-aware Reasoning in LSRG (Line 341), we plan to include this ablation experiment subsequently. \n\n\n**Question #2:** Have you considered reversing the order of graph encoding, i.e., local-to-global? If so, how would it potentially impact the results?\n\n**Response to #2:** The prevalent trend of dialogue modeling focuses largely on the global aspect. In order to strengthen the models with local semantic and interaction features, we formulated the notion of incremental refinement in reasoning. GLGR conducts discourse reasoning at the global level to facilitate the comprehension of structural utterance information. Subsequently, semantic-level local graph inference is employed to optimize comprehension of crucial semantics. Such an incremental refinement process is based on the hypothesis that global-to-local mode exhibits greater explicability compared to local-to-global. However, the local-to-global graph reasoning approach is interesting, and we intend to experiment with it in subsequent phases.\n\n\n**Question #3:** Given that longer dialogues can introduce more complex relationships between utterances and entities, have you explored any denoising methods to address this issue?\n\n**Response #3:** We appreciate your observation regarding the potential complexities arising from extended dialogues. Our interpretation of the denoising approach revolves around the endeavor to meticulously distinguish linguistic structures within multi-party conversations. This serves to mitigate potential disturbances caused by extraneous discourse. The GLGR model employs a global-to-local strategy, i.e., commencing with global-level utterance reasoning and subsequently delving into semantic comprehension of localized utterances. It leverages question-related cues to discern critical linguistic elements associated with the question. In contrast to a comprehensive global graph reasoning conducted all at once, GLGR adopts a more refined approach, intensifying the perception of the interaction between the question and the dialogue. Essentially, this process embodies a coarse-to-fine denoising progression.\n\n\nWe are grateful for your insightful comments, which undoubtedly strengthen the quality of our paper. Your guidance has been invaluable in shaping our work into a more coherent and impactful work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The idea of constructing multi-grained graphs for reasoning is straightforward yet effective.\n\nThe paper proposes an approach to construct Question-aware Interlocutor-Utterance Graph and Local Semantic Role Graph, which are crucial for understanding the underlying semantic structure necessary for reasoning.\n\nThe global-to-local reasoning method applied to both graphs results in improved performance in multi-party dialogue reading comprehension.\n\nThe paper includes detailed experiments and an ablation study to demonstrate the effectiveness of the proposed approach.", "reasons_to_reject": "The proposed graph construction heavily relies on external tools, such as co-reference, which may impact the stability of the approach.\n\nAs dialogues become longer, the relationships between utterances and entities can become complex and introduce additional noise, thereby limiting the applicability and generalization ability of the approach.\n\nThe effect of Question-aware Reasoning (line 341) has not been sufficiently validated.", "questions_to_authors": "A: Have you conducted an ablation study on Question-aware Reasoning (line 341) to verify its effectiveness?\n\nB: Have you considered reversing the order of graph encoding, i.e., local-to-global? If so, how would it potentially impact the results?\n\nC: Given that longer dialogues can introduce more complex relationships between utterances and entities, have you explored any denoising methods to address this issue?", "paper_topic_and_main_contributions": "The paper introduces two types of graphs to facilitate reasoning in the context of multi-party dialogue reading comprehension. These graphs model global information across utterances and local information within individual utterances. A two-stage encoder network is employed to reason from the global level to the local level. Experimental results demonstrate that the global-to-local reasoning approach using the two types of graphs leads to significant improvements compared to baseline models.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4hPJWOEmCO", "reviewer": "Reviewer_28uk", "summary": "", "questions": "Please address the concerns in reasons to reject.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our heartfelt appreciation for dedicating your valuable time to reviewing our paper and offering us invaluable insights and suggestions. Your feedback is of immense significance to us and has played an important role in refining our work. We have taken careful consideration of each of your comments, and we provide our responses to each point as below.\n\n----------Response to Comments----------\n\n**Comment #1:** It would be good to see some additional qualitative analysis of the outputs.\n\n**Response to #1:** Our interpretation of qualitative analysis is to understand how GLGR influences the behavior of the baseline models. In the case study mentioned in Line 569 of the paper, we present a detailed instance along with corresponding heatmaps. They effectively visualize the impact of using GLGR on answer extraction. If this style of qualitative analysis aligns with your requirements, we can provide additional instances for demonstration as long as the length of the paper is allowed. Furthermore, we can also add an appendix to dissect scenarios where the baseline models fall short and elucidate how GLGR enhances performance in those aspects.\n\n\n**Comment #2:** The decrease in performance as context length increases is a concern for practical applicability of these models. Have any experiments been tried with enc-dec models (e.g. FlanT5) or decoder-only models that have been pretrained with longer context lengths? Would that solve the issue?\n\n**Response to #2:** It is interesting to utilize encoder-decoder models like FlanT5 or decoder-only models within a generative framework. However, it is important to note that FlanT5 often comes with larger parameter counts, which could potentially pose limitations in scenarios with limited resources. More importantly, the datasets we used, i.e., Molweni and FriendsQA, are designed specifically for extractive machine reading comprehension. FlanT5 is developed within a generative framework. It is difficult to fairly evaluate the performance of FlanT5 using Molweni and FriendsQA. It is because the proper answers generated by FlanT5 may be determined as incorrect when they don’t occur in the dialogues. By contrast, BERT and ELECTRA are constructed with transformer encoders, and they are widely used in the extractive framework in this field. Therefore, we present the performance of our models that use BERT and ELECTRA as backbones.\n\n**Comment #3:** The technique proposed seems to include a lot of moving parts and engineering effort compared to the performance improvement over other baselines considered, which again impacts practical applicability. I would like to hear the authors' opinion on this.\n\n**Response to #3:**  We consider that this concern pertains to the necessity of employing NLP models to extract coreference and semantic role structures from the dialogues before reasoning. Detailed information on this aspect can be found in Lines 276 and 296. Specifically, the coreference resolution model we employ attains the following performance on the Conll-2012 test set, including the precision of 0.79, recall of 0.78 and F1-score of 0.79. More details can be found by accessing the hyperlink of “https://demo.allennlp.org/coreference-resolution”. In addition, the semantic role labeling model achieves an F1-score of 0.86 on the Ontonotes 5.0 test set. More details can be found by accessing the hyperlink of “https://demo.allennlp.org/semantic-role-labeling”. The above performance demonstrates that there's potential for improvement in the performance of these models. Furthermore, the models have been designed for general applicability and have not been specifically trained for multi-party dialogue contexts. Nevertheless, the coreference and semantic role label information obtained by these techniques contribute to reasoning in the MDRC task. As demonstrated in Table 4, incorporating both coreference and semantic role information leads to improvement. Briefly, we kindly suggest that the utilization of general information extraction techniques endows GLGR with a degree of adaptability in practical scenarios.\n\nWe extend our heartfelt gratitude for your insightful comments, which will help us improve the quality of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The two-stage reasoning framework going from global graph to local graph is intuitive and well-motivated for connecting coarse-grained and fine-grained semantics.\n* Good empirical performance improvements over baselines.\n* Generalization across models with different pretraining schemes (BERT and ELECTRA) which is promising", "reasons_to_reject": "* It would be good to see some additional qualitative analysis of the outputs.\n* The decrease in performance as context length increases is a concern for practical applicability of these models. Have any experiments been tried with enc-dec models (ex. FlanT5) or decoder only models that have been pretrained with longer context lengths? Would that solve the issue?\n* The technique proposed seems to include a lot of moving parts and engineering effort compared to the performance improvement over other baselines considered, which again impacts practical applicability. I would like to hear the authors' opinion on this.", "questions_to_authors": "Please address the concerns in reasons to reject.", "paper_topic_and_main_contributions": "* The paper proposes a question-aware global-to-local graph reasoning approach (GLGR) for multi-party dialogue reading comprehension (MDRC). \n\n* The approach encodes two heterogeneous graphs - a Question-aware Interlocutor-Utterance Graph (QIUG) that captures global interactive relations, and a Local Semantic Role Graph (LSRG) that captures local semantic relations within utterances. By developing a two-stage encoder network to implement progressive graph reasoning from global QIUG to local LSRG, this technique allows integrating both global and local clues for answer prediction.\n\n* By running experiments on two MDRC datasets (FriendsQA and Molweni) show that GLGR achieves significant improvements over BERT and ELECTRA baselines and state-of-the-art methods. The results show the contributions of global and local graph reasoning, as well as their integration via the two-stage reasoning process.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "B1lGh1rJa9", "reviewer": "Reviewer_wxqV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere appreciation for your dedicated effort in reviewing our paper and offering valuable suggestions to enhance its quality. Your feedback holds significant value for us, and we genuinely consider each suggestion you provide. We take your feedback seriously, and our response is as follows:\n\n\n----------Response to Comments----------\n\n**Comment #1:** The method of comparison is too old-fashioned and the latest baseline BiDEN is not for MDRC(Multi-party Dialogue Reading Comprehension).\n\n**Response to #1:** BiDEN is a model designed for dialog comprehension. It has been extensively tested using Molweni dataset, although the experiment isn’t conducted by us. More details can be found in Li et al (EMNLP, 2022) [1]’s work which provides the experimental results. Considering that Multi-party Dialogue Reading Comprehension (MDRC) can be regarded as a subtask of dialog comprehension, we kindly suggest that the utilization of BiDEN for MDRC is feasible.\n\n\n**Comment #2:** Insufficient task-related baseline comparisons.\n\n**Response to #2:** Within Table 2, an extensive comparative analysis is performed, which involves three baseline models, including BERT-base, BERT-large and ELECTRA. These models constitute all the pre-trained language models employed in the previous work. We systematically compare and analyze these baseline models, and reveal that GLGR exhibits noticeable performance enhancements. Furthermore, we have conducted comprehensive comparisons with all the task-related approaches that were published before the deadline of EMNLP 2023.\n\n\n**Comment #3:** The method of construction graph used by the authors seems to be a relatively common technique used in neural networks.\n\n**Response to #3:** Graph-based neural networks are commonly employed to address comprehension tasks, the ones that involve intricate interactions among various forms of structured information. However, the effectiveness of these methods can be contingent on the task and dataset at hand. The direct transfer of existing graph-based models often encounters limitations due to the distinctive characteristics of different tasks or datasets. In MDRC, the utilization of global utterance graphs is a conventional approach. Nevertheless, when faced with the challenge of accurately capturing local semantic interactions, we take steps to model localized semantic graphs. More importantly, we introduce a novel approach by combining global and local graphs, incorporating a multi-level graph reasoning methodology that progresses from coarse to fine-grained reasoning. In the MDRC task, our contribution lies in the novel design of graph construction and modeling at the level of dialogue structure. To our best knowledge, there are no other studies that have adopted similar strategies.\n\n\n**Comment #4:** In line 292~294, why is the threshold set to 0.5? Are $W^{l}$ (in Equation 3) and $W_t^{l}$ in line 251 the same?\n\n**Response to #4:** The setting of the threshold (0.5) at Line 291 is a result of empirical observation. This value is established after manually assessing around 100 graph nodes. For example, there is an overlap rate greater than 0.5 between “there is any way to disable tapping from my touchpad” and “disable tapping from my touchpad”, whereas the overlap rate between “it's really annoying” and “disable tapping from my touchpad” is less than 0.5. Regarding the differentiation between the two trainable weights $W^{l}$ and $W_t^{l}$, the latter is a specialized description of the former. $W_t^{l}$ is a specific weight of the t-th edge. \n\n\n**Comment #5:** According to the parameter settings, L1 is set to 2. But the two-layer graph neural network cannot obtain global information, it can only obtain the node information of the second hop. Does the author need to discuss the impact of the number of graph neural network layers on model performance? Does global information help the model?\n\n**Response to #5:** We kindly suggest that the comment is about the configuration of neural network layers of QIUG and LSRG. The experimental results demonstrate that a two-layer architecture (L1=2) outperforms a single-layer structure. However, extending the structure with more layers (beyond 2) doesn’t yield substantial performance improvements. We consider this phenomenon to be attributed to the characteristics of Molweni and FriendsQA datasets. In these datasets, the majority of questions entail reasoning steps that do not surpass two iterations. Furthermore, we kindly suggest that introducing multiple layers into the graph neural network will significantly increase the number of parameters. Considering the delicate equilibrium between parameter count and performance, we designate both L1 and L2 as 2 layers.\n\n\n----------Reference----------\n\n[1] Yiyang Li, Hai Zhao and Zhousheng Zhang. Back to the future: Bidirectional information decoupling network for multi-turn dialogue modeling. In EMNLP 2022.\n\nWe sincerely appreciate the constructive critique provided by the reviewers, which has undeniably enriched the final version of the paper. Thank you for your time and consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Manuscript  is well organized and written.\n\n2. GLGR integrates global reasoning and local graph reasoning methods, and achieves substantial improvements.\n\n3. The authors did some meaningful integration work.", "reasons_to_reject": "1. The method of comparison is too old-fashioned and the latest baseline BiDEN is not for MDRC(Multi-party Dialogue Reading Comprehension). \n\n2. Insufficient task-related baseline comparisons.\n\n3. The method of construction graph used by the authors seems to be a relatively common technique used in neural networks.\n\n4. In line 292~294, why is the threshold set to 0.5? Are $W^l$ (in Equation 3) and $W^l_t $ in line 251 the same?\n\n5. According to the parameter settings, L1 is set to 2. But the two layers graph neural network cannot obtain global information, it can only obtain the node information of the second hop. Does the author need to discuss the impact of the number of graph neural network layers on model performance? Does global information help the model?", "questions_to_authors": "", "paper_topic_and_main_contributions": "This  manuscript proposes a question-aware global-to-local graph reasoning approach GLGR by integrating QIUG and LSRG. Compared to PLM-based baselines, GLGR achieves better results.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZqPDYYbQRr", "reviewer": "Reviewer_BDx5", "summary": "", "questions": "A: Will the proposed method have a higher computational cost compared to other baselines in the paper?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We wholeheartedly appreciate your dedicated time in reviewing our paper and offering valuable suggestions to enhance its quality. Your feedback holds immense value for us, and we approach it with utmost seriousness. Each suggestion you provided has been thoroughly considered, and we provide our responses as follows:\n\n----------Response to Comments----------\n\n**Comment #1:** The adoption of a two-stage encoder network for implementing the reasoning process may be computationally expensive and potentially limit its feasibility in large-scale applications.\n\n**Response to #1:** Our computations are predominantly centered around graph modeling, where both QIUG and LSRG incorporate two layers of graph neural networks. Specifically, if we utilize the ELECTRA backbone (335 million parameters), GLGR’s parameter count would only rise by around 45 million. Furthermore, the backbone we utilize is notably more lightweight compared to LLM such as Llama (7 billion parameters). Our models were trained and tested in a mid-range server that possesses a V100 GPU.\n\n\n----------Response to Questions----------\n\n**Question #1:** Will the proposed method have a higher computational cost compared to other baselines in the paper?\n\n**Response to #1:** The computations in GLGR primarily revolve around the graph reasoning modules QIUG and LSRG. Each module has two layers of graph neural networks. Consequently, the increase in parameter count remains manageable. If we use the ELECTRA backbone (335 million parameters), the parameter count would rise by 45 million. The overview of parameter counts of the previous work is as follows.\n\nSKIDB--- It employs a multi-task learning framework with speaker and key utterance prediction tasks. More parameters come from speaker modeling and utterance prediction. Specifically, SKIDB employs 3 to 5 transformer blocks for speaker information flow and extra layers for utterance prediction. Compared to SKIDB, GLGR's parameter count increase is modest. \n\nESA--- With speaker masking, speaker graph and discourse graph, ESA adds three channels to the backbone using R-GCN for graph modeling. By contrast, GLGR uses two graphs (QIUG and LSRG), possibly resulting in a parameter count similar to or lower than ESA.\n\nBiDeN--- It uses three channels to capture latent temporal information in the dialogue, including past-to-present, present-to-present and future-to-present channels. Each channel consists of multiple transformer blocks. The majority of additional parameters in BiDeN come from these transformer blocks. Considering that the parameter count for a single layer of a graph neural network is comparable to that of a transformer block, the incremental parameter increase in our GLGR is not substantial, compared to BiDeN.\n\nULM-UOP--- It utilizes BERT (12 transformer encoder blocks). Due to additional pretraining, it consumes more resources during training than GLGR.\n\nDADgraph--- It uses BERT without fine-tuning, relying solely on its embeddings. It employs a global utterance graph for discourse and speaker structure, resulting in lower training computational demand compared to GLGR. However, GLGR obtained considerable performance enhancements compared to DADgraph. \n\nTherefore, the computational cost of GLGR is insignificantly higher than most of the previous work.\n\n\nThank you again for your insightful feedback, which undoubtedly contributes to enhancing the quality and enriching the final version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper proposes an innovative reasoning approach, GLGR, tackling the problem of MDRC that has not been well-addressed.\n- The paper makes a decent contribution by modeling the Question-aware Interlocutor-Utterance Graph (QIUG) and Local Semantic Role Graph (LSRG) and demonstrating improvements in MDRC performance compared to PLM-based baselines.\n- The solution proposed in this paper displays versatility as it shows general improvements regardless of the PLM adopted as the backbone.", "reasons_to_reject": "- The adoption of a two-stage encoder network for implementing the reasoning process may be computationally expensive and potentially limit its feasibility in large-scale applications.", "questions_to_authors": "A: Will the proposed method have a higher computational cost compared to other baselines in the paper?", "paper_topic_and_main_contributions": "The paper, \"GLGR: Question-aware Global-to-Local Graph Reasoning for Multi-party Dialogue Reading Comprehension (MDRC),\" addresses the problem of multi-hop reasoning over global conversational structures and the omission of token-level semantic relations in utterances. The authors propose a Global-to-Local Graph Reasoning approach (GLGR) with pre-trained language models to encode Question-aware Interlocutor-Utterance Graph (QIUG) and Local Semantic Role Graph (LSRG). The primary contributions of the paper are the proposition of a question-aware global-to-local graph reasoning approach, the implementation of this approach using a two-stage encoder network, and an experimental validation of this approach using benchmark datasets.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Paper Topic And Main Contributions:
* The paper proposes a question-aware global-to-local graph reasoning approach (GLGR) for multi-party dialogue reading comprehension (MDRC).
* It introduces two types of graphs to facilitate reasoning. These graphs model global information across utterances and local information within individual utterances. A two-stage encoder network is employed to reason from the global level to the local level. 
* Experiments on two MDRC datasets (FriendsQA and Molweni) show that GLGR achieves significant improvements over BERT and ELECTRA baselines and state-of-the-art methods. The results show the contributions of global and local graph reasoning, as well as their integration via the two-stage reasoning process.

Reasons to accept:
* The paper is well organized and written.
* The two-stage reasoning framework going from global graph to local graph is intuitive and well-motivated for connecting coarse-grained and fine-grained semantics.
* The paper includes detailed experiments and an ablation study to demonstrate the effectiveness of the proposed approach.
* Generalization across models with different pretraining schemes (BERT and ELECTRA), which is promising.
* The global-to-local reasoning method applied to both graphs results in improved performance in multi-party dialogue reading comprehension.

Reasons to reject:
* The proposed graph construction heavily relies on external tools, such as co-reference, which may impact the stability of the approach. The technique proposed seems to include a lot of moving parts and engineering effort compared to the performance improvement over other baselines considered, which again impacts practical applicability. The authors explain in the rebuttal that their models are more lightweight than LLMs such as Llama.
* The decrease in performance as context length increases is a concern for practical applicability of these models. As dialogues become longer, the relationships between utterances and entities can become complex and introduce additional noise, thereby limiting the applicability and generalization ability of the approach. The authors explain in the rebuttal that their model performs better than the baselines in this regard.
Decision: Accept-Findings