Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model
Authors: Zhuoer Wang
Yicheng Wang
Ziwei Zhu
James Caverlee
Venue: EMNLP 2023 Findings (2023)
Keywords: Candidate Answer Extraction
Self-consitency Learning
Unsupervised Learning
Masker-Reconstructor Model
TL;DR: Unsupervised Differentiable Masker-Reconstructor(DMR) model achieves competitive performance on candidate answer extraction task.

Abstract: Question generation is a widely used data augmentation approach with extensive applications, and extracting qualified candidate answers from context passages is a critical step for most question generation systems. However, existing methods for candidate answer extraction are reliant on linguistic rules or annotated data that face the partial annotation issue and challenges in generalization. To overcome these limitations, we propose a novel unsupervised candidate answer extraction approach that leverages the inherent structure of context passages through a Differentiable Masker-Reconstructor (DMR) Model with the enforcement of self-consistency for picking up salient information tokens. We curated two datasets with exhaustively-annotated answers and benchmark a comprehensive set of supervised and unsupervised candidate answer extraction methods. We demonstrate the effectiveness of the DMR model by showing its performance is superior among unsupervised methods and comparable to supervised methods.

Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstruutor Model
Zhuoer Wang'
Yicheng Wang'
Ziwei Zhu? James Caverlee
Teexas A&M University
GGerrge Mason University
(wang, wwwereceeeeereeeeeeeeeeeeeeeeeee edu zzhu20@gmu. edu
Abstract
from the context passage an indispensable pre- requisite to ensure that the generated questions are of high quality and relevant to the user's interests of salient information contained in the context pas- sage.
Question generation a widely used data aug- mentation approach with extensive applica- tions, and extracting qualified candidate an- swers from context passages critical step for most question generation systems. How- ever, existing methods for candidate answer extraction are reliant on linguistic rules or an- notated data that face the partial annotation is- sue and challenges in generalization. To over- these limitations, we propose novel unsupervised candidate answer extraction ap- proach that leverages the inherent structure of context passages through Diffcrentiable (DMR) Model with the enforcement of ceffsconsit for picking up salient information tokens. We curated two datasets with www.uutivvel--nnoattte answers and benchmark comprehensive of super- vised and unsupervised candidate answer ex- traction methods. We demonstrate the effec- tiveness of the DMR model by showing its performance is superior among unsupervised methods and comparable to supervised meth- ods. Our code and data publicly available at http:://dddllwwe..ihhhbb...e
Traditional methods for answer extraction rely on linguistic rules and models to discover the syn- tactic structure of the input passage's sentences. Constituency tags of Noun Phrases and Named Entity Recognition (NER) tags of person, time, location, etc., are popular choices of candidate an- swers. However, this kind of answer extraction method can only extract limited types of answers and usually disregards the importance of different possible answer tokens on the context and domain basis. More recently, with the development of ma- chine learning and large language models, candi- date answer extraction has been formalized as sequence labeling problem and tackled with su- pervised sequence learning and classification with the use annotated answers from MRC datasets like SQuAD (Rajpurkar et al., 2016). Nevertheless, the answer annotations of existing MRC and QA datasets have the partial annotation issue due to the annotation protocols that did not enforce exhaus- tive answer extraction from the context passage, leading to the overlooking of other essential details that could be beneficial in helping readers grasp the context (Bao et al., 2022). Training models with partially annotated data could result in degraded performance as the data provide misleading super- vision signal (Yang et al., 2018; Zhang et al., 2017), Furthermore, the supervised learning methods still face the generalization challenge when applying them
1 Introduction
Question Generation (QG) is burgeoning field of Natural Language Understanding and Genera- tion. The objective of Question Generation to produce well-structured, coherent, and valuable questions that correspond to specific context pas- sage and the intended answer. QG systems play a vital upstream role in enhancing the robustness and generalizability of Question Answering (QA) and Machine Reading Comprehension (MRC) mod- cls (Du et al., 2017; Dong et al., 2023), empower- ing chatbots and virtual assistants to answer more user needs (Gottardi al., 2022), and powering Al-driven tutoring systems for educational pur- poses (Kurdi et al., 2019). For most existing QG systems, extracting qualified candidate answers
domains where acquiring additional human annotations is time-consuming and expen- sive.
To overcome the aforementioned limitations and challenges, we propose novel unsupervised can- didate answer extraction approach that leverages the inherent structure of context passage. We posit

---

that passage tokens can be categorized into two types backbone tokens and information tokens. Backbone tokens are structural and common across various passages within the same domain, and such tokens are easily recoverable when masked. In contrast, information tokens are difficult to recover when masked, and such tokens are crucial infor- mation of specific context passage, making them excellent candidate answers. In Section 3, we in- troduce the Differentiable Wasker-Reconstrrn model, in which the masker module learns mask out tokens that the reconstructor module can then readily recover through the enforcement of self- consistency. For comprehensive assessments, we exhaustively annotated total of 100 passages with candidate answers SQuAD and WikiHow cook- ing texts as detailed in Section 4.1. We demonstrate the competitive performance of the DMR model in Section 5 by comparing it with a comprehensive list strong baselines that cover recent advancements both supervised and unsupervised candidate an- swer extraction.
2017: Lewis et al., 2019; Nie et al., 2022), Among them, Yang et al. and Nic al. also consider more diverse types of phrases including Adjective Phrases (AP), Verb Phrases (VP), and sub-clauses (S) extracted from the consistency parsing models of the context passage. Specially, Nic et al. fur- ther expand recognized named entities into longer constituents for more diverse candidate answers. Despite the reliance on NER and parsing models, these linguistic and syntactic rules based candidate answer extraction methods fail take the impor- tance of the information into account. With the advancements of supervised learning methods for neural networks, multiple studies (Du and Cardie, 2017: Subramanian et al., 2018; Wang et al., 2019) utilize the annotated answer phrases of SQuAD, MRC dataset, to train neural models that are capable of tagging and classifying candidate an- swer tokens. However, Bao et al. point out that the answer annotations of existing MRC and QA datasets have the partial annotation issue due to the annotation protocols that did not enforce exhaus- tive answer extraction from the context passage, leading to the overlooking of other essential details that could be beneficial in helping readers grasp the context. Training models with partially anno- tated data could result in degraded performance as the data provide misleading supervision signal. To address the partial annotation problem, previ- ous work resorts to -antabie-nllleeeddd (PU) learn- ing (Kiryo et al., 2017), which uses modified risk estimators to re-balance the weights of positively labeled answer tokens and the remaining deemed- as- unluellbeledd tokens for unbiased learning of unbiased binary classifier. Nevertheless, as com- mon problem across different tasks, the supervised learning methods still face the generalization chal- lenge when applying them to new domains where acquiring additional human annotations is time- consuming and expensive.
In summary, we make the following contribu- tions:
We propose a novel Differentiable Masker- Reconstructor model, in light of recent progress on self-consistency learning and masked language models, for unsupervised candidate answer extraction.
We release two newly created datasets with exhaustivel--annotatee candidate answers.
We benchmark a comprehensive list of super- vised and unsupervised candidate answer ex- traction methods and show the strong perfor- mance of our DMR model.
2 Related Work
Candidate Answer Extraction focuses on the ex- traction of salient information tokens that usually contain key information and knowledge the readers may seek from the context passage. In previous work, Named Entities are the most popular type of candidate answers (Yang et al., 2017; Lewis et al., 2019; Fabbri et al., 2020; Nic et al., 2022). In such cases, tokens of the context passage are first pro- cessed by existing NER models that tag tokens of person, organization, location, date/time, and nu- merical expressions. Noun Phrase (NP) is another popular choice that has been used a lot (Yang et al.,
coff-consistency Learning has been adopted the field of Natural Language Processing in recent years for tasks like text encoding (Li et al., 2015), sentence compression and summarization (Baziotis et al., 2019; Malireddy et al., 2020), NER (Tovine et al., 2022), and data-to-text generation (Wang et al., 2023), Models that leverage self-consistency learning usually include two modules that are re- versals of each other. Specifically, one compressor module takes a sentence, paragraph, or document as the initial input and compresses the text into

---

Article Title
Content of the step Heat the Olive Oil
Add 2 tablespoons (30 ml) of olive oil large pot. Allow the oil to heat for 2 to 3 minutes medium-high heat, or until it begins shimmer. You can substiture vegetable or canola oil for the olive oil if you prefer.
How Make
Caldo Tlalpeno
How to Make
Place alarge skillet on the stove, and add tablespoons (30 ml) of olive oil it. Turn the burner to medium, and allow the oil heat for 3 5 minutes until it starts to shimmer.
Slow Cooker Spaghetti Sauce
Add 3 tahlespoons (45 ml) of olive oil large pot Dutch oven, and place the stove. Turn the heat medium, and allow the oil to heat for minutes, or until it starts shimmer. If you prefer, you can substitute butter for the olive oil.
How to Make
Shrimp Bisque
Place large skillet on the stove, and add tablespoon (15 ml) of olive oil. Turn the heat to medium-higl high, and allow the oil to heat until it starts to shimmer, which should take approximately 5 minutes. You substitute butter for the olive oil if you prefer.
How Make
Omelette in Jar
Add 2 tablespoons (30 ml) of olive oil to medium, heavy-bottomed saucepan. Place the pan on the stove, and heat it over medium heat until it begins to shimmer. which should take approxinately 5 minutes. You can substitute vegetable oil for the olive oil.
How to Make a White Pizza
Table I: Content of multiple WikiHow articles with different expressions of the step Heat the Olive Oil
intermediate outputs of dense encoding, abstrac- tive text, structured data that preserve the key information of the original input text. Sequentially, another reversal module attempts to reconstruct the initial input based on the first modulc's s inter- mediate condensed outputs. The two modules are progressively trained by enforcing the consistency between the initial input and reconstructed input. Notably, the differentiability of the intermediate outputs greatly affects the back- propppropagttion of the learning signals from the reversal module to the compressor module. Except the application of self- consistency learning for text encoding, most other applications including ours have the challenge of handling non-differentiable discrete intermediate outputs. To deal with this issue, lovine et al. and Wang et al. adopt cycle training that alternatively changes the roles of the two modules, and Baziotis al. draw support from Gunbel-Softmax with the strike-through approach for differentiable sam- pling from the categorical distribution. However, the ablation study conducted by Baziotis et al. sug- gests that the generation of a relevant and fluent summary was mainly driven by a topic loss.
ing BERT, RoBERTa (Liu et al., 2019) also uses MLM its pretraining objective. It improves upon BERT by using dynamic masking that changes the masking pattern applied to the input text rather than static masking that always masking the same words, which results in a more robust LLM. Along this vein, MLM becomes one of the most popular pre- training objectives for large language models, in- cluding ALBERT (Lan et al., 2020), BART (Lewis et al., 2020), Longformer (Beltagy et al., 2020), etc. that optimized BERT from different aspects.
Approach
Our approach is based on the intuition and obser- vation that the text passages within the same do- main have inherent structure shaped by the under- lying writing style, knowledge space, and expres- sion formulation. We convey these concepts by showing a typical example we sourced from the WikiHow' cooking articles. WikiHow is popular online resource that offers user-curated, step-by- step guides on how to perform certain tasks. As shown in Table 1, multiple WikiHow articles on the cooking of five different dishes have the same step Heat the Olive Oil. We can observe explicit templates, like Add AMOUNT of olive oil to CONTAINER and Place a CONTAINER on the stove, and add AMOUNT of olive oil, from these passages. For the step of Heat the Olive Oil, is easier for the Masked Language Models to fill the masks of [MASK] 3 tablespoons (45 ml) [MASK] [MASK]
Masked Language Modeling (MLM) is a fun- damental technique for the pretraining of Large Language Models (LLM), first popularized by the BERT model (Devlin et al., 2019). During the pretraining stage, some percentage of the input tokens are masked at random, and the model's objective is to predict the original tokens based on their context tokens. This is different from traditional language models that typically predict the next word in a sequence, enabling models to bidirectionally understand the context and achieve strong language understanding capability. Follow-
large pot or Dutch oven with correct tokens than filling the masks of Add [MASK][MASK][MASS of olive
hhtps//wwwwwwikkhhww.com

---

Original Passage
Allow the oil heat for minutes medium heat
Masker Module
Roconzmuztinn
Longth Penalty
Masked Passage
Allow the oil [M] [M] [M] minutes [M] [M]
Reconstructor Module
Reconstructed
Allow the oll heat for 2 3 minutes high heat
Passage
Figure 1: The Masker- Reconstructor Model (CM) represents the special token [MASK]). Reconstruction loss guides the learning of the Reconstructor module as well as penalizes the Masker module for masking out hard-to-recover tokens (illustrated by [M] and high in red). Length penalty enforces the learning of the Masker module so that those easy-to-recover tokens (illustrated by Allow the oil in red) are more likely be masked out as masking them would yield the gain of both length and reconstruction loss.
[MASK][AAK]]]]]] [MASK]. The reason is that the black tokens are repeatedly seen in the domain and result in statistically higher prediction probability when the colored tokens are given as the context for the MLM. On the other hand, the colored tokens are precise and specific information of particular passage that do not have the prediction probability as high the black to- kens. Therefore, tokens of the text passages can be categorized into the following two types:
oil to
Masker Masker-Reconstruee Reconstructor Model to discriminate back- bone tokens and information tokens based on the idea of self-consistency lcarning as mentioned Section 2. As illustrated in Figure 1, the Masker- Reconstructor Model consists of a Masker module and Reconstructor module. The Masker mod- ule is token classification model Cas powered by LLM. It takes the original context passage P in- put and makes binary classification for each token with indicating the token should be preserved and indicating the token should be masked. Based on the classification results of C's and the original pas- sage P, the Masker module outputs an intermediate passage 1 with masked tokens represented by the special token [MASK]. Sequentially, the Reconstruc- tor module, Mask-Filling model CR powered by LLM as well, takes the intermediate masked pas- sage P as input and predicts the conditional proba- bility of possible original surface tokens for each [MASK]. Relate to other works in sonssccony learning, in our work, the Masker module and the Reconstructor module are reversals of each other, and the two modules can therefore be progressively trained by enforcing the self-consistency between the initial input and reconstructed input. Specif- ically, the reconstructed passage P' is compared with the original passage for the calculation of the reconstruction loss as follows:
Backbone Tokens are those black tokens of the content shown in Table 1, are structural and com- mon across various passages within the same do- main, and such tokens are easily recoverable when masked.
Information Tokens, in contrast to Backbone To- kens. are difficult to recover when masked, and such tokens are crucial information of specific context passage, making them excellent candidate answers. Examples of Information Tokens are col- ored tokens Table that express the AMOUNT, CONTAINER, TIME, HEAT-LEVEL, SUBSTITUTION of the specific context. Besides the cooking do- main, Wikipedia articles regarding a person may have similar structure that has date of birth, place of birth, occupation, education, etc. as informa- tion tokens. News articles may express who, what, where, when of events as information tokens with a shared explicit or implicit template.
[P]
1
L(P,
l CM(P)PP))
P
3.1 The Masker-Reconstruuctn Model
[Prob(P:) Log Log(Cn(PP))))
To extract candidate answers without accessing ground truth labels and upstream constituency parser or named entity recognizer, we develop the
The reconstruction loss not only guides the leamn ing of the Reconstructor module but also ensures

---

that the Masker module is getting more penalty if it masked out tokens that are harder for the Recon- structor module to recover. Also, this learning pro- cedure is solely dependent on the self-consistency between the original input passage and the asso- ciated reconstructed passage. However, a short cut for the Masker module to achieve low recon- struction loss is to classify all the tokens 1 that preserves all the tokens, which betrays our goal of finding those haro-to-recover information tokens for the candidate answer extraction purpose. To en- force the learning of the Masker module, length penalty is needed in conjunction with the recon- struction loss. In our work, the length loss is calcu- lated as:
one-hot (hard) vectors. Meanwhile, in the back- ward pass, can utilize the gradient calculated from the continuous approximation of the standard Gumbel-Softmal distribution.
Through this diffcrentiable self-consistency en- forced learning paradigm, we demonstrate in Sec- tion 5 that the proposed DMR model is capable of picking up information tokens that excellent candidate answers in an unsupervised manner.
Experimental Setup
4.1 Data
A prevailing challenge of the candidate answer ex- traction task is the absence of specific dataset for comprehensive evaluation and training. Although the answer annotations from MRC and QA datasets have been ubiquitously adapted for the task of can- didate answer extraction, researchers have pointed out that these data have the partial annotation prob- lem for the candidate answer extraction task (Bao et al., 2022). As the annotation protocols used for the construction of MRC and QA datasets did not require their annotators to find an exhaustive list of candidate answers and come up with associ- ated questions, using data with missing annotations could provide wrong supervision signals to the can- didate answer extraction models, and evaluating with this kind of data could lead to wrong conclu- sion. Bao et al.'s analysis found that 48.89% and 62. 22.44% of candidate answers are missing from the SQuAD dataset and the DROP dataset (Dua et al., 2019) respectively. Our answer/context ratio analy- sis (available Appendix A) on existing MRC and QA datasets also suggests that their annotated an- swers only cover a very small portion, with 13.35% being the highest and as low as 0.26% of the infor- mation contained in the context passages.
1 IP|
Lennghh
Σ (MIPP)
Therefore, the final loss of the Masker module is:
LCM = L(P, Llength
where Å is a weighting factor that balances the two loss terms. Higher À enforces the model to mask out more tokens.
3.2 Differentiable Seff-consistency Learning An unaddressed challenge at this point is the non- differentiability of the intermediate outputs as the Masker's binary classification is discrete step, which obstructs the backpropagation of the main training signal, reconstruction loss, the Masker module. To achieve differentiable self-consistency learning, our recosstrocon model em- ploys the Straight-Throon mator(Jang et al., 2017).
Gumbel-Softnax esti-
Gumbel-Softmax IiaeffmaaxMMaadiiiia (Maddison et al., 2017), re- laxation of discrete sampling operations, introduces stochasticity into the model by generating differen- tiable approximations of discrete random variables. This permits the rasostrrreo model to sample different tokens in a differentiable manner. However, standard Gumbel-Softmax does not allow for hard decisions, i.e., decisions with binary val- ues (1 for preserved tokens, 0 for masked tokens), which are desirable in our case. Therefore, we use estima-
In consideration of the training and evaluation challenge, we prepared and curated the following three datasets facilitate the assessment of our work:
Original (Parttally-annotteed)) SQuAD: SQuAD is a large scale MRC and QA dataset with 97,095 pairs of question-nnswer pairs corresponding to 20,94 20,947 context passages. It is also the most fre- quently used dataset for candidate answer extrac- tion in previous work. Despite the partial annoation issue, SQuAD has the highest answer/context ra- tio of 13.55% so far, and its partially annotated answers can still provide valuable supervision and bring some generalizability through its large scale.
the Straight-Through Gumbel-Softhee tor that offers elegant way of approximating hard decisions while keeping the process differentiable. During the forward pass, the Straight- -Though Combel-Soffmanl uses the Gumbel-SSfftax func- tion with temperature approaching zero, yielding

---

Dataset
SQuAD
WH-C
bly due the instructional and task-oriented nature
Source
Wikipedia
WikiHow
of content.
Domain
Open-domain
Cooking
Context Passage Amount
20.947
16,642
4.2 Baselines
Average Passage Length
135 Tokens
66 Tokens
Answer/Context Ratio
We compare our approach with a comprehensive list of methods representing recent progress on both supervised and unsupervised candidate answer ex- traction.
13.35%
N/A
(Original)
Answer/Context Ratio
35.11%
50.51%
(Exhaustivcl--nnotttell
FT-LLM leverages the recent advancement in Large Language Models (LLM) and the associ- ated fine-tuning (FT) technique. It is a represen- tative and robust supervised method. We train the
Table 2: Dataset Statistics
We use the original SQuAD dataset mainly for the purpose of training and comparing with supervised methods, and we also report the performance of different candidate answer extraction methods on its partially annotated answers reference. Exhaustively-annolttee SQuAD: For a more comprehensive evaluation, we have exhaustively annotated a subset of SQuAD. We employed two annotators, both possessing Bachelor's degrees from accredited universities and having experience in Natural Language Processing. We followed Bao et al.'s annotation guidelines to encourage annota- tors to label similar types of information as well as any important information missed in the original SQuAD dataset. We conducted the annotation on 50 SQuAD passages using the text annotation tool POTATO (Pei et al., 2022). The annotation resulted in a high agreement of 87.8% and a satisfactory Cohen's 's kappa score (Cohen, 1960; Carletta, 1996; Artstein and Poesio, 2008) of 0.7486. We com- bined the the two annotators agreed set of candidate answers, balancing extensiveness and importance, with the original SQuAD answers the final labels for each passage.
RoBERTa classification model with the original SQuAD dataset.
SCOPE (Bao et al., 2022) is the state-of -rrt supervised method that attempt to learn a candidate answer extraction model from partially annotated data with the Positive-Unlabeled Learning tech- nique. Specifically, we train the RoBERTa classifi- cation model with the Original SQuAD dataset and the PU objective function described in the SCOPE paper with prior distribution π of all positive sam- ples calculated from our exhaustively annotated SQuAD data.
Noun Phrases extracted by the off-the-shelf library spaCy (Honnibal et al., 2020).
Named Entities extracted by the uff-the-shelf li- brary spaCy.
Extended NE (Nie et al., 2022) extends each recog- nized named entity by finding a longer constituent that contains the named entity in the constituency parse tree with at least 80% of the sentence length. DiverseQA refers to another candidate answer ex- traction method of Nie et al. that combines Ex- tended NE and constituents tagged NP, ADJP, VP, and S in the constituency parse tree as candi- date answers as described in the DiverseQA paper. We also report the standalone performance of ADJP, VP, and as the reference.
WH-C: We obtained 16, 142 cooking-related passages from WikiHow for the self-consistency learning. For the evalua- tion of candidate answer extraction on WikiHow, we annotated 50 passages following the same anno- tation schema as mentioned above. The annotation resulted in agreement of 89.67% and satisfac- tory Cohen's 's kappa score of 0.7908. As it shows in Table 2, compared SQuAD, the WikiHow cook- ing data pertains a more specialized domain with shorter context passage. However, our annotation results suggest that passages of WH-C are more informationnintensive than SQuAD. This is proba-
ahhastivell-annotaaae
ChatGPT (Ouyang et al., 2022) is recent break- through that has demonstrated leading zero-shot learning capability. We prompt ChatGPT 3 with the context passage accompanied by the prepended in- struction "Extracting qualified candidate answers from context passages is a critical step for most question generation systems. Please extract an ex- haustive list of candidate answers (substrings from the following context passage): ....... For repro- ducible results, we set the temperature to 0 when
'Ideally. we would like to use Bao et al.'s annotation on SQuAD. However, its first author informed us that the anno- tated data was unfortunately lost due to a storage issuc after his graduation.
CChaiGPT model version: gpt-3. 5-turbo (relcased on March 1st, 2023).

---

Dataset Metric
Original (Partially-nnnotteed)
Exhaustively-annottted
Precision
Recall
F1
Precision
Recall
F1
Supervised Methods
FT-LLM
o2.4
(0.07
(0.3
19.90
29.84(1.99
SCOPE
62.
69.38
44.09
45.81am
19.00
51.57
Unsupervisce iisdd Methods
Noun Phrases
18.91
75.45
30.24
37.90
69.42
49.04
Named Entities
25.37
38.00
30.42
43.54
26.02
32.60
Extended NE
20.07
44.04
27.57
40.19
36.08
38.02
DiverseQA
16.87
94.12
28.62
35.69
89.26
50.99
I- ADJP
17.85
5.64
8.57
36.74
4.91
8.66
I- VP
16.23
47.24
24.16
30.93
38.84
34.43
I-S
15.86
28.22
20.31
29.01
24.00
26.27
ChatGPT
23.75
53.79
32.95
41.00
39.80
40.39
DMR (Ours)
18.14g00)
80.63
29.61
37.94.a3z)
77.98
51.04
Table 3: Experiment Results Original (Left) and Exhaustively- (Right) SQuAD data. We bold the highest among supervised and unsupervised methods respectively and underline the second highest among unsupervised methods. trained models, we report average and standard deviation (in parenthesized subscripts) of each metric for repeated runs with different random seeds.
querying ChatGPT.
Metric
Precision
Recall
F1
Supervised Methods
4.3 Training Parameters
FT-LLM SCOPE
83.64,229
20.65,246:
33.03
For the models trained with either supervised learning methods or the self-consistency learning method, we use the RoBERTa-base model which has 12 layers, a hidden size of 768, 12 self-attention heads, and approximately 125M parameters. We use the AdamW optimizer with linear weight de- cay, maximum input length of 512, learning rate of Se-5, and effective batch size of 256. We train each model until convergence and report the average per- formance and standard deviation across three runs that have different random seeds. For the weighting parameter À of the DMR model, we initialize it as 0.35 and gradually increase it every epoch for final value of 0.65. Data-wise, supervised learning methods have access both context passage and annotated answers while other methods only have access to the context passage.
(7.23
(8.59
(1.47
Unsupervised Methods
Noun Phrases
60.69
71.17
65.51
Named Entities
88.18
13.83
23.91
Extended NE
71.57
15.74
25.80
DiverseQA
52.28
87.65
65.49
I ADJP
69.61
7.53
13.58
I- VP
46.42
$3.58
49.74
S-
47.76
33.28
39.23
ChatGPT
75.91
$2.78
62.27
DMR (Ours)
58.20c109)
78.04pm3
66.64
Table 4: Experiment Results on Exhaustively-amnotaa WH-C data (format is the same Table
annotated dataset. These results signify the partial annotation problem that brings misleading training signal and evaluation challenge. The high precision of Named Entities suggests that tokens of person, organization, location, datc/time, and numerical ex- pressions are key pieces information in passage that the reader might seek. However, the low recall implies that key information can also be expressed in various forms beyond named entities. The high recall of Noun Phrases reflects the dominant role of noun words in conveying information, but the low precision hints at the difference in information im- portance from the reader's perspective. Also, it is noteworthy that the recall of Noun Phrases dropped by 6. basis points when evaluation on SQuAD data changed to the exhaustively aahaustively-annottted annotated version. This drop suggests that many complex and diverse
5 Results and Analysis
Table displays the candidate answer extraction performance of different methods on Original, partially-annotated, SQuAD data and Exhaustively- annotated SQuAD data. Table 4 shows the counter- part on Exhhusstively-amootae
WH-C C data.
The SCOPE model consistently outperforms the FT-LLM model across all three datasets. The per- formance gain of +5. 02, +21. 73, and +39. 44 in F1 is mainly driven by the improvement recall, and the performance gaps on exhaustively-annotaa datasets are larger than the gap on the partially-

---

forms of candidate answers have been omitted by the crowd-sourcing workers of SQuAD.
Conclusions
In this work, we target the underexplored task of unsupervised candidate answer extraction and introduce a novel Differentiable Masker- Reconstructor model inspired by recent progress in celf-consistency learning and masked language models. We have created two exhaustively anno- tated datasets that mitigate the partial annotation is- sue of existing MRC/QA datasets, thus allowing for a fairer evaluation of candidate answer extraction methods. We demonstrate that the DMR model, without the reliance on annotated data and external tools, achieves superior performance compared to other unsupervised methods, and its performance is also comparable that of supervised methods.
For the partially-anne -anootted data, as the precision of each method might be underestimated due to the partial annotation of answers, mainly look at the recall metric to assess their coverage of the existing answers. Methods that intend to extract a more comprehensive set of candidate answers including SCOPE, DMR, and DiverseQA have achieved high recall as expected. As for the eval- uation on exhaustively-annotace data, the afore- mentioned methods were able to maintain high recall. Notably, SCOPE consistently achieved the best performance, measured by FI, with the help of mass amount of annotated answers from the original SQuAD dataset. When such annotated data not available, our DMR model achieved the best performance among the unsupervised methods, Compared to DiverseQA, another competitive un- supervised method, the performance gain of DMR is mainly accomplished by balancing the precision, which emphasizes the importance of potential can- didate answers, and recall, which emphasizes the extensiveness and diversity of potential candidate answers.
Limitations and Future Work
Despite the effort we have made, our work still has the following limitations:
Relatively Low Precision: Although the DMR model is capable of extracting extensive and di- verse candidate answers, the precision of our model has large gap with the supervised methods as well as Named Entities based methods. Part of the rea- son is that the reconstructor model usually relies on the first word of each sentence and sometimes few other structural words for a success reconstruc- tion of the original passage. Such tokens are not valuable candidate answers and could add noise to the extractions. Hence, it's necessary explore alternative ways to better encode the underlying structure beyond using preserved tokens. Also, given the fact that partially annotated answers are available for some domains. It worth exploring pos- sible ways of incorporating those valuable labels with the DMR model.
We also would like to point out that, compared to unsupervised methods with second highest FI, our DMR model has the performance gain of +0. .00 and +1. 13 inlaustively-annotttee SQuAD and WH-C respectively. The gain the former dataset is quite insignificant while substantial on the later dataset. We attribute this difference to the fact that WH-C is more focused on a specialized cooking domain while SQuAD consists of open-domain data. Therefore, WH-C should have a more promi- nent underlying structure that can be captured by the DMR model through self-consistency learning discriminate backbone tokens and information tokens. This finding could guide the future devel- opment and application of DMR model.
Fra Fragmented Candidate Answer Phrases: The DMR model sometimes would mask structural tokens within a candidate answer phrase. For in- stance, it may mask out the token "to" for the phrase "3 to 5 minutes", which results frag- mented answers. As a future work, we would like to explore the possibilities and approaches mask- ing and reconstructing by constituents instead of doing it by tokens.
Lastly, it is also interesting to see that ChatGPT achieved the second highest precision across all the datasets among unsupervised methods. Upon checking ChatGPT extracted answers, we find ChatGPT significantly prefers to extract Noun Phrases and Named Entities as candidate answers. This preference results in the omission of more diverse answer forms and lower recall.
Effect on downstream applications: The effec- tiveness of different candidate answer extraction models on downstream applications can be affected by various factors including candidate answer re- fining methods, capabilities of question generation models, capabilities of question answering mod-

---

els, etc. Due to the space and resource constraints, we were not able to conduct comprehensive ex-
Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:200..05150.
periments and analysis to quantify the effect of different candidate answer extraction methods on downstream applications. In the future, we would like to explore if the DMR model itself can be used refining method in collaboration with other extraction methods, and we also would like to de- sign controlled experiments to see the effect of different candidate answer extraction methods on downstream QG and QA tasks.
Jean Carletta. 1996. Assessing agreement on classi-
fication tasks: The kappa statistic. Computational Linguistics, 22(2):249-254.
Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Mea- surement, 20:37 46.
Jacob Devlin, Ming-Wo Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies. Volume (Long and Short Papers), pages 4171 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Ethics Statement
Although our DMR model has achieved strong per- formance, it may still extract undesired or incorrect tokens as candidate answers. The errors generated by our model can propagate and amplify in down- stream models, such as those used in QG and QA systems. As result, QG systems may generate in- appropriate offensive answers and QA systems may provide inaccurate responses or misleading information, which could lead to real-world conse- quences. We urge downstream users and end users to use our models and data with caution.
Xiangjue Dong, Jiaying Lu, Jianling Wang, and James Caverlee. 2023, Closed-book question generation via contrastive learning. In Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, pages 3150- 3162, Dubrovnik, Croatia. Association for Computa- tional Linguistics.
Xinya Du and Claire Cardie. 2017. Identifying where to focus reading comprehension for neural question generation. In Proceedings of the 2017 Conference Empirical Methods in Natural Language Process- ing, pages 2067-2073, Copenhagen, Denmark. Asso- ciation for Computational Linguistics.
Acknowledgements
We would like to express our gratitude to the three reviewers and the meta reviewer for their valuable suggestions.
Xinya Du, Junru Shao, and Claire Cardie. 2017. Learn- ing to ask: Neural question generation for reading comprehension, In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1342-1352, Vancouver, Canada. Association for Computational Linguistics.
References
Ron Artstein and Massimo Poesio. 2008. Survey article: Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555-33 596..
Qiaoben Bao, Jiangjie Chen, Linfang Liu, Jingping Liu, Jiaqing Liang, and Yanghua Xiao. 2022. Harvesting more answer spans from paragraph beyond annota- tion. In Proceedings of the Fifleenth ACM Interna- tional Conference
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. DROP: reading comprehension benchmark requir- ing discrete reasoning over paragraphs. In Proceed- ings of the 2019 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume (Long and Short Papers), pages 2368-2378, Min- neapolis, Minnesota. Association for Computational Linguistics.
Web Search and Data Mining, WSDM '22, page 27-36. New York, NY, USA. As- sociation for Computing Machinery.
Christos Baziotis, Ion Androutsopoulos, Toannis Kon- stas, and Alexandros Potamianos. 2019. SEQ"3: Differentiable
au- toencoder for unsupervised abstractive sentence com- pression. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 673-681, Minneapolis, Minnesota. Association for Computational Linguistics.
Alexander Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, and Bing Xiang. 2020. Template-based question generation from retrieved sentences for im- proved unsupervised question answering. In Proceed- ings of the 58th Anmual Meeting of the Association for Computational Linguistics, pages 4508 451884513, On- line, Association for Computational Linguistics.

---

Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eun- sol Choi, and Danqi Chen. 2019. MRQA 2019 shared task: Evaluating generalization in reading compre- hension. In Proceedings of 2nd Machine Reading for Reading Comprehension (MRQA) Workshop at EMNLP.
pages 7871-7880, Online, Association for Computa-
tional Linguistics.
Patrick Lewis, Ludovic Denoyer, and Sebastian Riedel. 2019. Unsupervised question answering by cloze translation. In Proceedings of the 57th Annual Meet- ing of the Association for Compulational Linguisties, pages 4896-49101 Florence, Italy. Association for Computational Linguistics.
Anna Gottardi, Osman Ipek, Giuseppe Castellucci, Shui Hu, Lavina Vaz, Yao Lu, Anju Khatri, An- jali Chadha, Desheng Zhang, Sattvik Sahai, Pre- ma Dwivedi, Hangjie Shi, Lucy Hu, Andy Huang, Luke Dai, Bofei Yang. Varun Somani, Pankaj Ra- jan, Ron Rezac. Michael Johnston, Savanna Stiff, Leslie Ball, David Carmel, Yang Liu, Dilek Hakkani- Tür, Oleg Rokhlenko. Kate Bland, Bugene Agichtein, Reza Ghanadan, and Yoelle Maarek. 2022. Alexa, let's work together: Introducing the first alexa prize taskbot challenge on conversational task assistance. In Alexa Prize TaskBot Challenge Proceedings.
Jiwei Li, Thang Luong, and Dan Jurafsky. 2015. A hierarchical neural autoencoder for paragraphs and documents. In Proceedings of the 53rd Annual Meet- ing of the Association for Computational Linguistics and the 7th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages I 100--115,, Beijing, China. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: robustly optimized BERT pretraining approach. CoRR, 1907.11699
Matthew Honnibal, Ines Montani, Sofie Van Lan- deghem, Adriane Boyd, et al. 2020. spacy: Industrial- strength natural language processing in python.
Andrea lovine, Anjie Fang, Besnik Fetahu, Oleg Rokhlenko, and Shervin Malmasi. 2022. Cyclener: An unsupervised training approach for named entity recognition. In Proceedings of the ACM Web Confer- ence 2022, WWW '22, page 2916 2924, New York, NY, USA. Association for Computing Machinery.
Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. 2017. The concrete distribution: A continuous re- laxation of discrete random variables. In Sth Inter- national Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Con- ference Track Proceedings. OpenReview.net.
Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categori- cal reparameterization with gumbel- softmax. 5th International Conference on Learning Representa- rions, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.nee
Chanakya Malireddy, Tirth Maniar, and Manish Shri- vastava. 2020. SCAR: Sentence compression using autoencoders for reconstruction. In Proceedings of the 58th Anmual Meeting of the Association for Com- putational Linguistics: Student Research Workshop. pages 88 94, Online. Association for Computational Linguistics.
Ryuichi Kiryo, Gang Niu, Marthinus C. du Plessis, and Masashi Sugiyama. 2017. Positive-unlabeler learn- ing with non-negative risk estimator. In Proceed- ings of the 31st International Conference on Neu- ral Information Processing Syxtems, NIPS' 17, page 1674-1684, Red Hook, NY, USA. Curran Associates Inc.
Yuxiang Nie, Heyan Huang, Zewen Chi, and Xian-Ling Mao. 2022. Unsupervised question answering via answer diversifying. In Proceedings of the 29th Inter- national Conference Computational Linguistics, pages 1732-1742, Gyeongju, Republic of Korea. In- ternational Committee on Computational Linguistics.
Ghader Kurdi, Jared Leo, Bijan Parsia, Uli Sattler, and Salam Al-Fmarii. 2019. A systematic review of auto- matic question generation for educational purposes. International Journal of Artificial Intelligence in Ed- ucation, 30:12 204.
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddic Simens, Amanda Askell, Peter Welin- der, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instruc- tions with human feedback. In NewrIPS.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In 8th Inter- national Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26-30, 2020. OpenReview.net.
Jiaxin Pei, Aparna Ananthasubrmmanimm, Xingyao Wang, Naitian Zhou, Apostolos Dedeloudis, Jack- son Sargent, and David Jurgens. 2022. Potato: The portable text annotation tool. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations.
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, AbdeIrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. pre-training for natural language generation, translation, and com- prehension. Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics,
BART: Denoising sequence-to-sequeee
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for

---

machine comprehension of text. Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 2383-2392 Austin, Texas. Association for Computational Linguisties.
Appendix
A Analysis of Extractive MRC/QA Datasets
Sandeep Subramanian, Tong Wang. Xingdi Yuan, Saizheng Zhang, Adam Trischler, and Yoshua Ben- gio. 2018. Neural models for key phrase extraction and question generation. In Proceedings the Work- shop on Machine Reading for Question Answering, pages 78-88, Melbourne, Australia. Association for Computational Linguistics.
In this section, we analyze extractive MRC/QA datasets that has been used at the MRQA 2019 Shared Task (Fisch et al., 2019). Our analysis is based on the preprocessed data released by the MRQA 2019 organizer, and the detailed statistics and comparison can be found in Table (continued in next page).
Siyuan Wang, Zhongyu Wei, Zhihao Fan, Yang Liu, and Xuanjing Huang. 2019. multi-agent communica- tion framework for question-worthy phrase extraction and question generation. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):7168- 7175.
Zhuoer Wang. Marcus Collins, Nikhita Vedula, Simone Filice, Shervin Malmasi, and Oleg Rokhlenko. 2023. Faithful low-resource data-to-text generation through cycle training. In Proceedings of the 61st Annual Meeting of Association for Computational Lin- guisticy (ACL), Toronto, Canada. Association for Computational Linguistics.
Yaosheng Yang, Wenliang Chen, Zhenghua Li, Zhengqiu He, and Min Zhang. 2018. Distantly su- pervised NER with partial annotation learning and reinforcement learning. In Proceedings the 27th International Conference on Computational Linguis- tics, pages 2159-2169, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Zhilin Yang. Junjie Hu, Ruslan Salakhutdinov, and William Cohen. 2017. Semi-supervised QA with generative Comain-adaptivy nets. In Proceedings of the 55th Annual Meeting the Association for Computational Linguistics (Volume I: Long Papers), pages 1040-1050, Vancouver, Canada. Association for Computational Linguistics.
Yue Zhang, Zhenghua Li, Jun Lang, Qingrong Xia, and Min Zhang. 2017. Dependency parsing with partial annotations: An empirical comparison. In Proceed- ings the Eighth International Joint Conference Natural Language Processing (Volume Long Pa- pers), pages 49-58, Taipei. Taiwan. Asian Federation of Natural Language Processing.

---

Question/Answer
Context
Context Token Length
Dataset
Anser/Context Ratio
Source
Source
SQuAD
Crowdsourced
Wikipedia
134.88
13.35%
NewsQA
Crowdsourced
News Articles
589.82
6.69%
TriviaQA
Trivia
Web Snippets
753.42
0.31%
SearchQA
Jeopardy
Web Snippets
738.57
0.26%
HotpotQA
Crowdsourced
Wikipedia
176.84
1.80%
Natural Questions
Search Logs
Wikipedia
205.03
7.61%
BioASQ
Domain Experts
Science Articles
241.29
1.23%
DROP
Crowdsourced
Wikipedia
225.66
7.75%
DuoRC
Crowdsourced
Movie Plots
692.14
2.33%
RACE
Domain Experts
Examinations
343.42
0.93%
Relation Extraction
Synthetic
Wikipedia
29.25
11.06%
TextbookQA
Domain Experts
Textbook
$71.21
1.28%
Table 5: Statistics and comparison of MRC/QA datasets.


Reviews:
[{"review_id": "MedyTTIaIW", "reviewer": "Reviewer_wx9Z", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate reviewer wx9Z's in-depth review and the recognition of our paper's *clear research motivation*, *good innovation*, and *verified validity with sufficient experiments*. \n\nWe acknowledge the reviewer's concern regarding the elaboration of the mechanism. We intended to maintain a balance between brevity and clarity. However, we understand the importance of ensuring the potential audience of that paper could fully understand the mechanism of our approach. In the revised version, we will expand Section 3 to include more elaboration on the mechanism of self-consistency learning in our context and add associated pointers to some of the related work we have mentioned in Section 2.\n\nThanks again for your time and valuable suggestions. We hope our response could address your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The method proposed in this paper uses the inherent structure of context paragraphs to divide words ( tokens ) into structural words ( backbone tokens ) and information words ( information tokens ). It combines self-consistency learning with mask language modeling, the mask module shields words, and the reconstruction module can recover words through self-consistency to extract key information in an unsupervised way.\n2. Two annotation datasets are created to mitigate the partial annotation issue existing MRC/QA datasets.\n3. This structure of the paper is rational and clear writing.\n4. The experiment of the paper is sufficient, and the validity of the proposed model is verified by comparing supervised and unsupervised methods.", "reasons_to_reject": "The mechanism of this paper is not indepth and needs to be further elaborated.", "questions_to_authors": "", "paper_topic_and_main_contributions": "According to the partial labeling problem and generalization challenge faced by the existing candidate answer extraction methods for language rules or unlabeled data, this paper proposes a novel unsupervised candidate answer extraction method ( DMR ).This method utilizes the inherent structure of the context paragraph, uses the  Differentiable Masker-Reconstructor Model to enhance self-consistency to extract key information, and release two newly created datasets with exhaustively-annotated candidate answers. In general, the research motivation of this paper is clear, the research status is relatively sufficient, and it has good innovation.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "msmpD2azAF", "reviewer": "Reviewer_HZLu", "summary": "", "questions": "Is there any example of answer candidates that cannot be extracted by NER (or other existing methods) but can be extracted the proposed model? I could not immediately think of a case where a token other than a named entity could be an answer candidate because most gold answer candidates may be NEs.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate reviewer HZLu's in-depth review and are delighted to see that the reviewer deems our proposed method as an *interesting approach with better performance* and acknowledges the *new datasets* we contributed. \n\nRegarding the reviewer's concerns, we would like to highlight that our study targets an understudied part of the existing Question Generation framework. Although augmenting data for QA is a major use case of QG, QG also has a much broader impact on building chatbots, virtual assistants, and educational systems as we have noted in lines 32~39 of the paper. We recognize the significance of the study on our approach's effect on downstream applications like QA. However, take QA as an example, many factors including but not limited to the generation capability of the QG system and the comprehension capability of the QA system would affect the final QA performance. The fair assessment of different candidate answer extraction approaches' effectiveness based on downstream tasks would require careful design of a large set of experiments to test different QG and QA models to attribute, differentiate, and quantify the impacts of each factor on the final QA performance, which we believe is a different scope and can formalize a separate full paper. To ensure our work has an adequate focus and effective evaluation, we opt to test and compare different methods directly on the target task of candidate answer extraction on the ground-truth annotation. A similar scenario in the field of NLP is that POS tagging has been a fundamental task that could be incorporated and benefit many downstream applications. However, the effectiveness of a newly proposed POS tagging method may only be directly evaluated with the task's ground-truth annotation instead of a full stack of downstream tasks.\n\nAs for the small gain of F1 over existing methods on SQuAD, we would like to emphasize that although the performance gain varies on different datasets (+0.07 on SQuAD, +1.13 on WH-C compared to the best-performing unsupervised methods), our method is capable of adapting to the new domains through unsupervised learning from the unlabeled text that leverages domain-specific inherent structure, which empowers it to constantly achieve competitive performance that balances the extensiveness and importance of the extracted candidate answers. \n\n\n*Is there any example of answer candidates that cannot be extracted by NER (or other existing methods) but can be extracted the proposed model? I could not immediately think of a case where a token other than a named entity could be an answer candidate because most gold answer candidates may be NEs.*\n\nAs you can see from the recall of Table 3 and Table 4, Named Entities constitute 38% of the ground-truth candidate answer tokens in Original partially-annotated SQuAD, 26.02% of Exhaustively-annotated SQuAD, and 13.83% of Exhaustively-annotated WH-C. For Noun Phrases (NP covers most NE), those numbers are 75.45, 69.42, and 71.17 respectively. Accordingly, there still exists a large number of answer candidates that cannot be extracted by NER. \n\nFor example, in *\"The fact that chloroplasts are surrounded by a double membrane is often cited as evidence that they are the descendants of endosymbiotic cyanobacteria.\"*, (*chloroplasts are*) *surrounded by a double membrane* could be a good candidate answer as it contains key information regarding what is often cited as evidence of them as descendants of endosymbiotic cyanobacteria. This cannot be extracted by the NER/NP, but our method could capture it as the information is very context-specific and uncommon across different passages. Although DiverseQA may capture this candidate answer by incorporating VP, it also extracts structural, unimportant phrases like \"The fact\" as a standalone candidate answer as the method considers all those different kinds of phrases regardless of their importance in the context while our method adaptively achieved a better balance of the extensiveness and importance as mentioned before.\n\nThanks again for your time and valuable suggestions. We hope our response could address your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- this paper proposes a interesting unsupervised approach for answer candidate span extractions.\n- the proposed model achieves better performance in terms of F1 score.\n- the authors will release a new datasets for the answer candidate generation task.", "reasons_to_reject": "- lack of experiments on downstream QA tasks.", "questions_to_authors": "Is there any example of answer candidates that cannot be extracted by NER (or other existing methods) but can be extracted the proposed model? I could not immediately think of a case where a token other than a named entity could be an answer candidate because most gold answer candidates may be NEs.", "paper_topic_and_main_contributions": "This paper proposes a new approach for the answer candidate extraction task. This task aims to extract answer candidates from a passage. Existing works rely on a rule-based approach, NER, or a supervised sequence tagger. The authors propose a Masker-Reconstructor Model, which jointly learns the \"masking model\" and \"reconstruction model\". The masking model decides which token to be masked and the reconstruction model tries to reproduce the original unmasked passage. If the masking model masks tokens that cannot be reconstructed from the context in a passage, the mask model gets a penalty so the proposed model is intended to properly mask tokens that can be inferred from the contexts.\n\nThe evaluation was conducted on several datasets including newly curated ones. The proposed method achieves better F1 scores.\n\nThe main contributions are:\n- this paper proposes an interesting unsupervised approach for answer candidate extractions.\n- the proposed model achieves better performance in terms of F1 score.\n- the authors will release new datasets for the answer candidate generation task.\n\nThe proposed model is reasonable and the paper is very well-written.\n\nHowever, the answer extraction task is quite narrow task and mainly a subpart of data augmentation techniques for improving downstream QA tasks, i.e., extracted answers are then used for the question generation module for obtaining synthetic QA pairs for augmentation. Thus, the experiments to check the effects on the downstream QA task are necessary for this type of study (This is also mentioned in the limitation section by the authors.). \n\nFrom the results in Table 3, the F1 score gained only 0.07 points (from 50.99 to 51.04) when the authors compared it with an existing best-performing model. I am interested in such a rather small gain attribute to the downstream QA tasks.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fMzmvWB7RO", "reviewer": "Reviewer_P2SC", "summary": "", "questions": "1. Is it possible that there are cases where the token is readily to recover, but it contains core information? For example, if the question is about common sense, it seems that the information is likely to be important, although it is easy to recover.\n\n2. Does the length penalty actually work to prevent masking the first n tokens of a sentence? According to the equation, it seems to operate on the entire sentence instead of forcing the first n tokens of the sentence. \n\n3. In order to train the Masker Module, is it  utilize a pre-annotated dataset in supervised manner or to have the model itself determine the tokens to be masked in an unsupervised manner?\n\n4. What is the meaning of P_i in the equation in section 3.1? In this paper, P_i generally means the token, but in this equation, it seems like the probability. It can cause confusion.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate reviewer P2SC's in-depth review and constructive suggestions. We also thank the reviewer’s acknowledgement of our *exploration* and the *contribution of new datasets*.\n\nFor Reasons To Reject 1 and Question For The Authors 4, we thank the reviewer for pointing out the potential ambiguities of P_i. In the loss functions, P_i means the probability of the i-th token of the Passage. We note that the shared P initials of Probability and Passage may cause confusions, and we will use more distinctive symbols and add some additional textual explanations in the revised version of our paper.\n\nFor Reasons To Reject 2 and Question For The Authors 2, if we understand it correctly, the reviewer refers to the illustration we were given in Figure 1. The length penalty does affect the masking ratio of the entire sentence jointly with the reconstruction loss. Without the length penalty, the masker module could take the shortcut of not masking any tokens or very few tokens to achieve low reconstruction loss as we have mentioned in lines 316~321 of our paper. When the length penalty is enforced, the masker model is more likely to mask out those successfully recovered tokens, as we have illustrated in the figure, since masking this kind of token would yield the gain of both length and reconstruction loss. We will add an explaining caption for Figure 1 to make these points more clear.\n\nFor Reasons To Reject 3, we would like to first clarify that our work is targeting the task of extracting candidate answers mainly for QG systems. Such candidate answers naturally exist in the text passage regardless of the existence or performance of any QA systems. As for the evaluation on downstream applications, we would like to highlight that our study targets an understudied part of the existing Question Generation framework. Although augmenting data for QA is a major use case of QG, QG also has a much broader impact on building chatbots, virtual assistants, and educational systems as we have noted in lines 32~39 of the paper. We recognize the significance of the study on our approach's effect on downstream applications like QA. However, take QA as an example, many factors including but not limited to the generation capability of the QG system and the comprehension capability of the QA system would affect the final QA performance. The fair assessment of different candidate answer extraction approaches' effectiveness based on downstream tasks would require careful design of a large set of experiments to test different QG and QA models to attribute, differentiate, and quantify the impacts of each factor on the final QA performance, which we believe is a different scope and can formalize a separate full paper. To ensure our work has an adequate focus and effective evaluation, we opt to test and compare different methods directly on the target task of candidate answer extraction on the ground-truth annotation. A similar scenario in the field of NLP is that POS tagging has been a fundamental task that could be incorporated and benefit many downstream applications. However, the effectiveness of a newly proposed POS tagging method may only be directly evaluated with the task's ground-truth annotation instead of a full stack of downstream tasks.\n\nFor Question For The Authors 1, we acknowledge the existence of the reverse cases, however the general pattern aligns with our observation on the different recovering difficulties of backbone tokens and information tokens. For the preservation of common sense related tokens, it depends on if it has been reiterated a lot in the corpus of the target domain to formalize an underlying structure. An example is the corpus of WikiPedia, although it contains much commonsense knowledge, most of such knowledge is only elaborated very few times compared to those structural expressions under the background of the huge text space. \n\nFor Question For The Authors 3, the Masker module does NOT rely on any pre-annotated datasets in a supervised manner. It learns from the enforcement of self-consistency (reconstruction loss) and the length penalty.\n\nThanks again for your time and valuable suggestions. We hope our response could address your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Well written and easy-to-read paper\n2. Exploring in candidate answer extraction with differential masker-reconstructor model\n3. Newly released exhaustively-annotated dataset", "reasons_to_reject": "1. The meaning of the word in the equation is not clear. In this paper, P_i generally means the token, but in section 3.1's equation, it seems like the probability. \n\n2. It is not clear whether the length penalty actually works for the first n tokens. According to the equation, it seems to be more influential in determining the masking ratio throughout the entire sentence.\n\n3. It is not clear whether the exhaustively-annotated data actually helps QA. Instead of carrying out the entire QA framework, the authors declared that they only focused on the answer candidate selection. However, it seems necessary to verify at least the groundtruth of the exhaustively-annotated data set. This is because the more answer candidate tokens exist, the more likely the performance of the QA is to be worse. Rather than omitting the answer token, it seems necessary to have grounds to support the authors' argument that extracting more answer candidate tokens, which contain answer, is beneficial for QA.", "questions_to_authors": "1. Is it possible that there are cases where the token is readily to recover, but it contains core information? For example, if the question is about common sense, it seems that the information is likely to be important, although it is easy to recover.\n\n2. Does the length penalty actually work to prevent masking the first n tokens of a sentence? According to the equation, it seems to operate on the entire sentence instead of forcing the first n tokens of the sentence. \n\n3. In order to train the Masker Module, is it  utilize a pre-annotated dataset in supervised manner or to have the model itself determine the tokens to be masked in an unsupervised manner?\n\n4. What is the meaning of P_i in the equation in section 3.1? In this paper, P_i generally means the token, but in this equation, it seems like the probability. It can cause confusion.", "paper_topic_and_main_contributions": "In this work, the authors tend to tackle the unsupervised candidate answer extraction for QA(question answering) though DMR(Differentiable Masker-Reconstructor) model. The proposed model enforce of self-consistency for picking up salient information tokens to overcome the limitations of existing NER methods, which can only extract a limited number of types, and deep learning methods, which do not work well on partial annotation datasets. The proposed DMR model classifies the text token to two types: (a) backbone tokens and (b) information tokens. (a) The backbone tokens are readily recoverable because of structural and common tokens within the same domain. (b) On the other hand, information tokens are difficult to recover when masked because it have crucial information of a specific context passage. In addition, the proposed methods are verified by supervised/un-supervised experiment with SQuAD dataset and WH-C dataset. Furthermore, authors release two newly created datasets with exhaustively-annotated candidate answers.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper is focused on the topic of unsupervised candidate answer extraction for QA. It introduces a novel approach using the DMR model. The reviewers largely appreciate the paper's innovation, clarity, and contributions (the DMR model and annotated datasets). The reviewers offered many constructive comments, and there were major questions regarding better understanding the experimental results and the significance of the proposed approach on downstream QA tasks. Overall this is an interesting work that is worth sharing with the community.
Decision: Accept-Findings