Enhancing text comprehension for Question Answering with Contrastive Learning
Year: 2023

Abstract:
Although Question Answering (QA) have advanced to the human-level language skills in NLP tasks, there is still a problem: the QA model gets confused when there are similar sentences or paragraphs. Existing studies focus on enhancing the text understanding of the candidate answers to improve the overall performance of the QA models. However, since these methods focus on re-ranking queries or candidate answers, they fail to resolve the confusion when many generated answers are similar to the expected answer. To address these issues, we propose a novel contrastive learning framework called ContrastiveQA that alleviates the confusion problem in answer extraction. We propose a supervised method where we generate positive and negative samples from the candidate answers and the given answer, respectively. We thus introduce ContrastiveQA, which uses contrastive learning with sampling data to reduce incorrect answers. Experimental results on four QA benchmarks show the effectiveness of the proposed method.

heading: 1 Introduction
text: Large language model (LLM) is developing rapidly, and ChatGPT (Ouyang et al., 2022), which can perform almost all natural language processing tasks, is receiving tremendous attention. ChatGPT can be easily used by anyone through a conversation interface, and ChatGPT responds appropriately to the user’s query. It is clear that ChatGPT has led to tremendous development and success, but it has a fatal problem called hallucination (Alkaissi and McFarlane, 2023; Bang et al., 2023). A generative model such as ChatGPT can have serious consequences if poor decisions are made when it is used in specific domains such as finance, law, and medicine (Shahriar and Hayawi, 2023). To avoid this hallucination problem in specific domains, finding the correct
∗Corresponding author
answer within a given document is more reliable. In this paper, we focus on extractive question answering (QA), which extracts the correct answer to a given question in a context. In the field of natural language processing, there are several attempts to solve extractive QA problems that can increase the ability to answer and reason like humans, such as HotpotQA dataset (Yang et al., 2018; Gao et al., 2021; Saxena et al., 2020). However, it is still challenging for the extractive QA model to have a human-level understanding or comprehension skills. In general, humans can easily find and select the correct answer among many similar answers related to the given question. On the contrary, the current QA model is somewhat less capable of selecting the correct answer if the correct answer is confused within the context (Gupta et al., 2018; Kratzwald et al.,
75
2019; Mao et al., 2021; Glass et al., 2022). To deal with these issues, the conventional methods for improving QA performance use answer re-ranking methods to find the correct answers (Iyer et al., 2021; Majumder et al., 2021). In (Iyer et al., 2021), they use a cross-attention method to build an answer re-ranking model. The (Majumder et al., 2021) seeks to improve the performance of the QA model by explicitly assigning scores to candidate sentences according to the overlap of named entities in the question and sentence. Although the answer re-ranking methods are simple and intuitive, they employ methods of re-ranking answers over a smaller set of candidates rather than considering confusing cases of answer selection. As shown in Fig. 1, a typical QA model explores each sentence to extract an answer to a question. For complex cases like Fig. 1, the QA model may confuse selecting a correct answer during answer extraction and extracting other similar answers. These cases are mainly because the distractor words confuse finding the correct answer.
With this motivation, we propose a novel QA model called ContrastiveQA that uses contrastive learning to pull semantically similar sentences closer and push different sentences further. In addition, the proposed model introduces a method of positive and negative sampling with candidate answers. ContrastiveQA performs the contrastive learning by generating positive and negative samples with the given answer and candidate answers obtained from a pre-finetuned QA model. The proposed model extracts an answer close to the correct answer because the QA model uses a contrastive objective to alleviate the confusing problem of selecting the correct answer.
We demonstrate the effectiveness of our approach on four extractive QA datasets: HotpotQA (Yang et al., 2018), SQuAD (Rajpurkar et al., 2016), TriviaQA (Joshi et al., 2017) and NewsQA (Trischler et al., 2017). We obtain notable performance for the proposed ContrastiveQA, a method learned by contrastive learning, on extractive QA datasets. In particular, the experimental results demonstrate that the proposed method performs significantly better than the previous approaches due to the positive and negative sampling.
Contributions. The main contributions of the proposed ContrastiveQA are as follows :
• We propose a novel method called, Con-
trastiveQA that effectively solves the problem of ambiguous answer selection and confusion.
• We introduce positive and negative sampling to perform contrastive learning better using QA datasets.
• Experimental results show that the proposed models outperform the baseline models on the extractive QA datasets.

heading: 2 Related Works
text: Contrastive learning for Question Answering. Contrastive learning has achieved remarkable performance in supervised/unsupervised learning and is actively used in image processing (Liang et al., 2020; Wang et al., 2021, 2022). The main goal of contrastive learning is to learn representations that keep similar samples close together and dissimilar samples far apart. In the field of natural language processing, contrastive learning is used to train text representation. Contrastive Domain Adaptive For Question Answering (CAQA) (Yue et al., 2021) proposes a framework for answering out-of-domain questions combining question generation and contrastive domain adaptation to learn domain invariants. This setup can transfer knowledge to the target domain without additional training that can effectively answer the out-of-domain questions. Cross Momentum Contrastive Learning (xMoCo) (Yang et al., 2021) used fast and slow encoders to learn and optimize phrase-question and question-phrase matching. They demonstrated its effectiveness in open-domain QA similar to MoCo (He et al., 2020). It effectively retained large speech samples requiring different question and phrase encoders. (Hu et al., 2022) proposed Momentum Contrastive Pre-Training for Question AnSwering (MCROSS) that employed contrastive learning to align knowledge learned from cloze-type samples to answer natural language questions. This momentum contrastive learning method improved the performance of pre-trained models when answering questions by maximizing the consistency of the distribution of predicted answers. It showed remarkable performance in QA tasks for supervised and zero-shot settings. (Caciularu et al., 2022) applied contrastive learning to reinforce the similarity between the question and evidence sentences. They introduced information encoding to relate the questions to the sentences in order to optimize questions.
They also used similarity mechanisms in specific subspaces by linear projections of raw representations. This method showed consistent improvement in benchmarks with long document context (Yang et al., 2018; Dasigi et al., 2021).
Unlike previous studies, our proposed approach aims to avoid ambiguous answer extraction by processing candidates similar to the correct answer in a set of candidates using contrastive learning.

heading: 3 Proposed model
text: In this work, we propose a novel questionanswering (QA) model, called ContrastiveQA. The proposed model allows accurate answers to be extracted by adding a contrastive loss term, which alleviates the confusion and errors that may occur when using the existing QA model.
To achieve this purpose, we divide the training phase into three main steps: candidate answer extraction, positive and negative sampling, and ContrastiveQA to perform contrastive learning. In particular, we introduce a method for creating data to perform ContrastiveQA, where the correct answer is known, in the positive and negative sampling step.
The overall flow of the proposed ContrastiveQA model is shown in Fig. 2.

heading: 3.1 Candidate answer extraction
text: We use the pre-finetuned QA model to perform candidate answer extraction 1. Like the existing QA models, inputs are questions and contexts, and outputs are answers. When extracting answers, we do not extract only one answer but multiple answers in the order of high confidence score. The QA model gives a confidence score for each of the output answers. And, from among the multiple answers obtained, the most optimal number of candidate answers required for learning are heuristically obtained through experiments (see as subsection 5.2).
Such an extracted candidate answer has a start position and an end position that indicates the answer location within a given context. We represent the candidate answer set by A that has Astart and Aend in Eq. (1) :
1The QA model used to extract the candidate answer is Longformer (Beltagy et al., 2020), which is the backbone model in the experiments. To train the pre-finetuned QA model, we use the data except for the unanswerable of SQuAD v2.0 (Rajpurkar et al., 2018) for training to avoid overlapping data contents. Here, we use the answerable context-questionanswer pair of SQuAD v2.0.
Aik(Astartik ,Aendik ) = QA(qi, ci) (1)
In Aik, i is the number of contexts and k is the number of candidate answer sets.

heading: 3.2 Positive and Negative Sampling
text: In this subsection, we describe the process of obtaining positive and negative samples to perform contrastive learning. In general, dropout, masking, random change, and back-translation are used to obtain positive and negative samples for contrastive learning in NLP tasks. These methods can be easily applied to obtain samples, but it is challenging to apply to QA tasks because finding correct answers to questions is crucial. We thus utilize the answer’s span (start, end) position information to reconstruct the data.
We refer to positive and negative samples as the
Algorithm 1 Procedure of positive and negative sampling for Supervised ContrastiveQA
Input: Candidate answer (Aik), Given answer (Ai) Output: Positive, Negative samples
1: Compare the span positions of the candidate answer and the given answer. 2: Given answer (start, end) position : (Asi , A e i ) 3: Candidate answer (start, end) position : (Asik,Aeik) 4: if (Asi , Aei ) = (Asik,Aeik) then 5: Positive sample 6: else if Ai = Aik and (Asi , Aei ) ̸= (Asik,Aeik)
then 7: Negative sample 8: end if 9: return Positive, Negative samples for each
context
set of extracted candidates to mitigate the confusing problem of QA models in subsection 3.1. We distinguish between positive and negative samples by comparing the given answers in the training data with the candidate answers.
We consider a candidate answer with the same token as a correct answer token and the same span position (start, end) as a positive sample. We consider a positive sample when the candidate answer and the actual correct answer have the same token positions. Conversely, if the candidate answer and the actual correct answer do not have the same token position, it is considered a negative sample.
To utilize the positive and negative samples created in long sequences, the sentences with each sample are also designated as candidate sentences corresponding to the candidate answers. We define a candidate sentence set as Eq. (2) :
Sik = {sij |Aik ⊂ sij , 1 ≤ j ≤ M} (2)
where the i-th context has M sentences, and Sik indicates the sentence to which Aik belongs. In Algorithm 1, we explain the overall flow for the positive and negative sampling phase.

heading: 3.3 ContrastiveQA using contrastive learning
text: We attempt to address QA problems that can be misleading for distractor words or similar answers using positive and negative samples. First, we use Longformer (Beltagy et al., 2020) as a backbone model to extract an answer with a question and context. We use the context consisting of 15 candidate sentences rather than using the given context. We detail the experiments and analysis of context construction in subsection 5.2. As in previous studies (Beltagy et al., 2020; Zaheer et al., 2020; Caciularu et al., 2021, 2022), we use the special tokens <q> and 
 to indicate question and sentence boundaries to represent sentences in a long context or document as one long sequence. We configure the ContrastiveQA model’s input as [CLS] <q> q1, q2, q3, · · · 
 s11, s12, · · · 
 s21, s22, · · · 
 sn1, sn2, · · · by adding a special token like the
input format introduced in Longformer (Beltagy et al., 2020). In the input format, qi represents a question token, and sjk represents the kth token of the jth sentence. The special tokens are added to the vocabulary and initialized before fine-tuning the model. We define LCE using the cross-entropy loss and LQC using the InfoNCE loss (Oord et al., 2018) to perform contrastive objectives. Then, we define the final loss, LCL, by adjusting LCE and LQC with λ. In the final loss (LCL), λ is weight hyperparameter. We heuristically adjust the λ value to 0.6.
LCL = λLCE + (1− λ)LQC (3) LCE calculates the difference between the probability distribution of y and ŷ, and during training, it learns to reduce this difference. We define it as Eq. (4) :
LCE = − 1
N
N∑
i=1
yi log(ŷi) + (1− yi) log(1− ŷi)
(4) In LQC , positive sentences (s+) and question (q) representations are pulled to be close to each other, while negative samples (s−) are pushed to be far apart from each other. We define it as Eq. (5) :
LQCi = −log esim(qi,s
+)/τ
esim(qi,s+)/τ + ∑k−1
j e sim(qi,s
− j )/τ
(5)
where sim(·,·) represents similarity between two elements and τ is a configurable temperature parameter. We use Sentence-BERT (SBERT)2 to calculate the similarity between two sentences. In detail, we experiment with the similarity function in subsection 5.4 and discuss it. Formally, we define the contrastive loss as the sum of the negative log-likelihood losses over all input examples, as introduced in (Caciularu et al., 2022). Here, each loss term distinguishes between positive and negative samples. LQCi serves one example each, and the final loss (LQC) is obtained as the average loss value over all the examples.

heading: 4 Experiment Setup
text: In this section, we provide experimental datasets and detail about the experiments. We summarize the number of train datasets and the source of the dataset in Table 1.

heading: 4.1 Datasets
text: We use HotpotQA (Yang et al., 2018), SQuAD (Rajpurkar et al., 2016), TriviaQA (Joshi et al., 2017), and NewsQA (Trischler et al., 2017) validation datasets for evaluation.
HotpotQA (Yang et al., 2018) is an multi-hop QA dataset that requires reasoning. In this experiment, we use the HotpotQA-distractor setting. HotpotQA includes yes and no as the answer type, so we add ‘yes no’ character at the beginning of the context.
SQuAD (Rajpurkar et al., 2016) is a set of QA pairs derived from Wikipedia articles. In SQuAD, the correct answer can be found as a sequence of tokens within the given context. This dataset consists of over 100,000 question-answer pairs.
TriviaQA (Joshi et al., 2017) is a realistic textbased QA dataset obtained from Wikipedia and the Web. TriviaQA has relatively complex and struc-
2SBERT(Reimers and Gurevych, 2019) is modified BERT that uses siamese and triplet network structure to derive semantically meaningful sentence embeddings. The code is available at https://www.sbert.net/.
tured questions, with considerable syntactic and lexical variability.
NewsQA (Trischler et al., 2017) is a reading comprehension dataset for CNN news articles. Some answers to questions in the NewsQA dataset may sometimes require reasoning.

heading: 4.2 Training details
text: In the experiment, we evaluate the ContrastiveQA to alleviate the problem of confusion in answer extraction. We use the Longformer-base model of the Huggingface Transformer as the backbone model3 (Wolf et al., 2020). We use four RTX 8000 GPUs in our experiments. We experiment with a batch size of 8 for the Longformer-based model and 32 for the BERT-based model. We conduct experiments in different optimal experimental settings for each dataset. The learning rate for the datasets HotpotQA and TriviaQA is set to 3e−5, and the model is trained for five epochs. For other two datasets, SQuAD and NewsQA, the learning rates are set to 5e−5 and 1e−4, respectively, and the model is trained for three epochs and five epochs, respectively.

heading: 4.3 Baselines
text: We compare the ContrastiveQA model with eight baselines below :
• Long-context QA Models
- Longformer (Beltagy et al., 2020) encodes long inputs using both sliding-window local attention and global attention. - CDLM (Caciularu et al., 2021) is a crossdocument language model that learns relationships between documents by pre-training a set of related documents. - BigBird (Zaheer et al., 2020) uses up to 4,096 tokens by applying block sparse attention to efficiently look at the tokens.
• Extractive QA Models 3https://huggingface.co/
- BERT (Devlin et al., 2019) is a pre-trained language model that can understand the context by pre-training MLM (masked language model) and NSP (next sentence prediction). - RoBERTa (Liu et al., 2019) is a Robustly Optimized BERT Pretraining Approach, and it shows better performance than the existing BERT by adjusting the hyperparameters and training data size of BERT. - SpanBERT (Joshi et al., 2020) is a model that masks random spans unlike BERT, which masks random tokens and learns a span boundary representation to represent and predict text spans well.
• Previous Multi-hop QA Models
- HGN4 (Fang et al., 2020) is a hierarchical graph network that reinforces answer/evidence prediction through graphs subdivided step by step in a hierarchical framework. - SLED5 (Ivgi et al., 2023) is a method of processing long texts using a fusion-in-decoderbased encoder-decoder language model.

heading: 5 Model Analysis

heading: 5.1 Extractive QA Results
text: We compare previously superior baseline models for extractive QA with ContrastiveQA.
4https://github.com/yuwfan/HGN 5https://github.com/Mivg/SLED
Supervised ContrastiveQA. We evaluate the performance of models using the F1-score for four benchmarks. Table 2 shows the results of HotpotQA trained by the supervised learning method. Our proposed model results demonstrate improved performance compared to the baseline. In particular, HopotQA, which we used in this experiment, is based on a distractor setting. HotpotQA contains distractor words within the context, causing more confusion in QA tasks. The proposed ContrastiveQA model improves performance by avoiding confusion. In addition, we show excellent performance compared to the existing baseline model based on F1-score for the SQuAD, TriviaQA, and NewsQA datasets in Table 3. Those results demonstrate the effectiveness of positive and negative sampling and contrastive objectives to create positive and negative samples to avoid confusion in a supervised setting.

heading: 5.2 Effect of Selecting Candidate sets
text: We conduct an ablation study with experimental data to prove the effectiveness of the components of the proposed model, ContrastiveQA. We perform an ablation experiment to find the optimal number of candidates in candidate answer extraction introduced in section 3.1. As shown in Fig. 3, the HotpotQA data of the distractor setting has two gold paragraphs and eight distractor paragraphs, so the number of total sentences is very large. However, in practice, the number of sentences required to infer the correct answer is significantly less than
the total number of sentences. For this reason, we adopt a method of extracting and using sentences related to the correct answer rather than considering the entire sentence. We experiment using the HotpotQA validation set that has the longest sequence with ten paragraphs as a representative example. We experiment by setting k, the number of candidate sets, to 5, 10, 15, 20, and all data. One becomes a positive sample from the total number of candidates, and the rest becomes negative samples. As shown in Table 4, we get the best performance when the number of candidates is 15 in all three models. Since HotpotQA data are complex and require reasoning, the performance may be reduced if the candidate set is too small. Moreover, if the
entire dataset is used up, performance may be deteriorated because the sequence length exceeds the capacity (e.g., 512, 1024, 4096) the model can accommodate. We obtain the optimal k of 15 in the ablation experiment. This means we set one positive sample and the rest as negative samples, 15 samples in the experiment.

heading: 5.3 Effect of Contrastive Loss
text: In this subsection, we analyze the effect of contrastive loss, a key component of our proposed model. Through experiments, we derive the most optimal result when λ is 0.6. First, we set the weight as 0.5 between the cross-entropy and contrastive loss to find the optimal λ value. Then, we experiment with 0.4 and 0.6 and confirm that the performance is lower when the weight is 0.4 than when the weight is 0.5, but the performance is better when the weight is 0.6. As shown in Table 5, when λ is 1, this is the result of using only the cross-entropy loss, and we demonstrate that the QA result is improved by using the contrastive loss through this ablation experiment. In Table 6, we show an example of both when the contrastive loss is applied and when it is not to demonstrate the effect of contrastive loss more intuitively.
Longformer and ContrastiveQA are indicated in pink and cyan , respectively.

heading: 5.4 Effect of Similarity Function
text: As briefly described in subsection 3.3, we perform experiments on the similarity function of the contrastive loss term. We experiment using SBERT (Reimers and Gurevych, 2019), which applies not only cosine-similarity, commonly used to calculate the similarity of two sentences (question, sentence), but also sentence-level embedding. In Table 7, we show the results obtained using cosine-similarity and SBERT on the HotpotQA validation set. As a result of HotpotQA, we demonstrate that SBERT performs better than cosine similarity in all three items: answer span, supporting facts, and joint F1.

heading: 6 Conclusion
text: In this work, we proposed a novel ContrastiveQA to alleviate the problem of confusion in answer extraction using contrastive learning. The proposed model ContrastiveQA consists of three tasks: 1) candidate answer extraction, 2) positive and negative sampling, 3) ContrastiveQA using contrastive
learning. We demonstrated the effectiveness of the proposed model by outperforming the performance of the baseline models on four benchmark datasets. In the future, we would like to adopt a method to specify the number of samples required in contrastive learning dynamically. This approach created an adaptive mechanism to use the number of samples for each input. This method could improve the overall performance of the model.

heading: 7 Limitations
text: We obtain notable QA performance through experiments. However, we conduct many experiments to find the optimal candidate for ContrastiveQA. Many of these experiments inevitably consume a lot of time and energy, and we have to heuristically determine the number of candidate sets through experiments in a limited environment. We intend to alleviate the current problems by adding a module that can solve these problems in our future research. For example, the Longformer model takes almost a day to process long text for each epoch to train. Therefore, we use smaller batch sizes with a limited number of GPUs to train the LongFormer model. However, due to the lack of GPU resources, the optimal weight of the proposed framework cannot be learned. Thus, it is necessary for further research on model weight reduction to mitigate computational resource problems.

heading: Acknowledgements
text: This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. 2021-0-02068, Artificial Intelligence Innovation Hub), and Center for Applied Research in Artificial Intelligence (CARAI) grant funded by DAPA and ADD (UD230017TD).

heading: A Experimental result examples
text: A.1 Supervised ContrastiveQA examples
Data fields Example Answer Mark Masons’ Hall Our proposed model answer prediction Mark Masons’ Hall
Baseline model answer prediction 86 St James’s Street
Context
Mark Masons’ Hall in London is the headquarters of The Grand Lodge of Mark Master Masons of England and Wales, which also controls the Royal Ark Mariner degree. It is located in 86 St James’s Street in the central London district of St James’s, opposite St James’s Palace. While Freemasons’ Hall is the headquarters of the United Grand Lodge of England and the Supreme Grand Chapter of Royal Arch Masons of England, Mark Masons’ Hall is the home of several other important appendant orders of Freemasonry in England and Wales. St James’s Palace is the most senior royal palace in the United Kingdom. Located in the City of Westminster, although no longer the principal residence of the monarch, it is the ceremonial meeting place of the Accession Council and the London residence of several members of the royal family.
Question What building is opposite the ceremonial meeting place of the Accession Council in the UnitedKingdom? Answer Adelaide
Our proposed model answer prediction Adelaide
Baseline model answer prediction Adelaide, about 4 km east of the Adelaide city centre. The suburb is in the City of Norwood
Context
Frewville is a small suburb in the South Australian city of Adelaide. It is three xa0kilometres south-east of Adelaide’s central business district (CBD). Norwood is a suburb of Adelaide, about 4 km east of the Adelaide city centre. The suburb is in the City of Norwood Payneham & St Peters, the oldest South Australian local government municipality, with a city population over 34,000. Walter Frank Giffen (20 September 1861 in Norwood – 28 June 1949 in Adelaide) was an Australian cricketer who played in 3 Tests between 1887 and 1892. He was the brother of the great all-rounder George Giffen. Glenunga is a small southern suburb of 2,539 people in the South Australian city of Adelaide. It is located five kilometres southeast of the Adelaide city centre. Glenunga is a small southern suburb of 2,539 people in the South Australian city of Adelaide. It is located five kilometres southeast of the Adelaide city centre.
Question Walter Giffen is from a suburb of which South Australian city? Answer 2013
Our proposed model answer prediction 2013
Baseline model answer prediction 2015
Context
Frozen Fever is a 2015 American computer-animated musical fantasy short film produced by Walt Disney Animation Studios and released by Walt Disney Pictures. It is a sequel to the 2013 feature film "Frozen", and tells the story of Anna’s birthday party given by Elsa with the help of Kristoff, Sven, and Olaf. Chris Buck and Jennifer Lee again served as the directors with Kristen Bell, Idina Menzel, Jonathan Groff, and Josh Gad providing the lead voices. "Making Today a Perfect Day" is a song from the 2015 Walt Disney Animation Studios computer-animated short film "Frozen Fever", with music and lyrics by Kristen Anderson-Lopez and Robert Lopez and performed throughout most of the short. It was released as a single in the United States on March 12, 2015.
Question What is the year of the event that occured first, Making Today a Perfect Day was produced, orFrozen was produced?
Table A1: Example of answer extraction of Baseline and ContrastiveQA.
In Table A.1, we classify into positive and negative samples using the information of the given correct answer and train them with contrastive loss to derive a result that is closer to the original correct answer when compared with the baseline.

heading: Future Work
text: We intend to alleviate the current problems by adding a module that can solve these problems in our future research. For example, the Longformer model takes almost a day to process long text for each epoch to train. Therefore, we use smaller batch sizes with a limited number of GPUs to train the LongFormer model. However, due to the lack of GPU resources, the optimal weight of the proposed framework cannot be learned. Thus, it is necessary for further research on model weight reduction to mitigate computational resource problems