Contradiction Retrieval via Contrastive Learning with Sparsity
Authors: Haike Xu
Zongyu Lin
Kai-Wei Chang
Yizhou Sun
Piotr Indyk
Venue: ICML 2025 poster (2025)
Keywords: contradiction retrieval
sentence embedding
Primary area: deep_learning->other_representation_learning
TL;DR: We design SparseCL, a novel approach for contradiction retrieval, which leverages specially trained sentence embeddings and a combined metric of cosine similarity and sparsity to efficiently identify documents that contradict a given query.

Abstract: Contradiction retrieval refers to identifying and extracting documents that explicitly disagree with or refute the content of a query, which is important to many downstream applications like fact checking and data cleaning. To retrieve contradiction argument to the query from large document corpora, existing methods such as similarity search and cross-encoder models exhibit different limitations.
To address these challenges, we introduce a novel approach: SparseCL that leverages specially trained sentence embeddings designed to preserve subtle, contradictory nuances between sentences. Our method utilizes a combined metric of cosine similarity and a sparsity function to efficiently identify and retrieve documents that contradict a given query. This approach dramatically enhances the speed of contradiction detection by reducing the need for exhaustive document comparisons to simple vector calculations. 
We conduct contradiction retrieval experiments on Arguana, MSMARCO, and HotpotQA, where our method produces an average improvement of $11.0\%$ across different models. We also validate our method on downstream tasks like natural language inference and cleaning corrupted corpora.
This paper outlines a promising direction for non-similarity-based information retrieval which is currently underexplored.

Contradiction Retrieval via Contrastive Learning with Sparsity
Haike Xu Zongyu Lin Kai-Wei Chang?
Yizhou Sun? 2 Piotr Indyk' !
Abstract
dings are mainly tailored to similarity retrieval, while as far we know, there hasn't 't been sentence embeddings for non-similarity based retrieval. In this paper, we study the problem of contradiction retrieval, typical case of non- similarity based retrieval. Given a large document corpus and a query passage, the goal is to retrieve document(s) in the corpus that contradict the query, assuming they exist. This problem has large number of applications, includ- ing counter-argument detection (Wachsmuth et al., 2018) and fact verification (Thorne al., 2018). The standard approaches to retrieving contradictions are two-fold. One is to use a bi-encoder (Xiao et al., 2023; Li & Li, 2023; Li et al., 2023) that maps each document to a feature space such that two contradicting documents are mapped close to each other (c.g., according to the cosine metric) and use nearest neighbor search algorithms. The second approach is to train cross-encoder model (Xiao et al., 2023) that determines whether two documents contradict each other, and apply it to each document or passage in the corpus.
Contradiction retrieval refers to identifying and extracting documents that explicitly disagree with or refute the content of query, which important to many downstream applications like fact check- ing and data cleaning. To retrieve contradiction argument to the query from large document cor- pora, existing methods such as similarity search and cross-encoder models exhibit different limita- tions. To address these challenges, we introduce a novel approach: SparseCL. that leverages spe- cially trained sentence embeddings designed preserve subtle, contradictory nuances between sentences. Our method utilizes a combined met- ric of cosine similarity and a sparsity function to efficiently identify and retrieve documents that contradict given query. This approach dramati- cally enhances the speed of contradiction detec- tion by reducing the need for exhaustive docu- ment comparisons to simple vector calculations. We conduct contradiction retrieval experiments on Arguana, MSMARCO, and HotpotQA, where our method produces an average improvement of 11.0% across different models. We also validate our method on downstream tasks like natural lan- guage inference and cleaning corrupted corpora. This paper outlines promising direction for non- similarity- based information retrieval which is currently underexplored.
Unfortunately, both methods suffer from limitations. The first approach (cosine similarity search on sentence embed- dings) is inherently incapable of representing the "contra- diction relation" between the documents, due to the fact that the cosine metric is "transitive" (See Appendix B for formal analysis): if A is similar to B, and B is similar to C, then A is also similar to C. As an example, consider an origi- nal sentence and its paraphrase in Table 12. Both of them contradict the sentence in the third column but they are not contradicting each other. The second approach, which uses cross-encoder model, can capture the contradiction between sentences to some extent, but it is much more computa- tionally expensive. Our experiment in Appendix F shows that compared with standard vector computation, running a cross cross-encoder is at least 200 times slower.
1. Introduction
Training sentence embedding for similarity retrieval has been well studied in the literature (Gao et al., 2021; Xiong et al., 2020; Karpukhin et al., 2020), where a standard practice is to use contrastive learning to map those simi- lar sentences together and those dissimilar sentences far from each other. However, these existing sentence embed-
In this paper, we propose to overcome these limitations by introducing SPARSECL for efficient contradiction retrieval using sparse- sentence embeddings. The key idea be- hind our approach is to train sentence embedding model to preserve sparsity of differences between the contradicted sentence embeddings. When answering a query, we calcu- late a score between the query and each document in the corpus, based on both the cosine similarity and the sparsity of the difference between their embeddings, and retrieve the ones with the highest scores. Our specific measure of
'qqual contribution 'MIT "Univrrity California, Los Ange- les. Correspondence to: Haike Xu <haikexu @ mit.edus.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Contradiction Retrieval via Contrastive Learning with Sparsity
sparsity is defined by the Hoyer measure of sparsity (Hurley & Rickard, 2009), which uses the scaled ratio of the & norm and the lg norm of vector as a proxy of the number of non-zero entries in the vector. Unlike the cosine metric, the Hoyer measure is not transitive (please refer to Appendix B for a detailed analysis), which avoids the limitations of the former. At the same time this method is much more effi- cient than a cross-encoder, as both the cosine metric and the Hoyer measure are easy compute given the embeddings. The Hoyer sparsity histogram of our trained embeddings is displayed in Figure 2.
2018), there has been few previous work on retrieving the best counter-argument for a given argument (Orbach et al., 2020; Shi et al., 2023). In terms of methods, (Wachsmuth et al., 2018) uses a weighted sum of different word and em- bedding similarities and (Shi et al., 2023) designs a "Bipolar- encoder" and a classification head. We believe that our method relying only on cosine similarity and sparsity is simpler than theirs and produces better results in the experi- ment. In addition, some analyses in the counter-argument retrieval papers are specific to the "debate" setting, e.g. they rely on topic, stance, premise/conclusion, and some other inherent structures in debates for help, which may prevent their methods from being generalized to broader scenarios.
We first evaluate our method on the counter-argument de- tection dataset Arguana (Wachsmuth et al., 2018), which to the best of our knowledge, is the only publicly avail- able dataset suitable for testing contradiction retrieval. In addition, we generate another two data sets, where con- tradictions for documents in MSMARCO (Nguyen et al., 2016) and HotpotQA (Yang et al., 2018) datasets are gener- ated using GPT-4 (Achiam et al., 2023). Our experiments demonstrate the efficacy of our approach in contradiction retrieval, as seen in Table 1. We also apply our method to corrupted corpus cleaning problem, where the goal is to filter out contradictory sentences in corrupted corpus and preserve good QA retrieval accuracy.
Fact verification and LLM hallucination Addressing the hallucination problem in Large Language Models has been a subject of many research efforts in recent years. According to the three types of different hallucinations in (Zhang et al., 2023b), here we only focus on those so called "Fact-Cone CoeetCoonliiting Hallucination" where the outputs of LLM contradict real world knowledge. The most straightforward way to mitigate this hallucination issue is to assume an external groundtruth knowledge source and augment LLM's outputs with an information retrieval system. There have been a few works on this line showing the success of this method (Ren et al., 2023; Mialon et al., 2023). This practice is very similar to "Fact- Veriiteeiiication" " (Thorne et al., 201 8; Schuster et al., 2021) where the task is to judge whether a claim is true or false based given knowledge base.
To summarize. our contributions can be divided into three folds:
We introduce novel contradiction retrieval method that employs specially trained sentence embeddings combined with metric that includes both cosine simi- larity and the Hoyer measure of sparsity. This approach effectively captures the essence of contradiction while being computationally efficient.
However, as pointed out by (Zhang et al., 2023b), in the era of LLM, the external knowledge base can encompass the whole internet. It is impossible to assume that all the information there are perfectly correct and there may exist conflicting information within the database. In the context of our paper, instead of using a groundtruth database to check an external claim, our goal is to check the internal contradictions between different documents in an unknown corpus.
Our method demonstrates superior contradiction re- trieval metrics over different datasets compared to isting methods. This underscores the effectiveness of our embedding and scoring approach.
We apply our contradiction retrieval method to two downstream settings: (1) corpus cleaning, where SPAR- SECL removes contradictions from comrupted datasets to maintain high-quality QA retrieval; and (2) natu- ral language inference, where SPARSECL assigns a higher Hoyer sparsity score between contradicted pairs. These applications highlight the practical benefits of our approach in real-world scenarios.
Learning augmented LLM and retrieval corpus attack Augmenting large language models with retrieval has been shown to be useful for many purposes. Recently, there have been a few works (Zhong et al., 2023; Zou et al., 2024) study- ing the vulnerability of retrieval system from adversarial attack. Specifically, they show that adding a few corrupted data points to the corpus will significantly drop the retrieval accuracy. This phenomenon brings our attention to the ne- cessity of checking the factuality of the knowledge database. Note that the type of corrupted documents considered by their papers are different from ours. While they consider the injection of adversarially generated documents, we consider the existence of contradicted documents as a natural part of the corpus. Also their purpose is to show the effect of
2. Related Work
Counter Argument Retrieval A direct application of our contradiction retrieval task in Soouter-argumeet retrieval". Since the curation of Arguana dataset by (Wachsmuth et al.,
2

---

Contradiction Retricval via Contrastive Learning with Sparsity
Sent1
Sent2
Ice hockey a contact team sport played on ice, usually in rink, in which two teams of skaters use their sticks to shoot vulcanized rubber puck into their opponent's net to score points...
Ice hockey is a non-contact team sport played grass, often in open field, where two teams players their feet to kick soft leather ball into their opponent's goal score points...
0
Cosine-similarity
Cosine-similarit
Sparsity
Classification Head
Emb1
Emb2
Emb1
Emb2
Text Encoder
Text Encoder
Text Encoder
Text Encoder
Encoder
Sent1
Sent2
Sent1
Sent2 Bi-Encoder (CL)
Sent1
Sent2
Cross-Encoder
SparseCL
Figure 1. Comparison of our SPARSECL with Cross-Encoder and Contrastive-Leariing based Bi-Encoder for contradiction retrieval. adversarial attack, while we provide a defense method for a certain kind of corrupted database.
function basically a negation a similarity function, so Wachsmuth et al. (2018) proposes several different simi- larity functions and sets the scoring function maximize of them and minimize another. Here, instead of enu- merating different similarity funetions, consider another notion: the "sparsity" of their embedding differences. The basic intuition is as follows. Suppose that all sentences are represented as vectors a "semantic" basis, where each coordinate represents one clearly identifiable seman- tic meaning. Then a contradiction between two passages should manifest itself a difference in few coordinates, while other coordinates should be quite close to each other. The issue, however, is that we do not know how to construct the appropriate basis, and the sparsity is defined with respect to a fixed coordinate system. Nevertheless, following this intuition, we fine-tune sentence embedding models using contrastive learning, by rewarding the sparsity of the differ- ence vectors between embeddings of contradicting passages. Please see Figure 2 for the Hoyer sparsity histogram of our trained embeddings.
3. Method
Problem Formulation We consider the contradiction re- trieval problem: given a passage corpus C' = {P1, P2. .P. and a query passage g, retrieve the "best" passage p* that contradicts We assume that several similar passages sup- porting g might exist in the corpus C.
Embedding based method Judging whether two pas- sages contradict each other is a standard Natural Language Inference task and can be easily tackled by many off-the- shelf language models (Touvron al., 2023; Xu et al., 2022). However, retrieve the best candidate from the corpus, we have to iterate the whole corpus, or at least send the candi- dates retrieved by similarity search to the language model to determine if they constitute contradiction. This is time consuming, given that there are potentially many similar passages in the corpus. Therefore, in our paper, we mainly focus those methods that only rely their passage em- want design simple scoring function F that given the embeddings of two passages, out- puts score between 0 and 1, indicating the likelihood that they are contradicting each other.
SPARSECL We use contrastive learning (Gao et al., 2021; Karpukhin et al., 2020) to fine-tune a pretrained sentence embedding model to generate the desired sparsity- awarr embeddings. The choice of positive and negative examples are exactly the reverse of the choice we make when the training sets are Natural Language Inference datasets. The positive example for a passage is its contradiction passage in the training set. The hard negative example for a passage is its similar passage in the training set. There are also other random i-batch passages as soft negative examples. The
beddings. Specifically,
Sparse Aware Embeddings Following the idea from counter-argument retrieval papers (Wachsmuth et al., 2018), such a score function should be a combination of similarity and dissimilarity functions. Observe that a dissimilarity
3

---

Contradiction Retricval via Contrastive Learning with Sparsity
Standard sentence embedding
Scoring function for contradiction retrieval For the score function for contradiction retrieval, we use weighted sum of the standard cosine similarity and our sparsity func- tion. Note that the cosine similarity is provided separately by any off-the-shelf sentence embedding model in a zeroshot manner. It can also be fine-tuned. Let B0 be the standard sentence embedding model and E,( be our sparse-aware sentence embedding model trained by SPARSECL. Then the final score function for contradiction retrieval is
1600
Random pairs Paraphrases Contradictions
1400
1200
Frequency
1000
800
400
P) = cos (E(g), E(p)) + a (o)
where a is a scalar tuned using the validation set. Note that the criterion for contradiction usually case-dependent, so it is necessary that we reserve a parameter to adapt to dif- ferent notions of contradiction. To get the answer passages, we calculate the score function for all passages and report the top 10 of them'.
0.18
0.19
0.20
0.21
0.22
0.23
0.24
0.25
Hoyer sparsity
Our sentence embedding
Random pairs Paraphrases Contradictions
3000
2500
4. Experiments
Frequency
2000
We test our SPARSECL method on a counterargument re- trieval task Arguana (Wachsmuth et al., 2018) and two con- tradiction retrieval datasets adapted from HotpotQA (Yang et al., 2018) and MSMARCO (Nguyen et al., 2016). Then, we apply our contradiction retrieval task to two downstream applications: retrieval corpus cleaning and natural language inference. Finally, we perform ablation studies to explain the functionality of each component of our method. Most of our experiments are not so computationally extensive, which can be by one single A6000 GPU. We run major experiments on A6000 and A A100 GPUs.
1500
1000
500
0
0.2
0.3
0.4
0.5
0.6
0.7
Hoyer sparsity
Figure 2. Histograms for the Hoyer sparsity of different pairs of sentence embedding differences on HotpotQA test set. The upper figure is the histogram produced by standard sentence embed- ding model ("bge-base-en-vil sity values for random pairs, paraphrases, and contradictions are 0.212, 0.211, 0.211. The lower figure is the histogram produced by our sentence embedding model fine-tuned from "bge-base-ent v1.5" using our SPARSECL method, where the median Hoyer sparsity values for random pairs, paraphrases, and contradictions are 0.212, 0.281, 0.662.
where the median Hoyer spar-
4.1. Counter-argument Retrieval
Dataset Arguana is dataset curated in (Wachsmuth et al., 2018), where the author provide a corpus of 6753 argument- countcrargument pairs, taken from 1069 debates with 15 themes on idebate.org. For each debate, the arguments are further divided into two opposing stances (pro and con). For each stance, there are paired arguments and counter- arguments. The dataset is split into the training set (60% of the data), the validation set (20%), and the test set (20%). This ensures that data from each individual debate is in- cluded in only one set and that debates from every theme are represented in every set. The task goal is: given an argument, retrieve its best counter-argument.
sparsity function we choose here is Hoyer sparsity function from (Hurley& & Rickard, 2009). Let h and h2 be two sentence embeddings and their embeddings have dimension d. We define
Hoyer(hı, = (VVd
|/1 hella
(Vd-1).
h2ll2
This is transformed version of the ratio of the l to the la norm, with output normalized to [0, 1].
Training We use Arguana's 's training set fine-tune our sparsity aware sentence embedding model via SPARSECL.
Finally, for each training tuple (0.,0: x,,5 with their em- beddings (he, btt h5 ), batch size N, and temperature v, its loss function is defined as
'In the actual implementation, for time efficiency, we first use FAISS (Douze et al., 2024) to retrieve the top K candidates with cosine similarity and then rerank them using our cosine + sparsity score function. We set very large K (e.g. K 1000) so that empirically this is almost equivalent searching for the maximal cosine + sparsity score in the whole corpus
HT)
4 = - -log
(Hoyathash))) D/T + + Hoyar(hs, 3/7) M/²) Σラ=
4

---

Contradiction Retrieval via Contrastive Learning with Sparsity
To construct our training data, for each argument and counter-argument pair xf) in the Arguana's training set, we set z to be the positive example of zt. We select all the other arguments and counter- counter-arguments from the same dehate and stance as x; 's hard negatives. We fine-tune three pretrained sentence embedding models of different sizes ("UAA-Large-Vi V1" (Li & Li, 2023), "GTE-large-en-vv. (Li et al., 2023), and Tbge-base-en-vl.5 (Xiao et al., 2023)).
also contributed to performance gains. For instance, fine- tuned GTE models showed an increase from 72.5 to 77.8 on the Arguana dataset using standard cosine similarity alone. This suggests that the Arguana dataset inherently favors scenarios where the counterargument is the most similar passage to the query, which may amplify the benefits of fine-tuning.
Interestingly, for the "Prompt + CL (Cosine)" method, fine- tuning with the appended prompt even results in perfor- mance drop. During the training process, we observed over- fitting and hypothesize that the special prompt "Not true:" introduces a shortcut, making it easier for the model to learn whether a text belongs to the "argument" class or the "counter-argument class. However, this class information is not useful when identifying pairwise contradiction rela- tionships.
Please refer to Table 10 for our training parameters.
Baselines We are not aware of any accurate methods for retrieving contradictions that only rely on sentence embed- dings. To the best of our knowledge, we provide two base- lines in our main experiment:
CL: standard contrastive learning with cosine similar- ity, using the same training data (contradictions as pos- itive examples and paraphrases as negative examples) that we use for our SPARSECL.
These findings highlight the robustness of our approach, par- ticularly when traditional similarity metrics are augmented with sparsity measures to capture subtle nuances in contra- diction. Further insights can be gleaned from our ablation study detailed in Section 4.6, where we analyze the impact of similar non-contradictory passages within the corpus.
Prompt + CL: standard contrastive learning with prompt "Not true: " attached front of the query during both training and testing.
4.2. Contradiction retrieval
We report the performance of several efficient (with fewer than IB parameters) and top-ranked pretrained sentence embedding models including "GTE--larg-ee--1155,, "UAE- Large-VI", bage-base-en-vl.5".
The task of "contradiction retrieval" generalizes beyond the argument and counter-argument relationship in the de- bate area, e.g. passages with conflicting factual information should also be considered as "contradictions" To test our method's validity for these more general forms of contra- dictions, we construct two datasets to test our method's performance.
Test The Arguana test set consists of 1401 query argu- ments and counter- argument pairs. Following the standard test setting, we search for an answer of a query within the whole corpus (training set + validation set + test set) and report NDCG@ scores. We select or based on the best NDCG@ 10 score on the validation set. When we directly use model to provide cosine similarity scores in zeroshot manner, we use its default pooler ("cls") for that model. When we use a fine-tuned model (via either CL or SPAR- SECL) to provide either cosine similarity scores or sparsity scores, we use the "avg" pooler.
Data set construction Given a QA retrieval dataset, e.g. MSMARCO (Nguyen et al., 2016), for each answer pas- sage I, of a query 9;- we use Large Language Models (specifically, GPT-4 (Achiam et al., 2023)) to generate 3 answers paraphrasing x; contradicting Z4- Let the generated paraphrases be { T22; 2t3) and the gener- ated contradictions be (22
T22: L23, J. We then delete
z4 from the corpus and add the set of generated passages {#j 2 23g. T T2z 253) to the corpus. In the test phrase, the queries are { t ) each of which has the same answers [ T22 F13. J. We generate the paraphrases and contradictions for the validation set, test set, and a randomly sampled 10000 documents from the training set. Please refer to Appendix G for details.
Results The detailed results are presented in Table I. Across all "areellss"GGE--lgeeeeee///// "UAE-Large- V1", and "bge-base-en-vi.55 ---- an average improvement of 3.6% in counter-argument retrieval were observed when incorporating our SPARSECL to either Zeroshot or CL. Fur- thermore, our CL (Cosine) + SPARSECL (Hoyer) method achieves NDCG@ 10 score 81.3 using GTE with only 400M parameters. For completeness, we also compare our results with (Shi et al., 2023) in Appendix C.
Training To prepare the training data for contrastive learning, for each paraphrase and contradiction set {at - T )55) generated from the same origi- nal passage, we form 9 pieces of training data (a 25. x for 9 different combinations of paraphrases, contradictions,
This pattern of enhancement was consistently observed re- gardless of whether the embedding models were tne-tuned or not. Notably, standard cosine similarity fine-tuning alone
5

---

Contradiction Retricval via Contrastive Learning with Sparsity
Model
Method
Arguana
MSMARCO
HotpotQA
Zeroshot (Cosine)
65.8
60.0
59.5
CL (Cosine)
68.7
52.7
56.2
BGE
Prompt CL (Cosine)
64.4
61.1
80.4
Zeroshot (Cosine) SPARSECL(Hoyer)
70.4
90.9
96.7
CL (Cosine) + SPA SSPRSECL(Hoyer
72.2
88.3
96.5
Zeroshot (Cosine)
68.3
59.7
58.7
CL (Cosine)
70.4
44.2
54.1
UAB
Prompt + CL (Cosine)
63.7
85.8
94.3
Zeroshot (Cosine) SPARSECIL(HOyer)
74.3
90.2
95.5
CL (Cosine) + SPARsECL(Hoyer)
74.4
86.9
94.3
Zeroshot (Cosine)
72.5
60.3
59.7
CL (Cosine)
77.8
65.1
59.7
GTE
Prompt + CL (Cosine)
71.1
88.1
69.0
Zeroshot (Cosine) SPARSECL
79.7
95.3
97.7
CL (Cosine) + SPA CEARRSEL..HE (Hoye)
81.3
95.2
97.9
Table Results for different models and methods the contradiction retrieval task. Experiments are on the Arguana dataset (Wachsmuth et al., 2018) and modified MSMARCO(Nguyen et al., 2016) and HotpotQAAYYag et al., 2018) datasets. We report "GTE" stands
NDCG@ 10 score here, the higher the better. "UAE" stands for "UAB-Large-VV for "gte-large-en cn-vi. 1,"",,""""""""""" The "Method" column denotes the score function used to retrieve contradictions. We consider two score functions: cosine similarity and cosine similarity plus Hoyer sparsity. "Zeroshot" denotes direct testing the model without any fine-tuning. "CL" denotes fine-tuning using standard contrastive learning. "Prompt+CL" denotes fine-tuning using standard contrastive learning with prompt "Not true:" attached front the query. "SpARseCI." denotes fine-tuning using Hoyer sparsity contrastive learning (our method).
"BGE" stands for Thge-base-en-vlee
and a randomly selected hard negative from the remaining two paraphrases. We then perform SPARSECL to fine-tune embedding.
rely too heavily on opposite word replacement, which is better captured by the prompt 'not true.' However, the counter- aariuernts the Arguana dataset use entirely dif- ferent wording.
Test Similar to the testing strategy for Arguana, we de- fine our corpus to consist of all generated text (train- query the para-
4.3. Zero-shot Generalization Test
ing set + validation set + test set). phrases {2i 222 a of the original passage 2; and set the groundtruth answers be the generated contradictions { [2: F3 We select the a parameter with the maxi- mal NDCG@ @ 10 score on the validation set and report the NDCG@ @ 10 score obtained by applying that a to the test set.
To evaluate the generalization capability of our sparse- aware embeddings, we also conduct zero-shot tests other datasets. Specifically,
train the embeddings thetic HotpotQA or MSMARCO datasets and then test them on the other dataset in a zero-shot manner. We have con- firmed that there is no data overlap between the two datasets. Please refer to Table 14 for the corresponding statistics. As presented Table 2, SparseCL. trained on MSMARCO or HotpotQA produces reasonable results the other dataset, albeit with slight performance drop. Furthermore, using the same parameter selected on the other dataset also gives reasonable test accuracy, showing the stability of a parameter across different datasets. This demonstrates that the sparse-aware embeddings trained on one dataset can capture contradiction relationships and generalize to unseen datasets.
syn-
Results The results reported in Table 1. For both MSMARCO and HotpotQA data sets, incorporating our SPARSECL method achieves over 14.6% percentage points gain compared with the two baselines. The large improve- ment is due to the existence of paraphrases in the corpus, that are strong confounders for the pure similarity- baased methods.
We also observe that Prompt CL (Cosine) performs much better on the MSMARCO and HotpotQA datasets compared to standard CL (Cosine). We propose two potential reasons: We have 12 times more paraphrase and contradiction pairs generated for fine-tuning fine-tuning, which makes the model less likely to overfit. 2. The contradictions generated by GPT-
4.4. Retrieval Corpus Cleaning
As an application of contradiction retrieval, we test how well our method can be used to find inconsistencies within corpus and clean the corpus for future training QA re-
6

---

Contradiction Retricval via Contrastive Learning with Sparsity
Method
Train Dataset
Test Dataset
NDCG@ 10
Data Cleaning Our sparsity-based method can only iden- tify contradictions within the data set, but we do not know which element in a contradiction pair is correct. To perform data cleaning, we make the assumption that for each origi- nal passage x E S, we are given one of its paraphrases as the groundtruth. Then, our task is reduced to searching for passages contradicting a given ground truth document and filtering them out.
7oroshot(Cocinee
MSMARCO
HatpotQA MSMARCO
88.1
+SyuneCL(Hoyer) +lixed
HorporQA
82.2
Cerishot(Cosiiee
MSMARCO
HatporQA
88.6
+SpareeCLHHoyert + tuned
HotpoiQA
MSMARCO
87.7
7erishoi(Cosine)
HotporQA
HotpotQA
96.7
H(oyee)C(HHoyrr)
MSMARCO
MSMARCO
90.9
N/A
HotpotQA MSMARCO
59.5
Cosishot(Cosine)
60.0
Table Results for zero-shot generalization experiment for con- tradiction retrieval running on "bge-hase-en-v1.5 model
Method We use the GTE-large-en-v1.5 model without fine-tuning to provide the cosine similarity score for this data cleaning experiment. We use the model from our con- tradiction retrieval experiment in section 4.2 trained on MS- MARCO and HotpotQA to provide the sparsity score. The a parameter also identical to the one used in section 4.2. For each ground truth document, we filter out the top scored documents from the corpus.
trieval. We first inject corrupted data contradicting existing documents into the corpus, and measure the retrieval accu- racy degradation for retrieved answers. Then, our contradiction retrieval method filter out corrupted data and measure the retrieval accuracy again.
Original Acc
Comrupted Corrupt
Cleaned
Data Similarly to the data generation in Section 4.2, we construct new corpus containing LLM-generated para- phrases and contradictions based MSMARCO and Hot- potQA data sets. We start with an original corpus C and its subset S. We then generate paraphrases and contradictions for S' as in Section 4.2.
Datasets
Acc
Acc
Corrupt
HotpotQA
67.6
56.7
44.3
65.2
2.0
MSMARCO 43.5
38.1
41.3
41.4
4.0
Table 3. Experimental results for the impact of corrupted data on QA retrieval and contradiction retrieval for filtration. "Acc" rep- resents the retrieval accuracy measured by the NDCG@ 10 score and "Corrupt" represents the fraction of returned passages that are corrupted, as measured by Recall @ 10.
For HotpotQA, contains all answer documents for the test set, 10000 answer documents sampled from the training set, and 1000 answer documents sampled from the development set. For MSMARCO, S contains all answer documents for the dev set, and 11000 answer documents sampled from the training set.
Table 3 shows the results. We observe that the retrieval accuracy the corrupted corpus drops significantly, the generated contradictions cause the embedding model to re- trieve them as query answers. The corupttion ratio measures the average fraction of the top-10 retrieved documents that correspond to the generated contradicting passages. This performance is above 40% for both datasets. After perform- ing our corpus cleaning procedure, which searches for the passages contradicting the given ground truth documents and removes the top-3 for each of them, we can recover more than 60% of the performance loss due to corruption and at the same time reduce the corruption ratio to less than 5%.
We then curate different versions of the corpus based on the original corpus C and the subset S.
The initial corpus C++: For each original answer docu- ment & in S, we remove x from C and instead add 3 LLM-generated paraphrases fæt xxt 2z to C. The result forms the initial corpus C++
The corrupted corpus C=: For each original an- swer document & in S, we generate 3 contradictions {aī , I2 23 } and add them to C+ to get the corrupted corpus C-.
4.5. Score Functions for Natural Language Inference
As an application of our SPARSECI. method, we demon- strate that our method can be useful for distinguishing con- tradictions from entailments and random pairs in natural language inference datasets. For SNLI (Bowman al., 2015) and MNLI (Williams al., 2018) datasets, extract entailment and contradiction pairs, fine-tune using standard contrastive learning and our SPARSECI., and then report the average cosine similarity Hoyer sparsity score between entailments, contradictions, and random pairs.
The cleaned corpus Ct We apply our data cleaning procedure to the corrupted corpus C=, obtaining the cleaned dataset C
Test We test the retrieval accuracy (NDCG@ @ 10) and the corruption ratio (Recall @ 10) for answering the original queries the test set. The goal of our experiment to show how retrieval algorithms behave on these three constructed corpora C++, C=, and C
We can observe from Table 5 that, in the zeroshot setting, the
7

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Model
Method
12/4
K4
Hoyer
Cosine (baseline)
BGE
Zeroshot (Cosine)+ + SPARSECL
67.5
68.4
70.4
65.7
BGE
CL (Cosine) + SPARSECL
70.2
70.7
72.2
68.7
Table @ scores for Arguana using SPARSECL with different sparsity functions. We also report two baselines that use only the cosine similarity (zeroshot and contrastive learning).
Contradiction
Entailment
Random
retrieve from are C1 C2. C3-
Zeroshot (Cosine)
54.6
37.6
SNLI
CL (Cuxinee
88.5
88.6
77.7
Methods
C2
C2
C3
SparseCT. (Hoyer)
Zeroshot (Cosine)
56.1
35.5
26.7
Zeroshol (Cosine)
65.9
81.8
37.8
Zeroshot (Cosine) (LoysECL((oyerr
68.2
67.9
67.5
MNLI
(Cosine)
73.3
SparseCL (Huyer)
42.2
36.4
24.4
CL (Cosine)
47.1
30.3
22.8
CL (Casine) SP.RSeCI.(Hoyer)
61.9
61.8
61.5
Table 5. Average Cosine / Hoyer scores between Contradiction / Entailment / Random pairs of texts. The experiment is run on
Table 6 Counter-argument retricval results on the augmented Ar- guana dataset with different numbers of similar arguments in the corpus. C2 denotes testing counter-argument retricval on the cor- pus with existing paraphrases (including itself) of the query argument. Experiments were run on "bge-base-en-vl. 1.5" model.
grge-baseeen-l1. model. Texts pairs are from SNLI and MNLI datasets
average cosine similarity of contradiction pairs lies between the ranges of random and entailment pairs. For the fine- tuned model using standard contrastive learning (CL), the average cosine similarity of contradiction pairs is almost indistinguishable from that of entailment pairs. Finally, after being fine-tuned using SPARSECL, the model exhibits higher average Hoyer sparsity scores for contradiction pairs compared to other two types of relationships.
We present our overall experimental results in Table 6. Please also refer to Appendix D for an example case study. As the number of paraphrases in corpus increases from I to 3, the performance of the similarity- significantly. Thus it is reasonable to deduce that, as the number of similar arguments in the corpus increases further, the NDCG @ 10 scores for similarity- methods will converge to O. On the other hand, the performance of our sparsity-based method is stable with respect to the number of paraphrases in the corpus.
method drops
4.6. Ablation Studies
We perform the following three ablation studies to further understand sparsity- based retrieval method.
Different sparsity functions Our intuition in Section 3 does not give clear guidelines on which sparsity function to use in our SPARSECL. Thus, we also experiment with different choices of sparsity functions, selected from Hurley & Rickard (2009). Specifically, we consider two other spar- sity functions (12/11 and K4), which are scale invariant and differentiable (see Table III in (Hurley & Rickard, 2009)). Note that both of these two sparsity functions have ranges [0, 1]. and higher values of those functions correspond to sparser vectors.
Arguana retrieval results analysis In the standard Ar- guana dataset, even though the task is to retrieve the counter- argument for the query, the retrieval based solely on simi- larity still gives reasonable results. This means that counter- arguments are also the most similar arguments to the query, which makes the data set an imperfect test bed for testing contradiction retrieval.
To further compare our sparsity- -bsed method and the pure similarity- method, we augment Arguana by adding ar- guments' paraphrases to the corpus. Specifically, for any ar- gument x and its counter-argument x in the original corpus C, we use GPT-4 to generate three paraphrases (#1, 222 ) of œ. We then form three new corpora with an increasing number of paraphrases added to the corpus: C1 contains all 11 and x= C2 contains all æ1, 42, and and C3 contains all T1, T22 F3, and x
12
h2 l2 11
h1
h1 11
h2
R4
1
h2
1/2
As per Table compared to the cosine similarity method, the combination of the cosine similarity score with the spar- sity score trained by SPARSECI, yields higher NDCG @ 10 scores for each sparsity function. However, Hoyer sparsity yields the highest accuracy. We believe that simple sparsity functions have a more benign optimization landscape and thus are easier for models to optimize.
In the testing phase, we query the counter- aornter-arguments for one of I's paraphrases, the answer of which should still be x We observe how the performance varies when the corpora
8

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Different retrieval methods for contradiction retrieval We experiment with 5 retrieval methods in our ablation study. The methods evaluated are as follows: "Prompt" involves appending the "Not true: "prompt to the query dur- ing testing, followed by standard similarity search. "Prompt + CL (Cosine)" extends this by incorporating contrastive learning with the "Not true: prompt included in the train- ing data. "Gen" uses GPT-4 generate contradictions to the query (details in Appendix G) and applies similarity search for testing. "Gen CL (Cosine)" fine-tunes using contrastive learning with the generated contradictions in the training data before similarity search. Finally, "SparseCL (Hoyer)" employs SparseCL fine-tuning and retrieves doc- uments based on the maximal Hoyer sparsity score during testing.
5. Conclusion
In this work, we introduced a novel approach to contradic- tion retrieval that leverages the sparsity between sentence embeddings, combined with cosine similarity, efficiently identify contradictions in large document corpora. This method addresses the limitations of the traditional similar- ity search as well as computational inefficiencies of the cross-encoder models, proving its effectiveness on bench- mark datasets like Arguana and on synthetic contradictions retrieval from MSMARCO and HotpotQA.
As shown in Table 7, observe that generally "Gen" and "Prompt" don't t improve much upon standard similarity search. For the "Gen + CL (Cosine)" method, a diverse set of counter-argumentt exist for a given argument, making hard to generate a single counter arguumett that closely matches the true ground truth counter-argument. For the "Prompt + CL (Cosine)" method, fine-tuning with the ap- pended prompt even results in performance drop. During the training process, observed overfitting and hypothe- size that the special prompt "Not true:" introduces short- cut, making it easier for the model to learn whether a text belongs to the "argument" class or the aunnterrargument" class. However, this class information is not useful when identifying pairwise contradiction relationships. Finally, di- rectly using Hoyer sparsity retrieve contradictions doesn't 't yield good results well, because we believe contradictions involve combination of similarity and dissimilarity.
Model
Method
Arguana
Prompt + Zeroshot (Cosine)
65.7
BGE
Gen + Zeroshot (Cosine)
64.7
Zeroshot (Cosine)
65.8
Prompt CL (Cosine)
64.5
BGE
Gen CL (Cosine)
70.0
CL (Cosine)
68.7
SparseCL (Hoyer)
56.
BGE
CL (Cosine) + SparseCL (Hoyer)
72.2
Table 7. Counter-argument retrieval results (NDCG@ 10 scores) on Arguana dataset with different retrieval methods. "Gen" means using GPT-4 to generate contradiction of the query argument 4, "Prompt" means appending the "Not truc prompt in the front of the query text. "Zeroshot" refers to direct testing and "CL" and "SparseCL" refer to tine-tuning with respective methods.

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Acknowledgements
Jha, R., Lovering, C., and Pavlick, E. Does data augmen- tation improve generalization in nip? arXiv preprint arXiv:2004.1500 2020.
Haike Xu was supported by the Mathworks Fellowship. Piotr Indyk was supported in part by the NSF TRIPODS program (award DMS-2022448). Zongyu Lin was partially supported by DARPA ANSR program FA8750-23-2-0004, NSF 2331966 NSF 2211557, NSF 21 2119643, NSF 2303037, NSF 2312501, SRC JUMP 2.0 Center, Amazon Research Awards, and Snapchat Gifts.
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04900 2020.
Laban, P., Schnabel, T., Bennett, P. N., and Hearst, M. A. SummaC: Re-visiting NLI-based models for inconsis- tency detection in summarization. Transactions of the Association for Computational Linguistics, 10:163-177, 2022. doi: 10.1162/tac__ Lactieel__004445 00453. URL https:
Impact Statement
This paper introduces novel approach the contradiction retrieval task that relies solely on text embeddings. To the best of our knowledge, we are the first to consider such non-similarity-base retrieval problem. We consider this an important yet underexplored area and hope our work will spark greater interest in this field.
aclanthology. org/ wttp:////......1.....1.e macc-11100
Li, X. and Li, J. Angle-optimized text embeddings. arXiv preprint arXiv:2309. 1207,,
Li, Z., Zhang, X., Zhang, Y., Long, D., Xie, P., and Zhang,
M. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.0322 2023.
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774. 2023.
Mialon, G., Dessi, R., Lomeli, M., Nalmpantis, C., Pa- sunuru, R., Raileanu, R., Rozière, B., Schick, T., Dwivedi- Yu, J., Celikyilmaz, A., Grave, E., LeCun, Y., and
Asai, A., Schick, T., Lewis, P., Chen, X., Izacard, G., Riedel, S., Hajishirzi, H., and Yih, W.-t. Task-aware retrieval with instructions. In Rogers, A., Boyd-Graben J., and Okazaki, N. (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 3650-3675, Toronto, Canada, July 2023. Association for Computa- tional Linguistics. doi: 10.18653/ 225. URL https: ac http//aaanatooolooo/////00000 aereinngs-cc.2200
Scialom, T. Augmented language models: a survey, 2023.
Muennighoff, N.. Tazi, N.. Magne, L., and Reimers. N. MTEB: Massive text embedding benchmark. In Vla- chos, A. and Augenstein, I. (eds.), Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, pp. 2014-2037, Dubrovnik, Croatia, May 2023. Association for Compu- tational Linguisties. doi: 10.18663/w1/0023.eaam 148. URL https:: http://aalaatttt ogy. 2og.aantholooyy.or/222222222 eacl main. 148.
Bowman, S., Angeli, G., Potts, C., and Manning, C. D. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 632 642, 2015.
Nguyen, T., Rosenberg, M., Song, X.. Gao, J., Tiwary, S., Majumder, R., and Deng, L. MS MARCO: A human generated machine reading comprehension dataset. CoRR, abs/1611 abs/1611.09266,, 109268, 2016. URL. http:://axxi..rrrr
09268.
Cui, L. and Lee, D. Coaid: Covid-19 19 healthcare misinfor- mation dataset, 2020.
Orbach, M., Bilu, Y.. Toledo. A., Lahav, D., Jacovi, M., Aharonov, R., and Slonim, N. Out of the echo chamber: Detecting countering debate speeches. arXiv preprint arXiv:2005.00 01 157, 2020.
Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.-E., Lomeli, M., Hosseini, L., and Jégou, H. The faiss library. 2024.
Ren, R., Wang. Y., Qu, Y., Zhao, W. X., Liu, J., Tian, H., Wu, H., Wen, J.-R., and Wang, H. Investigating the factual knowledge boundary of large language models with retrieval augmentation, 2023.
Gao, T., Yao, X., and Chen, D. Simcse: trastive learning of sentence embeddings. arXiv preprint arXiv:2104.0882. 2021.
Simple con-
Hurley, N. and Rickard, S. Comparing measures of sparsity. IEEE Transactions on Information Theory, 55(10)44723- 4741, 2009.
Schuster, T., Fisch, A., and Barzilay, R. Get your vi- tamin C! robust fact verification with contrastive evi- dence. In Toutanova, K., Rumshisky, A., Zettlemoyer,
10

---

Contradiction Retricval via Contrastive Learning with Sparsity
L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y. (eds.), Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Techmologies, pp. 624-643, Online, June 2021. Association for Computational Linguistics. URL https:
Wang, Z., Lin, Z., Liu, P., ZHeng, G., Wen, J., Chen, X., Chen, Y., and Yang. Z. Learning to detect noisy labels using model-based features. arXiv preprint arXiv:2212.13767, 2022.
Williams, A., Nangia, N., and Bowman, S. A broad- coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long Papers), pp. 1112-1122. 2018.
doi: 10. 12.186653/122221.maaal //aclantho wwg. nwaclaaaa-aeeeeaa aasaacl-mainn main. 52.
Shahi, G. K. and Nandini, D. FakeCovid- A Multilingual Cross-domain Fact Check News Dataset for COVID- 19. ICWSM, Jun 2020. doi: 10.36190/2000. 14. URL.
Xiao, S., Liu, Z., Zhang, P., and Muennighoff, N. C-pack: Packaged resources to advance general chinese embed- ding, 2023.
https: petps://dols /ttp:///doiiiee /ttp:///.....or/......00000.0000
Shi, H., Cao, S., and Nguyen, C.-T. Revisiting the role of similarity and dissimilarity inbest counter argument retrieval. arXiv preprint XXXi:2334.088077 2023.
Xie, Q., Luong, M.-T., Hovy, E.,, and Le, Q. V. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10687-10698, 2020.
Singhal, A. et al. Modern information retrieval: A brief overview. IEEE Data Eng. Bull., 24(4):35-43, 2001.
Xiong, L., Xiong, C., Li, Y., Tang, K.-F., Liu, J., Bennett, P., Ahmed, J.,, and Overwijk, A. Approximate nearest neigh- bor negative contrastive learning for dense text retrieval. arXiv preprint arXiv:2007.0000 2020.
Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., and Gurevych, I. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track (Round 2), 2021. URL 1d=wCu6T5xF jeJ.
Xu, H., Lin, Z., Zhou, J., Zheng, Y., and Yang, Z. A uni- versal discriminator for zero-shot generalization. arXiv preprint arXiv:2211.08099, 2022.
n forum?
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. Salakhutdinov, R., and Manning, C. D. HotporQA: A dataset for diverse, explainable multi-hop question an- swering. In Riloff, E., Chiang, D., Hockenmaier, J., and Tsujii, J. (eds.), Proceedings of the 2018 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pp. 2369-2380, Brussels, Belgium, October- November 2018. Association for Computational Lin- guistics. doi: 10. 10.18653/11/118--2255 URL https: / //aclanthologe wwt/ 1259.
Thorne, J., Vlachos, A., Christodoulopoulls, C.,, and Mit-
tal, A. FEVER: a large-scale dataset for fact extrac- tion and VERification. In Walker, M., Ji, H., and Stent, A. (eds.). Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume 1 (Long Pupers), pp. 809-819, New Or- leans, Louisiana, June 2018. Association for Computa- tional Linguistics. doi: 10. 10.18653/v1/18--100 1074. URL https:
org/N1 -
Zhang, X., Thakur, N., Ogundepo, O., Kamalloo, E., Alfonso--ermelo, D., Li, X., Liu, Q., Rezagholizadeh, M., and Lin, J. MIRACL: A Multilingual Retrieval Dataset Covering 18 Diverse Languages. Transactions of the Association for Computational Linguistics, 11:1114 114
Touvron, H., Martin, I.., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09288 2023.
1131, 09 2023a. ISSN 2307-387X. doi: 10. 1662taa/ ____ 00595. URL ht http://.tioo..///..../// 1162//...11111100 taccl 00555.
Wachsmuth, H., Syed, S., and Stein, B. Retrieval of the best counterargument without prior topic knowledge. In Pro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 241-251, 2018
Zhang. Y., Li, Y., Cui, I.,, Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang. Y., Chen, Y., Wang. L., Luu, A. T. Bi, W., Shi, F., and Shi, S. Siren's 's song in the ai ocean: A survey on hallucination in large language models, 2023b.
Wang, X., Wang, J., Cao, W., Wang, K., Paturi, R., and Bergen, L. Birco: A benchmark of information retrieval tasks with complex objectives, 2024.
Zhong, Z., Huang, Z., Wettig, A., and Chen, D. Poisoning retrieval corpora by injecting adversarial passages. arXiy preprint arXiv:2310. 19/66, 2023.

---

Contradiction Retricval via Contrastive Learning with Sparsity
Zhou, J., Lin, Z.,, Zheng, Y., J., and Yang, Z. Not all tasks born equal: Understanding zero-shot generalization. In The Eleventh International Conference Learning Representations. 2022.
Zou, W., Geng, R., Wang, B., and Jia, J. Poisonedrag: Knowledge poisoning attacks to retrieval-augmented gen- eration of large language models, 2024.
12

---

Contradiction Retricval via Contrastive Learning with Sparsity
A. Additional related work
Complex retrieval tasks Information retrieval a well-studied area (Singhal et al., 2001) and there have been many benchmarks for testing retrieval performance such as BEIR (Thakur et al., 2021), MTEB (Muennighoff et al., 2023), and MIRACL (Zhang et al., 2023a). However, most of the datasets, through varying some degrees, focus only on "retrieving the most similar document".. People have noted that there exist some more complex retrieval tasks (e.g. Arguana (Wachsmuth et al., 2018) retrieves counter-arguments that refute query argument), and build retrieval benchmark focusing on complex retrival goals, e.g. BIRCO (Wang et al., 2024) and BERRI (Asai et al., 2023).
To retrieve according to different instructions, (Asai et al., 2023) trains TART, a multi-task retrieval system with task instructions attached as prompts in front of the query content. However, when answering queries, they are still searching for the most similar sentence embedding. though the prompt is different for different tasks. As far as we know, our paper studies the first aenn--millrrtttaaase -based search problem.
Data inconsistency and misinformation detection Data inconsistency, refers to the factually incorrectness in the content, might come from different sources, including their natural existence in the corpus (Shahi & Nandini, 2020; Cui & Lee, 2020), data augmentations (Jha et al., 2020; Zhou et al., 2022), and pseudo labeling (Xie et al., 2020; Wang et al., 2022), which might lead to negative influence if serving as training dataset. There have been a few datasets on detecting the factually wrong information. For example, (Laban et al., 2022) detects whether a given summary is consistent with the input document, (Shahi & Nandini, 2020; Cui & Lee, 2020) detects whether given COVID-19 related news is true or false. Most of these datasets lie specific domain and require external knowledge to judge the correctness of each piece of data. the contrary, the "data inconsistency" notion we consider in our paper doesn't 't depend on any external knowledge, but is relationship between different pieces of data in the same corpus. The goal of our method is to find such "contradiction pairs" in corpus efficiently, but not to judge which one is consistent with the real world knowledge.
B. The "non-transittvity" of Hoyer sparsity and the "transitivity" of cosine function
Here, we provide a simple example to demonstrate that using Hoyer sparsity to measure "contradiction" can bypass the challenging scenario for similarity metrics where "A contradicts C, contradicts C, but A doesn't contradict B". Specifically, Hoyer sparsity satisfies the following "non-transitivity" property.
Proposition B.1 ("non-transitivity" hoyer sparsity). There exist three vectors A, B, and C of dimensionality d, satisfying
1 < |||| 2, |B l2, |Oll2 < + O (2). such that Hoyer(A,C) > 1 1-00 0 at)) Hoyer (, 1 1 (+t)) and
Hoyer( A, B) 0 +
Proof. We construct the following dimensional vectors where e
2/ can be any parameter.
A (1, 0, 0. 0)
B (1, 0, ....
C = (0, 1, 0, 0)
Then, we calculate their I over la ratios:
|| A BIl1
Vd
A BI2
| Cl12
V
A ClI2
B Cll1
2 + (d 2)e 2)e
3
C||2
/2+ (d 2)
V2
Applying their 4 over l2 ratio bounds to the Hoyer sparsity formula will give us the desired relationship.
13

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Next, we provide another example to demonstrate that the cosine function exhibits the following "transitivity" property, which makes it hard characterize the scenario where "A contradicts C, B contradicts C, but A doesn't t contradict B" Proposition B.2 ("transitivity" property of cosine function). Given three unit vectors A, B, and C, (A, -- and cos B, C) 2 O(c), we have cos (A, B) 2 (c))
Proof. For any two vectors X and Y with unit norm, we have cos cos (X,Y) = =1 1 Because cos(AA, C) 2 1 1 (c)),, we have ||A C||2 s O(Ve). Finally, cos(A A, B) = 1 i
2
(IIA-Cila+ BlI+) O(t)
C. Experiment comparison with method from (Shi et al., 2023)
Shi et al. (2023) proposes "Bipolar-encodes" method to retrieve contradictions from the corpus. They also tested their method the Arguana dataset but used a different metrie, R 1. For completeness, we have translated our results into their Recall @1 metric for a fair comparison. As shown in Table 8, both CL (baseline method) and CL+SparseCL (our method) demonstrate significant improvement over the previous results in (Shi et al., 2023).
Model
Method
Arguana(Recall @
GTE GTE
CL+SparseCL (ours)
62.9
CL. (baseline)
56.3
(Shi et al., 2023)
Bipolar-encoder
49.0
Table 8. Comparison of experimental results the Arguana dataset
D. A case study for counter-argument retrieval from Arguana dataset
In this section we provide example to illustrate how our pearsity-basell retrieval method is better at retrieving counter- arguments, In the setting of the augmented Arguana dataset (see our ablation study in Section 4.6), we selected an example query with an II) "aeghh-pro033", for which we list the top 10 retrieved passages using the standard cosine similarity score and our sparsity-based score (a = 1.78 selected from the dev set). The first five letters of a passage ID represent the argument topic ID; "pro/con" denotes the argument stance; suffix "a/b" indicates the argument and its corresponding counter-arg arguuet;:
httpl///aaaa//araaa/// are three paraphrases generated by GPT4
As shown in Table 9, for the example query "aeghh-pro03a", its correct counter-argument, "aeghh-pro03b" (in red), ranks fourth using the cosine score but first using the cosine hoyer score. Meanwhile, its paraphrases "aeghh-pro03e pro03a-para00112 1/2 (in blue) achieve high cosine scores but low sparsity scores.
Method
CL((Cosine)
(L.Cossie)-SSpaeeeC.HHHyy)
Rank
Cosine
Passage ID
Overall
Cosine
Hoyer
Passage ID
0.940
aeghh-proolaa-pun
1.683
0.794
0.499
aeghh-pro03b
0.926
aeghh-pro03a-ppra2
1.644
0.719
0.519
aeghh-conn0a-ppraa0
0.916
aeghh-pro0Ba-ppraal
1.617
0.716
0.506
aeghh-con00a-ppraa
0.794
acghh-pro03b
1.606
0.940
0.374
acghh-pro@0a-paraa
0.719
aeghh-con02a-parad
1.602
0.718
0.496
acghh-con02a-paraal
6
0.718
aeghh-con02b
1.528
0.718
0.454
aeghh-con02b
7
0.718
aeghh-con02a-parre
1.494
0.916
0.324
0.716
aeght-coom02a-para2
1.426
0.926
0,280
aeghh-pro003-ppra22
0,696
acghh-con02a
1.396
0.669
0.408
dhwif-pro02b
10
0.692
reghh-pr004appra0
1.344
0.628
0.402
thggl-con03b
Table 9, An example query analysis for counter-argun argument retrieval. The passage ID in red represents the ground ground-truth counter argummet. while the passage IDs blue are paraphrases of the query argument.
14

---

Contradiction Retricval via Contrastive Learning with Sparsity
E. #lyper-parameters for training and inference
Here we present the training details (Table 10) for our experiments on Arguan, HotpoiQA, and MSMARCO.
CL
SPARSECL
Models
Model Size
Backbone
temp
bz
ep
lr
Ir
GTE-
434M
BERT RoPE GLU
1
le-5
2c-5
0.01
64
UAE-Large-V V1
335M
BERT
1
2e-5
2e-5
0.02
64
bge-base-en-vl.5
109M
BERT
1
2e-5
2e-5
0.02
64
Table 10. Training parameters for Arguana. We set max sequence length to be 512 for Arguana dataset and 256 for HotpotQA and MSMARCO datasets.
F. Efficiency test of cross-encoder and vector calculation
To further compare the efficiency of cross-encoders and Hoyer sparsity calculations, we perform the following experiments:
We choose "fge-reranker-base and "bge-reranker-large"" to be cross-encoders. We them calculate the similarity between one query from Arguana's 's test set and 100 documents from Arguana's corpus. We report the average running time of this method for 100 queries.
We choose Snge-base-en-vl.. and "bge-large-en-vll to our bi-encoders. Suppose we have preprocessed all the sentence embeddings. We use it calculate the Hoyer sparsity between query embedding from Arguana's test and 100 document embeddings from Arguana's corpus. We report the average running time of this method for 100 queries.
Please Table 11 for the running time of different methods. We that the calculation of Hoyer sparsity is least 200 times faster than running cross-encoder.
Cross-cncoder
Model size
Time
bge-reranker-base
278M 560M
0.8832s
Dge-reranker-larg
1.6022s
Bi-encoder
Embedding dimension
Time
-ge-
768
0.0029s
bge-large-en-vl.5
1024
0.0036s
Table 11. Average running time for calculating the functions hetween Arguana query and 100 Arguana documents
G. Data generation details for MSMARCO and HotpotQA experiments in Section 4.2
use "gpt-4-turbo" to generate paraphrases and contradictions for our experiment in Section 4.2. The prompts we use are Table 13. We set temperature 1 and n (to generate 3 outputs). Please see Table 12 for some examples of generated paraphrases and contradictions and Table 14 for the number of unique passages generated from each dataset split. We have verified that there is overlap between different splits different dataset. For generated data quality, sampled 500 data points from the dataset and 2 people annotated the data to check the quality. Our agreement score (using Cohen's Kappa) is 0.98, indicating the quality of the generated data. We present the first 20 passages and their generated paraphrases and contradictions from MSMARCO and HotpotQA datasets the end of this section for readers' reference.
15

---

Contradiction Retricval via Contrastive Learning with Sparsity
Datasets
Orginal
Paraphrase
Contradiction
MSMARCO
In addition to the high financial value of higher education, higher education also makes individuals much more intelligent than what they would be with just high school education...
Beyond significant
Besides the low financial significance of higher education, higher education often renders individuals no more intelligent than they would be with juth high school education...
monetary worth, higher education substantially enhances person's intelligence compared to merely completing high school...
HotpotQA
Ice hockey is contact team sport played on ice, usually in rink, in which two teams of skaters use their sticks to shoot a vulcanized rubber puck into their opponent's net score points...
Ice hockey is contact sport where two teams compete on an ice surface, typically in rink, using sticks to hit vulcanized rubber puck into the opposing team's 's net earn points...
Ice hockey is non-contact team sport played on grass, often in an open field, where two teams of players use their feet to kick a soft leather ball into their opponent's goal to score points...
Table 12. Examples of passages from MSMARCO and HotpotQA datasets, with their generated paraphrases, and generated contradictions.
Highlighted key-words represent exact matchings or contradictions
Task
Prompt
Generating paraphrases
Paraphrase the given paragraph keeping its original meaning. Do not add information that is not present in the original paragraph. Your response should be as indistinguishable to the original paragraph as possible in terms of length, language style, and format. Begin your answer directly without any introductory words.
Generating contradictions
Rewrite the given paragraph to contradict the original content. Ensure the revised paragraph changes the factuality of the original. Your response should be as indistinguishable to the original paragraph as possible in terms of length, language style, and format, Begin your answer directly without any introductory words.
Table 13. Prompts used generate paraphrases and contradictions for MSMARCO and HotpotQA documents.
Dataset
Train
Dev
Test
MSMARCO
59500
5950
44101
HotpotQA
59074
5896
81673
Table 14. Number of unique passages generated from MSMARCO and HotpotQA corpus via GPT-4. We have verified that there is overlap between different splits or different datasets.
16

---

Contradiction Retricval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
MSMARCO-
What is BIOS? BIOS is an acronym for Basic Input Out- put System. On virtually ev- ery computer available, the BIOS makes sure all the other chips, hard drives, ports and CPU func- tion together. What BIOS Does The BIOS software has num- ber of different roles, but its most important role is to load the op- erating system. When you turn on your computer and the micro- processor tries to execute its first instruction, it has to get that in- struction from somewhere.
BIOS stands for Basic In- put/Output System. It exists on almost all computers and is responsible for ensuring that all the various components such as chips, hard drives, ports, and the CPU work in unison. The primary function of BIOS soft- ware is to initiate the operating system. Upon powering up your computer, the microprocessor seeks to perform its initial instruction, which it retrieves through the BIOS.
What a BIOS? BIOS stands for Basic Input / Output System. On nearly no modern computer, the BIOS is used to allow all the other chips, hard drives, ports, and CPU to operate disjointedly. What BIOS Does Not Do The BIOS software has limited func- tions, and its least significant role is to load the operating system. When you turn on your computer and the microprocessor attempts to execute its first instruction, does not rely on the BIOS ob- tain that instruction.
6725993
MSMARCO-
Causes of nitrates urine. 1 Ni- trites/nitrates in urine refer to a byproduct formed due to the ac- tion of bacteria occurring in the urinary tract. The kidneys per- form the vital function of clean- ing the blood by filtering out the unnecessary, toxic matter from blood. The nitrite developed by bacteria cannot be filtered by the kidneys.
Reasons for nitrates in urine. Nitrates nitrites in urine are byproducts produced from bac- terial activity within the urinary tract. The kidneys have the crit- ical role of purifying the blood by removing unwanted, harmful substances. However, the nitrites created by such bacteria are not able to be filtered out by the kid- neys.
Causes of nitrates in urine. 1 Ni- trites/nitrates in urine signify byproduct that is not associated with bacterial action in the uri- nary tract. The kidneys, unable to filter these components once formed, do not influence their presence in the urine. Instead, ni- trites converted from nitrates in food enter the urinary system in- dependent of any bacterial inter- vention.
6909689
MSMARCO 594175
Food Additives. Food additives are substances added intention- ally to foodstuff's to perform cer- tain technological functions, for example colour, sweeten help preserve foods. In the European Union all food addi- tives are identified by an E num- ber. Food additives are always included in the ingredient lists of foods which they are used.
Food Additives. Substances known as food additives are de- liberately incorporated into foods to fulfill specific technological roles such coloring, sweeten- ing, preserving the food. In the European Union, these addi- tives are marked with an E num- ber. The presence of food addi- tives in products is consistently disclosed in the ingredient lists of the foods where they are utilized. monosubstituted alkene. (organic chemistry). An alkene charac- terized by the general formula RHC=CH2, where R represents any organic group; a single car- bon atom is directly bonded to one carbon of the double bond be- tween carbons. Facebook.2 Twit-
Food Additives. Food additives are substances naturally present in food items to perform specific biological purposes, such as to enhance flavor, to sour, or de- crease the shelf life of foods. In the European Union, no specific identification like an E number is utilized for food additives. Food additives are rarely disclosed in the ingredient lists of foods in which they appear.
MSMARCO- monosubstituted alkene. (or- 6208116
disubstituted alkene. (organic chemistry). An alkene with the general formula RHC=CHR, where R represents any organic group: each carbon atom in the carbon-to-carbor double bond is directly bonded its own distinct organic group. Facebook. Twitter.
ganic chemistry). An alkene with the general formula RHC=CH 2, where R is any organic group; only one carbon atom is bonded directly to one of the carbons of the carbon-to-carbon double bond. 1 Facebook.2 Twitter.
17

---

Contradiction Retricval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
MSMARCO-
Depending on the specific germ, the incubation period is between 12 hours and 5 days, usually 48 hours. To answer your question, the common cold is contagious between 24 hours before onset symptoms until days after on- set.
The incubation period varies from 12 hours to 5 days, typically around 48 hours, depending on the germ involved. Regarding your question, one can spread the common cold from 24 hours be- fore symptoms start up days following their appearance.
Depending on the specific germ, the incubation period is between 6 days and 2 weeks, usually 7 days. To answer your question, the common cold is contagious from 5 days before the onset of symptoms until 10 days after on- set.
3366870
MSMARCO-
Noun. 1. decrescendo (music) a gradual decrease in loudness. diminuendo. softness-a sound property that is free from loud- ness or stridency; and in softness almost beyond hearing. music-an artistic form of auditory commu- nication incorporating instrumen- tal or vocal tones in structured and continuous manner.
Noun. 1. decrescendo (music) progressive reduction in volume. diminuendo. softness charac- teristic of sound marked by low volume and an absence of harsh- ness; approaching a level that is barely audible. music an art form of auditory communication that uses instrumental vocal sounds in structured and sus- tained way.
crescendo (mu- sic) gradual increase in loud- ness. crescendo. loudness-a sound property characterized by high volume and stridency; and in loudness almost beyond en- durance. silence-a a form of non- auditory communication devoid of instrumental or vocal tones, un- structured and intermittent.
Noun.
6494839
MSMARCO- Due to its important role in curdling milk, rennin enzyme widely used the food in- dustry, notably in the produc- tion of cheese. Rennin for cheese- making was once derived mainly from the dried stomachs of calves and from some non- animal sources. are produced by the stomach cells of young mammals. Rennin secreted in large amounts right after the birth and then its pro- duction gradually drops off. It is then eclipsed in importance by the Pepsin enzyme.
Rennin enzyme. critical for milk curdling, is extensively utilized in the food sector, especially for cheese production. Histori- cally, cheese-making rennin was sourced primarily from the dried stomachs of young calves and var- ious non-animal origins. This en- zyme is secreted by the stomach cells of young mammals, with secretion levels peaking immedi- ately following birth before grad- ually declining. Over time, the significance of rennin decreases as it is overshadowed by the en- zyme Pepsin.
Despite its crucial function in cur- dling milk, the rennin enzyme is minimally used in the food industry, particularly in the pro- duction of cheese. Rennin for cheese- cheese-making used to be primar- ily sourced synthetically and not from natural origins such as the dried stomachs of calves. Ren- nin enzymes are produced by the stomach cells of older mammals. Rennin is secreted in minimal amounts right after birth, and then its production progressively in- creases. It continues to retain its importance over the Pepsin en- zyme.
934134
enzymes
MSMARCO
Winter Moth (Operophtera bru- mata). up in traps, at least, in southeastern NH, coastal Maine, place in southeastern CT and out Long Island. Mas- sachusetts still appears to have the largest and most damaging pop- ulations of this pest. spanworm (Fidt al.
Winter Moth (Operophtera bru- mata) has been caught in traps primarily in southeastern New Hampshire, coastal Maine, loca- tion in southeastern Connecticut, and on Long Island. The most sig- nificant and harmful populations of this pest seem to still be in Mas- sachusetts, according to Fidt et al.
Summer Moth (Operophtera sol- sticia). down nets, at least, in northwestern NH, inland Maine, place in northwestern CT and out on Long Island. Mas- sachusetts seems to have the smallest and least harmful pop- ulations of this pest. inchworm (Fidt al.
4473191
18

---

Contradiction Retricval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
MSMARCO-
Actor Charles Bronson dies 81 Death Wish movie star Charles Bronson, the coal miner from Pennsylvania who drifted into films as villain and became a hard-faced action star, has died. Bronson, 81, died of pneumonia Cedars-Sinas Medical Centre in Los Angeles, with his wife at his bedside, publicist Lori Jonas said. He had been in the hospital for weeks.
Charles Bronson, star of 'Death Wish' and former Pennsylvania coal miner who turned to acting as villain before becoming a renowned action hero, has passed away at the age of 81. Bronson died from pneumomia at Cedars- Sinai Medical Centre in Los An- geles while his wife was by his side, his publicist Lori Jonas con- firmed. He had been hospitalized for several weeks.
Actor Charles Bronson thrives at 81 Death Wish movie star Charles Bronson, the coal miner from Pennsylvania who drifted into films as a hero and became beloved romantic lead, thriv- ing. Bronson, 81, recently recov- ered from mild cold at his home in Los Angeles, without needing hospital care, publicist Lori Jonas said. He has been in excellent health.
7066556
MSMARCO-
(Redirected from Primary Wave Music) Primary Wave Entertain- ment is full-service entertain- company with expertise in Talent Management, Literary Management, Music Publishing, Branding, Digital Marketing and Licensing based in the United States.
Primary Wave Entertainment is comprehensive entertainment firm based in the United States, specializing in Talent Manage- ment, Literary Management, Mu- sic Publishing, Branding, Digital Marketing, and Licensing.
Secondary Echo Entertainment is a specialized entertainment company lacking capabilitiess in Talent Management, Literary Management, Music Publishing, Branding, Digital Marketing, and Licensing, located outside the United States.
2539424
MSMARCO- 5893222
As 2010, median salaries for unit secretaries ranged from $ 23,098-$ 31,604, while me- dian hourly wages for hos- pital unit secretaries ranged from 14.83-14.67, according to PayScale.com.
In 2010, the median annual salaries for unit secretaries var- ied between 23,098 and $ 31 1,6604, and their median hourly pay in hospitals was between $ 10.83 and 14.67, as reported by PayScale.com.
As of 2010, median salaries for unit secretaries varied from $ 35,000-S 42,500, while me- dian hourly wages for hospital unit secretaries were between S 16.50-$ 19.75, according to PayScale.com.
MSMARCO- 3476298
Answers.com@ is making the world better one answer at a time. cleffa evolves level 30 you can find it mt. coronet. 2 people found this useful.
Answers.com@) improves the world by providing one answer at time. Cleffa evolves level 30 and can be found Mt. Coro- net. This information was useful to 2 people.
Answers.come often misleads more than it informs. Cleffa evolves with high friendship. not at level 30, and it is found in mul- tiple locations, not just Mt. Coro- net. people found this mislead- ing.
19

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Passage ID MSMARCO-
Orginal
Paraphrase
Contradiction
In the sidebar, make sure you se- lect Music under the iPod (in- dented below the iPod). That's the iPod's content list and shows the list of songs on the iPod. Do NOT select Music under LI- BRARY. If you want to delete ALL of the songs on the iPod, click any song that song Dost.Do Select All, which is Cmd-A on the keyboard or Se- lect All from the Edit menu (in menu bar). With all of the songs selected, press Delete on the key- board.
In the sidebar, choose Music lo- cated beneath the iPod (indented under the iPod). This displays the list of songs stored on your iPod. Be sure not to select Music listed under LIBRARY. To remove all songs from the iPod, click on any track in that music list. Perform Select All by pressing Cmd-A on your keyboard or by choosing Select All from the Edit menu in the menu bar. Once all songs are highlighted, hit the Delete key on your keyboard.
In the sidebar, ensure that you choose Music under LIBRARY, not under the iPod (indented be- low the iPod). That selection links to your computer's music collection instead of displaying the list of songs stored on the iPod. Always select Music un- der LIBRARY. If you wish to pre- serve all the songs on the iPod, avoid clicking on any song in that song list. Do not use Select All, which would be Cmd-A on the keyboard or Select All from the Edit menu (in menu bar). With none of the songs selected, do not press Delete on the keyboard. En- sure you do not alter the content list that appears under the iPod selection in the sidebar.
6150895
MSMARCO- 1814356
The Associate of Applied Science Degree requires that you have completed the TSI requirements. Course Requirements Graduation with the Associate of Applied Sci- ence Degree in Welding or the completion of the Com-bination or Structural Welding Certificate re-quires successful completion of Comprehensive Exit Exam.
The Associate of Applied Science Degree mandates fulfillment of TSI prerequisites. To graduate with Associate of Applied Sci- ence Degree in Welding or to re- ceive either the Combination or Structural Welding Certificate, it is essential to pass a Comprehen- sive Exit Exam.
The Associate of Applied Sci- ence Degree does not require you have completed the TSI requirements. Course Require- ments Graduation with the As- sociate of Applied Science De- gree in Welding or the completion of the Combination or Structural Welding Certificate does not re- quire the successful completion of Comprehensive Exit Exam.
MSMARCO-
Pluto's ' rotation period, its day, is equal to 6.39 Earth days. Like Uranus, Pluto rotates on its side on its orbital plane, with an ax- ial tilt of 120A", and so its sea- sonal variation is extreme; at its solstices, one-fourth of its surface is in continuous daylight, whereas another fourth is in continuous darkness.lute luto (minor- sminor-planet des- ignation: 134340 Pluto) is a dwarf planet in the Kuiper belt, a ring of bodies beyond Neptune.
Pluto's 's day, its rotation period, spans 6.39 Earth days. Similar to Uranus, Pluto has a rotation along its orbital plane on its side, with an axial tilt of 120 degrees. Con- sequently, it experiences extreme seasonal changes; during its sol- stices, quarter of its surface joys perpetual daylight while an- other quarter remains in constant darkness. Pluto, designated as minor-planet 134340, classified as a dwarf planet located the Kuiper belt, a collection of ob- jects that orbits around Neptune.
Pluto's 's rotation period, its day, is equal to 153.3 Earth hours. Un- like Uranus, Pluto rotates upright in its orbital plane, with an ax- ial tilt of 30" and so its seasonal variation is moderate; its sol- stices, nearly all of its surface ex- periences alternating daylight and darkness. Pluto (minor-plance designation: 134340 Pluto) is not considered a dwarf planet but a major planet, centrally situated in the Kuiper belt, sparse ring of bodies beyond Neptune.
6108 145
20

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
MSMARCO-
Domestic Costa Rica flights. Your premier Costa Rica airline choice for travel and vacation flights within Costa Rica, offer- ing 74 daily flights to 17 destina- tions in Costa Rica! Nature Air the Costa Rica Domestic Airline, offers domestic flights 17 des- tinations in Costa Rica. Nature Air Costa Rica Airline, offers 74 daily domestic flights including to and from Juan Santamaria In- ternational Airport, which allows easy connections to International flights.
Costa Rica domestic flights with Nature Air, your top airline choice for traveling and vacation- ing across Costa Rica. Providing 74 daily flights, this Costa Rica airline connects to 17 local desti- nations. As the premier domestic airline in Costa Rica, Nature Air facilitates 74 daily flights, includ- ing routes to and from Juan San- tamaria International Airport, en- abling seamless connections with international flights.
International Costa Rica flights. Your last choice for Costa Rica airline for travel and vacation flights outside of Costa Rica, of- fering no daily flights to any des- tinations outside Costa Rica! Na- ture Air, the International Costa Rica Airline, provides interna- tional flights to destinations out- side of Costa Rica. Nature Air Costa Rica Airline, offers zero daily international flights exclud- ing Juan Santamaria International Airport, which prevents any con- nections to international flights. There was only one group of Cubists throughout the peak of the movement, from 1909 to 1914. Pablo Picasso (1881-1973) and Georges Braque (1882-1963) are regarded as Independent Cu- bists because they showcased their works without any exclusive agreements, avoiding ties with major galleries like that of Daniel- Henri Kahnweiler's.
5618382
MSMARCO-
There two groups of Cubists during the height of the move- ment, 1909 to 1914. Pablo Pi- casso (1881-1973) and Georges Braque (1882-1963 1963) are known as the Gallery Cubists because they exhibited under contract with Daniel-Henri Kahnweiler's gallery.
During the peak of the Cubist movement, from 1909 to 1914,
585206
there existed two factions of Cu- bists. Pablo Picasso (1881-1973) and Georges Braque (1882-1963)
were identified as the Gallery Cu- bists due their contractual exhi- bitions with the gallery of Daniel- Henri Kahnweiler.
MSMARCO-
Cody is a city in Park County, Wyoming, United States. It is named after William Frederick Cody, primarily known as Buffalo Bill, from his part in the creation of the original town.
Cody, located in Park County, Wyoming, USA, is named for William Frederick Cody, better known as Buffalo Bill, due to his significant role in founding the initial settlement.
Cody is township in Park County, Wyoming, United States. It is named after James Frederick Cody, primarily known Bison James, from his opposition the establishment of the original set- tlement.
4547162
MSMARCO-
Hemoglobin is what actually gives your red blood cells the ability to carry oxygen. A high hematocrit simply means that you have higher concentration of hemoglobin the blood. First of all, it could indicate dehydration, normal amount of hemoglobin, but too little blood (plasma) vol- ume. If this is the case, depending on how high the hematocrit is, re- hydration might be needed.
Hemoglobin enables red blood cells to transport oxygen effec- tively. An elevated hematocrit indicates a greater concentration of hemoglobin within the blood- stream. This could either sug- gest dehydration normal level of hemoglobin coupled with re- duced volume of blood plasma. Should this be the situation, and depending on the severity of the hematocrit levels, rehydration may be necessary.
Hemoglobin actually restricts your red blood cells' capacity to transport oxygen. A low hemat- ocrit indicates a diminished con- centration of hemoglobin in the blood. Primarily, it could sig- nify overhydration, excessive blood (plasma) volume with too little hemoglobin. If this situation arises, depending on how low the hematocrit is, dehydration mea- sures might be required.
6813425
21

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
MSMARCO-
Imperial system of measurement, on the other hand refers to the sys- tem used in the British Empire in the 19th and 20th centuries. How- ever, after adoption of metric sys- tem, Imperial system has been re- duced few countries of the world, notably UK, and surpris- ingly US.
The Imperial measurement sys- tem, alternatiely,, was the stan- dard in the British Empire dur- ing the 19th and 20th centuries. Yet, with the adoption of the met- ric system, the Imperial system's usage has declined and is now primarily limited to handful of countries, notably the UK and, in- terestingly, the US.
Metric system of measurement, however, refers to the system used globally. Still, the adop- tion of the imperial system has been expanding, being notably preferred an increasing num- ber of countries. including the UK and, interestingly, the US.
1760626
HotpotQA- 305193
St. Moritz (also German: "Sankt Moritz", Romansh: "" Italian: "San Maurizio", French: "Saint- Moritz" ) is a high Alpine resort in the Engadine in Switzerland, an elevation of about 1800 above sea level. It is Upper Enga- dine's 's major village and munic- ipality in the district of Maloja in the Swiss canton of Graubünden.
St. Moritz (German: "Sankt
St. Moritz (also German: "Sankt Moritz", Romansh: "", Italian: "San Maurizio", French: "Saint- Moritz") is a lowland resort in the plains of Lower Engadine in Switzerland,
Moritz", Romansh: "" Italian:
"San Maurizio". French: "Saint- Moritz") is a high Alpine resort town situated in the Engadine in Switzerland, located approxi- mately 1800 above sea level. It serves as the primary village of Upper Engadine and is munic- ipality within the district of Mal- oja in the canton of Graubünden, Switzerland.
elevation of about 300 above sea level. It is Lower Engadine's 's minor village and a small community in the dis- trict of Maloja in the Swiss canton of Graubünden.
HotpotQA- 32060208
John Adedayo B. Adegboyega (born 17 March 1992), known professionally John Boyega, is an English actor and producer best known for playing Finn in the 2015 film "", the seventh film of the "Star Wars" series. Boyega rose to prominence in his native United Kingdom for his role as Moses in the 2011 sci-fi comedy film Attaack the Block".
John Adedayo B. Adegboyega, born on 17 March 1992 and professionally known John Boyega, is an English actor and producer. He is most recognized for his portrayal of Finn in the 2015 movie "Star Wars: The Force Awakens", which is the seventh installment in the "Star Wars" saga. Boyega first gained fame in the UK for his perfor- mance as Moses in the 2011 1 sci- ence fiction comedy "A Attaakk the Block".
John Adedayo B. Adegboyega (born 17 March 1992), profes- sionally known John Boyega, is Scottish singer and song- writer best recognized for his de- but in the 2018 album "Horizons", the fourth album in his musical ca- reer. Boyega first gained attention in the international music scene for his performance the Edin- burgh Festival Fringe in the musi- cal "Glimpse of the Stars".
HotpotQA- 42146101
Sarah Davis (born 1976) is an American politician and a Repub- lican member of the Texas House of Representatives; she was first elected in the Tea Party wave of 2010. Davis' district contains contains The Galleria and the Texas Medical Center.
Sarah Davis, born in 1976, is a Republican politician serving in the Texas House of Representa- tives. She entered office follow- ing the Tea Party election wave in 2010. Her district includes key areas such as The Galleria and the Texas Medical Center.
Sarah Davis (born 1976) is an American politician and Demo- cratic member of the Texas House of Representatives; she was first elected in the progressive wave of 2010. Davis' district excludes The Galleria and the Texas Medi-
cal Center.
22

---

"Yes More Happy Songs" is song by American boy band Big Mix from the group's fifth live al- bum, "Tragic Nights" (2017). The song was written by John Doe, Alexander Back, Richard Smith- son, and Lily Johnson; produced by Acoustic and Jane Doe. A cover version, lacking any con- tributions from Canadian singer Contradiction Retricval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
HotpotQA- 1982071
"Lola" is song written by Ray Davies and performed by English rock band the Kinks on their al- bum "Lola Versus Powerman and the Moneygoround, Part One". The song details a romantic en- counter between a young man and a possible transvestite, whom he meets in club in Soho, London. In the song, the narrator describes his confusion towards a person named Lola who "walked like a woman and talked like a man". Although Ray Davies claims that the incident was inspired by a true encounter experienced by the band's manager, alternate expla- nations for the song have been given by drummer Mick Avory.
"Lola" is track penned by Ray Davies and executed by the British rock group the Kinks, fea- tured on their album "Lola Ver- sus Powerman and the Moneygor- ound, Part One." The song nar- rates a romantic interaction be- tween young man and a likely transvestite he meets in a Soho, London club. It portrays the young man's perplexity toward an individual named Lola, who "walked like a woman and talked like man." Ray Davies has at- tributed the inspiration for the song to actual event that the band's manager encountered, al- though drummer Mick Avory has offered different explanations for the song's s origins.
"Lola" is song written by Ray Davies and performed by English rock band the Kinks on their al- bum "Lola Versus Powerman and the Moneygoround, Part One". The song narrates the mundane interactions between middle- aged man and woman he knows, which they experience during typical day. In the song, the nar- rator simply mentions a person named Lola without hinting at any gender ambiguity, stating that she "walked like woman and talked like woman". Although Ray Davies has denied any real- life inspiration behind the song, claiming it to be purely fictional, consistent explanations for the song's 's origin have been supported by drummer Mick Avory.
HotpotQA- 8540095
"No More Sad Songs" is song by British girl group Little Mix from the group's fourth studio al- bum, "Glory Days" (2016). The song was written by Emily War- ren, Edvard Førre Erfjord, Hen- rik Michelsen and Tash Phillips; produced by Electric and Joe Kearns. A remix version, featur- ing newly recorded vocals from American rapper Machine Gun Kelly, was released as the third single from the album on 3 March 2017, through Syco Music.
"No More Sad Songs" is a track by the British girl band Little Mix, featured
their fourth studio album titled "Glory Days" from 2016. The creators of the song include Emily Warren, Edvard Førre Erfjord, Henrik Michelsen, and Tash Phillips, with produc- tion handled by Electric and Joe Kearns. On 3 March 2017, remixed version of the song, in- corporating new vocals by Amer- ican rapper Machine Gun Kelly, was issued as the album's 's third single through Syco Music.
Jon Bellion, was dismissed as the fourth single from the album on December 2018, through Rhythm Records.
23

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
HotpotQA- 228538
I Might Be Wrong: Live Record- ings is a live album by the En- glish rock band Radiohead, re- leased on 12 November 2001 by Parlophone Records in the United Kingdom and a day later by Capitol Records in the United States. Recorded during Radio- head's 2001 tour, it comprises performances of songs from the band's fourth and fifth albums "Kid A" (2000) and "Amnesiac" (2001); plus the song "True Love Waits", which would not be re- leased on studio album until "A Moon Shaped Pool" (2016).
Might Be Wrong: Live Record- ings is live album from the British rock group Radiohead, which was published on 12 November 2001 by Parlophone Records the United Kingdom and one day later in the United States by Capitol Records. Cap- tured during the group's 2001 tour, it features live renditions of tracks from their fourth and fifth albums, "Kid A" (2000) and "Amnesiac" (2001), as well as the track "True Love Waits," which was not released on a studio al- bum until "A Moon Shaped Pool" in 2016.
Might Be Right: Studio Ses- sions is studio album by the
American pop band Radiohead, released on 12 November 2005 by Parlophone Records in the United Kingdom and a day ear- lier by Capitol Records in the United States. Recorded during Radiohead's studio sessions in 2005, it features remixes of songs from the band's sixth and sev- enth albums "Hail to the Thief" (2003) and "In Rainbows" (2007), excluding the song "True Love Waits", which had already been released on studio album prior to "A Moon Shaped Pool" (2016). MMy Brave Face" is a track from Paul McCartney's 's 1989 album, "Flowers in the Dirt", which he wrote without any contributions from Elvis Costello. "My Brave Face" is one of the least recog- nized songs from "Flowers in the Dirt". It did not make a signifi- cant impact on the charts, failing to enter the top 40 in the United Kingdom and the United States. It was not McCartney's s final top 40 hit on the "Billboard" Hot 100, as he continued to achieve suc- cess in the charts, particularly with his subsequent collabora- tions. As of 2017, other former Beatles have also achieved top 40 hits on Billboard in lead roles.
HotpotQA- 14741307
"My Brave Face is single from Paul McCartney's 's 1989 album, "Flowers in the Dirt". Written by McCartney and Elvis Costello, "My Brave Face" is one of the most popular songs from "Flow- ers in the Dirt". It peaked at #18 in the United Kingdom a week after its debut, and #25 in the United States 7 weeks after its de- but. was McCartney's 's last top 40 hit on the "Billboard" Hot 100 until his 2014 collaboration with Kanye West, "Only One", and as of 2017 is the last Billboard top 40 hit with any former Beatle in the lead credit.
"My Brave Face" is a track from Paul McCartney's s 1989 release, "Flowers in the Dirt". The song, co-written by McCartney and Elvis Costello, is among the album's most notable tracks, Shortly after its release, it reached #18 in the UK charts and as- cended to #25 in the US charts seven weeks post-debut. It repre- sents McCartney's 's final top 40 ap- pearance on the "Billboard" Hot 100 until his 2014 partnership with Kanye West on the track "OOly One". As of 2017, it re- mains the most recent top 40 hit on the Billboard chart featuring a former Beatle as the lead artist.
HotpotQA- 54385691
Lonesome Dove: The Series an American western drama tele- vision series that debuted in first- run syndication on September 26, 1994. It serves as continuation of the story of the miniseries of the same name. The television series starred Scott Bairstow and Eric McCormack, and its execu- tive producers were Suzanne de Passe and Robert Halmi Jr. The series was produced by Telegenic Programs Inc. and RHI Entertain- ment in association with Rysher TPE, in conjunction with Cana- dian television network CTV.
Lonesome Dove: The Series, an American Western drama TV se- ries, premiered in first-run syndi- cation on September 26, 1994. It continues the narrative from the similarly titled miniseries. Fea- turing Scott Bairstow and Eric McCormack, the series was ex- ecutive produced by Suzanne de Passe and Robert Halmi Jr. Pro- duction was handled by Telegenic Programs Inc. and RHI Enter- tainment, in collaboration with Rysher TPE and the Canadian TV network CTV.
Lonesome Dove: The Series is a Canadian comedy television se- ries that premiered exclusively on the premium cable network HBO on October 15, 1995. It is pre- sented as a parody of the original miniseries bearing the same title. The television series featured Tim Curry and Hugh Grant as leads, with executive production helmed by John Smith and Jane Doe. The series was produced by Generic Studios Ltd. and ABC Entertain- ment without any collaboration with the American television net- work NBC.
24

---

Contradiction Retricval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
HotpotQA-
The Revolution was villainous stable in Total Nonstop Action Wrestling (TNA), consisting of members James Storm, Abyss, The Great Sanada, Khoya, Manik and Serena Deeb.
nefarious group in Total Nonstop Action Wrestling (TNA) that included James Storm, Abyss, The Great Sanada, Khoya, Manik, and Ser- ena Deeb as members.
The Revolution
heroic stable in Total Nonstop Action Wrestling (TNA), consisting of members James Storm, Abyss, The Great Sanada, Khoya, Manik and Serema Deeb.
The Revolution
44751816
HotpotQA- 394493
Jerrald King "Jerry" Goldsmith (February 10, 1929July 21, 2004) American composer and conductor most known for his work in film and television scor- ing. He composed scores for such noteworthy films as "", "The Sand Pebbles", "Logan's Run", "Planet of the Apes". "Patton", "Papillon". "Chinatown", "The Wind and the Lion", "The Omen", "The Boys from Brazil", "Capricorn One", "Alien", "Outland", "Poltergeist", "The Secret of NIMH", "Grem- lins", "Hoosiers", "Total Recall", "Basic Instinet", "Rudy", "Air Force One", "L.A. Confidential", "Mulan", "The Mummy", three "Rambo" films, "Explorers" and four other "Star Trek" films.
Jerrald King "Jerry" Goldsmith (February 10, 1929 July 21, 2004) was American com- poser and conductor renowned primarily for his contributions to film and television music. He crafted the scores for notable films including "The Sand Peb- bles", "Logan's 's Run", "Planet of the Apes", "Patton", "Papillon", "Chinatown", "The Wind and the Lion", "The Omen", "The Boys from Brazil", "Capricorn One", "Alien", "Outland", "Poltergeist", "The Secret of NIMH", "Grem- lins", "Hoosiers", "Total Recall", "Basic Instinct", "Rudy", "Air Force One", "L.A. Confidential"", "Mulan", "The Mummy". three "Rambo" films, "Explorers", and four "Star Trek" films.
Jerrald King "Jerry" Goldsmith (February 10, 1929 July 21, 2004)
American painter and sculptor, primarily known for his abstract art exhibitions. He produced artworks for such note- worthy galleries as the MoMA, the Louvre, the Tate Modern, Guggenheim, the Institute of Chicago, the National Gallery, the Whitney Museum, the San Francisco Museum of Modern Art, the Getty Center, the Mu- seum of Fine Arts, Boston, the Philadelphia Museum Art, the Cleveland Museum of Art, the Detroit Institute of Arts, the Walker Art Center, the Houston Museum of Fine Arts, the Dal- las Museum of Art, the Denver Art Museum, the Seattle Art Mu- seum, the Miami Art Museum, the Barnes Foundation, the Kim- bell Art Museum, the Nelson- Atkins Museum of Art, and con- tributed to exhibits in the Smithso- nian and several other prestigious international venues.
HotpotQA- 11792076
"Livin' Our Love Song" is a song co-written and recorded by Amer- ican country music artist Jason Michael Carroll. It was released in April 2007 the second sin- gle from his album "Waitin' in the Country". Carroll co-wrote the song with Glen Mitchell and Tim Galloway.
"Livin" Our Love Song" was re- leased by Jason Michael Carroll, an American country music artist, as the second single from his al- bum "Waitin' in the Country" in April 2007. The song, which Car- roll co- wrote with Glen Mitchell and Tim Galloway, showcases his contribution both as a writer and performer.
"Livin" Our Love Song" is a track solely written and performed by British pop artist Emma Louise Clark. It was released in Octo- ber 2010 the final single from her album "Cityscape Dreams".. Clark independently created the song without collaborations.
25

---

Contradiction Retricval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
HotpotQA-
Malik Izaak Taylor (November 20, 1970March 22, 2016), known professionally as Phife Dawg (or simply Phife), was an American rapper and a member of the group Tribe Called Quest with high school friends Q-Tip and Ali Sha- heed Muhammad (and for short time Jarobi White). He was also known the "Five Foot Assas- sin" and "The Five Footer", be- cause he stood at ft.
Malik Izaak Taylor (November 20, 1970 - March 22, 2016), rec- ognized by his stage name Phife Dawg (or just Phife), was an American rapper and belonged to the group A Tribe Called Quest alongside his high school companions Q-Tip, Ali Shaheed Muhammad, and briefly Jarobi White. He was also nicknamed the "Five Foot Assassin" and "The Five Footer" due to his height of 5 feet.
Malik Izaak Taylor (November 20, 1970March 22, 2016), known professionally as Phife Dawg (or simply Phife), was a British singer and a member of the pop group The Harmonic Voices with college acquaintances Liam Tweed and Noah Parker (and briefly Emily Stone). He was also known as the "Tall Lyricist" and "The Towering Tenor", because stood at ft in.
1472458
HotpotQA- 3500070
The Fourth Dimension non- fiction work written by Rudy Rucker, the Silicon Valley profes- sor of mathematies and computer science, and was published in 1984 by Houghton Mifflin. The book subtitled as guided tour the higher universes. The fore- word included by Martin Gard- ner, and the 200+ illustrations by David Povilaitis. Like other books by Rucker, "The Fourth Dimension" " is dedicated to Ed- win Abbott Abbott, author the novella "Flatland".
"The Fourth Dimension," " a non- fiction book by Rudy Rucker, a mathematics and computer sci- ence professor from Silicon Val- ley, was released in 1984 by Houghton Mifflin. Subtitled "a guided tour of the higher uni- verses," it includes a foreword by Martin Gardner and features over 200 illustrations by David Povi- laitis. As with his other works, Rucker dedicates this book to Ed- win Abbott Abbott, who wrote the novella "Flatland."
The Fifth Dimension is fictional novel authored by Rudy Rucker, the Silicon Valley professor of literature and art, and was re- leased in 1990 by Penguin Ran- dom House. The book is alterna- tively titled as a narrative journey through imaginary realms. The afterword provided is by Douglas Hofstadter, and the 150+ paint- ings by Sarah Jensen. Un- like other works by Rucker, "The Fifth Dimension" pays homage to Lewis Carroll, author of the novel "Through the Looking-Glass".
26

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
HotpotQA-
Music of the Sun is the debut stu- dio album by Barbadian singer Rihanna. It was released on August 30, 2005 in the United States through Def Jam Record- ings. Prior to signing with Def Jam, Rihanna was discovered by record producer Evan Rogers in Barbados, who helped Rihanna record demo tapes to send out to several record labels. Jay-Z, the former chief executive offi- cer (CEO) and president of Def Jam, was given Rihanna's demo by Jay Brown, his A&R Def Jam, and invited her audition for the label after hearing what turned out to be her first single, "Pon de Replay". She auditioned for Jay-Z and L.A. Reid, the for- mer CEO and president of record label group The Island Def Jam Music Group, and was signed on the spot to prevent her from sign- ing with another record label.
Music of the Sun, the inaugural studio album by Barbadian artist Rihanna, was launched on Au- gust 30, 2005 the United States by Def Jam Recordings. Be- fore her association with Def Jam, Rihanna was discovered in Bar- bados by record producer Evan Rogers, who assisted her in cre- ating demo tapes that were dis- tributed to various record labels, Jay-Z, who was then the chief ex- ecutive officer (CEO) and pres- ident of Def Jam, received Ri- hanna's 's demo from Jay Brown, his A&R at Def Jam, and after lis- tening to what eventually became her debut single, "Pon de Replay", he asked her to audition for the la- bel. During her audition for Jay-Z and L.A. Reid, the then CEO and president of The Island Def Jam Music Group, she was immedi- ately signed to the label to avoid her potentially signing elsewhere.
Music of the Sun is not the debut studio album of Barbadian singer Rihanna; rather, it is her second album. It was released on Au- gust 30, 2006, outside the United States and was not associated with Def Jam Recordings. Before her agreement with different la- bel, Rihanna was noticed by talent scout in Canada, unrelated to record producer Evan Rogers, and she independently produced her initial demos. Jay-Z, who had already stepped down as the chief executive officer (CEO) and pres- ident of Def Jam, never encoun- tered Rihanna's ' demo, which was not passed by Jay Brown, his for- mer A&R at Def Jam. Further- more, she did not perform an au- dition before the executives of the label, and as result, she was cautiously offered contract days after her negotiations with vari- ous other record labels had con- cluded.
2585886
HotpotQA- 5994297
Barbara Bouchet (born Barbara Gutscher, 15 August 1944) is a Gerran-American actress and en- trepreneur who lives and works in Italy.
Barbara Bouchet, born as Barbara Gutscher on August 15, 1944, is a GerraanAmerican actress and entrepreneur, currently residing and working in Italy.
Barbara Bouchet (born Barbara Gutscher, August 1944) is a German-American scientist and politician who resides and oper- ates in the United States.
HotpotQA- 26188218
The 2009-10 Spanish football season is Xerez's first season ever in Liga BBVA.
The 2009-10 season marks Xerez's 's inaugural appearance in Liga BBVA, which is Spain's 's top football division.
The 2009-10 Spanish football season is not Xerez's 's first season in Liga BBVA.
HotpotQA- 20049848
Kim Coco Iwamoto is a commis- sioner on the Hawaii Civil Rights Commission, appointed by Gov- ernor Neil Abererombic to serve the four-year term from 2012 to 2016.
Kim Coco Iwamoto served as a commissioner on the Hawaii Civil Rights Commission, having been appointed by Governor Neil Abercrombie for four- four-year term spanning from 2012 to 2016.
Kim Coco Iwamoto was not a commissioner on the Hawaii Civil Rights Comnission, and she was not appointed by Gover- nor Neil Abercrombie for term from 2012 to 2016.
27

---

Contradiction Retrieval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
HotpotQA-
Oscar De La Hoya (; born Febru- ary 4, 1973) is a former profes- sional boxer who competed from 1992 to 2008. He holds dual American and Mexican citizen- ship. Nicknamed "The Golden Boy." De La Hoya represented the United States at the 1992 Summer Olympics, winning gold medal in the lightweight division shortly after graduating from James A. Garfield High School.
Oscar De La Hoya (born Febru- ary 4, 1973) is an ex-professional boxer who competed between 1992 and 2008. He has both American and Mexican citizen- ship. Known as "The Golden Boy," " De La Hoya competed for the United States in the 1992 Summer Olympics and secured gold medal in the lightweight category soon after his gradua- tion from James A. Garfield High School.
Oscar De La Hoya (: born Febru- ary 4, 1973) is current profes- sional wrestler who began com- peting in 2010. He holds solely Mexican citizenship. Nicknamed "The Silver Star," " De La Hoya represented Mexico the 2008 Summer Olympies, losing in the initial rounds in the heavyweight division just before enrolling at James A. Garfield High School.
95310
HotpotQA- 526562
Donny Edward Hathaway (Oc- tober 1, 1945 - January 13. 1979) was American jazz. blues, soul and gospel singer, songwriter, arranger and pianist. Hathaway signed with Atlantic Records 1969 and with his first single for the Atco label, "The Ghetto", in early 1970, "Rolling Stone" magazine "marked him a major new force in soul mu- sic." His enduring songs include "The Ghetto", "This Christmas", "Someday We'll All Be Free", "Little Ghetto Boy", "I Love You More Than You'll '1l Ever Know", signature versions of A Song for You" and "For All We Know". and "Where Is the Love" and "The Closer I Get to You", two of many collaborations with Roberta Flack. "Where Is the Love" won the Grammy Award for Best Pop Performance by Duo or Group with Vocals in 1973. At the height of his career Hathaway was diag- nosed with paranoid schizophre- nia and was known to not take his prescribed medication regularly enough to properly control his symptoms. On January 13, 1979, Hathaway's 's body was found out- side the luxury hotel Essex House in New York City; his death was ruled suicide.
Donny Edward Hathaway (Oc- tober 1, 1945 - January 13, 1979) was an acclaimed Ameri- can singer, songwriter, arranger, and pianist, known for his contri- butions to jazz, blues, soul, and gospel music. Signing with At- lantic Records in 1969, Hathaway released his debut single "The Ghetto" under Atco label in early 1970, which led "Rolling Stone" magazine to recognize him as significant new voice in soul mu- sic. Among his most memorable tracks are "The Ghetto", "This Christmas", "Someday We'll All Be Free", "Little Ghetto Boy". "I Love You More Than You 'll Ever Know", definitive renditions of "A Song for You" and "For All We Know", alongside "Where Is the Love" and "The Closer I Get to You", results of frequent col- laborations with Roberta Flack. "Where Is the Love" earned him Grammy Award for Best Pop Per- formance by a Duo or Group with Vocals in 1973. During his peak, Hathaway suffered from paranoid schizophrenia, struggling with in- consistent use of his prescribed medications to manage his condi- tion effectively. On January 13, 1979, Hathaway died by suicide, with his body discovered outside New York City's 's luxury Essex House hotel.
Donny Edward Hathaway (Octo- ber 1, 1945 - January 13, 1979) was American classical, pop, rock and country musician, com- poser, conductor, and keyboardist. Hathaway began his career with Columbia Records in 1975, and with his debut album under the Harmony label. "A Quiet Storm", in late 1976, "Rolling Stone" mag- azine dismissed him ing figure in the music world. His obscure songs include "A Quiet Storm", "Winter Wonder- land", "Always Free", "Big City Boy", Hate You Less Than You'll 'lll Ever Know", obscure ren- ditions of "A Tune for Me" and "What We Don't Know", and "Where Is the Hate" and "The Further Go from You", a few isolated attempts to work with Roberta Flack. "Where Is the Hate" lost the Grammy Award for Worst Pop Performance by Duo or Group with Vocals in 1973. At low point in his ca- reer, Hathaway was diagnosed with acute stress response and was known for his strict adher- ence to prescribed medication, ef- fectively managing his symptoms.
fleet-
On January 13, 1979, Hathaway's 's body was discovered inside the modest Cortlandt Hotel in New York City: his death was declared a natural cause.
28

---

Contradiction Retricval via Contrastive Learning with Sparsity
Passage ID
Orginal
Paraphrase
Contradiction
HotpotQA-
Randy Edelman (born June 10, 1947) is American musician, producer, and composer for film and television known for his work comedy films. He has been nominated Golden Globe Award, a BAFTA Award, and is the recipient of twelve BMI Awards.
Randy Edelman, born June 10, 1947, is American composer, producer, and musician famed for his contributions to film and tele- vision comedies. He has received nominations for a Golden Globe Award and BAFTA Award, and has twelve BMI Awards.
Randy Edelman (born June 10, 1947) is American musician, producer, and composer for film and television known for his work in dramatic films. He has never been nominated for a Golden Globe Award, BAFTA Award, and has not received any BMI Awards.
1834726
29


Reviews:
[{"review_id": "gQ5OJ5b6Gm", "reviewer": "Reviewer_V6ZT", "summary": "The paper is about non-similarity based information retrieval which is currently under explored. In the paper they introduce SparseCL, a novel approach to address shortcomings in existing similarity search and cross-encoder models when retrieving arguments contradictory to the query from large document corpora.  The proposed method utilizes a combined metric of cosine similarity and sparsity function to identify and retrieve documents that contradict a given query. \n\nThe proposed method shows an 11.0% improvement when evaluated on existing retrieval benchmarks.\n\nIn summary;\n- The paper introduces a novel approach using sentence embeddings together with cosine similarity and Hoyer measure of sparsity to capture the essence of contradiction.\n- The embedding and scoring proposed approach shows an improved performance as compared to existing methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "The authors make a compelling argument that supports the proposed approach of training a sentence embedding model to preserve sparsity of differences between the contradicted sentence embeddings. This is represented in Figure 2. Hoyer sparsity histogram.", "methods_and_evaluation_criteria": "The proposed method is well supported by evaluations on well known benchmarks of Arguana (Wachsmuth et al., 2018) a counterargument retrieval task and two contradiction retrieval datasets adapted from HotpotQA (Yang et al., 2018) and MSMARCO (Nguyen et al., 2016).", "theoretical_claims": "NA", "experimental_designs_or_analyses": "The experimental design is reasonable, the authors test the performance of the proposed method on well known benchmarks of Arguana (Wachsmuth et al., 2018) a counterargument retrieval task and two contradiction retrieval datasets adapted from HotpotQA (Yang et al., 2018) and MSMARCO (Nguyen et al., 2016).  \nThey apply the proposed contradiction retrieval to downstream applications ie., retrieval corpus cleaning and natural language inference. In addition authors perform ablation to explain the functionality of each component of the proposed method.", "relation_to_broader_scientific_literature": "Similarity retrieval is a well studied task where contrastive learning is used to map similar sentences together and dissimilar ones far from each other in the embedding space (Gao et al., 2021, Karpukhin et al., 2020 and Xiong et al., 2020). In the same spirit, the authors try to address a rather different problem of non-similarity based retrieval.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "The study makes an in-depth analysis of non-similarity based information retrieval, the analysis suggests that a combined metric of cosine similarity and sparsity function to identify and retrieve documents that contradict a given query improves performance as compared to existing methods.\n\nThe authors cite and explain the existing studies, they also make a good literature study of the existing and related studies and how their proposed approach addresses the similarity problem differently.\n\nThe paper is well written, with clear figures, and the experimental results support the findings.", "other_comments_or_suggestions": "NA", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "NA"}}, {"review_id": "aShYztcwrj", "reviewer": "Reviewer_RTuC", "summary": "* Introduces a contradiction retrieval method using cosine similarity and Hoyer measure of sparsity. The novel idea being using sparsity of embedding differences as a function to model contradiction.\n* Discusses other approaches namely bi-encoders and cross-encoder models along with their shortcomings and addresses how SparseCL overcomes those.\n* Shows robustness of the proposed fine tuning method on three pretrained sentence embedding models of different sizes across three datasets: Arguana, adapted HotpotQA and MSMARCO.\n* Discusses imperfections of Arguana dataset, augments with paraphrases to make it harder and shows differential performance of methods as the number of paraphrases increase. \n* Shows generalization capability of the sparse-aware embeddings by training on HotpotQA and testing on MSMARCO and, vice versa. \n* Applies the method on two downstream applications of corpus cleaning and NLI.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your insightful comments on our work!\n\nThank you for pointing out the typos. \n\n**\"Some arguments lack evidence/clarity e.g. “embeddings designed to preserve subtle, contradictory nuances between sentences”, meaning of the scores from Zero Shot (Cosine) vs CL (Cosine) methods.\"**\n\nThis sentence serves as a motivating hypothesis that explains why we believe embeddings for contradictions exhibit sparse differences. We acknowledge the current lack of evidence in explaining the semantic meaning behind each coordinate, which is an intriguing area for further investigation.\n\nWe will adjust the figure captions and the notations in the score function as you suggested.", "claims_and_evidence": "* Claims that transitivity of cosine function makes it inherently incapable of representing the contradiction relation, backs it up with proofs and shows an average improvement of 3.6% with sparsity aware embeddings. \n* The method enhances the speed of contradiction detection, supported by an experiment showing 200 times speed up compared to cross encoder methods.\n* Mentions “embeddings designed to preserve subtle, contradictory nuances between sentences” and “effectively captures the essence of contradiction”, but lacks qualitative evidence to support these claims.", "methods_and_evaluation_criteria": "* Cosine similarity being limiting and using spasticity of embedding difference on top, is a good idea to retrieve contradictions. Although it is not very clear if Hoyer Sparsity is able to model and distinguish between contradictory differences vs orthogonal differences between two embeddings. \n* Breadth of benchmarking datasets and data augmentations shows effectiveness of the approach. \n* For the evaluation metric, most of the earlier work used Recall@1 and the paper uses NDCG@10, explanation for the choice of hyperparameter would make the shift clearer.", "theoretical_claims": "NA", "experimental_designs_or_analyses": "Checked the soundness of Counter-argument Retrieval, Contradiction retrieval, Zero-shot Generalization Tests and Arguana retrieval results analysis. \n\nVery thorough job explaining details except in Contradiction retrieval task, It is not clear whether both, Zeroshot(cosine) and CL(cosine) are expected to give high scores for similar vs contradictory examples (seemingly zeroshot/pre-trained embeddings are trained on semantic similarity whereas CL uses contradictions as positives), and if either scenario impacts the validity of comparing these two methods.", "relation_to_broader_scientific_literature": "* Introduces a method for contradiction retrieval task, using cosine similarity and Hoyer measure of sparsity, presenting sparsity of embedding differences as a function to capture contradiction, being the novel idea.\n* Discusses prior similarity based methods for contradiction retrieval like bi encoders and cross encoders and their limitations.\n* Runs two small experiments to show their method for contradiction retrieval can be applied to downstream applications of corpus cleaning and NLI.\n* Talks about how contradiction retrieval is related to some major LLM based research efforts in recent years namely Fact verification and LLM hallucination, and augmented LLM and retrieval corpus attack.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "* A novel idea to use sparsity of differences between the sentence embeddings to model contradiction. \n* Well written, provides theoretical and quantitative backing(supplement section C and G) on claims against the existing bi-encoder and cross-encoder models. \n* Shows robustness of the proposed method on three embedding models of different sizes and across three datasets.\n* An insightful ablation into the limitations of Arguana dataset (section 4.6), proposed augmentations and effectiveness of sparsity-based retrieval over similarity based methods(supplement section E). \n* Some arguments lack evidence/clarity e.g. “embeddings designed to preserve subtle, contradictory nuances between sentences”, meaning of the scores from Zero Shot (Cosine) vs CL (Cosine) methods.", "other_comments_or_suggestions": "* [Typo] Figure 2: sub figures are referred to as left and right, instead of top and bottom. \n* Scoring function for contradiction retrieval: In the Problem Formulation, query passage is referred to as q and corpus passages with p1, p2 and so on. The score function should probably be calculated between q and corpus passages, pi.", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "None"}}, {"review_id": "NMQfzpv9cf", "reviewer": "Reviewer_gpUP", "summary": "This paper introduces SparseCL, a novel approach for contradiction retrieval using sparse-aware sentence embeddings.  The method addresses limitations of traditional similarity search and cross-encoder methods by training sentence embeddings to preserve sparsity of differences between embeddings of contradicting sentences. SparseCL utilizes a combined metric of cosine similarity and Hoyer sparsity to score and retrieve documents that contradict a query, aiming for efficiency and improved contradiction detection.\n\n## update after rebuttal\n\nResponses did not really address the methodological questions I had (which is to be expected, since the issues go to the heart of the experiments presented)\n\nIt seems other reviewers didn't find the dataset *HotPotQA-synthetic-rephrase* being labelled *HotPotQA* throughout as much of a problem as I did.  Clearly Figure 2 is impressive, though I feel that this also points to a separation between matching/contradicting passages that is too-good-to-be-true (and hence my skepticism about the datasets overall).\n\nRating remains \"1: Reject\"", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your insightful comments provided for our work. Here are our responses to the proposed weaknesses and questions.\n\n**L271 statement:**  \nThis means that our SparseCL method can be combined with either a zero-shot embedding model or a fine-tuned embedding model, and we demonstrate that in both cases, the retrieval performance with SparseCL surpasses the performance without SparseCL. We will clarify this in our final version.\n\n**Samples provided in Appendix H:**  \nThanks for pointing out this observation. We did a simple check that among the 40 examples we listed in Appendix H, 8 of the original passages have the string “no/not/non” and 17 of the generated contradictions have the string “no/not/non”. It is true that in some cases, contradictions are formed by adding negation words, but it is also worth noticing that in more than half of the cases, contradictions are judged in other non-trivial ways like opposite words or phrase replacement. We believe this represents the essence of a well-defined contradiction rather than a flaw.\n\nSecond, it is still unclear how similarity-based retrieval methods can solve this task even if a certain style exists for generated contradictions. As we are looking for the opposite side of the query passage, rather than passages sharing the same style.\n\n**Concern about Arguana:**  \nWe agree that textual patterns exist for contradictions from any specific area. The reason that our method shows less dramatic performance in Arguana is that there aren't enough paraphrases within the corpus to confound the similarity-based retrieval method, because then it is likely that the counterargument is still the most similar passage within the corpus. In our ablation study (agreed by Reviewer RTuC), we manually add more paraphrases to the Arguana corpus, and we then observe a significant performance drop for the similarity-based retrieval method, and our method’s performance is unaffected. Please refer to Table 6.\n\n**W1: Synthetic Dataset Reliance:**  \nWe want to emphasize that, in addition to Arguana, our experiments on NLI tasks and data cleaning are conducted on real datasets. Please refer to Sections 4.4 and 4.5. For our data cleaning experiment, although the corrupted passages are generated by an LLM, they are similar enough to confuse current embedding models and cause a performance drop.\n\n**W2: Marginal Improvement on Arguana:**  \nSee above (Concern about Arguana)\n\n**W3: Need for Deeper Analysis:**  \nPlease refer to our ablation study, including sparsity function variants, data augmentation, and a case study in Appendix E to show its superiority against traditional cosine-based contrastive learning on the contradiction retrieval problem.\n\n**Typo:**  \nWe will fix all typos and citations for the camera ready version.\n\n**Q1 Synthetic Data Artifacts (Critical Question):**  \nWe have put in the prompt to encourage the generated content to be indistinguishable “in terms of length, language style, and format”. From the examples in Appendix H, we have manually checked and didn’t find any obvious language style difference between the paraphrases and contradictions. Given that it is hard to “prove” artifacts exist or not in generated passages, a manual check is the best we can do. We agree that certain key words like “no/not/non” may be used more often in contradictions, but this is the common practice to form contradictions, which also exist in human generated contradictions like Arguana.\n\n**Q2: the generalizability of your findings to real-world contradiction scenarios, given the reliance on synthetic data:**  \nThe motivation for us to generate contradiction pairs using LLM is the lack of natural contradiction pairs in real-world datasets besides Arguana and NLI datasets.\n\n**Q3: further validate that SparseCL captures genuine contradiction beyond LLM-specific artifacts:**  \nWe have sampled 500 passages from LLM generated contradictions and 2 people annotated the data to check the quality. Please refer to Appendix H. Therefore, we believe that the experiments run on our synthetic datasets are fair.", "claims_and_evidence": "The embedding training separation shown in Figure 2 is very impressive - the training is clearly effective according to the metrics.\n\nIn L271, the authors state \"This pattern of enhancement was consistently observed regardless of whether the embedding models were fine-tuned or not.\"  This is a very puzzling statement, which prompted digging deeper.  \n\nLooking at the samples provided in Appendix H: It seems like many of the LLM-generated contradiction samples are 'simple contradictions' - including \"no\" and \"not\" far more than the straight paraphrases.  Which in turn leads to a reconsideration of what the MSMARCO and HotpotQA dataset results actually represent.  A plausible explanation for the experimental results is simply that the paraphrasing (positive or contradictory) preserves L2 similarity, while the sparse contradiction elements are just learning to recognise the signs of LLM-generated-contradiction.  This seems like a fundamental experimental flaw.\n\nThe fact that SparseCL *also* shows improvements on the Arguana dataset, which is a human-curated dataset, provides some counter-evidence to this artifact-only argument.  However, Arguana is also focused on counter-argument detection within debates, which might still exhibit specific textual patterns different from more general contradiction.  Moreover, the Arguana results show less dramatic performance - which is what raised the artifact questions above in the first place.", "methods_and_evaluation_criteria": "The proposed SparseCL method, combining cosine similarity with a Hoyer sparsity measure on contrastively learned embeddings, is a sensible approach to address the limitations of purely similarity-based methods for contradiction retrieval. The use of contrastive learning with reversed positive/negative examples (contradictions as positive, paraphrases as negative) is also well-motivated for learning contradiction-aware embeddings.", "theoretical_claims": "The paper includes a theoretical analysis in Appendix C about the non-transitivity of Hoyer sparsity and the transitivity of cosine similarity.  I have not rigorously checked the correctness of the proofs in Appendix C, but the intuition presented and the examples seem reasonable.  The theoretical motivation for using Hoyer sparsity to circumvent the transitivity of cosine similarity is clearly presented and makes sense.", "experimental_designs_or_analyses": "As noted in 'Claims and Evidence', the MSMARCO and HotpotQA experimental design and results are overshadowed by the synthetic data offering 'short-cuts' to detection.", "relation_to_broader_scientific_literature": "The paper is well-positioned within the broader scientific literature on information retrieval and semantic representation learning. It correctly identifies the limitations of traditional similarity search for non-similarity-based tasks like contradiction retrieval.  \n\nThe novelty lies in the specific combination of contrastive learning with a sparsity-inducing objective, tailored for contradiction detection, and the use of Hoyer sparsity.  The paper appropriately cites relevant works in these areas.", "essential_references_not_discussed": "Nothing specific.", "other_strengths_and_weaknesses": "### Strengths:\n\n- **Novel Approach:** The SparseCL method is a novel and well-motivated approach for contradiction retrieval, addressing a gap in existing methods.\n- **Clear Presentation:** The paper is generally well-written and clearly presents the method, experiments, and results.\n- **Efficiency:** The method offers a computationally efficient alternative to cross-encoders.\n\n### Weaknesses:\n\n- **Synthetic Dataset Reliance:**  The heavy reliance on synthetically generated datasets (MSMARCO, HotpotQA) for evaluation is a significant weakness regarding generalizability and the potential for artifact exploitation.\n- **Marginal Improvement on Arguana:** While improvements are shown on Arguana, they are less dramatic than on synthetic datasets, which could be interpreted as supporting the artifact-exploitation concern.\n- **Need for Deeper Analysis:**  The paper could benefit from a deeper analysis of *why* SparseCL works, particularly on the synthetic datasets, and more detailed error analysis.", "other_comments_or_suggestions": "### Typos / Fixes\n* L043: Fix citation double bracketing\n* L009: \"while as far as we know\" ... seems like the authors should be clear on this point\n* L011: \"non-simlarity\" -> \"non-similarity\"\n* L100: \"In specific, they\" -> \"Specifically, they\"\n* L102: \"This phenomenon bring our attention\" -> \"This phenomenon brought our attention\"\n* L105: \"papers are different\" -> \"papers was different\" (tense+number)\n* L148: Fix \",.\"\n* L157: \"a score between [0, 1]\" -> \"a score in (0, 1)\"\n* L134: \"so the authors of (Wachsmuth et al 2018) designed\" -> \"so Wachsmuth et al (2018) proposed\"\n* L154: Fix citation double bracketing\n* L155: \"fine-tune any pretrained\" -> \"fine-tune a pretrained\"\n* L421: Fix bracketing \"selected from (Hurley...)\" -> \"selected from Hurley ... ()\"\n* L655: \"(Shi et al., 2023) proposes\" -> \"Shi et al. (2023) proposes\"\n\n* Appendix C. \"Two examples demonstrating the ...\" -> \"The ...\" (these are proofs, not examples)", "questions_for_authors": "### Synthetic Data Artifacts (Critical Question):\n\nThe MSMARCO and HotpotQA datasets are synthetically generated.  One concern is that SparseCL might be exploiting artifacts from the LLM's negative paraphrasing rather than capturing generalizable contradiction.  Could you please elaborate on how you have considered or mitigated this potential issue?  Specifically:\n1) Have you analyzed failure cases or error types to see if SparseCL is overly sensitive to specific LLM-generated patterns?\n1) Could you discuss the generalizability of your findings to real-world contradiction scenarios, given the reliance on synthetic data?\n1) Are there any experiments or analyses you could perform to further validate that SparseCL captures genuine contradiction beyond LLM-specific artifacts?  For instance, could you compare performance on a subset of the synthetic data that is *manually* verified for contradiction quality, or explore datasets of contradictions not generated by LLMs?\n\n*If your response clearly shows that I am mistaken about the potential issues with identifiable LLM 'smell' for the synthetic contradictions, then I would be happy to update my score*", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "APpMzrzE5L", "reviewer": "Reviewer_isYx", "summary": "This paper addresses the task of contradiction retrieval—retrieving documents or passages that explicitly refute a given query. The authors introduce SPARSECL, a method that augments standard contrastive learning for sentence embeddings with a sparsity measure (specifically, the Hoyer measure) to capture subtle contradictory nuances. The proposed approach combines traditional cosine similarity with a sparsity-based score to more effectively identify contradictions. Extensive experiments are performed on the Arguana dataset as well as on synthetic contradiction datasets constructed from MSMARCO and HotpotQA, with additional applications in corpus cleaning and natural language inference.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your insightful comments on our work. Here are our responses to the proposed weaknesses and questions.\nW1 & W2 see below Q2 and Q1\n**W3: Some parts of the theoretical analysis could be more tightly integrated with empirical findings.**\n\nWe are not claiming that we have theoretical analysis for characterizing the contradiction relationship. The two propositions in Appendix serve as motivating examples to show the fundamental limitation of the cosine metric in representing contradictions and how the sparsity metric can bypass that.\n\n**Q1: hyperparameter sensitivity**\n\nHere is the way we tune the $\\alpha$ parameter: From the range [0, 10], we first divide the range into 10 intervals, calculating the NDCG@10 score on the validation set for each interval’s midpoint, and then diving into that interval for a finer search. We stop when the interval range is smaller than 0.01. We will collect all the $\\alpha$ values and investigate their sensitivity in the final version.\n\n**Q2: data generation concerns**\n\nPlease see our experimental results on Arguana, NLI datasets, where contradictions are collected from natural human-written sources. \n\n**Q3: High-Dimensional Effects**\n\nWe believe that the current embeddings are already in a high-dimensional space (e.g. bge-base has dimensionality 768) and our experiments work well there. Please see our ablation study in Table 4 on different sparsity functions. All of them yield some improvement compared with the baseline, while our Hoyer sparsity yields the best performance. We believe that’s because simple sparsity functions have a more benign optimization landscape and thus are easier for models to optimize, which is worth future study.", "claims_and_evidence": "Key Claims:\n1. SPARSECL can enhance contradiction retrieval over standard similarity-based methods.\n2. Incorporating the Hoyer sparsity measure with cosine similarity yields measurable improvements (e.g., in NDCG@10) across multiple datasets.\n3. The method generalizes across datasets and has practical downstream applications (e.g., corpus cleaning).\n\nEvidence:\n1. The paper presents quantitative improvements (e.g., up to 11.0% average gain in retrieval metrics) compared to baselines such as standard contrastive learning and prompt-augmented approaches.\n2. Ablation studies and additional zero-shot tests are provided, reinforcing the claim that the sparsity measure contributes uniquely to capturing contradictions.", "methods_and_evaluation_criteria": "The authors fine-tune pre-trained sentence embedding models via a modified contrastive learning loss that rewards high Hoyer sparsity for contradiction pairs while using similar (non-contradictory) pairs as negatives. A combined scoring function—cosine similarity plus an α-weighted Hoyer score—is used at test time to rank candidate passages.\n\nRetrieval performance is measured using NDCG@10 on Arguana, as well as on modified MSMARCO and HotpotQA datasets. Additional evaluations include ablation studies, zero-shot generalization tests, and experiments on corpus cleaning.", "theoretical_claims": "The paper argues that cosine similarity’s transitivity limits its ability to capture contradiction (since similar sentences may both contradict a third without contradicting each other).\n\nIn contrast, the Hoyer sparsity measure is shown (via propositions in the Appendix) to be non-transitive, allowing it to better represent the localized differences that signify contradiction.\n\nThe provided proofs (Propositions C.1 and C.2) appear sound under the stated assumptions, but the practical implications in high-dimensional settings might warrant further discussion.", "experimental_designs_or_analyses": "Design Strengths:\nAblation studies dissect the contribution of each component (e.g., effect of adding paraphrases, alternative sparsity functions). Downstream applications (such as corpus cleaning) are tested, demonstrating the method’s utility beyond standard retrieval benchmarks.\n\nPotential Issues:\nSome experiments rely on synthetic data generated by GPT-4, which might not fully capture the complexities of naturally occurring contradictions. The evaluation on the Arguana dataset suggests that in some cases, even similarity-based methods perform reasonably well, potentially masking the unique contribution of the sparsity component.", "relation_to_broader_scientific_literature": "The paper is well-situated within the literature on contrastive learning and sentence embeddings, and it draws appropriate connections to fact verification and counter-argument retrieval. Its focus on non-similarity-based retrieval differentiates it from prior work, although a deeper discussion comparing its approach with alternative methods for detecting contradictions (e.g., adversarial training methods) would further clarify its novelty.", "essential_references_not_discussed": "While the paper cites relevant work on counter-argument retrieval (e.g., Shi et al. (2023) on the Bipolar-encoder), it might benefit from referencing additional recent studies on robust retrieval or adversarial defenses in retrieval systems, which could provide context regarding alternative approaches to handling contradictory information.\n\nA discussion on recent developments in fact-checking and misinformation detection could also broaden the context of its contributions.", "other_strengths_and_weaknesses": "Strengths:\n1. Novel use of a sparsity measure (Hoyer) to capture contradiction nuances.\n2. Comprehensive experimental validation across multiple datasets and applications.\n3. Clear exposition of the limitations of standard cosine-based retrieval and the motivation for a new metric.\n\nWeaknesses:\n1. Dependence on synthetic data for some evaluations might limit external validity.\n2. The sensitivity to the hyperparameter α and potential computational implications in high-dimensional spaces are not fully explored.\n3. Some parts of the theoretical analysis could be more tightly integrated with empirical findings.", "other_comments_or_suggestions": "Failure modes: Discuss potential limitations or failure cases in more detail, particularly concerning the generalization to varied contradiction types.", "questions_for_authors": "1. Hyperparameter Sensitivity: How sensitive is the retrieval performance to the choice of the α parameter? Could you provide a more detailed analysis of its tuning across different datasets?\n\n2. Data Generation Concerns: Given that some datasets rely on GPT-4–generated contradictions, how do you expect SPARSECL to perform on datasets containing naturally occurring contradictions?\n\n3. High-Dimensional Effects: Can you elaborate on any potential issues that might arise from applying the Hoyer sparsity measure in high-dimensional embedding spaces, and whether alternative sparsity measures might offer complementary advantages?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xpP6FmW9tW", "reviewer": "Reviewer_6o5M", "summary": "The paper proposes a new method named SparseCL to leverage specially trained sentence embeddings designed to preserve subtle, contradictory nuances between sentences. SparseCL utilizes a combined metric of cosine similarity and a sparsity function to efficiently identify and retrieve documents that contradict a given query. Experiments are performed on the Arguana, MSMARCO, and HotpotQA datasets to demonstrate the effectiveness of the proposed method. The authors apply the proposed contradiction retrieval method to two downstream settings including corpus cleaning and natural language inference to highlight the practical benefits of the proposed approach in real-world scenarios.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your insightful comments provided for our work. Here are our responses to the proposed weaknesses and questions.\n\n**W1: The overall design is rather simple with a well-known sparsity metric.**\n\nAs shown in Table 4, the idea of SparseCL is not limited to Hoyer sparsity, but also works on many other sparsity functions. We believe that simple idea with superior performance is actually an advantage.\n\n**W2: Besides the provided baselines (BGE, UAE, GTE), does the proposed method perform well on an LLM-based encoding method?**\n\nThank you for pointing out this consideration! As of September 2024, the 'gte-large-v1.5' model, achieving an NDCG score of 72, performed best on the Arguana dataset among models with fewer than 1 billion parameters. Additionally, LLM-based embeddings were not effective on Arguana during that time; for instance, SFR-Embedding-Mistral scored 67 in NDCG, and Voyage-lite-02-instruct scored 70. We believe that the three models we tested, varying in size, backbone, and performance, sufficiently demonstrate the effectiveness of our SparseCL method.\n\n**Typo**\n\nThank you for correcting the typos. We will fix them in the final version.", "claims_and_evidence": "Yes. The claims are mostly well-supported by the experimental results.", "methods_and_evaluation_criteria": "Yes. The proposed method can improve the baseline without introducing much burden.", "theoretical_claims": "Yes. Theoretical Claims are well supported.", "experimental_designs_or_analyses": "Yes. The experimental designs are overall reasonably designed.", "relation_to_broader_scientific_literature": "The proposed contradiction retrieval method may benefit areas like corpus cleaning and natural language inference.", "essential_references_not_discussed": "No. The references are well-cited and discussed.", "other_strengths_and_weaknesses": "Strengths:\n- The paper is overall well-written and organized.\n- Besides the cosine similarity, the paper proposes to use Hoyer sparsity as the metric for contradiction retrieval without complex pair-wise computation for reranking.\n- The experimental validation is performed on three public datasets and demonstrates the effectiveness of the proposed method.\n\nWeakness:\n- The overall design is rather simple with a well-known sparsity metric.\n- Besides the provided baselines (BGE, UAE, GTE), does the proposed method perform well on an LLM-based encoding method?", "other_comments_or_suggestions": "- Figure 1 is not referenced.\n- The caption of Figure 2 L196 and L199: left/right should be upper/bottom?", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a novel approach to contradiction retrieval called SparseCL. The approach combines standard cosine similarity metrics with a sparsity function that allows for efficient and effective retrieval of documents that refute the given query. The approach is evaluated on both human-curated and synthetically generated data sets, showing consistent improvements over baselines.

The reviewers found the proposed approach to be novel and well motivated, the empirical results to be strong, the method to demonstrate convincing efficiency properties, and the presentation to be clear.

The reviewers also noted a number of weaknesses, the primary one being the reliance on synthetic data for some of the evaluations, and the fact that the approach did not work quite as well on manually curated data.

Overall, though, the strengths of the paper outweighed the weaknesses. Efficient contradiction retrieval is a practically important direction and an interesting research direction. This paper represents a meaningful contribution and therefore should be accepted for publication.
Decision: Accept (poster)