Question: What dataset is used to compare these neural network designs?
Evidence: We conducted sentence pair modeling experiments on eight popular datasets: two NLI datasets, three PI datasets, one STS dataset and two QA datasets. Table 2 gives a comparison of these datasets:
SNLI BIBREF7 contains 570k hypotheses written by crowdsourcing workers given the premises. It focuses on three semantic relations: the premise entails the hypothesis (entailment), they contradict each other (contradiction), or they are unrelated (neutral).
Multi-NLI BIBREF33 extends the SNLI corpus to multiple genres of written and spoken texts with 433k sentence pairs.
Quora BIBREF34 contains 400k question pairs collected from the Quora website. This dataset has balanced positive and negative labels indicating whether the questions are duplicated or not.
Twitter-URL BIBREF35 includes 50k sentence pairs collected from tweets that share the same URL of news articles. This dataset contains both formal and informal language.
PIT-2015 BIBREF5 comes from SemEval-2015 and was collected from tweets under the same trending topic. It contains naturally occurred (i.e. written by independent Twitter users spontaneously) paraphrases and non-paraphrases with varied topics and language styles.
STS-2014 BIBREF36 is from SemEval-2014, constructed from image descriptions, news headlines, tweet news, discussion forums, and OntoNotes BIBREF37 .
WikiQA BIBREF8 is an open-domain question-answering dataset. Following He and Lin he-lin:2016:N16-1, questions without correct candidate answer sentences are excluded, and answer sentences are truncated to 40 tokens, resulting in 12k question-answer pairs for our experiments.
TrecQA BIBREF38 is an answer selection task of 56k question-answer pairs and created in Text Retrieval Conferences (TREC). For both WikiQA and TrecQA datasets, the best answer is selected according to the semantic relatedness with the question.
•SNLI (Bowman et al., 2015) contains 570k hypotheses written by crowdsourcing workers given the premises. It focuses on three semantic relations: the premise entails the hypothesis (entailment), they contradict each other (contradiction), or they are unrelated (neutral). •Multi-NLI (Williams et al., 2017) extends the SNLI corpus to multiple genres of written and spoken texts with 433k sentence pairs. •Quora (Iyer et al., 2017) contains 400k question pairs collected from the Quora website. This dataset has balanced positive and negative labels indicating whether the questions are duplicated or not. •Twitter-URL (Lan et al., 2017) includes 50k sentence pairs collected from tweets that share the same URL of news articles. This dataset contains both formal and informal language. •PIT-2015 (Xu et al., 2015) comes from SemEval-2015 and was collected from tweets under the same trending topic. It contains naturally occurred (i.e. written by independent Twitter users spontaneously) paraphrases and non-paraphrases with varied topics and language styles. •STS-2014 (Agirre et al., 2014) is from SemEval-2014, constructed from image descriptions, news headlines, tweet news, discussion forums, and OntoNotes (Hovy et al., 2006). •WikiQA (Yang et al., 2015) is an open-domain question-answering dataset. Following He and Lin (2016), questions without correct candidate answer sentences are excluded, and answer sentences are truncated to 40 tokens, resulting in 12k question-answer pairs for our experiments. •TrecQA (Wang et al., 2007) is an answer selection task of 56k question-answer pairs and created in Text Retrieval Conferences (TREC). For both WikiQA and TrecQA datasets, the best answer is selected according to the semantic relatedness with the question.
Answer: Eight datasets are used including SNLI (Bowman et al., 2015),Multi-NLI (Williams et al., 2017),Quora (Iyer et al., 2017),Twitter-URL (Lan et al., 2017),PIT-2015 (Xu et al., 2015),STS-2014 (Agirre et al., 2014),WikiQA (Yang et al., 2015) and TrecQA (Wang et al., 2007).