qinchuanhui/UDA-QA huggingface dataset:
Question: Which two datasets does the resource come from?

Evidence:
raw_evidence: The resource is composed of data from two different surveys. In both surveys subjects were asked to draw on a map (displayed under a Mercator projection) a polygon representing a given geographical descriptor, in the context of the geography of Galicia in Northwestern Spain (see Fig. FIGREF1 ). However, the surveys were run with different purposes, and the subject groups that participated in each survey and the list of descriptors provided were accordingly different.
The first survey was run in order to obtain a high number of responses to be used as an evaluation testbed for modeling algorithms. It was answered by 15/16 year old students in a high school in Pontevedra (located in Western Galicia). 99 students provided answers for a list of 7 descriptors (including cardinal points, coast, inland, and a proper name). Figure FIGREF2 shows a representation of the answers given by the students for “Northern Galicia” and a contour map that illustrates the percentages of overlapping answers.
The second survey was addressed to meteorologists in the Galician Weather Agency BIBREF12 . Its purpose was to gather data to create fuzzy models that will be used in a future NLG system in the weather domain. Eight meteorologists completed the survey, which included a list of 24 descriptors. For instance, Figure FIGREF3 shows a representation of the answers given by the meteorologists for “Eastern Galicia” and a contour map that illustrates the percentage of overlapping answers.

Answers:
two surveys by two groups - school students and meteorologists to draw on a map a polygon representing a given geographical descriptor
Question: What model was used by the top team?

Evidence:
raw_evidence: BIBREF9 Two different BERT models were developed. For Friends, pre-training was done using a sliding window of two utterances to provide dialogue context. Both Next Sentence Prediction (NSP) phase on the complete unlabeled scripts from all 10 seasons of Friends, which are available for download. In addition, the model learned the emotional disposition of each of six main six main characters in Friends (Rachel, Monica, Phoebe, Joey, Chandler and Ross) by adding a special token to represent the speaker. For EmotionPush, pre-training was performed on Twitter data, as it is similar in nature to chat based dialogues. In both cases, special attention was given to the class imbalance issue by applying “weighted balanced warming” on the loss function.

Answers:
Two different BERT models were developed
Question: What is the size of the second dataset?

Evidence:
raw_evidence: The two datasets used for the challenge are Friends and EmotionPush, part of the EmotionLines corpus BIBREF4. The datasets contain English-language dialogues of varying lengths. For the competition, we provided 1,000 labeled dialogues from each dataset for training, and 240 unlabeled dialogues from each dataset for evaluation. The Friends dialogues are scripts taken from the American TV sitcom (1994-2004). The EmotionPush dialogues are from Facebook Messenger chats by real users which have been anonymized to ensure user privacy. For both datasets, dialogue lengths range from 5 to 24 lines each. A breakdown of the lengths of the dialogues is shown in Table .

Answers:
1 000 labeled dialogues for training and 240 unlabeled dialogues for evaluation
Question: How large is the first dataset?

Evidence:
raw_evidence: The two datasets used for the challenge are Friends and EmotionPush, part of the EmotionLines corpus BIBREF4. The datasets contain English-language dialogues of varying lengths. For the competition, we provided 1,000 labeled dialogues from each dataset for training, and 240 unlabeled dialogues from each dataset for evaluation. The Friends dialogues are scripts taken from the American TV sitcom (1994-2004). The EmotionPush dialogues are from Facebook Messenger chats by real users which have been anonymized to ensure user privacy. For both datasets, dialogue lengths range from 5 to 24 lines each. A breakdown of the lengths of the dialogues is shown in Table .

Answers:
1 000 labeled dialogues for training and 240 unlabeled dialogues for evaluation
Question: What is MRR?

Evidence:
raw_evidence: I first measured the effectiveness of the embeddings at the skip-gram's training task, predicting context words INLINEFORM0 given input words INLINEFORM1 . This task measures the methods' performance for predictive language modeling. I used four datasets of sociopolitical, scientific, and literary interest: the corpus of NIPS articles from 1987 – 1999 ( INLINEFORM2 million), the U.S. presidential state of the Union addresses from 1790 – 2015 ( INLINEFORM3 ), the complete works of Shakespeare ( INLINEFORM4 ; this version did not contain the Sonnets), and the writings of black scholar and activist W.E.B. Du Bois, as digitized by Project Gutenberg ( INLINEFORM5 ). For each dataset, I held out 10,000 INLINEFORM6 pairs uniformly at random, where INLINEFORM7 , and aimed to predict INLINEFORM8 given INLINEFORM9 (and optionally, INLINEFORM10 ). Since there are a large number of classes, I treat this as a ranking problem, and report the mean reciprocal rank. The experiments were repeated and averaged over 5 train/test splits.

Answers:
mean reciprocal rank
Question: Which techniques for word embeddings and topic models are used?

Evidence:
raw_evidence: To design an interpretable word embedding model for small corpora, we identify novel connections between word embeddings and topic models, and adapt advances from topic modeling. Following the distributional hypothesis BIBREF23 , the skip-gram's word embeddings parameterize discrete probability distributions over words INLINEFORM0 which tend to co-occur, and tend to be semantically coherent – a property leveraged by the Gaussian LDA model of BIBREF21 . This suggests that these discrete distributions can be reinterpreted as topics INLINEFORM1 . We thus reinterpret the skip-gram as a parameterization of a certain supervised naive Bayes topic model (Table TABREF2 , top-right). In this topic model, input words INLINEFORM2 are fully observed “cluster assignments,” and the words in INLINEFORM3 's contexts are a “document.” The skip-gram differs from this supervised topic model only in the parameterization of the “topics” via word vectors which encode the distributions with a log-bilinear model. Note that although the skip-gram is discriminative, in the sense that it does not jointly model the input words INLINEFORM4 , we are here equivalently interpreting it as encoding a “conditionally generative” process for the context given the words, in order to develop probabilistic models that extend the skip-gram.
As in LDA, this model can be improved by replacing the naive Bayes assumption with a mixed membership assumption. By applying the mixed membership representation to this topic model version of the skip-gram, we obtain the model in the bottom-right of Table TABREF2 . After once again parameterizing this model with word embeddings, we obtain our final model, the mixed membership skip-gram (MMSG) (Table TABREF2 , bottom-left). In the model, each input word has a distribution over topics INLINEFORM0 . Each topic has a vector-space embedding INLINEFORM1 and each output word has a vector INLINEFORM2 (a parameter, not an embedding for INLINEFORM3 ). A topic INLINEFORM4 is drawn for each context, and the words in the context are drawn from the log-bilinear model using INLINEFORM5 : DISPLAYFORM0

Answers:
 skip-gram, LDA
Question: Why is big data not appropriate for this task?

Evidence:
raw_evidence: Word embeddings have risen in popularity for NLP applications due to the success of models designed specifically for the big data setting. In particular, BIBREF0 , BIBREF1 showed that very simple word embedding models with high-dimensional representations can scale up to massive datasets, allowing them to outperform more sophisticated neural network language models which can process fewer documents. In this work, I offer a somewhat contrarian perspective to the currently prevailing trend of big data optimism, as exemplified by the work of BIBREF0 , BIBREF1 , BIBREF3 , and others, who argue that massive datasets are sufficient to allow language models to automatically resolve many challenging NLP tasks. Note that “big” datasets are not always available, particularly in computational social science NLP applications, where the data of interest are often not obtained from large scale sources such as the internet and social media, but from sources such as press releases BIBREF11 , academic journals BIBREF10 , books BIBREF12 , and transcripts of recorded speech BIBREF13 , BIBREF14 , BIBREF15 . A standard practice in the literature is to train word embedding models on a generic large corpus such as Wikipedia, and use the embeddings for NLP tasks on the target dataset, cf. BIBREF3 , BIBREF0 , BIBREF16 , BIBREF17 . However, as we shall see here, this standard practice might not always be effective, as the size of a dataset does not correspond to its degree of relevance for a particular analysis. Even very large corpora have idiosyncrasies that can make their embeddings invalid for other domains. For instance, suppose we would like to use word embeddings to analyze scientific articles on machine learning. In Table TABREF1 , I report the most similar words to the word “learning” based on word embedding models trained on two corpora. For embeddings trained on articles from the NIPS conference, the most similar words are related to machine learning, as desired, while for embeddings trained on the massive, generic Google News corpus, the most similar words relate to learning and teaching in the classroom. Evidently, domain-specific data can be important.
I have proposed a model-based method for training interpretable corpus-specific word embeddings for computational social science, using mixed membership representations, Metropolis-Hastings-Walker sampling, and NCE. Experimental results for prediction, supervised learning, and case studies on state of the Union addresses and NIPS articles, indicate that high-quality embeddings and topics can be obtained using the method. The results highlight the fact that big data is not always best, as domain-specific data can be very valuable, even when it is small. I plan to use this approach for substantive social science applications, and to address algorithmic bias and fairness issues.

Answers:
Training embeddings from small-corpora can increase the performance of some tasks
Question: What is an example of a computational social science NLP task?

Evidence:
raw_evidence: I also performed several case studies. I obtained document embeddings, in the same latent space as the topic embeddings, by summing the posterior mean vectors INLINEFORM0 for each token, and visualized them in two dimensions using INLINEFORM1 -SNE BIBREF24 (all vectors were normalized to unit length). The state of the Union addresses (Figure FIGREF27 ) are embedded almost linearly by year, with a major jump around the New Deal (1930s), and are well separated by party at any given time period. The embedded topics (gray) allow us to interpret the space. The George W. Bush addresses are embedded near a “war on terror” topic (“weapons, war...”), and the Barack Obama addresses are embedded near a “stimulus” topic (“people, work...”).

Answers:
Visualization of State of the union addresses
Question: What were the previous state of the art benchmarks?

Evidence:
raw_evidence: The comparison of our method with various baselines and state-of-the-art methods is provided in table TABREF26 for VQA 1.0 and table TABREF27 for VQG-COCO dataset. The comparable baselines for our method are the image based and caption based models in which we use either only the image or the caption embedding and generate the question. In both the tables, the first block consists of the current state-of-the-art methods on that dataset and the second contains the baselines. We observe that for the VQA dataset we achieve an improvement of 8% in BLEU and 7% in METEOR metric scores over the baselines, whereas for VQG-COCO dataset this is 15% for both the metrics. We improve over the previous state-of-the-art BIBREF35 for VQA dataset by around 6% in BLEU score and 10% in METEOR score. In the VQG-COCO dataset, we improve over BIBREF5 by 3.7% and BIBREF36 by 3.5% in terms of METEOR scores.

Answers:
BIBREF35 for VQA dataset, BIBREF5, BIBREF36
Question: How/where are the natural question generated?

Evidence:
raw_evidence: Our method is based on a sequence to sequence network BIBREF38 , BIBREF12 , BIBREF39 . The sequence to sequence network has a text sequence as input and output. In our method, we take an image as input and generate a natural question as output. The architecture for our model is shown in Figure FIGREF4 . Our model contains three main modules, (a) Representation Module that extracts multimodal features (b) Mixture Module that fuses the multimodal representation and (c) Decoder that generates question using an LSTM-based language model.

Answers:
Decoder that generates question using an LSTM-based language model
Question: What is the input to the differential network?

Evidence:
raw_evidence: Our method is based on a sequence to sequence network BIBREF38 , BIBREF12 , BIBREF39 . The sequence to sequence network has a text sequence as input and output. In our method, we take an image as input and generate a natural question as output. The architecture for our model is shown in Figure FIGREF4 . Our model contains three main modules, (a) Representation Module that extracts multimodal features (b) Mixture Module that fuses the multimodal representation and (c) Decoder that generates question using an LSTM-based language model.

Answers:
image
Question: How do the authors define a differential network?

Evidence:
raw_evidence: The proposed Multimodal Differential Network (MDN) consists of a representation module and a joint mixture module.
We use a triplet network BIBREF41 , BIBREF42 in our representation module. We refereed a similar kind of work done in BIBREF34 for building our triplet network. The triplet network consists of three sub-parts: target, supporting, and contrasting networks. All three networks share the same parameters. Given an image INLINEFORM0 we obtain an embedding INLINEFORM1 using a CNN parameterized by a function INLINEFORM2 where INLINEFORM3 are the weights for the CNN. The caption INLINEFORM4 results in a caption embedding INLINEFORM5 through an LSTM parameterized by a function INLINEFORM6 where INLINEFORM7 are the weights for the LSTM. This is shown in part 1 of Figure FIGREF4 . Similarly we obtain image embeddings INLINEFORM8 & INLINEFORM9 and caption embeddings INLINEFORM10 & INLINEFORM11 . DISPLAYFORM0
The Mixture module brings the image and caption embeddings to a joint feature embedding space. The input to the module is the embeddings obtained from the representation module. We have evaluated four different approaches for fusion viz., joint, element-wise addition, hadamard and attention method. Each of these variants receives image features INLINEFORM0 & the caption embedding INLINEFORM1 , and outputs a fixed dimensional feature vector INLINEFORM2 . The Joint method concatenates INLINEFORM3 & INLINEFORM4 and maps them to a fixed length feature vector INLINEFORM5 as follows: DISPLAYFORM0

Answers:
The proposed Multimodal Differential Network (MDN) consists of a representation module and a joint mixture module.
Question: How do the authors define exemplars?

Evidence:
raw_evidence: Exemplars aim to provide appropriate context. To better understand the context, we experimented by analysing the questions generated through an exemplar. We observed that indeed a supporting exemplar could identify relevant tags (cows in Figure FIGREF3 ) for generating questions.
We improve use of exemplars by using a triplet network. This network ensures that the joint image-caption embedding for the supporting exemplar are closer to that of the target image-caption and vice-versa. We empirically evaluated whether an explicit approach that uses the differential set of tags as a one-hot encoding improves the question generation, or the implicit embedding obtained based on the triplet network. We observed that the implicit multimodal differential network empirically provided better context for generating questions. Our understanding of this phenomenon is that both target and supporting exemplars generate similar questions whereas contrasting exemplars generate very different questions from the target question. The triplet network that enhances the joint embedding thus aids to improve the generation of target question. These are observed to be better than the explicitly obtained context tags as can be seen in Figure FIGREF2 . We now explain our method in detail.

Answers:
Exemplars aim to provide appropriate context., joint image-caption embedding for the supporting exemplar are closer to that of the target image-caption
Question: Is this a task other people have worked on?

Evidence:
raw_evidence: We propose a novel problem of relationship recommendation (RSR). Different from the reciprocal recommendation problem on DSNs, our RSR task operates on regular social networks (RSN), estimating long-term and serious relationship compatibility based on social posts such as tweets.

Answers:
No
Question: Where did they get the data for this project?

Evidence:
raw_evidence: Since there are no publicly available datasets for training relationship recommendation models, we construct our own. The goal is to construct a list of user pairs in which both users are in relationship. Our dataset is constructed via distant supervision from Twitter. We call this dataset the Love Birds dataset. This not only references the metaphorical meaning of the phrase `love birds' but also deliberately references the fact that the Twitter icon is a bird. This section describes the construction of our dataset. Figure 1 describes the overall process of our distant supervision framework.

Answers:
Twitter
Question: Which major geographical regions are studied?

Evidence:
raw_evidence: We examine other features regarding the characteristics of the studied colleges, which might be significant factors of sexual harassment. Four factual attributes pertaining to the 200 colleges are extracted from the U.S. News Statistics, which consists of Undergraduate Enrollment, Male/Female Ratio, Private/Public, and Region (Northeast, South, West, and Midwest). We also use the normalized rape-related cases count (number of cases reported per student enrolled) from the stated government resource as another attribute to examine the proximity of our dataset to the official one. This feature vector is then fitted in a linear regression to predict the normalized #metoo users count (number of unique users who posted #MeToo tweets per student enrolled) for each individual college.

Answers:
Northeast U.S, South U.S., West U.S. and Midwest U.S.
Question: How are the topics embedded in the #MeToo tweets extracted?

Evidence:
raw_evidence: In order to understand the latent topics of those #MeToo tweets for college followers, we first utilize Latent Dirichlet Allocation (LDA) to label universal topics demonstrated by the users. We determine the optimal topic number by selecting the one with the highest coherence score. Since certain words frequently appear in those #MeToo tweets (e.g., sexual harassment, men, women, story, etc.), we transform our corpus using TF-IDF, a term-weighting scheme that discounts the influence of common terms.

Answers:
Using Latent Dirichlet Allocation on TF-IDF transformed from the corpus
Question: How many tweets are explored in this paper?

Evidence:
raw_evidence: In this study, we limit the sample size to the followers identified as English speakers in the U.S. News Top 200 National Universities. We utilize the Jefferson-Henrique script, a web scraper designed for Twitter to retrieve a total of over 300,000 #MeToo tweets from October 15th, when Alyssa Milano posted the inceptive #MeToo tweet, to November 15th of 2017 to cover a period of a month when the trend was on the rise and attracting mass concerns. Since the lists of the followers of the studied colleges might overlap and many Twitter users tend to reiterate other's tweets, simply putting all the data collected together could create a major redundancy problem. We extract unique users and tweets from the combined result set to generate a dataset of about 60,000 unique tweets, pertaining to 51,104 unique users.

Answers:
60,000
Question: Which geographical regions correlate to the trend?

Evidence:
raw_evidence: Observing the results of the linear regression in Table 2, we find the normalized governmental reported cases count and regional feature to be statistically significant on the sexual harassment rate in the Twitter data ($p-value<0.05$). Specifically, the change in the number of reported cases constitutes a considerable change in the number of #MeToo users on Twitter as p-value is extremely small at $5.7e-13$. This corresponds to the research by Napolitano (2014) regarding the "Yes means yes" movement in higher education institutes in recent years, as even with some limitations and inconsistency, the sexual assault reporting system is gradually becoming more rigorous BIBREF17. Meanwhile, attending colleges in the Northeast, West and South regions increases the possibility of posting about sexual harassment (positive coefficients), over the Midwest region. This finding is interesting and warrants further scrutiny.

Answers:
Northeast U.S., West U.S. and South U.S.
Question: How many followers did they analyze?

Evidence:
raw_evidence: In this study, we limit the sample size to the followers identified as English speakers in the U.S. News Top 200 National Universities. We utilize the Jefferson-Henrique script, a web scraper designed for Twitter to retrieve a total of over 300,000 #MeToo tweets from October 15th, when Alyssa Milano posted the inceptive #MeToo tweet, to November 15th of 2017 to cover a period of a month when the trend was on the rise and attracting mass concerns. Since the lists of the followers of the studied colleges might overlap and many Twitter users tend to reiterate other's tweets, simply putting all the data collected together could create a major redundancy problem. We extract unique users and tweets from the combined result set to generate a dataset of about 60,000 unique tweets, pertaining to 51,104 unique users.

Answers:
51,104
Question: What two components are included in their proposed framework?

Evidence:
raw_evidence: In this paper, we present an extraction-then-synthesis framework for machine reading comprehension shown in Figure 1 , in which the answer is synthesized from the extraction results. We build an evidence extraction model to predict the most important sub-spans from the passages as evidence, and then develop an answer synthesis model which takes the evidence as additional features along with the question and passage to further elaborate the final answers.

Answers:
evidence extraction and answer synthesis
Question: Which framework they propose in this paper?

Evidence:
raw_evidence: In this paper, we present an extraction-then-synthesis framework for machine reading comprehension shown in Figure 1 , in which the answer is synthesized from the extraction results. We build an evidence extraction model to predict the most important sub-spans from the passages as evidence, and then develop an answer synthesis model which takes the evidence as additional features along with the question and passage to further elaborate the final answers.

Answers:
 extraction-then-synthesis framework