qinchuanhui/UDA-QA huggingface dataset:
Question: Do the images have multilingual annotations or monolingual ones?

Evidence:
raw_evidence: We experiment using a dataset derived from Google Images search results. The dataset consists of queries and the corresponding image search results. For example, one (query, image) pair might be “cat with big ears” and an image of a cat. Each (query, image) pair also has a weight corresponding to a relevance score of the image for the query. The dataset includes 3 billion (query, image, weight) triples, with 900 million unique images and 220 million unique queries. The data was prepared by first taking the query-image set, filtering to remove any personally identifiable information and adult content, and tokenizing the remaining queries by replacing special characters with spaces and trimming extraneous whitespace. Rare tokens (those that do not appear in queries at least six times) are filtered out. Each token in each query is given a language tag based on the user-set home language of the user making the search on Google Images. For example, if the query “back pain” is made by a user with English as her home language, then the query is stored as “en:back en:pain”. The dataset includes queries in about 130 languages.

Answers:
monolingual
Question: Could you learn such embedding simply from the image annotations and without using visual information?

Evidence:
raw_evidence: Another approach for generating query and image representations is treating images as a black box. Without using pixel data, how well can we do? Given the statistics of our dataset (3B query, image pairs with 220M unique queries and 900M unique images), we know that different queries co-occur with the same images. Intuitively, if a query $q_1$ co-occurs with many of the same images as query $q_2$ , then $q_1$ and $q_2$ are likely to be semantically similar, regardless of the visual content of the shared images. Thus, we can use a method that uses only co-occurrence statistics to better understand how well we can capture relationships between queries. This method serves as a baseline to our initial approach leveraging image understanding.

Answers:
Yes
Question: How is the generative model evaluated?

Evidence:
raw_evidence: We train an LSTM with and without attention on the training set. After training, we take the best model in terms of BLEU score BIBREF16 on the development set and calculate the BLEU score on the test set. To our surprise, we found that using attention yields only a marginally higher BLEU score (43.1 vs. 42.8). We suspect that this is due to the fact that generating entailed sentences has a larger space of valid target sequences, which makes the use of BLEU problematic and penalizes correct solutions. Hence, we manually annotated 100 random test sentences and decided whether the generated sentence can indeed be inferred from the source sentence. We found that sentences generated by an LSTM with attention are substantially more accurate ( $82\%$ accuracy) than those generated from an LSTM baseline ( $71.7\%$ ). To gain more insights into the model's capabilities, we turn to a thorough qualitative analysis of the attention LSTM model in the remainder of this paper.

Answers:
Comparing BLEU score of model with and without attention
Question: How do they evaluate their method?

Evidence:
raw_evidence: In order to fairly compare and evaluate the proposed methods in terms of effectiveness in representation of tweets, we fix the number of features to 24 for all methods and feed these representations as an input to 3 different clustering algorithms namely, k-means, Ward and spectral clustering with cluster numbers of 10, 20 and 50. Distance metric for k-means clustering is chosen to be euclidean and the linkage criteria for Ward clustering is chosen to be minimizing the sum of differences within all clusters, i.e., recursively merging pairs of clusters that minimally increases the within-cluster variance in a hierarchical manner. For spectral clustering, Gaussian kernel has been employed for constructing the affinity matrix. We also run experiments with tf-idf and BoWs representations without further dimensionality reduction as well as concatenation of all word embeddings into a long feature vector. For evaluation of clustering performance, we use Calinski-Harabasz score BIBREF42 , also known as the variance ratio criterion. CH score is defined as the ratio between the within-cluster dispersion and the between-cluster dispersion. CH score has a range of $[0, +\infty ]$ and a higher CH score corresponds to a better clustering. Computational complexity of calculating CH score is $\mathcal {O}(N)$ .
For visual validation, we plot and inspect the t-Distributed Stochastic Neighbor Embedding (t-SNE) BIBREF52 and Uniform Manifold Approximation and Projection (UMAP) BIBREF53 mappings of the learned representations as well. Implementation of this study is done in Python (version 3.6) using scikit-learn and TensorFlow libraries BIBREF54 , BIBREF55 on a 64-bit Ubuntu 16.04 workstation with 128 GB RAM. Training of autoencoders are performed with a single NVIDIA Titan Xp GPU.

Answers:
Calinski-Harabasz score, t-SNE, UMAP
Question: What is an example of a health-related tweet?

Evidence:
raw_evidence: FLOAT SELECTED: Fig. 1. Proposed representation learning method depicting the overall flow starting from a tweet to the learned features, including the architecture of the convolutional autoencoder.
FLOAT SELECTED: Fig. 1. Proposed representation learning method depicting the overall flow starting from a tweet to the learned features, including the architecture of the convolutional autoencoder.

Answers:
The health benefits of alcohol consumption are more limited than previously thought, researchers say
Question: Was the introduced LSTM+CNN model trained on annotated data in a supervised fashion?

Evidence:
raw_evidence: We train our models on Sentiment140 and Amazon product reviews. Both of these datasets concentrates on sentiment represented by a short text. Summary description of other datasets for validation are also as below:
We have provided baseline results for the accuracy of other models against datasets (as shown in Table 1 ) For training the softmax model, we divide the text sentiment to two kinds of emotion, positive and negative. And for training the tanh model, we convert the positive and negative emotion to [-1.0, 1.0] continuous sentiment score, while 1.0 means positive and vice versa. We also test our model on various models and calculate metrics such as accuracy, precision and recall and show the results are in Table 2 . Table 3 , Table 4 , Table 5 , Table 6 and Table 7 . Table 8 are more detail information with precisions and recall of our models against other datasets.

Answers:
Yes
Question: What is the challenge for other language except English

Evidence:
raw_evidence: Given the fact that the research on offensive language detection has to a large extent been focused on the English language, we set out to explore the design of models that can successfully be used for both English and Danish. To accomplish this, an appropriate dataset must be constructed, annotated with the guidelines described in BIBREF0 . We, furthermore, set out to analyze the linguistic features that prove hard to detect by analyzing the patterns that prove hard to detect.

Answers:
not researched as much as English
Question: How many categories of offensive language were there?

Evidence:
raw_evidence: In sub-task C the goal is to classify the target of the offensive language. Only posts labeled as targeted insults (TIN) in sub-task B are considered in this task BIBREF17 . Samples are annotated with one of the following:
Individual (IND): Posts targeting a named or unnamed person that is part of the conversation. In English this could be a post such as @USER Is a FRAUD Female @USER group paid for and organized by @USER. In Danish this could be a post such as USER du er sku da syg i hoved. These examples further demonstrate that this category captures the characteristics of cyberbullying, as it is defined in section "Background" .
Group (GRP): Posts targeting a group of people based on ethnicity, gender or sexual orientation, political affiliation, religious belief, or other characteristics. In English this could be a post such as #Antifa are mentally unstable cowards, pretending to be relevant. In Danish this could be e.g. Åh nej! Svensk lorteret!
Other (OTH): The target of the offensive language does not fit the criteria of either of the previous two categories. BIBREF17 . In English this could be a post such as And these entertainment agencies just gonna have to be an ass about it.. In Danish this could be a post such as Netto er jo et tempel over lort.

Answers:
3
Question: How large was the dataset of Danish comments?

Evidence:
raw_evidence: We published a survey on Reddit asking Danish speaking users to suggest offensive, sexist, and racist terms for a lexicon. Language and user behaviour varies between platforms, so the goal is to capture platform-specific terms. This gave 113 offensive and hateful terms which were used to find offensive comments. The remainder of comments in the corpus were shuffled and a subset of this corpus was then used to fill the remainder of the final dataset. The resulting dataset contains 3600 user-generated comments, 800 from Ekstra Bladet on Facebook, 1400 from r/DANMAG and 1400 from r/Denmark. In light of the General Data Protection Regulations in Europe (GDPR) and the increased concern for online privacy, we applied some necessary pre-processing steps on our dataset to ensure the privacy of the authors of the comments that were used. Personally identifying content (such as the names of individuals, not including celebrity names) was removed. This was handled by replacing each name of an individual (i.e. author or subject) with @USER, as presented in both BIBREF0 and BIBREF2 . All comments containing any sensitive information were removed. We classify sensitive information as any information that can be used to uniquely identify someone by the following characteristics; racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership, genetic data, and bio-metric data.

Answers:
3600 user-generated comments
Question: Who were the annotators?

Evidence:
raw_evidence: We base our annotation procedure on the guidelines and schemas presented in BIBREF0 , discussed in detail in section "Classification Structure" . As a warm-up procedure, the first 100 posts were annotated by two annotators (the author and the supervisor) and the results compared. This was used as an opportunity to refine the mutual understanding of the task at hand and to discuss the mismatches in these annotations for each sub-task.
In light of these findings our internal guidelines were refined so that no post should be labeled as offensive by interpreting any context that is not directly visible in the post itself and that any post containing any form of profanity should automatically be labeled as offensive. These stricter guidelines made the annotation procedure considerably easier while ensuring consistency. The remainder of the annotation task was performed by the author, resulting in 3600 annotated samples.

Answers:
the author and the supervisor
Question: Is is known whether Sina Weibo posts are censored by humans or some automatic classifier?

Evidence:
raw_evidence: In cooperation with the ruling regime, Weibo sets strict control over the content published under its service BIBREF0. According to Zhu et al. zhu-etal:2013, Weibo uses a variety of strategies to target censorable posts, ranging from keyword list filtering to individual user monitoring. Among all posts that are eventually censored, nearly 30% of them are censored within 5–30 minutes, and nearly 90% within 24 hours BIBREF1. We hypothesize that the former are done automatically, while the latter are removed by human censors.

Answers:
No
Question: Which matching features do they employ?

Evidence:
raw_evidence: In this paper, we propose the MIMN model for NLI task. Our model introduces a multi-turns inference mechanism to process multi-perspective matching features. Furthermore, the model employs the memory mechanism to carry proceeding inference information. In each turn, the inference is based on the current matching feature and previous memory. Experimental results on SNLI dataset show that the MIMN model is on par with the state-of-the-art models. Moreover, our model achieves new state-of-the-art results on the MPE and the SCITAL datasets. Experimental results prove that the MIMN model can extract important information from multiple premises for the final judgment. And the model is good at handling the relationships of entailment and contradiction.

Answers:
Matching features from matching sentences from various perspectives.
Question: How often are the newspaper websites crawled daily?

Evidence:
raw_evidence: The Logoscope retrieves newspaper articles from several RSS feeds in French on a daily basis. The newspaper articles are preprocessed such that only the journalistic content is kept. The articles are then segmented into paragraphs and word forms. The resulting forms are filtered based on an exclusion list (French words found in several lexicons and corpora). They are then reordered in such a way that those words which are the most likely new word candidates appear on top, using a supervised classification method which will be described more in detail in Section SECREF71 .

Answers:
RSS feeds in French on a daily basis
Question: Which countries and languages do the political speeches and manifestos come from?

Evidence:
raw_evidence: Ideally one would choose for each topic a sample of reports from the entire political spectrum in order to form an unbiased opinion. But ordering media content with respect to the political spectrum at scale requires automated prediction of political bias. The aim of this study is to provide empirical evidence indicating that leveraging open data sources of german texts, automated political bias prediction is possible with above chance accuracy. These experimental results confirm and extend previous findings BIBREF0 , BIBREF1 ; a novel contribution of this work is a proof of concept which applies this technology to sort news article recommendations according to their political bias.

Answers:
german
Question: Do changes in policies of the political actors account for all of the mistakes the model made?

Evidence:
raw_evidence: In order to investigate the errors the models made confusion matrices were extracted for the predictions on the out-of-domain evaluation data for sentence level predictions (see tab:confusion) as well as topic level predictions (see tab:confusiontopic). One example illustrates that the mistakes the model makes can be associated with changes in the party policy. The green party has been promoting policies for renewable energy and against nuclear energy in their manifestos prior to both legislative periods. Yet the statements of the green party are more often predicted to be from the government parties than from the party that originally promoted these green ideas, reflecting the trend that these legislative periods governing parties took over policies from the green party. This effect is even more pronounced in the topic level predictions: a model trained on data from the 18th Bundestag predicts all manifesto topics of the green party to be from one of the parties of the governing coalition, CDU/CSU or SPD.

Answers:
Yes
Question: What model are the text features used in to provide predictions?

Evidence:
raw_evidence: Bag-of-words feature vectors were used to train a multinomial logistic regression model. Let INLINEFORM0 be the true label, where INLINEFORM1 is the total number of labels and INLINEFORM2 is the concatenation of the weight vectors INLINEFORM3 associated with the INLINEFORM4 th party then DISPLAYFORM0

Answers:
 multinomial logistic regression
Question: Does each attention head in the decoder calculate the same output?

Evidence:
raw_evidence: On the other hand, in the case of MHD, instead of the integration at attention level, we assign multiple decoders for each head and then integrate their outputs to get a final output. Since each attention decoder captures different modalities, it is expected to improve the recognition performance with an ensemble effect. The calculation of the attention weight at the head INLINEFORM0 in Eq. ( EQREF21 ) is replaced with following equation: DISPLAYFORM0

Answers:
No
Question: Which distributional methods did they consider?

Evidence:
raw_evidence: Most unsupervised distributional approaches for hypernymy detection are based on variants of the Distributional Inclusion Hypothesis BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , BIBREF4 . Here, we compare to two methods with strong empirical results. As with most DIH measures, they are only defined for large, sparse, positively-valued distributional spaces. First, we consider WeedsPrec BIBREF8 which captures the features of INLINEFORM0 which are included in the set of a broader term's features, INLINEFORM1 : DISPLAYFORM0
Second, we consider invCL BIBREF11 which introduces a notion of distributional exclusion by also measuring the degree to which the broader term contains contexts not used by the narrower term. In particular, let INLINEFORM0
Although most unsupervised distributional approaches are based on the DIH, we also consider the distributional SLQS model based on on an alternative informativeness hypothesis BIBREF10 , BIBREF4 . Intuitively, the SLQS model presupposes that general words appear mostly in uninformative contexts, as measured by entropy. Specifically, SLQS depends on the median entropy of a term's top INLINEFORM0 contexts, defined as INLINEFORM1
For completeness, we also include cosine similarity as a baseline in our evaluation.

Answers:
WeedsPrec BIBREF8, invCL BIBREF11, SLQS model, cosine similarity
Question: Do they repot results only on English data?

Evidence:
raw_evidence: We evaluate the proposed architecture on two publicly available datasets: the Adverse Drug Events (ADE) dataset BIBREF6 and the CoNLL04 dataset BIBREF7. We show that our architecture is able to outperform the current state-of-the-art (SOTA) results on both the NER and RE tasks in the case of ADE. In the case of CoNLL04, our proposed architecture achieves SOTA performance on the NER task and achieves near SOTA performance on the RE task. On both datasets, our results are SOTA when averaging performance across both tasks. Moreover, we achieve these results using an order of magnitude fewer trainable parameters than the current SOTA architecture.

Answers:
Yes
Question: What were the variables in the ablation study?

Evidence:
raw_evidence: To further demonstrate the effectiveness of the additional task-specific BiRNN layers in our architecture, we conducted an ablation study using the CoNLL04 dataset. We trained and evaluated in the same manner described above, using the same hyperparameters, with the following exceptions:
We used either (i) zero NER-specific BiRNN layers, (ii) zero RE-specific BiRNN layers, or (iii) zero task-specific BiRNN layers of any kind.
We increased the number of shared BiRNN layers to keep the total number of model parameters consistent with the number of parameters in the baseline model.
We average the results for each set of hyperparameter across three trials with random weight initializations.

Answers:
(i) zero NER-specific BiRNN layers, (ii) zero RE-specific BiRNN layers, or (iii) zero task-specific BiRNN layers of any kind
Question: What is barycentric Newton diagram?

Evidence:
raw_evidence: By using the probability values that emerge from the activation function in the neural network, rather than just the final classification, we can draw a barycentric Newton diagram, as shown in Figure 4 . The basic idea of the visualization, drawing on Isaac Newton’s visualization of the color spectrum BIBREF8 , is to express a mixture in terms of its constituents as represented in barycentric coordinates. This visualization allows an intuitive interpretation of which country a recipe belongs to. If the probability of Japanese is high, the recipe is mapped near the Japanese. The countries on the Newton diagram are placed by spectral graph drawing BIBREF9 , so that similar countries are placed nearby on the circle. The calculation is as follows. First we define the adjacency matrix $W$ as the similarity between two countries. The similarity between country $i$ and $j$ is calculated by cosine similarity of county $i$ vector and $j$ vector. These vector are defined in next section. $W_{ij} = sim(vec_i, vec_j)$ . The degree matrix $D$ is a diagonal matrix where $D_{ii} = \sum _{j} W_{ij}$ . Next we calculate the eigendecomposition of $D^{-1}W$ . The second and third smallest eingenvalues and corresponded eingevectors are used for placing the countries. Eigenvectors are normalized so as to place the countries on the circle.

Answers:
 The basic idea of the visualization, drawing on Isaac Newton’s visualization of the color spectrum BIBREF8 , is to express a mixture in terms of its constituents as represented in barycentric coordinates.
Question: Do they propose any solution to debias the embeddings?

Evidence:
raw_evidence: We have presented the first study on social bias in KG embeddings, and proposed a new metric for measuring such bias. We demonstrated that differences in the distributions of entities in real-world knowledge graphs (there are many more male bankers in Wikidata than female) translate into harmful biases related to professions being encoded in embeddings. Given that KGs are formed of real-world entities, we cannot simply equalize the counts; it is not possible to correct history by creating female US Presidents, etc. In light of this, we suggest that care is needed when applying graph embeddings in NLP pipelines, and work needed to develop robust methods to debias such embeddings.

Answers:
No