qinchuanhui/UDA-QA huggingface dataset:
Question: So this paper turns unstructured text inputs to parameters that GNNs can read?

Evidence:
raw_evidence: In this section, we will introduce the general framework of GP-GNNs. GP-GNNs first build a fully-connected graph $\mathcal {G} = (\mathcal {V}, \mathcal {E})$ , where $\mathcal {V}$ is the set of entities, and each edge $(v_i, v_j) \in \mathcal {E}, v_i, v_j \in \mathcal {V}$ corresponds to a sequence $s = x_0^{i,j}, x_1^{i,j}, \dots , x_{l-1}^{i,j}$ extracted from the text. After that, GP-GNNs employ three modules including (1) encoding module, (2) propagation module and (3) classification module to proceed relational reasoning, as shown in Fig. 2 .

Answers:
Yes
Question: What empirical data are the Blending Game predictions compared to?

Evidence:
raw_evidence: In this paper we have investigated duality of patterning at the lexicon level. We have quantified in particular the notions of combinatoriality and compositionality as observed in real languages as well as in a large-scale dataset produced in the framework of a web-based word association experiment BIBREF1 . We have paralleled this empirical analysis with a modeling scheme, the Blending Game, whose aim is that of identifying the main determinants for the emergence of duality of patterning in language. We analyzed the main properties of the lexicon emerged from the Blending Game as a function of the two parameters of the model, the graph connectivity $p_{link}$ and the memory scale $\tau $ . We found that properties of the emerging lexicon related to the combinatoriality, namely the words length distribution, the frequency of use of the different forms and a measure for the combinatoriality itself, reflect both qualitatively and quantitatively the corresponding properties as measured in human languages, provided that the memory parameter $\tau $ is sufficiently high, that is that a sufficiently high effort is required in order to understand and learn brand new forms. Conversely, the compositional properties of the lexicon are related to the parameter $p_{link}$ , that is a measure of the level of structure of the conceptual graph. For intermediate and low values of $p_{link}$ , semantic relations between objects are more differentiated with respect to the situation of a more dense graph, in which every object is related to anyone else, and compositionality is enhanced. In summary, while the graph connectivity strongly affects the compositionality of the lexicon, noise in communication strongly affects the combinatoriality of the lexicon.

Answers:
words length distribution, the frequency of use of the different forms and a measure for the combinatoriality
Question: Does the paper report translation accuracy for an automatic translation model for Tunisian to Arabish words?

Evidence:
raw_evidence: The 10-fold cross validation with this setting gave a token-level accuracy of roughly 71%. This result is not satisfactory on an absolute scale, however it is more than encouraging taking into account the small size of our data. This result means that less than 3 tokens, on average, out of 10, must be corrected to increase the size of our corpus. With this model we automatically transcribed into Arabic morphemes, roughly, 5,000 additional tokens, corresponding to the second annotation block. This can be manually annotated in at least 7,5 days, but thanks to the automatic annotation accuracy, it was manually corrected into 3 days. The accuracy of the model on the annotation of the second block was roughly 70%, which corresponds to the accuracy on the test set. The manually-corrected additional tokens were added to the training data of our neural model, and a new block was automatically annotated and manually corrected. Both accuracy on the test set and on the annotation block remained at around 70%. This is because the block added to the training data was significantly different from the previous and from the third. Adding the third block to the training data and annotating a fourth block with the new trained model gave in contrast an accuracy of roughly 80%. This incremental, semi-automatic transcription procedure is in progress for the remaining blocks, but it is clear that it will make the corpus annotation increasingly easier and faster as the amount of training data will grow up.

Answers:
Yes
Question: Was this experiment done in a lab?

Evidence:
raw_evidence: We recruited 102 pairs of participants from Amazon Mechanical Turk and randomly assigned speaker and listener roles. After we removed 7 games that disconnected part-way through and 12 additional games according to our pre-registered exclusion criteria (due to being non-native English speakers, reporting confusion about the instructions, or clearly violating the instructions), we were left with a sample of 83 full games.

Answers:
No
Question: How does this compare to contextual embedding methods?

Evidence:
raw_evidence: In this paper, we propose to represent each word with an expressive multimodal distribution, for multiple distinct meanings, entailment, heavy tailed uncertainty, and enhanced interpretability. For example, one mode of the word `bank' could overlap with distributions for words such as `finance' and `money', and another mode could overlap with the distributions for `river' and `creek'. It is our contention that such flexibility is critical for both qualitatively learning about the meanings of words, and for optimal performance on many predictive tasks.

Answers:
 represent each word with an expressive multimodal distribution, for multiple distinct meanings, entailment, heavy tailed uncertainty, and enhanced interpretability. For example, one mode of the word `bank' could overlap with distributions for words such as `finance' and `money', and another mode could overlap with the distributions for `river' and `creek'.
Question: Does the new system utilize pre-extracted bounding boxes and/or features?

Evidence:
raw_evidence: In this section, we elaborate our model consisting of four parts: (a) image feature pre-selection part which models the tendency where people focus to ask questions, (b) question encoding part which encodes the question words as a condensed semantic embedding, (c) attention-based feature fusion part performs second selection on image features and (d) answer generation part which gives the answer output.
We propose to perform saliency-like pre-selection operation to alleviate the problems and model the RoI patterns. The image is first divided into $g\times g$ grids as illustrated in Figure. 2 . Taking $m\times m$ grids as a region, with $s$ grids as the stride, we obtain $n\times n$ regions, where $n=\left\lfloor \frac{g-m}{s}\right\rfloor +1$ . We then feed the regions to a pre-trained ResNet BIBREF24 deep convolutional neural network to produce $n\times n\times d_I$ -dimensional region features, where $d_I$ is the dimension of feature from the layer before the last fully-connected layer.

Answers:
Yes
Question: To which previous papers does this work compare its results?

Evidence:
raw_evidence: holistic: The baseline model which maps the holistic image feature and LSTM-encoded question feature to a common space and perform element-wise multiplication between them.
TraAtt: The traditional attention model, implementation of WTL model BIBREF9 using the same $3\times 3$ regions in SalAtt model.
RegAtt: The region attention model which employs our novel attention method, same as the SalAtt model but without region pre-selection.
ConAtt: The convolutional region pre-selection attention model which replaces the BiLSTM in SalAtt model with a weight-sharing linear mapping, implemented by a convolutional layer.
Besides, we also compare our SalAtt model with the popular baseline models i.e. iBOWIMG BIBREF4 , VQA BIBREF1 , and the state-of-the-art attention-based models i.e. WTL BIBREF9 , NMN BIBREF21 , SAN BIBREF14 , AMA BIBREF33 , FDA BIBREF34 , D-NMN BIBREF35 , DMN+ BIBREF8 on two tasks of COCO-VQA.

Answers:
holistic, TraAtt, RegAtt, ConAtt, ConAtt, iBOWIMG , VQA, VQA, WTL , NMN , SAN , AMA , FDA , D-NMN, DMN+
Question: Do they consider other tasks?

Evidence:

Answers:
No
Question: What were the model's results on flood detection?

Evidence:
raw_evidence: We have used the following hardware for the experimentation: Windows 10 Education desktop consisting of intel core i-7 processor and 16GB RAM. We have used python 3.6 and Google colab notebook to execute our model and obtained the results discussed below: The train and test data have divided into 70-30 ratio and we got these results as shown in Table TABREF17 for the individual dataset and the combination of both. The pre-trained network was already trained and we used the target data Queensland flood which provided 96% accuracy with 0.118 Test loss in only 11 seconds provided we used only 70% of training labeled data. The second target data is Alberta flood with the same configuration of train-test split which provided 95% accuracy with 0.118 Test loss in just 19 seconds. As we can see it takes very less time to work with 20,000 of tweets (combined) and at times of emergency it can handle a huge amount of unlabeled data to classify into meaningful categories in minutes.

Answers:
Queensland flood which provided 96% accuracy, Alberta flood with the same configuration of train-test split which provided 95% accuracy
Question: What dataset did they use?

Evidence:
raw_evidence: Data Collection: We are using the disaster data from BIBREF5. It contains various dataset including the CrisiLexT6 dataset which contains six crisis events related to English tweets in 2012 and 2013, labeled by relatedness (on-topic and off-topic) of respective crisis. Each crisis event tweets contain almost 10,000 labeled tweets but we are only focused on flood-related tweets thus, we experimented with only two flood event i.e. Queensland flood in Queensland, Australia and Alberta flood in Alberta, Canada and relabeled all on-topic tweets as Related and Off-topic as Unrelated for implicit class labels understanding in this case. The data collection process and duration of CrisisLex data is described in BIBREF5 details.

Answers:
 disaster data from BIBREF5, Queensland flood in Queensland, Australia and Alberta flood in Alberta, Canada
Question: What exactly is new about this stochastic gradient descent algorithm?

Evidence:
raw_evidence: DNNs are usually trained with first-order online methods like stochastic gradient descent (SGD). This method yields a crucial advantage in crisis situations, where retraining the whole model each time a small batch of labeled data arrives is impractical. Algorithm "Online Learning" demonstrates how our CNN model can be trained in a purely online setting. We first initialize the model parameters $\theta _0$ (line 1), which can be a trained model from other disaster events or it can be initialized randomly to start from scratch.
As a new batch of labeled tweets $B_t= \lbrace \mathbf {s}_1 \ldots \mathbf {s}_n \rbrace $ arrives, we first compute the log-loss (cross entropy) in Equation 11 for $B_t$ with respect to the current parameters $\theta _t$ (line 2a). Then, we use backpropagation to compute the gradients $f^{\prime }(\theta _{t})$ of the loss with respect to the current parameters (line 2b). Finally, we update the parameters with the learning rate $\eta _t$ and the mean of the gradients (line 2c). We take the mean of the gradients to deal with minibatches of different sizes. Notice that we take only the current minibatch into account to get an updated model. Choosing a proper learning rate $\eta _t$ can be difficult in practice. Several adaptive methods such as ADADELTA BIBREF6 , ADAM BIBREF7 , etc., have been proposed to overcome this issue. In our model, we use ADADELTA.

Answers:
CNN model can be trained in a purely online setting. We first initialize the model parameters $\theta _0$ (line 1), which can be a trained model from other disaster events or it can be initialized randomly to start from scratch.

As a new batch of labeled tweets $B_t= \lbrace \mathbf {s}_1 \ldots \mathbf {s}_n \rbrace $ arrives, we first compute the log-loss (cross entropy) in Equation 11 for $B_t$ with respect to the current parameters $\theta _t$ (line 2a). Then, we use backpropagation to compute the gradients $f^{\prime }(\theta _{t})$ of the loss with respect to the current parameters (line 2b). Finally, we update the parameters with the learning rate $\eta _t$ and the mean of the gradients (line 2c). We take the mean of the gradients to deal with minibatches of different sizes. Notice that we take only the current minibatch into account to get an updated model.
Question: What codemixed language pairs are evaluated?

Evidence:
raw_evidence: We use the Universal Dependencies' Hindi-English codemixed data set BIBREF9 to test the model's ability to label code-mixed data. This dataset is based on code-switching tweets of Hindi and English multilingual speakers. We use the Devanagari script provided by the data set as input tokens.

Answers:
Hindi-English
Question: How do they compress the model?

Evidence:
raw_evidence: For model distillation BIBREF6 , we extract sentences from Wikipedia in languages for which public multilingual is pretrained. For each sentence, we use the open-source BERT wordpiece tokenizer BIBREF4 , BIBREF1 and compute cross-entropy loss for each wordpiece: INLINEFORM0
where INLINEFORM0 is the cross-entropy function, INLINEFORM1 is the softmax function, INLINEFORM2 is the BERT model's logit of the current wordpiece, INLINEFORM3 is the small BERT model's logits and INLINEFORM4 is a temperature hyperparameter, explained in Section SECREF11 .
To train the distilled multilingual model mMiniBERT, we first use the distillation loss above to train the student from scratch using the teacher's logits on unlabeled data. Afterwards, we finetune the student model on the labeled data the teacher is trained on.

Answers:
we extract sentences from Wikipedia in languages for which public multilingual is pretrained. For each sentence, we use the open-source BERT wordpiece tokenizer BIBREF4 , BIBREF1 and compute cross-entropy loss for each wordpiece: INLINEFORM0
Question: What is the multilingual baseline?

Evidence:
raw_evidence: We discuss two core models for addressing sequence labeling problems and describe, for each, training them in a single-model multilingual setting: (1) the Meta-LSTM BIBREF0 , an extremely strong baseline for our tasks, and (2) a multilingual BERT-based model BIBREF1 .

Answers:
 the Meta-LSTM BIBREF0
Question: Which features do they use?

Evidence:
raw_evidence: In this paper, we present the problem of DAR from the viewpoint of extending richer CRF-attentive structural dependencies along with neural network without abandoning end-to-end training. For simplicity, we call the framework as CRF-ASN (CRF-Attentive Structured Network). Specifically, we propose the hierarchical semantic inference integrated with memory mechanism on the utterance modeling. The memory mechanism is adopted in order to enable the model to look beyond localized features and have access to the entire sequence. The hierarchical semantic modeling learns different levels of granularity including word level, utterance level and conversation level. We then develop internal structured attention network on the linear-chain conditional random field (CRF) to specify structural dependencies in a soft manner. This approach generalizes the soft-selection attention on the structural CRF dependencies and takes into account the contextual influence on the nearing utterances. It is notably that the whole process is differentiable thus can be trained in an end-to-end manner.

Answers:
beyond localized features and have access to the entire sequence
Question: By how much do they outperform state-of-the-art solutions on SWDA and MRDA?

Evidence:
raw_evidence: The results show that our proposed model CRF-ASN obviously outperforms the state-of-the-art baselines on both SwDA and MRDA datasets. Numerically, Our model improves the DAR accuracy over Bi-LSTM-CRF by 2.1% and 0.8% on SwDA and MRDA respectively. It is remarkable that our CRF-ASN method is nearly close to the human annotators' performance on SwDA, which is very convincing to prove the superiority of our model.

Answers:
improves the DAR accuracy over Bi-LSTM-CRF by 2.1% and 0.8% on SwDA and MRDA respectively
Question: What type and size of word embeddings were used?

Evidence:
raw_evidence: We used the public tool, word2vec, released by Mikolov-2013 to obtain the word embeddings. Their neural network approach is similar to the feed-forward neural networks BIBREF5 , BIBREF6 . To be more precise, the previous words to the current word are encoded in the input layer and then projected to the projection layer with a shared projection matrix. After that, the projection is given to the non-linear hidden layer and then the output is given to softmax in order to receive a probability distribution over all the words in the vocabulary. However, as suggested by Mikolov-2013, removing the non-linear hidden layer and making the projection layer shared by all words is much faster, which allowed us to use a larger unlabeled corpus and obtain better word embeddings.
Among the methods presented in Mikolov-2013, we used the continuous Skip-gram model to obtain semantic representations of Turkish words. The Skip-gram model uses the current word as an input to the projection layer with a log-linear classifier and attempts to predict the representation of neighboring words within a certain range. In the Skip-gram model architecture we used, we have chosen 200 as the dimension of the obtained word vectors. The range of surrounding words is chosen to be 5, so that we will predict the distributed representations of the previous 2 words and the next 2 words using the current word. Our vector size and range decisions are aligned with the choices made in the previous study for Turkish NER by Demir-2014. The Skip-gram model architecture we used is shown in Figure FIGREF3 .

Answers:
word2vec, 200 as the dimension of the obtained word vectors
Question: What data was used to build the word embeddings?

Evidence:
raw_evidence: In the unsupervised stage, we used two types of unlabeled data to obtain Turkish word embeddings. The first one is a Turkish news-web corpus containing 423M words and 491M tokens, namely the BOUN Web Corpus BIBREF9 , BIBREF10 . The second one is composed of 21M Turkish tweets with 241M words and 293M tokens, where we combined 1M tweets from TS TweetS by Sezer-2013 and 20M Turkish Tweets by Bolat and Amasyalı.

Answers:
Turkish news-web corpus,  TS TweetS by Sezer-2013 and 20M Turkish Tweets by Bolat and Amasyalı
Question: How are templates discovered from training data?

Evidence:
raw_evidence: Our framework includes three key modules: Retrieve, Fast Rerank, and BiSET. For each source article, Retrieve aims to return a few candidate templates from the training corpus. Then, the Fast Rerank module quickly identifies a best template from the candidates. Finally, BiSET mutually selects important information from the source article and the template to generate an enhanced article representation for summarization.
This module starts with a standard information retrieval library to retrieve a small set of candidates for fine-grained filtering as cao2018retrieve. To do that, all non-alphabetic characters (e.g., dates) are removed to eliminate their influence on article matching. The retrieval process starts by querying the training corpus with a source article to find a few (5 to 30) related articles, the summaries of which will be treated as candidate templates.
The above retrieval process is essentially based on superficial word matching and cannot measure the deep semantic relationship between two articles. Therefore, the Fast Rerank module is developed to identify a best template from the candidates based on their deep semantic relevance with the source article. We regard the candidate with highest relevance as the template. As illustrated in Figure FIGREF6 , this module consists of a Convolution Encoder Block, a Similarity Matrix and a Pooling Layer.

Answers:
For each source article, Retrieve aims to return a few candidate templates from the training corpus. Then, the Fast Rerank module quickly identifies a best template from the candidates.
Question: What is WNGT 2019 shared task?

Evidence:
raw_evidence: The Transformer network BIBREF3 is a neural sequence-to-sequence model that has achieved state-of-the-art results in machine translation. However, Transformer models tend to be very large, typically consisting of hundreds of millions of parameters. As the number of parameters directly corresponds to secondary storage requirements and memory consumption during inference, using Transformer networks may be prohibitively expensive in scenarios with constrained resources. For the 2019 Workshop on Neural Generation of Text (WNGT) Efficiency shared task BIBREF0, the Notre Dame Natural Language Processing (NDNLP) group looked at a method of inducing sparsity in parameters called auto-sizing in order to reduce the number of parameters in the Transformer at the cost of a relatively minimal drop in performance.

Answers:
efficiency task aimed  at reducing the number of parameters while minimizing drop in performance