qinchuanhui/UDA-QA huggingface dataset:
Question: Which datasets did they use?

Evidence:
raw_evidence: Stanford - Twitter Sentiment Corpus (STS Corpus): STS Corpus contains 1,600K training tweets collected by a crawler from BIBREF0 . BIBREF0 constructed a test set manually with 177 negative and 182 positive tweets. The Stanford test set is small. However, it has been widely used in different evaluation tasks BIBREF0 BIBREF5 BIBREF13 .
Sanders - Twitter Sentiment Corpus: This dataset consists of hand-classified tweets collected by using search terms: INLINEFORM0 , #google, #microsoft and #twitter. We construct the dataset as BIBREF14 for binary classification.
Health Care Reform (HCR): This dataset was constructed by crawling tweets containing the hashtag #hcr BIBREF15 . Task is to predict positive/negative tweets BIBREF14 .

Answers:
Stanford - Twitter Sentiment Corpus (STS Corpus), Sanders - Twitter Sentiment Corpus, Health Care Reform (HCR)
Question: Are results reported only on English datasets?

Evidence:
raw_evidence: Stanford - Twitter Sentiment Corpus (STS Corpus): STS Corpus contains 1,600K training tweets collected by a crawler from BIBREF0 . BIBREF0 constructed a test set manually with 177 negative and 182 positive tweets. The Stanford test set is small. However, it has been widely used in different evaluation tasks BIBREF0 BIBREF5 BIBREF13 .
Sanders - Twitter Sentiment Corpus: This dataset consists of hand-classified tweets collected by using search terms: INLINEFORM0 , #google, #microsoft and #twitter. We construct the dataset as BIBREF14 for binary classification.
Health Care Reform (HCR): This dataset was constructed by crawling tweets containing the hashtag #hcr BIBREF15 . Task is to predict positive/negative tweets BIBREF14 .
Table IV shows the result of our model for sentiment classification against other models. We compare our model performance with the approaches of BIBREF0 BIBREF5 on STS Corpus. BIBREF0 reported the results of Maximum Entropy (MaxEnt), NB, SVM on STS Corpus having good performance in previous time. The model of BIBREF5 is a state-of-the-art so far by using a CharSCNN. As can be seen, 86.63 is the best prediction accuracy of our model so far for the STS Corpus.
For Sanders and HCR datasets, we compare results with the model of BIBREF14 that used a ensemble of multiple base classifiers (ENS) such as NB, Random Forest (RF), SVM and Logistic Regression (LR). The ENS model is combined with bag-of-words (BoW), feature hashing (FH) and lexicons. The model of BIBREF14 is a state-of-the-art on Sanders and HCR datasets. Our models outperform the model of BIBREF14 for the Sanders dataset and HCR dataset.

Answers:
Yes
Question: Which three Twitter sentiment classification datasets are used for experiments?

Evidence:
raw_evidence: Stanford - Twitter Sentiment Corpus (STS Corpus): STS Corpus contains 1,600K training tweets collected by a crawler from BIBREF0 . BIBREF0 constructed a test set manually with 177 negative and 182 positive tweets. The Stanford test set is small. However, it has been widely used in different evaluation tasks BIBREF0 BIBREF5 BIBREF13 .
Sanders - Twitter Sentiment Corpus: This dataset consists of hand-classified tweets collected by using search terms: INLINEFORM0 , #google, #microsoft and #twitter. We construct the dataset as BIBREF14 for binary classification.
Health Care Reform (HCR): This dataset was constructed by crawling tweets containing the hashtag #hcr BIBREF15 . Task is to predict positive/negative tweets BIBREF14 .

Answers:
Stanford - Twitter Sentiment Corpus (STS Corpus), Sanders - Twitter Sentiment Corpus, Health Care Reform (HCR)
Question: Which knowledge graph completion tasks do they experiment with?

Evidence:
raw_evidence: We evaluate the effectiveness of our LAN model on two typical knowledge graph completion tasks, i.e., link prediction and triplet classification. We compare our LAN with two baseline aggregators, MEAN and LSTM, as described in the Encoder section. MEAN is used on behalf of pooling functions since it leads to the best performance in BIBREF9 ijcai2017-250. LSTM is used due to its large expressive capability BIBREF26 .

Answers:
link prediction , triplet classification
Question: Apart from using desired properties, do they evaluate their LAN approach in some other way?

Evidence:

Answers:
No
Question: Do they evaluate existing methods in terms of desired properties?

Evidence:

Answers:
Yes
Question: What is the dataset used to train the model?

Evidence:
raw_evidence: Clean-labeled Datasets. We use three clean labeled datasets. The first one is the movie sentence polarity dataset from BIBREF19. The other two datasets are laptop and restaurant datasets collected from SemEval-2016 . The former consists of laptop review sentences and the latter consists of restaurant review sentences. The original datasets (i.e., Laptop and Restaurant) were annotated with aspect polarity in each sentence. We used all sentences with only one polarity (positive or negative) for their aspects. That is, we only used sentences with aspects having the same sentiment label in each sentence. Thus, the sentiment of each aspect gives the ground-truth as the sentiments of all aspects are the same.
Noisy-labeled Training Datasets. For the above three domains (movie, laptop, and restaurant), we collected 2,000 reviews for each domain from the same review source. We extracted sentences from each review and assigned review's label to its sentences. Like previous work, we treat 4 or 5 stars as positive and 1 or 2 stars as negative. The data is noisy because a positive (negative) review can contain negative (positive) sentences, and there are also neutral sentences. This gives us three noisy-labeled training datasets. We still use the same test sets as those for the clean-labeled datasets. Summary statistics of all the datasets are shown in Table TABREF9.

Answers:
 movie sentence polarity dataset from BIBREF19, laptop and restaurant datasets collected from SemEval-201, we collected 2,000 reviews for each domain from the same review source
Question: Is the model evaluated against a CNN baseline?

Evidence:
raw_evidence: Baselines. We use one strong non-DNN baseline, NBSVM (with unigrams or bigrams features) BIBREF23 and six DNN baselines. The first DNN baseline is CNN BIBREF25, which does not handle noisy labels. The other five were designed to handle noisy labels.
The comparison results are shown in Table TABREF12. From the results, we can make the following observations. (1) Our NetAb model achieves the best ACC and F1 on all datasets except for F1 of negative class on Laptop. The results demonstrate the superiority of NetAb. (2) NetAb outperforms the baselines designed for learning with noisy labels. These baselines are inferior to ours as they were tailored for image classification. Note that we found no existing method to deal with noisy labels for SSC. Training Details. We use the publicly available pre-trained embedding GloVe.840B BIBREF48 to initialize the word vectors and the embedding dimension is 300.

Answers:
Yes
Question: Has STES been previously used in the literature to evaluate similar tasks?

Evidence:
raw_evidence: We propose a pipeline called SMERTI (pronounced `smarty') for STE. Combining entity replacement (ER), similarity masking (SM), and text infilling (TI), SMERTI can modify the semantic content of text. We define a metric called the Semantic Text Exchange Score (STES) that evaluates the overall ability of a model to perform STE, and an adjustable parameter masking (replacement) rate threshold (MRT/RRT) that can be used to control the amount of semantic change.

Answers:
No
Question: What are the baseline models mentioned in the paper?

Evidence:
raw_evidence: We evaluate on three datasets: Yelp and Amazon reviews BIBREF1, and Kaggle news headlines BIBREF2. We implement three baseline models for comparison: Noun WordNet Semantic Text Exchange Model (NWN-STEM), General WordNet Semantic Text Exchange Model (GWN-STEM), and Word2Vec Semantic Text Exchange Model (W2V-STEM).

Answers:
Noun WordNet Semantic Text Exchange Model (NWN-STEM), General WordNet Semantic Text Exchange Model (GWN-STEM), Word2Vec Semantic Text Exchange Model (W2V-STEM)
Question: What kind of settings do the utterances come from?

Evidence:
raw_evidence: CN-Celeb specially focuses on Chinese celebrities, and contains more than $130,000$ utterances from $1,000$ persons.
CN-Celeb covers more genres of speech. We intentionally collected data from 11 genres, including entertainment, interview, singing, play, movie, vlog, live broadcast, speech, drama, recitation and advertisement. The speech of a particular speaker may be in more than 5 genres. As a comparison, most of the utterances in VoxCeleb were extracted from interview videos. The diversity in genres makes our database more representative for the true scenarios in unconstrained conditions, but also more challenging.

Answers:
entertainment, interview, singing, play, movie, vlog, live broadcast, speech, drama, recitation and advertisement
Question: Do they experiment with cross-genre setups?

Evidence:

Answers:
No
Question: Does the new objective perform better than the original objective bert is trained on?

Evidence:
raw_evidence: Table 2 lists the accuracies of the all methods on two classifier architectures. The results show that, for various datasets on different classifier architectures, our conditional BERT contextual augmentation improves the model performances most. BERT can also augments sentences to some extent, but not as much as conditional BERT does. For we masked words randomly, the masked words may be label-sensitive or label-insensitive. If label-insensitive words are masked, words predicted through BERT may not be compatible with original labels. The improvement over all benchmark datasets also shows that conditional BERT is a general augmentation method for multi-labels sentence classification tasks.

Answers:
Yes
Question: Do the authors report performance of conditional bert on tasks without data augmentation?

Evidence:
raw_evidence: Table 2 lists the accuracies of the all methods on two classifier architectures. The results show that, for various datasets on different classifier architectures, our conditional BERT contextual augmentation improves the model performances most. BERT can also augments sentences to some extent, but not as much as conditional BERT does. For we masked words randomly, the masked words may be label-sensitive or label-insensitive. If label-insensitive words are masked, words predicted through BERT may not be compatible with original labels. The improvement over all benchmark datasets also shows that conditional BERT is a general augmentation method for multi-labels sentence classification tasks.

Answers:
Yes
Question: Do they cover data augmentation papers?

Evidence:

Answers:
No
Question: Do they survey visual question generation work?

Evidence:
raw_evidence: Visual Question Generation (VQG) is another emerging topic which aims to ask questions given an image. We categorize VQG into grounded- and open-ended VQG by the level of cognition. Grounded VQG generates visually grounded questions, i.e., all relevant information for the answer can be found in the input image BIBREF74 . A key purpose of grounded VQG is to support the dataset construction for VQA. To ensure the questions are grounded, existing systems rely on image captions to varying degrees. BIBREF75 and BIBREF76 simply convert image captions into questions using rule-based methods with textual patterns. BIBREF74 proposed a neural model that can generate questions with diverse types for a single image, using separate networks to construct dense image captions and to select question types.
In contrast to grounded QG, humans ask higher cognitive level questions about what can be inferred rather than what can be seen from an image. Motivated by this, BIBREF10 proposed open-ended VQG that aims to generate natural and engaging questions about an image. These are deep questions that require high cognition such as analyzing and creation. With significant progress in deep generative models, marked by variational auto-encoders (VAEs) and GANs, such models are also used in open-ended VQG to bring “creativity” into generated questions BIBREF77 , BIBREF78 , showing promising results. This also brings hope to address deep QG from text, as applied in NLG: e.g., SeqGAN BIBREF79 and LeakGAN BIBREF80 .

Answers:
Yes
Question: Do they survey multilingual aspects?

Evidence:

Answers:
No
Question: What learning paradigms do they cover in this survey?

Evidence:
raw_evidence: Past research took a reductionist approach, separately considering these two problems of “what” and “how” via content selection and question construction. Given a sentence or a paragraph as input, content selection selects a particular salient topic worthwhile to ask about and determines the question type (What, When, Who, etc.). Approaches either take a syntactic BIBREF11 , BIBREF12 , BIBREF13 or semantic BIBREF14 , BIBREF3 , BIBREF15 , BIBREF16 tack, both starting by applying syntactic or semantic parsing, respectively, to obtain intermediate symbolic representations. Question construction then converts intermediate representations to a natural language question, taking either a tranformation- or template-based approach. The former BIBREF17 , BIBREF18 , BIBREF13 rearranges the surface form of the input sentence to produce the question; the latter BIBREF19 , BIBREF20 , BIBREF21 generates questions from pre-defined question templates. Unfortunately, such QG architectures are limiting, as their representation is confined to the variety of intermediate representations, transformation rules or templates.
In contrast, neural models motivate an end-to-end architectures. Deep learned frameworks contrast with the reductionist approach, admitting approaches that jointly optimize for both the “what” and “how” in an unified framework. The majority of current NQG models follow the sequence-to-sequence (Seq2Seq) framework that use a unified representation and joint learning of content selection (via the encoder) and question construction (via the decoder). In this framework, traditional parsing-based content selection has been replaced by more flexible approaches such as attention BIBREF22 and copying mechanism BIBREF23 . Question construction has become completely data-driven, requiring far less labor compared to transformation rules, enabling better language flexibility compared to question templates.

Answers:
Considering "What" and "How" separately versus jointly optimizing for both.
Question: What are all the input modalities considered in prior work in question generation?

Evidence:
raw_evidence: Question generation is an NLG task for which the input has a wealth of possibilities depending on applications. While a host of input modalities have been considered in other NLG tasks, such as text summarization BIBREF24 , image captioning BIBREF25 and table-to-text generation BIBREF26 , traditional QG mainly focused on textual inputs, especially declarative sentences, explained by the original application domains of question answering and education, which also typically featured textual inputs.
Recently, with the growth of various QA applications such as Knowledge Base Question Answering (KBQA) BIBREF27 and Visual Question Answering (VQA) BIBREF28 , NQG research has also widened the spectrum of sources to include knowledge bases BIBREF29 and images BIBREF10 . This trend is also spurred by the remarkable success of neural models in feature representation, especially on image features BIBREF30 and knowledge representations BIBREF31 . We discuss adapting NQG models to other input modalities in Section "Wider Input Modalities" .

Answers:
Textual inputs, knowledge bases, and images.
Question: Do they survey non-neural methods for question generation?

Evidence:

Answers:
No
Question: What is their model?

Evidence:
raw_evidence: Most annotated corpus based NE recognition tasks can benefit a great deal from a known NE dictionary, as NEs are those words which carry common sense knowledge quite differ from the rest ones in any language vocabulary. This work will focus on the NE recognition from plain text instead of corpus based NE recognition. For a purpose of learning from limited annotated linguistic resources, our preliminary discovery shows that it is possible to build a geometric space projection between embedding spaces to help cross-lingual NE recognition. Our study contains two main steps: First, we explore the NE distribution in monolingual case. Next, we learn a hypersphere mapping between embedding spaces of languages with minimal supervision.

Answers:
cross-lingual NE recognition
Question: Do they evaluate on NER data sets?

Evidence:
raw_evidence: To evaluate the influence of our hypersphere feature for off-the-shelf NER systems, we perform the NE recognition on two standard NER benchmark datasets, CoNLL2003 and ONTONOTES 5.0. Our results in Table 6 and Table 7 demonstrate the power of hypersphere features, which contribute to nearly all of the three types of entities as shown in Table 6, except for a slight drop in the PER type of BIBREF22 on a strong baseline. HS features stably enhance all strong state-of-the-art baselines, BIBREF22 , BIBREF21 and BIBREF23 by 0.33/0.72/0.23 $F_1$ point and 0.13/0.3/0.1 $F_1$ point on both benchmark datasets, CoNLL-2003 and ONTONOTES 5.0. We show that our HS feature is also comparable with previous much more complicated LS feature, and our model surpasses their baseline (without LS feature) by 0.58/0.78 $F_1$ point with only HS features. We establish a new state-of-the-art $F_1$ score of 89.75 on ONTONOTES 5.0, while matching state-of-the-art performance with a $F_1$ score of 92.95 on CoNLL-2003 dataset.

Answers:
Yes
Question: What previously proposed methods is this method compared against?

Evidence:
raw_evidence: The baseline model for our experiments is explained in the paper by Alec Go [1]. The model uses the Naive Bayes, SVM, and the Maximum Entropy classifiers for their experiment. Their feature vector is either composed of Unigrams, Bigrams, Unigrams + Bigrams, or Unigrams + POS tags.

Answers:
Naive Bayes, SVM, Maximum Entropy classifiers
Question: How is effective word score calculated?

Evidence:
raw_evidence: We define the Effective Word Score of score x as
EFWS(x) = N(+x) - N(-x),
where N(x) is the number of words in the tweet with polarity score x.

Answers:
We define the Effective Word Score of score x as

EFWS(x) = N(+x) - N(-x),

where N(x) is the number of words in the tweet with polarity score x.
Question: Why is supporting fact supervision necessary for DMN?

Evidence:
raw_evidence: We speculate that there are two main reasons for this performance disparity, all exacerbated by the removal of supporting facts. First, the GRU only allows sentences to have context from sentences before them, but not after them. This prevents information propagation from future sentences. Second, the supporting sentences may be too far away from each other on a word level to allow for these distant sentences to interact through the word level GRU.

Answers:
First, the GRU only allows sentences to have context from sentences before them, but not after them. This prevents information propagation from future sentences. Second, the supporting sentences may be too far away from each other on a word level to allow for these distant sentences to interact through the word level GRU.
Question: What does supporting fact supervision mean?

Evidence:
raw_evidence: We analyze the DMN components, specifically the input module and memory module, to improve question answering. We propose a new input module which uses a two level encoder with a sentence reader and input fusion layer to allow for information flow between sentences. For the memory, we propose a modification to gated recurrent units (GRU) BIBREF7 . The new GRU formulation incorporates attention gates that are computed using global knowledge over the facts. Unlike before, the new DMN+ model does not require that supporting facts (i.e. the facts that are relevant for answering a particular question) are labeled during training. The model learns to select the important facts from a larger set.

Answers:
 the facts that are relevant for answering a particular question) are labeled during training.
Question: What changes they did on input module?

Evidence:
raw_evidence: For the DMN+, we propose replacing this single GRU with two different components. The first component is a sentence reader, responsible only for encoding the words into a sentence embedding. The second component is the input fusion layer, allowing for interactions between sentences. This resembles the hierarchical neural auto-encoder architecture of BIBREF9 and allows content interaction between sentences. We adopt the bi-directional GRU for this input fusion layer because it allows information from both past and future sentences to be used. As gradients do not need to propagate through the words between sentences, the fusion layer also allows for distant supporting sentences to have a more direct interaction.

Answers:
For the DMN+, we propose replacing this single GRU with two different components. The first component is a sentence reader, The second component is the input fusion layer
Question: What improvements they did for DMN?

Evidence:
raw_evidence: We analyze the DMN components, specifically the input module and memory module, to improve question answering. We propose a new input module which uses a two level encoder with a sentence reader and input fusion layer to allow for information flow between sentences. For the memory, we propose a modification to gated recurrent units (GRU) BIBREF7 . The new GRU formulation incorporates attention gates that are computed using global knowledge over the facts. Unlike before, the new DMN+ model does not require that supporting facts (i.e. the facts that are relevant for answering a particular question) are labeled during training. The model learns to select the important facts from a larger set.
In addition, we introduce a new input module to represent images. This module is compatible with the rest of the DMN architecture and its output is fed into the memory module. We show that the changes in the memory module that improved textual question answering also improve visual question answering. Both tasks are illustrated in Fig. 1 .

Answers:
the new DMN+ model does not require that supporting facts (i.e. the facts that are relevant for answering a particular question) are labeled during training., In addition, we introduce a new input module to represent images.
Question: How does the model circumvent the lack of supporting facts during training?

Evidence:
raw_evidence: We have proposed new modules for the DMN framework to achieve strong results without supervision of supporting facts. These improvements include the input fusion layer to allow interactions between input facts and a novel attention based GRU that allows for logical reasoning over ordered inputs. Our resulting model obtains state of the art results on both the VQA dataset and the bAbI-10k text question-answering dataset, proving the framework can be generalized across input domains.

Answers:
the input fusion layer to allow interactions between input facts and a novel attention based GRU that allows for logical reasoning over ordered inputs.
Question: Does the DMN+ model establish state-of-the-art ?

Evidence:
raw_evidence: We have proposed new modules for the DMN framework to achieve strong results without supervision of supporting facts. These improvements include the input fusion layer to allow interactions between input facts and a novel attention based GRU that allows for logical reasoning over ordered inputs. Our resulting model obtains state of the art results on both the VQA dataset and the bAbI-10k text question-answering dataset, proving the framework can be generalized across input domains.

Answers:
Yes