qinchuanhui/UDA-QA huggingface dataset:
Question: What cyberbulling topics did they address?

Evidence:
raw_evidence: Past works on cyberbullying detection have at least one of the following three bottlenecks. First (Bottleneck B1), they target only one particular social media platform. How these methods perform across other SMPs is unknown. Second (Bottleneck B2), they address only one topic of cyberbullying such as racism, and sexism. Depending on the topic, vocabulary and nature of cyberbullying changes. These models are not flexible in accommodating changes in the definition of cyberbullying. Third (Bottleneck B3), they rely on carefully handcrafted features such as swear word list and POS tagging. However, these handcrafted features are not robust against variations in writing style. In contrast to existing bottlenecks, this work targets three different types of social networks (Formspring: a Q&A forum, Twitter: microblogging, and Wikipedia: collaborative knowledge repository) for three topics of cyberbullying (personal attack, racism, and sexism) without doing any explicit feature engineering by developing deep learning based models along with transfer learning.
raw_evidence: Please refer to Table TABREF7 for summary of datasets used. We performed experiments using large, diverse, manually annotated, and publicly available datasets for cyberbullying detection in social media. We cover three different types of social networks: teen oriented Q&A forum (Formspring), large microblogging platform (Twitter), and collaborative knowledge repository (Wikipedia talk pages). Each dataset addresses a different topic of cyberbullying. Twitter dataset contains examples of racism and sexism. Wikipedia dataset contains examples of personal attack. However, Formspring dataset is not specifically about any single topic. All three datasets have the problem of class imbalance where posts labeled as cyberbullying are in the minority as compared to neutral posts. Variation in the number of posts across datasets also affects vocabulary size that represents the number of distinct words encountered in the dataset. We measure the size of a post in terms of the number of words in the post. For each dataset, there are only a few posts with large size. We truncate such large posts to the size of post ranked at 95 percentile in that dataset. For example, in Wikipedia dataset, the largest post has 2846 words. However, size of post ranked at 95 percentile in that dataset is only 231. Any post larger than size 231 in Wikipedia dataset will be truncated by considering only first 231 words. This truncation affects only a small minority of posts in each dataset. However, it is required for efficiently training various models in our experiments. Details of each dataset are as follows.

Answers:
personal attack, racism, and sexism
racism, sexism, personal attack, not specifically about any single topic
Question: Were any of the pipeline components based on deep learning models?

Evidence:
raw_evidence: Part-of-speech tagging happens after language detection. It labels each word with a tag that indicates its syntactic role in the sentence. For instance, a word could be a noun, verb, adjective or adverb (or other syntactic tag). We used Freeling BIBREF8 library to provide the tags. This library resorts to a Hidden Markov Model as described by Brants BIBREF9. The end result is a tag for each word as described by the EAGLES tagset .
We use the named entity recognition module after part-of-speech tagging. This module labels each part of the sentence into different categories such as "PERSON", "LOCATION", or "ORGANIZATION". We also used Freeling to label the named entities and the details of the algorithm are shown in the paper by Carreras et al BIBREF10. Aside from the three aforementioned categories, we also extracted "DATE/TIME" and "CURRENCY" values by looking at the part-of-speech tags: date/time words have a tag of "W", while currencies have "Zm".
Dependency parsing involves tagging a word based on different features to indicate if it is dependent on another word. The Freeling library also has dependency parsing models for Portuguese. Since we wanted to build a SRL (Semantic Role Labeling) module on top of the dependency parser and the current released version of Freeling does not have an SRL module for Portuguese, we trained a different Portuguese dependency parsing model that was compatible (in terms of used tags) with the available annotated.

Answers:
No
No
Question: How is the effectiveness of this pipeline approach evaluated?

Evidence:
raw_evidence: Figure FIGREF23 shows the proposed ontology, which, in our evaluation procedure, was populated with 3121 events entries from 51 documents.

Answers:
proposed ontology, which, in our evaluation procedure, was populated with 3121 events entries from 51 documents.
Question: What is the size of the parallel corpus used to train the model constraints?

Evidence:
raw_evidence: A parser model was trained for Hindi using the MSTParser BIBREF2 by a part of the the Hindi Dependency Treebank data (18708 sentences) from IIIT-Hyderabad BIBREF3 . A part of the Penn Treebank (28188 sentences) was used for training an English parser BIBREF4 . The treebanks were converted to MSTParser format from ConLL format for training. A part of the ILCI English-Hindi Tourism parallel corpus (1500 sentences) was used for training the classifiers. This corpus was POS-tagged using the Stanford POS Tagger BIBREF5 for English and using the Hindi POS Tagger BIBREF6 from IIIT-Hyderabad for Hindi. It was then automatically annotated with dependency parse trees by the parsers we had trained before English and Hindi.
raw_evidence: A parser model was trained for Hindi using the MSTParser BIBREF2 by a part of the the Hindi Dependency Treebank data (18708 sentences) from IIIT-Hyderabad BIBREF3 . A part of the Penn Treebank (28188 sentences) was used for training an English parser BIBREF4 . The treebanks were converted to MSTParser format from ConLL format for training. A part of the ILCI English-Hindi Tourism parallel corpus (1500 sentences) was used for training the classifiers. This corpus was POS-tagged using the Stanford POS Tagger BIBREF5 for English and using the Hindi POS Tagger BIBREF6 from IIIT-Hyderabad for Hindi. It was then automatically annotated with dependency parse trees by the parsers we had trained before English and Hindi.

Answers:
1500 sentences
1500 sentences
Question: How does enforcing agreement between parse trees work across different languages?

Evidence:
raw_evidence: Let us take a closer look at what we mean by agreement between the two parse trees. Essentially, if we have two words in the English sentence denoted by i and i', aligned to words j and j' in the parallel Hindi sentence respectively, we can expect a dependency edge between i and i' in the English parse tree to correspond to an edge between j and j' in the Hindi parse tree, and vice versa. Also, in order to accommodate structural diversity in languages BIBREF1 , we can expect an edge in the parse tree in one language to correspond to more than one edge, or rather, a path, in the other language parse tree. This has been captured in the examples in figures FIGREF6 (A) and FIGREF6 (B). For an edge in the English parse tree, we term the corresponding edge or path in the Hindi parse tree as the projection or projected path of the English edge on the Hindi parse tree, and similarly there are projected paths from Hindi to English. For matters of simplicity, we ignore the direction of the edges in the parse trees. The dual decomposition inference algorithm tries to bring the parse trees in the two languages through its constraints.

Answers:
we can expect a dependency edge between i and i' in the English parse tree to correspond to an edge between j and j' in the Hindi parse tree, dual decomposition inference algorithm tries to bring the parse trees in the two languages through its constraints
Question: How is the vocabulary of word-like or phoneme-like units automatically discovered?

Evidence:
raw_evidence: UTD aims to automatically identify and cluster repeated terms (e.g. words or phrases) from speech. To circumvent the exhaustive DTW-based search limited by INLINEFORM0 time BIBREF6 , we exploit the scalable UTD framework in the Zero Resource Toolkit (ZRTools) BIBREF7 , which permits search in INLINEFORM1 time. We briefly describe the UTD procedures in ZRTools by four steps below, and full details can be found in BIBREF7 .

Answers:
Zero Resource Toolkit (ZRTools) BIBREF7
Question: How well did the baseline perform?

Evidence:
raw_evidence: Table TABREF8 shows the results for the baseline models as well as the HUMAN measured performance on the task. The human performance provides an upper bound for the RUN task performance, while the simple baselines provide lower bounds. The best baseline model is NO-MOVE, reaching an accuracy of 30.3% on single sentences and 0.3 on complete paragraphs. For the HUMAN case, paragraph accuracy reaches above 80.

Answers:
accuracy of 30.3% on single sentences and 0.3 on complete paragraphs
Question: What is the baseline?

Evidence:
raw_evidence: We provide three simple baselines for the RUN task: (1) NO-MOVE: the only position considered is the starting point; (2) RANDOM: As in BIBREF4, turn to a randomly selected heading, then execute a number of *WALK actions of an average route; (3) JUMP: at each sentence, extract entities from the map and move between them in the order they appear. If the *WALK action is invalid we take a random *TURN action.
raw_evidence: We provide three simple baselines for the RUN task: (1) NO-MOVE: the only position considered is the starting point; (2) RANDOM: As in BIBREF4, turn to a randomly selected heading, then execute a number of *WALK actions of an average route; (3) JUMP: at each sentence, extract entities from the map and move between them in the order they appear. If the *WALK action is invalid we take a random *TURN action.

Answers:
NO-MOVE, RANDOM, JUMP
NO-MOVE: the only position considered is the starting point, RANDOM: As in BIBREF4, turn to a randomly selected heading, then execute a number of *WALK actions of an average route, JUMP: at each sentence, extract entities from the map and move between them in the order they appear
Question: what methods were used to reduce data sparsity effects?

Evidence:
raw_evidence: In this section, we describe the data augmentation methods we use to increase the amount of training data in order to make our NMT systems suffer less from the low-resourced situation in Japanese INLINEFORM0 Vietnamese translation. Although NMT systems can predict and generate the translation of unseen words on their vocabularies, but they only perform this well if the parallel corpus for training are sufficiently large. For many under-resourced languages, unfortunately, it hardly presents. In reality, although the monolingual data of Vietnamese and Japanese are immensely available due to the popularity of their speakers, the bilingual Japanese-Vietnamese corpora are very limited and often in low quality or in narrowly technical domains. Therefore, data augmentation methods to exploit monolingual data for NMT systems are necessary to obtain more bilingual data, thus upgrading the translating quality.
Back Translation
One of the approaches to leverage the monolingual data is to use a machine translation system to translate those data in order to create a synthetic parallel data. Normally, the monolingual data in the target language is translated, thus the name of the method: Back Translation BIBREF11 .
Mix-Source Approach
Another data augmentation method considered useful in this low-resourced setting is the mix-source method BIBREF12 . In this method, we can utilize the monolingual data of the target language in a multilingual NMT system by mixing the original source sentences with those target monolingual data. The multilingual framework then uses the share information across source and target languages to improve the decision of the target words to be chosen.
raw_evidence: In this section, we describe the data augmentation methods we use to increase the amount of training data in order to make our NMT systems suffer less from the low-resourced situation in Japanese INLINEFORM0 Vietnamese translation. Although NMT systems can predict and generate the translation of unseen words on their vocabularies, but they only perform this well if the parallel corpus for training are sufficiently large. For many under-resourced languages, unfortunately, it hardly presents. In reality, although the monolingual data of Vietnamese and Japanese are immensely available due to the popularity of their speakers, the bilingual Japanese-Vietnamese corpora are very limited and often in low quality or in narrowly technical domains. Therefore, data augmentation methods to exploit monolingual data for NMT systems are necessary to obtain more bilingual data, thus upgrading the translating quality.

Answers:
Back Translation, Mix-Source Approach
data augmentation
Question: what was the baseline?

Evidence:
raw_evidence: Baseline. For the baseline systems, the training data includes KyTea-segmented Japanese texts and pyvi-segmented Vietnamese texts. For comparison purpose, we build two baseline systems for each direction: one is use the traditional phrase-based statistical machine translation (SMT), the other one is the NMT system. Although our training set is small but we find that the NMT systems (2) are still more effective than the phrase-based SMT models (1) in both translation directions.
raw_evidence: Baseline. For the baseline systems, the training data includes KyTea-segmented Japanese texts and pyvi-segmented Vietnamese texts. For comparison purpose, we build two baseline systems for each direction: one is use the traditional phrase-based statistical machine translation (SMT), the other one is the NMT system. Although our training set is small but we find that the NMT systems (2) are still more effective than the phrase-based SMT models (1) in both translation directions.
Subword NMT. We applied VNBPE and JPBPE to the baseline's data and trained NMT systems. On Vietnamese INLINEFORM0 Japanese, we observed an improvement of 0.6 BLEU points when we used our VNBPE (3) instead of the pyvi's word segmentation (2). Furthermore, when we trained our NMT models using both BPE methods (4), we obtained a bigger gain of 1.15 BLEU points. The similar improvements can be found in the Japanese INLINEFORM1 Vietnamese as well: 0.29 BLEU points between (3) and (2) and 0.57 BLEU points between (4) and (3). This draws two conclusions: (i), despite using an unsupervised Vietnamese word segmentation which is fast, robust and does not require linguistic resources, our NMT systems performed better than those systems employing a complicate word segmentation method, (ii) BPE works significantly well for Japanese texts after we tokenized the texts.

Answers:
traditional phrase-based statistical machine translation (SMT), NMT system
traditional phrase-based statistical machine translation (SMT), NMT system
Question: did they collect their own data?

Evidence:
raw_evidence: We collected Japanese-Vietnamese parallel data from TED talks extracted from WIT3's corpus BIBREF15 . After removing blank and duplicate lines we obtained 106758 pairs of sentences. The validation set used in all experiments is dev2010 and the test set is tst2010.

Answers:
No
Question: what japanese-vietnamese dataset do they use?

Evidence:
raw_evidence: We collected Japanese-Vietnamese parallel data from TED talks extracted from WIT3's corpus BIBREF15 . After removing blank and duplicate lines we obtained 106758 pairs of sentences. The validation set used in all experiments is dev2010 and the test set is tst2010.

Answers:
WIT3's corpus
Question: Do they introduce errors in the data or does the data already contain them?

Evidence:
raw_evidence: We use three languages in our experiments: English, Estonian and Latvian. All three have different characteristics, for example Latvian and (especially) Estonian are morphologically complex and have loose word order, while English has a strict word order and the morphology is much simpler. Most importantly, all three languages have error-corrected corpora for testing purposes, though work on their automatic grammatical error correction is extremely limited (see Section SECREF3 ).
raw_evidence: Test Data and Metrics
We use the following error-corrected corpora both for scoring and as basis for manual analysis:
for English: CoNLL-2014 BIBREF5 and JFLEG BIBREF20 corpora
for Estonian: the Learner Language Corpus BIBREF21
for Latvian: the Error-annotated Corpus of Latvian BIBREF22
All of these are based on language learner (L2) essays and their manual corrections.

Answers:
 all three languages have error-corrected corpora for testing purposes
Data already contain errors
Question: What error types is their model more reliable for?

Evidence:
raw_evidence: To conclude this section, our model reliable corrects grammatical, spelling and word order errors on , with more mixed performance on lexical choice errors and some unnecessary paraphrasing of the input. The error types that the model manages well can be traced back to having a strong monolingual language model, a common trait of a good NMT model. As the model operates on the level of word parts and its vocabulary is limited, this leads to combining wrong word parts, sometimes across languages. This could be fixed by either using character-based NMT or doing automatic spelling correction prior to applying our current model.
raw_evidence: We showed that for GEC our approach reliably corrects spelling, word order and grammatical errors, while being less reliable on lexical choice errors. Applied to style transfer our model is very good at meaning preservation and output fluency, while reliably transferring style for English contractions, lexical choice and grammatical constructions. The main benefit is that no annotated data is used to train the model, thus making it very easy to train it for other (especially under-resourced) languages.

Answers:
grammatical, spelling and word order errors
spelling, word order and grammatical errors
Question: Do they experiment with their proposed model on any other dataset other than MovieQA?

Evidence:
raw_evidence: We also applied our model to MCTest dataset which requires machines to answer multiple-choice reading comprehension questions about fictional stories. The original paper describes that a baseline method uses a combination of a sliding window score and a distance based . They achieve 66.7% and 56.7% on MC500 and MC160 separately. Because of the restricted training set and development set, we trained our model on MovieQA training set and applied the result to test MCTest dataset. On MCTest dataset, we still outperform baseline and achieve 68.1% accuracy on MC160 and 61.5% accuracy on MC500.
raw_evidence: We also applied our model to MCTest dataset which requires machines to answer multiple-choice reading comprehension questions about fictional stories. The original paper describes that a baseline method uses a combination of a sliding window score and a distance based . They achieve 66.7% and 56.7% on MC500 and MC160 separately. Because of the restricted training set and development set, we trained our model on MovieQA training set and applied the result to test MCTest dataset. On MCTest dataset, we still outperform baseline and achieve 68.1% accuracy on MC160 and 61.5% accuracy on MC500.

Answers:
Yes
Yes
Question: What is the difference of the proposed model with a standard RNN encoder-decoder?

Evidence:
raw_evidence: While the RNN-based generators with DA gating-vector can prevent the undesirable semantic repetitions, the ARED-based generators show signs of better adapting to a new domain. However, none of the models show significant advantage from out-of-domain data. To better analyze model generalization to an unseen, new domain as well as model leveraging the out-of-domain sources, we propose a new architecture which is an extension of the ARED model. In order to better select, aggregate and control the semantic information, a Refinement Adjustment LSTM-based component (RALSTM) is introduced to the decoder side. The proposed model can learn from unaligned data by jointly training the sentence planning and surface realization to produce natural language sentences. We conducted experiments on four different NLG domains and found that the proposed methods significantly outperformed the state-of-the-art methods regarding BLEU BIBREF15 and slot error rate ERR scores BIBREF4 . The results also showed that our generators could scale to new domains by leveraging the out-of-domain data. To sum up, we make three key contributions in this paper:

Answers:
Introduce a "Refinement Adjustment LSTM-based component" to the decoder
Question: Does the model evaluated on NLG datasets or dialog datasets?

Evidence:
raw_evidence: We assessed the proposed models on four different NLG domains: finding a restaurant, finding a hotel, buying a laptop, and buying a television. The Restaurant and Hotel were collected in BIBREF4 , while the Laptop and TV datasets have been released by BIBREF22 with a much larger input space but only one training example for each DA so that the system must learn partial realization of concepts and be able to recombine and apply them to unseen DAs. This makes the NLG tasks for the Laptop and TV domains become much harder. The dataset statistics are shown in Table 1 .
raw_evidence: We assessed the proposed models on four different NLG domains: finding a restaurant, finding a hotel, buying a laptop, and buying a television. The Restaurant and Hotel were collected in BIBREF4 , while the Laptop and TV datasets have been released by BIBREF22 with a much larger input space but only one training example for each DA so that the system must learn partial realization of concepts and be able to recombine and apply them to unseen DAs. This makes the NLG tasks for the Laptop and TV domains become much harder. The dataset statistics are shown in Table 1 .

Answers:
NLG datasets
NLG datasets
Question: What tasks do they experiment with?

Evidence:
raw_evidence: We evaluate our models on two typical tasks: text classification and text semantic matching. The results show that our models are more expressive due to their learning to learn nature, yet without increasing the number of model's parameters. Moreover, we find certain composition operations can be learned implicitly by meta TreeNN, such as the composition of noun phrases and verb phrases.
raw_evidence: We evaluate our models on two typical tasks: text classification and text semantic matching. The results show that our models are more expressive due to their learning to learn nature, yet without increasing the number of model's parameters. Moreover, we find certain composition operations can be learned implicitly by meta TreeNN, such as the composition of noun phrases and verb phrases.

Answers:
text classification and text semantic matching
text classification and text semantic matching
Question: what data did they use?

Evidence:
raw_evidence: Following recent works, we compare on two widely used datasets, the Penn Treebank (PTB) BIBREF28 as prepared by BIBREF29 and WikiText2 (WT-2) BIBREF20 . For both datasets, we follow the same training, validation, and test splits as in BIBREF0 .
raw_evidence: Following recent works, we compare on two widely used datasets, the Penn Treebank (PTB) BIBREF28 as prepared by BIBREF29 and WikiText2 (WT-2) BIBREF20 . For both datasets, we follow the same training, validation, and test splits as in BIBREF0 .

Answers:
 Penn Treebank, WikiText2
Penn Treebank (PTB) , WikiText2 (WT-2)