qinchuanhui/UDA-QA huggingface dataset:
Question: How are the supervised scores of the words calculated?

Evidence:
raw_evidence: The effect of this multiplication is exemplified in Figure FIGREF7, showing the positions of word vectors in the VSM. Those “x" words are sentimentally negative words, those “o" words are sentimentally positive ones. On the top coordinate plane, the words of opposite polarities are found to be close to each other, since they have common words in their dictionary definitions. Only the information concerned with the dictionary definitions are used there, discarding the polarity scores. However, when we utilise the supervised score (+1 or -1), words of opposite polarities (e.g. “happy" and “unhappy") get far away from each other as they are translated across coordinate regions. Positive words now appear in quadrant 1, whereas negative words appear in quadrant 3. Thus, in the VSM, words that are sentimentally similar to each other could be clustered more accurately. Besides clustering, we also employed the SVD method to perform dimensionality reduction on the unsupervised dictionary algorithm and used the newly generated matrix by combining it with other subapproaches. The number of dimensions is chosen as 200 again according to the $U$ matrix. The details are given in Section 3.4. When using and evaluating this subapproach on the English corpora, we used the SentiWordNet lexicon BIBREF13. We have achieved better results for the dictionary-based algorithm when we employed the SVD reduction method compared to the use of clustering.

Answers:
(+1 or -1), words of opposite polarities (e.g. “happy" and “unhappy") get far away from each other
Question: what dataset was used?

Evidence:
raw_evidence: We would like to thank the organizers of the WASSA-2017 Shared Task on Emotion Intensity, for providing the data, the guidelines and timely support.

Answers:
WASSA-2017 Shared Task on Emotion Intensity
Question: how many total combined features were there?

Evidence:
raw_evidence: Based on these observations, the feature extraction step is implemented as a union of different independent feature extractors (featurizers) in a light-weight and easy to use Python program EmoInt . It comprises of all features available in the baseline model BIBREF2 along with additional feature extractors and bi-gram support. Fourteen such feature extractors have been implemented which can be clubbed into 3 major categories:
[noitemsep]
Lexicon Features
Word Vectors
Syntax Features

Answers:
Fourteen
Question: what pretrained word embeddings were used?

Evidence:
raw_evidence: Word Vectors: We focus primarily on the word vector representations (word embeddings) created specifically using the twitter dataset. GloVe BIBREF13 is an unsupervised learning algorithm for obtaining vector representations for words. 200-dimensional GloVe embeddings trained on 2 Billion tweets are integrated. Edinburgh embeddings BIBREF14 are obtained by training skip-gram model on Edinburgh corpus BIBREF15 . Since tweets are abundant with emojis, Emoji embeddings BIBREF16 which are learned from the emoji descriptions have been used. Embeddings for each tweet are obtained by summing up individual word vectors and then dividing by the number of tokens in the tweet.
raw_evidence: Word Vectors: We focus primarily on the word vector representations (word embeddings) created specifically using the twitter dataset. GloVe BIBREF13 is an unsupervised learning algorithm for obtaining vector representations for words. 200-dimensional GloVe embeddings trained on 2 Billion tweets are integrated. Edinburgh embeddings BIBREF14 are obtained by training skip-gram model on Edinburgh corpus BIBREF15 . Since tweets are abundant with emojis, Emoji embeddings BIBREF16 which are learned from the emoji descriptions have been used. Embeddings for each tweet are obtained by summing up individual word vectors and then dividing by the number of tokens in the tweet.

Answers:
Pretrained word embeddings  were not used
GloVe, Edinburgh embeddings BIBREF14, Emoji embeddings BIBREF16
Question: What datasets are used?

Evidence:
raw_evidence: Given that there is no public dataset available with financial intents in Portuguese, we have employed the incremental approach to create our own training set for the Intent Classifier. First, we applied the Wizard of Oz method and from this study, we have collected a set of 124 questions that the users asked. Next, after these questions have been manually classified into a set of intent classes, and used to train the first version of the system, this set has been increased both in terms of number of classes and samples per class, resulting in a training set with 37 classes of intents, and a total 415 samples, with samples per class ranging from 3 to 37.
We have created domain-specific word vectors by considering a set 246,945 documents, corresponding to of 184,001 Twitter posts and and 62,949 news articles, all related to finance .
raw_evidence: Given that there is no public dataset available with financial intents in Portuguese, we have employed the incremental approach to create our own training set for the Intent Classifier. First, we applied the Wizard of Oz method and from this study, we have collected a set of 124 questions that the users asked. Next, after these questions have been manually classified into a set of intent classes, and used to train the first version of the system, this set has been increased both in terms of number of classes and samples per class, resulting in a training set with 37 classes of intents, and a total 415 samples, with samples per class ranging from 3 to 37.

Answers:
Custom dataset with user questions; set of documents, twitter posts and news articles, all related to finance.
a self-collected financial intents dataset in Portuguese
Question: What is the state of the art described in the paper?

Evidence:
raw_evidence: In this section we discuss the state of the art on conversational systems in three perspectives: types of interactions, types of architecture, and types of context reasoning. Then we present a table that consolidates and compares all of them.
ELIZA BIBREF11 was one of the first softwares created to understand natural language processing. Joseph Weizenbaum created it at the MIT in 1966 and it is well known for acting like a psychotherapist and it had only to reflect back onto patient's statements. ELIZA was created to tackle five "fundamental technical problems": the identification of critical words, the discovery of a minimal context, the choice of appropriate transformations, the generation of appropriate responses to the transformation or in the absence of critical words, and the provision of an ending capacity for ELIZA scripts.
Right after ELIZA came PARRY, developed by Kenneth Colby, who is psychiatrist at Stanford University in the early 1970s. The program was written using the MLISP language (meta-lisp) on the WAITS operating system running on a DEC PDP-10 and the code is non-portable. Parts of it were written in PDP-10 assembly code and others in MLISP. There may be other parts that require other language translators. PARRY was the first system to pass the Turing test - the psychiatrists were able to make the correct identification only 48 percent of the time, which is the same as a random guessing.
A.L.I.C.E. (Artificial Linguistic Internet Computer Entity) BIBREF12 appeared in 1995 but current version utilizes AIML, an XML language designed for creating stimulus-response chat robots BIBREF13 . A.L.I.C.E. bot has, at present, more than 40,000 categories of knowledge, whereas the original ELIZA had only about 200. The program is unable to pass the Turing test, as even the casual user will often expose its mechanistic aspects in short conversations.
Cleverbot (1997-2014) is a chatbot developed by the British AI scientist Rollo Carpenter. It passed the 2011 Turing Test at the Technique Techno-Management Festival held by the Indian Institute of Technology Guwahati. Volunteers participate in four-minute typed conversations with either Cleverbot or humans, with Cleverbot voted 59.3 per cent human, while the humans themselves were rated just 63.3 per cent human BIBREF14 .

Answers:
ELIZA,  PARRY, A.L.I.C.E., Cleverbot
Question: What GAN models were used as baselines to compare against?

Evidence:
raw_evidence: We compared our model with MLE, RL and GAN baselines. Since COCO and EMNLP2017 WMT don't have input while WeiboDial regards posts as input, we chose the following baselines respectively:
MLE: a RNN model trained with MLE objective BIBREF4 . Its extension, Seq2Seq, can work on the dialogue dataset BIBREF2 .
SeqGAN: The first text GAN model that updates the generator with policy gradient based on the rewards from the discriminator BIBREF7 .
LeakGAN: A variant of SeqGAN that provides rewards based on the leaked information of the discriminator for the generator BIBREF11 .
MaliGAN: A variant of SeqGAN that optimizes the generator with a normalized maximum likelihood objective BIBREF8 .
IRL: This inverse reinforcement learning method replaces the discriminator with a reward approximator to provide dense rewards BIBREF12 .
RAML: A RL approach to incorporate MLE objective into RL training framework, which regards BLEU as rewards BIBREF17 .
DialogGAN: An extension of SeqGAN tuned to dialogue generation task with MLE objective added to the adversarial objective BIBREF16 .
DPGAN: A variant of DialogGAN which uses a language model based discriminator and regards cross-entropy as rewards BIBREF13 .
raw_evidence: We compared our model with MLE, RL and GAN baselines. Since COCO and EMNLP2017 WMT don't have input while WeiboDial regards posts as input, we chose the following baselines respectively:
SeqGAN: The first text GAN model that updates the generator with policy gradient based on the rewards from the discriminator BIBREF7 .
LeakGAN: A variant of SeqGAN that provides rewards based on the leaked information of the discriminator for the generator BIBREF11 .
MaliGAN: A variant of SeqGAN that optimizes the generator with a normalized maximum likelihood objective BIBREF8 .
DialogGAN: An extension of SeqGAN tuned to dialogue generation task with MLE objective added to the adversarial objective BIBREF16 .
DPGAN: A variant of DialogGAN which uses a language model based discriminator and regards cross-entropy as rewards BIBREF13 .

Answers:
MLE, SeqGAN, LeakGAN, MaliGAN, IRL, RAML, DialogGAN, DPGAN
SeqGAN, LeakGAN, MaliGAN, DialogGAN, DPGAN
Question: What is the corpus used for the task?

Evidence:
raw_evidence: The task, as framed above, requires to detect the semantic change between two corpora. The two corpora used in the shared task correspond to the diachronic corpus pair from BIBREF0: DTA18 and DTA19. They consist of subparts of DTA corpus BIBREF11 which is a freely available lemmatized, POS-tagged and spelling-normalized diachronic corpus of German containing texts from the 16th to the 20th century. DTA18 contains 26 million sentences published between 1750-1799 and DTA19 40 million between 1850-1899. The corpus version used in the task has the following format: "year [tab] lemma1 lemma2 lemma3 ...".
raw_evidence: The Diachronic Usage Relatedness (DURel) gold standard data set includes 22 target words and their varying degrees of semantic change BIBREF12. For each of these target words a random sample of use pairs from the DTA corpus was retrieved and annotated. The annotators were required to rate the pairs according to their semantic relatedness on a scale from 1 to 4 (unrelated - identical meanings) for two time periods. The average Spearman's $\rho $ between the five annotators was 0.66 for 1,320 use paris. The resulting word ranking of the DURel data set is determined by the mean usage relatedness across two time periods and is used as the benchmark to compare the models’ performances in the shared task.

Answers:
DTA18, DTA19
Diachronic Usage Relatedness (DURel) gold standard data set
Question: How is evaluation performed?

Evidence:
raw_evidence: The output of a system with the target words in the predicted order is compared to the gold ranking of the DURel data set. As the metric to assess how well the model's output fits the gold ranking Spearman's $\rho $ was used. The higher Spearman's rank-order correlation the better the system's performance.
raw_evidence: The output of a system with the target words in the predicted order is compared to the gold ranking of the DURel data set. As the metric to assess how well the model's output fits the gold ranking Spearman's $\rho $ was used. The higher Spearman's rank-order correlation the better the system's performance.

Answers:
As the metric to assess how well the model's output fits the gold ranking Spearman's $\rho $ was used
Spearman's rank-order correlation
Question: What is a normal reading paradigm?

Evidence:
raw_evidence: In the first task, participants were instructed to read the sentences naturally, without any specific task other than comprehension. Participants were told to read the sentences normally without any special instructions. Figure FIGREF8 (left) shows an example sentence as it was depicted on the screen during recording. As shown in Figure FIGREF8 (middle), the control condition for this task consisted of single-choice questions about the content of the previous sentence. 12% of randomly selected sentences were followed by such a comprehension question with three answer options on a separate screen.
raw_evidence: Corpus Construction ::: Experimental design ::: Normal reading (NR)
In the first task, participants were instructed to read the sentences naturally, without any specific task other than comprehension. Participants were told to read the sentences normally without any special instructions. Figure FIGREF8 (left) shows an example sentence as it was depicted on the screen during recording. As shown in Figure FIGREF8 (middle), the control condition for this task consisted of single-choice questions about the content of the previous sentence. 12% of randomly selected sentences were followed by such a comprehension question with three answer options on a separate screen.

Answers:
read the sentences normally without any special instructions
participants were instructed to read the sentences naturally, without any specific task other than comprehension
Question: Did they experiment with this new dataset?

Evidence:

Answers:
No
Question: What kind of sentences were read?

Evidence:
raw_evidence: We provide the first dataset of simultaneous eye movement and brain activity recordings to analyze and compare normal reading to task-specific reading during annotation. The Zurich Cognitive Language Processing Corpus (ZuCo) 2.0, including raw and preprocessed eye-tracking and electroencephalography (EEG) data of 18 subjects, as well as the recording and preprocessing scripts, is publicly available at https://osf.io/2urht/. It contains physiological data of each subject reading 739 English sentences from Wikipedia (see example in Figure FIGREF1). We want to highlight the re-use potential of this data. In addition to the psycholinguistic motivation, this corpus is especially tailored for training and evaluating machine learning algorithms for NLP purposes. We conduct a detailed technical validation of the data as proof of the quality of the recordings.
During the recording session, the participants read 739 sentences that were selected from the Wikipedia corpus provided by culotta2006integrating. This corpus was chosen because it provides annotations of semantic relations. We included seven of the originally defined relation types: political_affiliation, education, founder, wife/husband, job_title, nationality, and employer. The sentences were chosen in the same length range as ZuCo 1.0, and with similar Flesch reading ease scores. The dataset statistics are shown in Table TABREF2.
raw_evidence: During the recording session, the participants read 739 sentences that were selected from the Wikipedia corpus provided by culotta2006integrating. This corpus was chosen because it provides annotations of semantic relations. We included seven of the originally defined relation types: political_affiliation, education, founder, wife/husband, job_title, nationality, and employer. The sentences were chosen in the same length range as ZuCo 1.0, and with similar Flesch reading ease scores. The dataset statistics are shown in Table TABREF2.

Answers:
sentences that were selected from the Wikipedia corpus provided by culotta2006integrating
seven of the originally defined relation types: political_affiliation, education, founder, wife/husband, job_title, nationality, and employer
Question: why are their techniques cheaper to implement?

Evidence:
raw_evidence: In this paper, we have analyzed various ways to integrate monolingual data in an NMT framework, focusing on their impact on quality and domain adaptation. While confirming the effectiveness of BT, our study also proposed significantly cheaper ways to improve the baseline performance, using a slightly modified copy of the target, instead of its full BT. When no high quality BT is available, using GANs to make the pseudo-source sentences closer to natural source sentences is an efficient solution for domain adaptation.
raw_evidence: We now analyze the effect of using much simpler data generation schemes, which do not require the availability of a backward translation engine.

Answers:
They use a slightly modified copy of the target to create the pseudo-text instead of full BT to make their technique cheaper
They do not require the availability of a backward translation engine.
Question: what data simulation techniques were introduced?

Evidence:
raw_evidence: We use the following cheap ways to generate pseudo-source texts:
copy: in this setting, the source side is a mere copy of the target-side data. Since the source vocabulary of the NMT is fixed, copying the target sentences can cause the occurrence of OOVs. To avoid this situation, BIBREF24 decompose the target words into source-side units to make the copy look like source sentences. Each OOV found in the copy is split into smaller units until all the resulting chunks are in the source vocabulary.
copy-marked: another way to integrate copies without having to deal with OOVs is to augment the source vocabulary with a copy of the target vocabulary. In this setup, BIBREF25 ensure that both vocabularies never overlap by marking the target word copies with a special language identifier. Therefore the English word resume cannot be confused with the homographic French word, which is marked @fr@resume.
copy-dummies: instead of using actual copies, we replace each word with “dummy” tokens. We use this unrealistic setup to observe the training over noisy and hardly informative source sentences.
raw_evidence: We use the following cheap ways to generate pseudo-source texts:
copy: in this setting, the source side is a mere copy of the target-side data. Since the source vocabulary of the NMT is fixed, copying the target sentences can cause the occurrence of OOVs. To avoid this situation, BIBREF24 decompose the target words into source-side units to make the copy look like source sentences. Each OOV found in the copy is split into smaller units until all the resulting chunks are in the source vocabulary.
copy-marked: another way to integrate copies without having to deal with OOVs is to augment the source vocabulary with a copy of the target vocabulary. In this setup, BIBREF25 ensure that both vocabularies never overlap by marking the target word copies with a special language identifier. Therefore the English word resume cannot be confused with the homographic French word, which is marked @fr@resume.
copy-dummies: instead of using actual copies, we replace each word with “dummy” tokens. We use this unrealistic setup to observe the training over noisy and hardly informative source sentences.

Answers:
copy, copy-marked, copy-dummies
copy, copy-marked, copy-dummies
Question: what is their explanation for the effectiveness of back-translation?

Evidence:
raw_evidence: Comparing the natural and artificial sources of our parallel data wrt. several linguistic and distributional properties, we observe that (see Fig. FIGREF21 - FIGREF22 ):
artificial sources are on average shorter than natural ones: when using BT, cases where the source is shorter than the target are rarer; cases when they have the same length are more frequent.
automatic word alignments between artificial sources tend to be more monotonic than when using natural sources, as measured by the average Kendall INLINEFORM0 of source-target alignments BIBREF22 : for French-English the respective numbers are 0.048 (natural) and 0.018 (artificial); for German-English 0.068 and 0.053. Using more monotonic sentence pairs turns out to be a facilitating factor for NMT, as also noted by BIBREF20 .
The intuition is that properties (i) and (ii) should help translation as compared to natural source, while property (iv) should be detrimental. We checked (ii) by building systems with only 10M words from the natural parallel data selecting these data either randomly or based on the regularity of their word alignments. Results in Table TABREF23 show that the latter is much preferable for the overall performance. This might explain that the mostly monotonic BT from Moses are almost as good as the fluid BT from NMT and that both boost the baseline.

Answers:
when using BT, cases where the source is shorter than the target are rarer; cases when they have the same length are more frequent, automatic word alignments between artificial sources tend to be more monotonic than when using natural sources