qinchuanhui/UDA-QA huggingface dataset:
Question: How big is vSTS training data?

Evidence:
raw_evidence: The full dataset comprises both the sample mentioned above and the 819 pairs from our preliminary work, totalling 2677 pairs. Figure FIGREF14 shows the final item similarity distribution. Although the distribution is skewed towards lower similarity values, we consider that all the similarity ranges are sufficiently well covered.
We split the vSTS dataset into training, validation and test partitions sampling at random and preserving the overall score distributions. In total, we use 1338 pairs for training, 669 for validation, and the rest of the 670 pairs were used for the final testing. Similar to the STS task, we use the Pearson correlation coefficient ($\rho $) as the evaluation metric of the task.

Answers:
1338 pairs for training
Question: Which evaluation metrics do they use for language modelling?

Evidence:
raw_evidence: The functional dissimilarity score was computed using sentences from the test set in CoNLL 2017 Universal Dependencies task BIBREF20 for the relevant languages with the provided UPOS sequences. Furthermore, none of the evaluated models, including the proposed method, were trained with CoNLL2017 data.
We computed the nearest neighbours experiment for all languages in the training data for the above models. The results are shown in Table TABREF27. The results show that general purpose language models do capture syntax information, which varies greatly across languages and models.

Answers:
 functional dissimilarity score, nearest neighbours experiment
Question: Do they do quantitative quality analysis of learned embeddings?

Evidence:
raw_evidence: The high nearest neighbours accuracy indicates that syntax information was successfully captured by the embeddings. Table TABREF22 also shows that the syntactic information of multiple languages was captured by a single embedding model.

Answers:
Yes
Question: Do they evaluate on downstream tasks?

Evidence:
raw_evidence: Many NLP tasks utilize POS as features, but human annotated POS sequences are difficult and expensive to obtain. Thus, it is important to know if we can learn sentences-level syntactic embeddings for low-sources languages without treebanks.
We performed zero-shot transfer of the syntactic embeddings for French, Portuguese and Indonesian. French and Portuguese are simulated low-resource languages, while Indonesian is a true low-resource language. We reported the 1-NN and 5-NN accuracies for all languages using the same evaluation setting as described in the previous section. The results are shown in Table TABREF31 (top).

Answers:
Yes
Question: Which corpus do they use?

Evidence:
raw_evidence: To create our training dataset, we followed an approach similar to LASER. The dataset contains 6 languages: English, Spanish, German, Dutch, Korean and Chinese Mandarin. These languages use 3 different scripts, 2 different language orderings, and belong to 4 language families.
The dataset was created by using translations provided by Tatoeba and OpenSubtitles BIBREF16. They were chosen for their high availability in multiple languages.

Answers:
The dataset was created by using translations provided by Tatoeba and OpenSubtitles BIBREF16.
Question: How big is improvement over the old  state-of-the-art performance on CoNLL-2009 dataset?

Evidence:
raw_evidence: Table TABREF46 shows that our Open model achieves more than 3 points of f1-score than the state-of-the-art result, and RelAwe with DepPath&RelPath achieves the best in both Closed and Open settings. Notice that our best Closed model can almost perform as well as the state-of-the-art model while the latter utilizes pre-trained word embeddings. Besides, performance gap between three models under Open setting is very small. It indicates that the representation ability of BERT is so powerful and may contains rich syntactic information. At last, the Gold result is much higher than the other models, indicating that there is still large space for improvement for this task.

Answers:
our Open model achieves more than 3 points of f1-score than the state-of-the-art result
Question: What different approaches of encoding syntactic information authors present?

Evidence:
raw_evidence: The most intuitive way to represent syntactic information is to use individual dependency relations directly, like dependency head and dependency relation label, denoted as Dep and Rel for short.
In order to preserve the structural information of dependency trees as much as possible, we take the syntactic path between candidate arguments and predicates in dependency trees as linguistic knowledge. Referring to BIBREF9, we use the Tree-based Position Feature (TPF) as Dependency Path (DepPath) and use the Shortest Dependency Path (SDP) as Relation Path (RelPath).

Answers:
dependency head and dependency relation label, denoted as Dep and Rel for short, Tree-based Position Feature (TPF) as Dependency Path (DepPath), Shortest Dependency Path (SDP) as Relation Path (RelPath)
Question: What domain does the dataset fall into?

Evidence:
raw_evidence: We perform our experiments using ActivityNet Captions dataset BIBREF2 that is considered as the standard benchmark for dense video captioning task. The dataset contains approximately 20k videos from YouTube and split into 50/25/25 % parts for training, validation, and testing, respectively. Each video, on average, contains 3.65 temporally localized captions, around 13.65 words each, and two minutes long. In addition, each video in the validation set is annotated twice by different annotators. We report all results using the validation set (no ground truth is provided for the test set).

Answers:
YouTube videos
Question: What ASR system do they use?

Evidence:
raw_evidence: The dataset itself is distributed as a collection of links to YouTube videos, some of which are no longer available. Authors provide pre-computed C3D features and frames at 5fps, but these are not suitable for our experiments. At the time of writing, we found 9,167 (out of 10,009) training and 4,483 (out of 4,917) validation videos which is, roughly, 91 % of the dataset. Out of these 2,798 training and 1,374 validation videos (approx. 28 %) contain at least one speech segment. The speech content was obtained from the closed captions (CC) provided by the YouTube ASR system which can be though as subtitles.

Answers:
YouTube ASR system
Question: How big are datasets used in experiments?

Evidence:
raw_evidence: There is usually a trade-off between the number of training samples and the number of trainable parameters in a deep network model BIBREF16. In general, more data we have, better result are generated by supervised deep learning methods. Data augmentation helps to increase the number of training data, but a bigger dataset needs a better and most likely bigger network architecture in terms of generalization. Otherwise, the model might over-fitted or under-fitted on training data. Using our annotation software, we automatically extracted landmarks of 2000 images from the UOttawa database BIBREF14 had been annotated for image segmentation tasks. The database was randomly divided into three sets: 90$\%$ training, 5$\%$ validation, and 5$\%$ testing datasets. For testing, we also applied the TongueNet on the UBC database BIBREF14 without any training to see the generalization ability of the model. During the training process of TongueNet, we employed an online data augmentation, including rotation (-25 to 25 degrees), translation (-30 to 30 points in two directions), scaling (from 0.5 to 2 times), horizontal flipping, and combination of these transformations, and annotation point locations are also was transformed, correspondingly. From our extensive random search hyper-parameter tuning, learning rate, the number of iterations, mini-batch sizes, the number of epochs was determined as 0.0005, 1000, 30, 10, respectively. We deployed our experiments using Keras with TensorFlow as the backend on a Windows PC with Core i7, 4.2 GHz speed using one NVIDIA 1080 GPU unit, and 32 GB of RAM. Adam optimization with fixed momentum values of 0.9, was utilized for training.

Answers:
2000 images
Question: What previously annotated databases are available?

Evidence:
raw_evidence: There is usually a trade-off between the number of training samples and the number of trainable parameters in a deep network model BIBREF16. In general, more data we have, better result are generated by supervised deep learning methods. Data augmentation helps to increase the number of training data, but a bigger dataset needs a better and most likely bigger network architecture in terms of generalization. Otherwise, the model might over-fitted or under-fitted on training data. Using our annotation software, we automatically extracted landmarks of 2000 images from the UOttawa database BIBREF14 had been annotated for image segmentation tasks. The database was randomly divided into three sets: 90$\%$ training, 5$\%$ validation, and 5$\%$ testing datasets. For testing, we also applied the TongueNet on the UBC database BIBREF14 without any training to see the generalization ability of the model. During the training process of TongueNet, we employed an online data augmentation, including rotation (-25 to 25 degrees), translation (-30 to 30 points in two directions), scaling (from 0.5 to 2 times), horizontal flipping, and combination of these transformations, and annotation point locations are also was transformed, correspondingly. From our extensive random search hyper-parameter tuning, learning rate, the number of iterations, mini-batch sizes, the number of epochs was determined as 0.0005, 1000, 30, 10, respectively. We deployed our experiments using Keras with TensorFlow as the backend on a Windows PC with Core i7, 4.2 GHz speed using one NVIDIA 1080 GPU unit, and 32 GB of RAM. Adam optimization with fixed momentum values of 0.9, was utilized for training.

Answers:
the UBC database BIBREF14
Question: Do they argue that all words can be derived from other (elementary) words?

Evidence:
raw_evidence: In order to understand what INLINEFORM0 is, we need to understand the mathematics of grammar. The study of the mathematical structure of grammar has indicated that the fundamental things making up sentences are not the words, but some atomic grammatical types, such as the noun-type and the sentence-type BIBREF23 , BIBREF24 , BIBREF25 . The transitive verb-type is not an atomic grammatical type, but a composite made up of two noun-types and one sentence-type. Hence, particularly interesting here is that atomic doesn't really mean smallest...

Answers:
No
Question: Do they break down word meanings into elementary particles as in the standard model of quantum theory?

Evidence:
raw_evidence: In order to understand what INLINEFORM0 is, we need to understand the mathematics of grammar. The study of the mathematical structure of grammar has indicated that the fundamental things making up sentences are not the words, but some atomic grammatical types, such as the noun-type and the sentence-type BIBREF23 , BIBREF24 , BIBREF25 . The transitive verb-type is not an atomic grammatical type, but a composite made up of two noun-types and one sentence-type. Hence, particularly interesting here is that atomic doesn't really mean smallest...
On the other hand, just like in particle physics where we have particles and anti-particles, the atomic types include types as well as anti-types. But unlike in particle physics, there are two kinds of anti-types, namely left ones and right ones. This makes language even more non-commutative than quantum theory!

Answers:
No
Question: How big is the dataset used?

Evidence:
raw_evidence: We utilize the VATEX dataset for video captioning, which contains over 41,250 videos and 825,000 captions in both English and Chinese. Among the captions, there are over 206,000 English-Chinese parallel translation pairs. It covers 600 human activities and a variety of video content. Each video is paired with 10 English and 10 Chinese diverse captions. We follow the official split with 25,991 videos for training, 3,000 videos for validation and 6,000 public test videos for final testing.

Answers:
over 41,250 videos and 825,000 captions in both English and Chinese., over 206,000 English-Chinese parallel translation pairs
Question: How they prove that multi-head self-attention is at least as powerful as convolution layer? 

Evidence:
raw_evidence: The theorem is proven constructively by selecting the parameters of the multi-head self-attention layer so that the latter acts like a convolutional layer. In the proposed construction, the attention scores of each self-attention head should attend to a different relative shift within the set $\Delta \!\!\!\!\Delta _K = \lbrace -\lfloor K/2 \rfloor , \dots , \lfloor K/2 \rfloor \rbrace ^2$ of all pixel shifts in a $K\times K$ kernel. The exact condition can be found in the statement of Lemma UNKREF15.

Answers:
constructively by selecting the parameters of the multi-head self-attention layer so that the latter acts like a convolutional layer
Question: Is there any nonnumerical experiment that also support author's claim, like analysis of attention layers in publicly available networks? 

Evidence:

Answers:
No
Question: What numerical experiments they perform?

Evidence:
raw_evidence: The aim of this section is to validate the applicability of our theoretical results—which state that self-attention can perform convolution—and to examine whether self-attention layers in practice do actually learn to operate like convolutional layers, when being trained on standard image classification tasks. In particular, we study the relationship between self-attention and convolution with quadratic and learned relative positional encodings. We find that for both cases, the attention probabilities learned tend to respect the conditions of Lemma UNKREF15, corroborating our hypothesis.
We study a fully attentional model consisting of six multi-head self-attention layers. As it has already been shown by BIBREF9 that combining attention features with convolutional features improves performance on Cifar-100 and ImageNet, we do not focus on attaining state-of-the-art performance. Nevertheless, to validate that our model learns a meaningful classifier we compare it to the standard ResNet18 BIBREF14 on the CIFAR-10 dataset BIBREF15. In all experiments, we use a $2\times 2$ invertible down-sampling BIBREF16 on the input to reduce the size of the image as storing the attention coefficient tensor requires a large amount of GPU memory. The fixed size representation of the input image is computed as the average pooling of the last layer representations and given to a linear classifier.

Answers:
attention probabilities learned tend to respect the conditions of Lemma UNKREF15, corroborating our hypothesis, validate that our model learns a meaningful classifier we compare it to the standard ResNet18
Question: What dataset is used?

Evidence:
raw_evidence: We evaluate our model on both English and Chinese segmentation. For both languages we used standard datasets for word segmentation and language modeling. For all datasets, we used train, validation and test splits. Since our model assumes a closed character set, we removed validation and test samples which contain characters that do not appear in the training set. In the English corpora, whitespace characters are removed. In Chinese, they are not present to begin with. Refer to Appendix SECREF9 for dataset statistics.
The Brent corpus is a standard corpus used in statistical modeling of child language acquisition BIBREF15 , BIBREF16 . The corpus contains transcriptions of utterances directed at 13- to 23-month-old children. The corpus has two variants: an orthographic one (BR-text) and a phonemic one (BR-phono), where each character corresponds to a single English phoneme. As the Brent corpus does not have a standard train and test split, and we want to tune the parameters by measuring the fit to held-out data, we used the first 80% of the utterances for training and the next 10% for validation and the rest for test.
We use the commonly used version of the PTB prepared by BIBREF17 . However, since we removed space symbols from the corpus, our cross entropy results cannot be compared to those usually reported on this dataset.
Since Chinese orthography does not mark spaces between words, there have been a number of efforts to annotate word boundaries. We evaluate against two corpora that have been manually segmented according different segmentation standards.
The Beijing University Corpus was one of the corpora used for the International Chinese Word Segmentation Bakeoff BIBREF18 .
We use the Penn Chinese Treebank Version 5.1 BIBREF19 . It generally has a coarser segmentation than PKU (e.g., in CTB a full name, consisting of a given name and family name, is a single token), and it is a larger corpus.

Answers:
Brent corpus, PTB , Beijing University Corpus, Penn Chinese Treebank
Question: What language do they look at?

Evidence:
raw_evidence: We evaluate our model on both English and Chinese segmentation. For both languages we used standard datasets for word segmentation and language modeling. For all datasets, we used train, validation and test splits. Since our model assumes a closed character set, we removed validation and test samples which contain characters that do not appear in the training set. In the English corpora, whitespace characters are removed. In Chinese, they are not present to begin with. Refer to Appendix SECREF9 for dataset statistics.

Answers:
English, Chinese
Question: What dierse domains and languages are present in new datasets?

Evidence:
raw_evidence: We provide three pairs of short/long datasets from different domains (movies and restaurants) and from different languages (English and Korean) suitable for the task: Mov_en, Res_en, and Mov_ko. Most of the datasets are from previous literature and are gathered differently The Mov_en datasets are gathered from different websites; the short dataset consists of hand-picked sentences by BIBREF19 from document-level reviews from the Rotten Tomatoes website, while the long dataset consists of reviews from the IMDB website obtained by BIBREF20. The Res_en dataset consists of reviews from Yelp, where the short dataset consists of reviews with character lengths less than 140 from BIBREF21, while reviews in the long dataset are gathered from BIBREF20. We also share new short/long datasets Mov_ko, which are gathered from two different channels, as shown in Figure FIGREF4, available in Naver Movies. Unlike previous datasets BIBREF9, BIBREF22 where they used polarity/binary (e.g., positive or negative) labels as classes, we also provide fine-grained classes, with five classes of different sentiment intensities (e.g., 1 is strong negative, 5 is strong positive), for Res_en and Mov_ko. Following the Cross Domain Transfer setting BIBREF9, BIBREF23, BIBREF24, we limit the size of the dataset to be small-scale to focus on the main task at hand. This ensures that models focus on the transfer task, and decrease the influence of other factors that can be found when using larger datasets. Finally, following BIBREF22, we provide additional unlabeled data for those models that need them BIBREF9, BIBREF23, except for the long dataset of Mov_ko, where the labeled reviews are very limited. We show the dataset statistics in Table TABREF9, and share the datasets here: https://github.com/rktamplayo/LeTraNets.

Answers:
movies , restaurants, English , Korean
Question: Are their corpus and software public?

Evidence:
raw_evidence: Our system, including software and corpus, is available as an open source project for free research purpose and we believe that it is a good baseline for the development and comparison of future Vietnamese SRL systems. We plan to integrate this tool to Vitk, an open-source toolkit for processing Vietnamese text, which contains fundamental processing tools and are readily scalable for processing very large text data.

Answers:
Yes
Question: What are the currently available datasets for EAC?

Evidence:
raw_evidence: Nowadays, most of chatbots technologies were built by using neural-based approach. Emotional Chatting Machine (ECM) BIBREF15 was the first works which exploiting deep learning approach in building a large-scale emotionally-aware conversational bot. Then several studies were proposed to deal with this research area by introducing emotion embedding representation BIBREF24 , BIBREF25 , BIBREF26 or modeling as reinforcement learning problem BIBREF27 , BIBREF28 . Most of these studies used encoder-decoder architecture, specifically sequence to sequence (seq2seq) learning. Some works also tried to introduce a new dataset in order to have a better gold standard and improve system performance. BIBREF14 introduce EMPATHETICDIALOGUES dataset, a novel dataset containing 25k conversations include emotional contexts information to facilitate training and evaluating the textual conversational system. Then, work from BIBREF2 produce a dataset containing 1.5 million Twitter conversation, gathered by using Twitter API from customer care account of 62 brands across several industries. This dataset was used to build tone-aware customer care chatbot. Finally, BIBREF29 tried to enhance SEMAINE corpus BIBREF30 by using crowdsourcing scenario to obtain a human judgement for deciding which response that elicits positive emotion. Their dataset was used to develop a chatbot which captures human emotional states and elicits positive emotion during the conversation.

Answers:
EMPATHETICDIALOGUES dataset, a dataset containing 1.5 million Twitter conversation, gathered by using Twitter API from customer care account of 62 brands across several industries, SEMAINE corpus BIBREF30