qinchuanhui/UDA-QA huggingface dataset:
Question: What syntactic structure is used to model tones?

Evidence:
raw_evidence: Based on the above considerations, in this paper, we present CSSMCM, a sentence-level Chinese Mandarin lip reading network, which contains three sub-networks. Same as BIBREF5 , in the first sub-network, pinyin sequence is predicted from the video. Different from BIBREF5 , which predicts pinyin characters from video, pinyin is taken as a whole in CSSMCM, also known as syllables. As we know, Mandarin Chinese is a syllable-based language and syllables are their logical unit of pronunciation. Compared with pinyin characters, syllables are a longer linguistic unit, and can reduce the difficulty of syllable choices in the decoder by sequence-to-sequence attention-based models BIBREF6 . Chen et al. BIBREF7 find that there might be a relationship between the production of lexical tones and the visible movements of the neck, head, and mouth. Motivated by this observation, in the second sub-network, both video and pinyin sequence is used as input to predict tone. Then in the third sub-network, video, pinyin, and tone sequence work together to predict the Chinese character sequence. At last, three sub-networks are jointly finetuned to improve overall performance.

Answers:
syllables
Question: What visual information characterizes tones?

Evidence:
raw_evidence: As shown in Equation ( EQREF6 ), tone prediction sub-network ( INLINEFORM0 ) takes video and pinyin sequence as inputs and predict corresponding tone sequence. This problem is modeled as a sequence-to-sequence learning problem too. The corresponding model architecture is shown in Figure FIGREF8 .
In order to take both video and pinyin information into consideration when producing tone, a dual attention mechanism BIBREF3 is employed. Two independent attention mechanisms are used for video and pinyin sequence. Video context vectors INLINEFORM0 and pinyin context vectors INLINEFORM1 are fused when predicting a tone character at each decoder step.
The video encoder is the same as in Section SECREF7 and the pinyin encoder is: DISPLAYFORM0
The pinyin prediction sub-network transforms video sequence into pinyin sequence, which corresponds to INLINEFORM0 in Equation ( EQREF6 ). This sub-network is based on the sequence-to-sequence architecture with attention mechanism BIBREF8 . We name the encoder and decoder the video encoder and pinyin decoder, for the encoder process video sequence, and the decoder predicts pinyin sequence. The input video sequence is first fed into the VGG model BIBREF9 to extract visual feature. The output of conv5 of VGG is appended with global average pooling BIBREF10 to get the 512-dim feature vector. Then the 512-dim feature vector is fed into video encoder. The video encoder can be denoted as: DISPLAYFORM0

Answers:
video sequence is first fed into the VGG model BIBREF9 to extract visual feature
Question: How do they demonstrate the robustness of their results?

Evidence:
raw_evidence: The results are displayed in Figure FIGREF43 . In the standard setting (Figure FIGREF43 ), the models that have access to the context besides the content ( INLINEFORM0 ) and the models that are only allowed to access the context ( INLINEFORM1 ), always perform better than the content-based models ( INLINEFORM2 ) (bars above zero). However, when we randomly flip contexts of the test instances (Figure FIGREF43 ), or suppress them entirely (Figure FIGREF43 ), the opposite picture emerges: the content-based models always outperform the other models. For some classes (support, INLINEFORM3 ) the difference can exceed 50 F1 percentage points. These two studies, where testing examples are varied regarding their context (randomized-context or no-context) simulates what can be expected if we apply our systems for relation class assignment to EAUs stemming from heterogeneous sources. While the performances of a purely content-based model naturally stays stable, the performance of the other systems decrease notably – they perform worse than the content-based model.

Answers:
performances of a purely content-based model naturally stays stable
Question: What baseline and classification systems are used in experiments?

Evidence:
raw_evidence: The results in Table TABREF38 confirm the results of BIBREF13 and suggest that we successfully replicated a large proportion of their features.
The results for all three prediction settings (one outgoing edge: INLINEFORM0 , support/attack: INLINEFORM1 and support/attack/neither: INLINEFORM2 ) across all type variables ( INLINEFORM3 , INLINEFORM4 and INLINEFORM5 ) are displayed in Table TABREF39 . All models significantly outperform the majority baseline with respect to macro F1. Intriguingly, the content-ignorant models ( INLINEFORM6 ) always perform significantly better than the models which only have access to the EAUs' content ( INLINEFORM7 , INLINEFORM8 ). In the most general task formulation ( INLINEFORM9 ), we observe that INLINEFORM10 even significantly outperforms the model which has maximum access (seeing both EAU spans and surrounding contexts: INLINEFORM11 ).

Answers:
BIBREF13, majority baseline
Question: How are the EAU text spans annotated?

Evidence:
raw_evidence: Tree-based sentiment annotations are sentiment scores assigned to nodes in constituency parse trees BIBREF26 . We represent these scores by a one-hot vector of dimension 5 (5 is very positive, 1 is very negative). We determine the contextual ( INLINEFORM0 ) sentiment by looking at the highest possible node of the context which does not contain the EAU (ADVP in Figure FIGREF26 ). The sentiment for an EAU span ( INLINEFORM1 ) is assigned to the highest possible node covering the EAU span which does not contain the context sub-tree (S in Figure FIGREF26 ). The full-access ( INLINEFORM2 ) score is assigned to the lowest possible node which covers both the EAU span and its surrounding context (S' in Figure FIGREF26 ). Next to the sentiment scores for the selected tree nodes and analogously to the word embeddings, we also calculate the element-wise subtraction of the one-hot sentiment source vectors from the one-hot sentiment target vectors. This results in three additional vectors corresponding to INLINEFORM3 , INLINEFORM4 and INLINEFORM5 difference vectors.
Results

Answers:
Answer with content missing: (Data and pre-processing section) The data is suited for our experiments because the annotators were explicitly asked to provide annotations on a clausal level.
Question: Which Twitter corpus was used to train the word vectors?

Evidence:
raw_evidence: Twitter provides well-documented API, which allows to request any information about Tweets, users and their profiles, with respect to rate limits. There is special type of API, called Streaming API, that provides a real-time stream of tweets. The key difference with regular API is that connection is kept alive as long as possible, and Tweets are sent in real-time to the client. There are three endpoints of Streaming API of our interest: “sample”, “filter” and “firehose”. The first one provides a sample (random subset) of the full Tweet stream. The second one allows to receive Tweets matching some search criteria: matching to one or more search keywords, produced by subset of users, or coming from certain geo location. The last one provides the full set of Tweets, although it is not available by default. In order to get Twitter “firehose” one can contact Twitter, or buy this stream from third-parties.
In our case the simplest approach would be to use “sample” endpoint, but it provides Tweets in all possible languages from all over the World, while we are concerned only about one language (Russian). In order to use this endpoint we implemented filtering based on language. The filter is simple: if Tweet does not contain a substring of 3 or more cyrillic symbols, it is considered non-Russian. Although this approach keeps Tweets in Mongolian, Ukrainian and other slavic languages (because they use cyrillic alphabet), the total amount of false-positives in this case is negligible. To demonstrate this we conducted simple experiment: on a random sample of 200 tweets only 5 were in a language different from Russian. In order not to rely on Twitter language detection, we chose to proceed with this method of language-based filtering.
However, the amount of Tweets received through “sample” endpoint was not satisfying. This is probably because “sample” endpoint always streams the same content to all its clients, and small portion of it comes in Russian language. In order to force mining of Tweets in Russian language, we chose "filter" endpoint, which requires some search query. We constructed heuristic query, containing some auxiliary words, specific to Russian language: conjunctions, pronouns, prepositions. The full list is as follows:
russian я, у, к, в, по, на, ты, мы, до, на, она, он, и, да.

Answers:
They collected tweets in Russian language using a heuristic query specific to Russian
Question: How does proposed word embeddings compare to Sindhi fastText word representations?

Evidence:
raw_evidence: Generally, closer words are considered more important to a word’s meaning. The word embeddings models have the ability to capture the lexical relations between words. Identifying such relationship that connects words is important in NLP applications. We measure that semantic relationship by calculating the dot product of two vectors using Eq. DISPLAY_FORM48. The high cosine similarity score denotes the closer words in the embedding matrix, while less cosine similarity score means the higher distance between word pairs. We present the cosine similarity score of different semantically or syntactically related word pairs taken from the vocabulary in Table TABREF77 along with English translation, which shows the average similarity of 0.632, 0.650, 0.591 yields by CBoW, SG and GloVe respectively. The SG model achieved a high average similarity score of 0.650 followed by CBoW with a 0.632 average similarity score. The GloVe also achieved a considerable average score of 0.591 respectively. However, the average similarity score of SdfastText is 0.388 and the word pair Microsoft-Bill Gates is not available in the vocabulary of SdfastText. This shows that along with performance, the vocabulary in SdfastText is also limited as compared to our proposed word embeddings.
Moreover, the average semantic relatedness similarity score between countries and their capitals is shown in Table TABREF78 with English translation, where SG also yields the best average score of 0.663 followed by CBoW with 0.611 similarity score. The GloVe also yields better semantic relatedness of 0.576 and the SdfastText yield an average score of 0.391. The first query word China-Beijing is not available the vocabulary of SdfastText. However, the similarity score between Afghanistan-Kabul is lower in our proposed CBoW, SG, GloVe models because the word Kabul is the name of the capital of Afghanistan as well as it frequently appears as an adjective in Sindhi text which means able.

Answers:
Proposed SG model vs SINDHI FASTTEXT:
Average cosine similarity score: 0.650 vs 0.388
Average semantic relatedness similarity score between countries and their capitals: 0.663 vs 0.391
Question: How is the data collected, which web resources were used?

Evidence:
raw_evidence: The corpus is a collection of human language text BIBREF31 built with a specific purpose. However, the statistical analysis of the corpus provides quantitative, reusable data, and an opportunity to examine intuitions and ideas about language. Therefore, the corpus has great importance for the study of written language to examine the text. In fact, realizing the necessity of large text corpus for Sindhi, we started this research by collecting raw corpus from multiple web resource using web-scrappy framwork for extraction of news columns of daily Kawish and Awami Awaz Sindhi newspapers, Wikipedia dumps, short stories and sports news from Wichaar social blog, news from Focus Word press blog, historical writings, novels, stories, books from Sindh Salamat literary websites, novels, history and religious books from Sindhi Adabi Board and tweets regarding news and sports are collected from twitter.

Answers:
daily Kawish and Awami Awaz Sindhi newspapers, Wikipedia dumps, short stories and sports news from Wichaar social blog, news from Focus Word press blog, historical writings, novels, stories, books from Sindh Salamat literary website, novels, history and religious books from Sindhi Adabi Board,  tweets regarding news and sports are collected from twitter
Question: What trends are found in musical preferences?

Evidence:
raw_evidence: How the music taste of the audience of popular music changed in the last century? The trend lines of the MUSIC model features, reported in figure FIGREF12, reveal that audiences wanted products more and more contemporary, intense and a little bit novel or sophisticated, but less and less mellow and (surprisingly) unpretentious. In other words, the audiences of popular music are getting more demanding as the quality and variety of the music products increases.

Answers:
audiences wanted products more and more contemporary, intense and a little bit novel or sophisticated, but less and less mellow and (surprisingly) unpretentious
Question: Which decades did they look at?

Evidence:
raw_evidence: time: decade (classes between 1900s and 2010s) and year representative of the time when the genre became meainstream

Answers:
between 1900s and 2010s
Question: How many genres did they collect from?

Evidence:
raw_evidence: From the description of music genres provided above emerges that there is a limited number of super-genres and derivation lines BIBREF19, BIBREF20, as shown in figure FIGREF1.
From a computational perspective, genres are classes and, although can be treated by machine learning algorithms, they do not include information about the relations between them. In order to formalize the relations between genres for computing purposes, we define a continuous genre scale from the most experimental and introverted super-genre to the most euphoric and inclusive one. We selected from Wikipedia the 77 genres that we mentioned in bold in the previous paragraph and asked to two independent raters to read the Wikipedia pages of the genres, listen to samples or artists of the genres (if they did not know already) and then annotate the following dimensions:

Answers:
77 genres
Question: Does the paper mention other works proposing methods to detect anglicisms in Spanish?

Evidence:
raw_evidence: In terms of automatic detection of anglicisms, previous approaches in different languages have mostly depended on resource lookup (lexicon or corpus frequencies), character n-grams and pattern matching. alex-2008-comparing combined lexicon lookup and a search engine module that used the web as a corpus to detect English inclusions in a corpus of German texts and compared her results with a maxent Markov model. furiassi2007retrieval explored corpora lookup and character n-grams to extract false anglicisms from a corpus of Italian newspapers. andersen2012semi used dictionary lookup, regular expressions and lexicon-derived frequencies of character n-grams to detect anglicism candidates in the Norwegian Newspaper Corpus (NNC) BIBREF21, while losnegaard2012data explored a Machine Learning approach to anglicism detection in Norwegian by using TiMBL (Tilburg Memory-Based Learner, an implementation of a k-nearest neighbor classifier) with character trigrams as features. garley-hockenmaier-2012-beefmoves trained a maxent classifier with character n-gram and morphological features to identify anglicisms in German online communities. In Spanish, serigos2017using extracted anglicisms from a corpus of Argentinian newspapers by combining dictionary lookup (aided by TreeTagger and the NLTK lemmatizer) with automatic filtering of capitalized words and manual inspection. In serigos2017applying, a character n-gram module was added to estimate the probabilities of a word being English or Spanish. moreno2018configuracion used different pattern-matching filters and lexicon lookup to extract anglicism cadidates from a corpus of tweets in US Spanish.

Answers:
Yes
Question: What is the performance of the CRF model on the task described?

Evidence:
raw_evidence: Results on all sets show an important difference between precision and recall, precision being significantly higher than recall. There is also a significant difference between the results obtained on development and test set (F1 = 89.60, F1 = 87.82) and the results on the supplemental test set (F1 = 71.49). The time difference between the supplemental test set and the development and test set (the headlines from the the supplemental test set being from a different time period to the training set) can probably explain these differences.

Answers:
the results obtained on development and test set (F1 = 89.60, F1 = 87.82) and the results on the supplemental test set (F1 = 71.49)
Question: Does the paper motivate the use of CRF as the baseline model?

Evidence:
raw_evidence: A baseline model for automatic extraction of anglicisms was created using the annotated corpus we just presented as training material. As mentioned in Section 3, the task of detecting anglicisms can be approached as a sequence labeling problem where only certain spans of texts will be labeled as anglicism (in a similar way to an NER task). The chosen model was conditional random field model (CRF), which was also the most popular model in both Shared Tasks on Language Identification for Code-Switched Data BIBREF23, BIBREF24.

Answers:
the task of detecting anglicisms can be approached as a sequence labeling problem where only certain spans of texts will be labeled as anglicism (in a similar way to an NER task). The chosen model was conditional random field model (CRF), which was also the most popular model in both Shared Tasks on Language Identification for Code-Switched Data
Question: What are the handcrafted features used?

Evidence:
raw_evidence: The following handcrafted features were used for the model:
Bias feature
Token feature
Uppercase feature (y/n)
Titlecase feature (y/n)
Character trigram feature
Quotation feature (y/n)
Word suffix feature (last three characters)
POS tag (provided by spaCy utilities)
Word shape (provided by spaCy utilities)
Word embedding (see Table TABREF26)

Answers:
Bias feature, Token feature, Uppercase feature (y/n), Titlecase feature (y/n), Character trigram feature, Quotation feature (y/n), Word suffix feature (last three characters), POS tag (provided by spaCy utilities), Word shape (provided by spaCy utilities), Word embedding (see Table TABREF26)
Question: What are three new proposed architectures?

Evidence:
raw_evidence: Let us propose two further extensions of this baseline architecture. To improve reproducibility of the research the code of the studied models is open. Both extensions aim to improve the quality of information decomposition within the latent representation. In the first one, shown in Figure FIGREF12, a special dedicated discriminator is added to the model to control that the latent representation does not contain stylistic information. The loss of this discriminator is defined as
The second extension of the baseline architecture does not use an adversarial component $D_z$ that is trying to eradicate information on $c$ from component $z$. Instead, the system, shown in Figure FIGREF16 feeds the "soft" generated sentence $\tilde{G}$ into encoder $E$ and checks how close is the representation $E(\tilde{G} )$ to the original representation $z = E(x)$ in terms of the cosine distance. We further refer to it as shifted autoencoder or SAE. Ideally, both $E(\tilde{G} (E(x), c))$ and $E(\tilde{G} (E(x), \bar{c}))$, where $\bar{c}$ denotes an inverse style code, should be both equal to $E(x)$. The loss of the shifted autoencoder is
We also study a combination of both approaches described above, shown on Figure FIGREF17.

Answers:
special dedicated discriminator is added to the model to control that the latent representation does not contain stylistic information, shifted autoencoder or SAE, combination of both approaches
Question: How much does the standard metrics for style accuracy vary on different re-runs?

Evidence:
raw_evidence: On Figure FIGREF1 one can see that the outcomes for every single rerun differ significantly. Namely, accuracy can change up to 5 percentage points, whereas BLEU can vary up to 8 points. This variance can be partially explained with the stochasticity incurred due to sampling from the latent variables. However, we show that results for state of the art models sometimes end up within error margins from one another, so one has to report the margins to compare the results rigorously. More importantly, one can see that there is an inherent trade-off between these two performance metrics. This trade-off is not only visible across models but is also present for the same retrained architecture. Therefore, improving one of the two metrics is not enough to confidently state that one system solves the style-transfer problem better than the other. One has to report error margins after several consecutive retrains and instead of comparing one of the two metrics has to talk about Pareto-like optimization that would show confident improvement of both.

Answers:
accuracy can change up to 5 percentage points, whereas BLEU can vary up to 8 points
Question: Which baseline methods are used?

Evidence:
raw_evidence: Table 1 shows the BLEU scores of our model on different sequence lengths while varying $K$ . This is a study of the trade-off between computational time and representational power. A large $K$ allows us to compute complex source representations, while a $K$ of 1 limits the source representation to a single vector. We can see that performance consistently increases with $K$ up to a point that depends on the data length, with longer sequences requiring more complex representations. The results with and without position encodings are almost identical on the toy data. Our technique learns to fit the data as well as the standard attention mechanism despite having less representational power. Both beat the non-attention baseline by a significant margin.
All models are implemented using TensorFlow based on the seq2seq implementation of BIBREF15 and trained on a single machine with a Nvidia K40m GPU. We use a 2-layer 256-unit, a bidirectional LSTM BIBREF16 encoder, a 2-layer 256-unit LSTM decoder, and 256-dimensional embeddings. For the attention baseline, we use the standard parametrized attention BIBREF2 . Dropout of 0.2 (0.8 keep probability) is applied to the input of each cell and we optimize using Adam BIBREF17 at a learning rate of 0.0001 and batch size 128. We train for at most 200,000 steps (see Figure 3 for sample learning curves). BLEU scores are calculated on tokenized data using the multi-bleu.perl script in Moses. We decode using beam search with a beam

Answers:
standard parametrized attention and a non-attention baseline