niryuu/sni-each-converted huggingface dataset:
Very recently, the success of deep neural networks in many natural language processing tasks ( BIBREF20 ) has inspired new work in abstractive summarization . BIBREF2 propose a neural attention model with a convolutional encoder to solve this task.  More recently, BIBREF4 extended BIBREF2 's work with an RNN decoder, and BIBREF8 proposed an RNN encoder-decoder architecture for summarization. Both techniques are currently the state-of-the-art on the DUC competition.
Output: What is the state-of-the art?
The annotator carried out all annotation.
Output: How many annotators tagged each tweet?
. This work can be extended to predict future events with one day in advance, where we will use the same method for feature selection in addition to to time series analysis of the historical patterns of the word-pairs.
Output: Do the authors suggest any future extensions to this work?
This way, we can label 77 new pro-Russian edges by looking at 415 tweets, which means that 19% of the candidates are hits. For the pro-Ukrainian class, we can label 110 new edges by looking at 611 tweets (18% hits). Hence even though the quality of the classifier predictions is too low to be integrated into the network analysis right away, the classifier drastically facilitates the annotation process for human annotators compared to annotating unfiltered tweets (from the original labels we infer that for unfiltered tweets, only 6% are hits for the pro-Russian class, and 11% for the pro-Ukrainian class).
Output: How can the classifier facilitate the annotation task for human annotators?
We also observe that the best combination seems to consist in training our model on the original out-of-context dataset and testing it on the in-context pairs. In this configuration we reach an F-score (0.72) only slightly lower than the one reported in BIBREF3 (0.74), and we record the highest Pearson correlation, 0.3 (which is still not strong, compared to BIBREF3 's best run, 0.75). 
Output: What were the results of the first experiment?
The following handcrafted features were used for the model:

Bias feature

Token feature

Uppercase feature (y/n)

Titlecase feature (y/n)

Character trigram feature

Quotation feature (y/n)

Word suffix feature (last three characters)

POS tag (provided by spaCy utilities)

Word shape (provided by spaCy utilities)

Word embedding (see Table TABREF26)
Output: What are the handcrafted features used?
Single-Turn: This dataset consists of single-turn instances of statements and responses from the MiM chatbot developed at Constellation AI BIBREF5. Multi-Turn: This dataset is taken from the ConvAI2 challenge and consists of various types of dialogue that have been generated by human-computer conversations.
Output: what datasets did they use?
First, we adapt a dataset extracted from Freebase, i.e., FB24K, which used by BIBREF26. Then, we collect extra entities and relations that from DBpedia which that they should have at least 100 mentions BIBREF7 and they could link to the entities in the FB24K by the sameAs triples. Finally, we build a datasets named as DBP24K. In addition, we build a game datasets from our game knowledge graph, named as Game30K.
Output: What three datasets are used to measure performance?
At the end of the annotation process, we have annotated 700 tweets, where 175 tweets are in favor of and 175 tweets are against Target-1, and similarly 175 tweets are in favor of and 175 are against Target-2. 
Output: How many tweets did they collect?
To build the large ancient-modern Chinese dataset, we collected 1.7K bilingual ancient-modern Chinese articles from the internet. More specifically, a large part of the ancient Chinese data we used come from ancient Chinese history records in several dynasties (about 1000BC-200BC) and articles written by celebrities of that era.
Output: Where does the ancient Chinese dataset come from?
We feed the above-described hand-crafted features together with the task-specific embeddings learned by the deep neural neural network (a total of 1,892 attributes combined) into a Support Vector Machines (SVM) classifier BIBREF37 .
Output: what classifiers were used in this paper?
We end up eliminating 5.86% of total characters, and end up with 45,821 characters and 12,815 unique HLA, resulting in 945,519 total character-HLA pairs.  We end up eliminating 5.86% of total characters, and end up with 45,821 characters and 12,815 unique HLA, resulting in 945,519 total character-HLA pairs.
Output: How many different characters were in dataset?
In addition, we include state-of-the-art models : RegSum BIBREF0 and GCN+PADG BIBREF3. We outperform both in terms of ROUGE-1. For ROUGE-2 scores we achieve better results than GCN+PADG but without any use of domain-specific hand-crafted features and a much smaller and simpler model. Finally, RegSum achieves a similar ROUGE-2 score but computes sentence saliences based on word scores, incorporating a rich set of word-level and domain-specific features.
Output: How better are state-of-the-art results than this model?
We pre-trained our model on the Dutch section of the OSCAR corpus, a large multilingual corpus which was obtained by language classification in the Common Crawl corpus BIBREF16.
Output: What data did they use?
Therefore, we implement reinforcement learning and elaborately design two rewards to describe the irony accuracy and sentiment preservation, respectively.
Output: What is the combination of rewards for reinforcement learning?
We introduce a surrogate training objective that avoids these problems and as a result is fully continuous.  Specifically, we form a continuous function softLB that seeks to approximate the result of running our decoder on input INLINEFORM0 and then evaluating the result against INLINEFORM1 using INLINEFORM2 .
Output: Do they provide a framework for building a sub-differentiable for any final loss metric?
Turning to semantics, using visualizations of the activations created by different pieces of text, we show suggestive evidence that BERT distinguishes word senses at a very fine level.  We apply attention probes to the task of identifying the existence and type of dependency relation between two words.
Output: How were the feature representations evaluated?
This corpus has multiple versions, and we choose the following two versions as their test dataset has significantly larger number of instances of multiple relation tuples with overlapping entities. (i) The first version is used by BIBREF6 (BIBREF6) (mentioned as NYT in their paper) and has 24 relations. We name this version as NYT24. (ii) The second version is used by BIBREF11 (BIBREF11) (mentioned as NYT10 in their paper) and has 29 relations.
Output: Are there datasets with relation tuples annotated, how big are datasets available?
To fine-tune GPT2 for text generation, it is typical to concatenate the conditioning context $X = x_1 \ldots x_n$ and citing sentence $Y = y_1 \ldots y_m$ with a special separator token $\mho $. The abstract of each document is embedded into a single dense vector by averaging the contextualized embeddings provided by the SciBERT model of BIBREF11 and normalizing.
Output: Which baselines are explored?
In particular, we use Semeval 2014 BIBREF34 Twitter Sentiment Analysis Dataset for the training Sarcasm Datasets Used in the Experiment
This dataset was created by BIBREF8 . The tweets were downloaded from Twitter using #sarcasm as a marker for sarcastic tweets. It is a monolingual English dataset which consists of a balanced distribution of 50,000 sarcastic tweets and 50,000 non-sarcastic tweets.

Since sarcastic tweets are less frequently used BIBREF8 , we also need to investigate the robustness of the selected features and the model trained on these features on an imbalanced dataset. To this end, we used another English dataset from BIBREF8 . It consists of 25,000 sarcastic tweets and 75,000 non-sarcastic tweets.

We have obtained this dataset from The Sarcasm Detector. It contains 120,000 tweets, out of which 20,000 are sarcastic and 100,000 are non-sarcastic. We randomly sampled 10,000 sarcastic and 20,000 non-sarcastic tweets from the dataset. Visualization of both the original and subset data show similar characteristics.
Output: Which benchmark datasets are used?
We define the Effective Word Score of score x as

EFWS(x) = N(+x) - N(-x),

where N(x) is the number of words in the tweet with polarity score x.
Output: How is effective word score calculated?
An octave convolutional layer BIBREF0 factorizes the output feature maps of a convolutional layer into two groups. The resolution of the low-frequency feature maps is reduced by an octave – height and width dimensions are divided by 2. In this work, we explore spatial reduction by up to 3 octaves – dividing by $2^t$, where $t=1,2,3$ – and for up to 4 groups. We refer to such a layer as a multi-octave convolutional (MultiOctConv) layer, and an example with three groups and reductions of one and two octaves is depicted in Fig. FIGREF1.
Output: How is octave convolution concept extended to multiple resolutions and octaves?
The current state-of-the-art approach BIBREF14 , BIBREF15 uses maximum entropy and CRF models with a combination of language model and hand-crafted features to predict if each character in the hashtag is the beginning of a new word.
Output: What current state of the art method was used for comparison?
We evaluate our model on both English and Chinese segmentation
Output: What language do they look at?
The first type of classifier we evaluate in this work are feedforward neural networks (DNNs) consisting of 3 hidden layers, each with 512 rectified linear units (ReLUs) with a softmax activation function. As a second classifier to evaluate, we use convolutional neural networks (CNNs) with 2 convolutional and max pooling layers, followed by 2 fully-connected ReLU layers with 512 nodes.
Output: What type of models are used for classification?
This work also contributes a new dataset of INLINEFORM0 pairs of free-form natural language instructions and high-level navigation plans. This dataset was collected through Mechanical Turk using 100 simulated environments with a corresponding topological map and, to the best of our knowledge, it is the first of its kind for behavioral navigation. The dataset opens up opportunities to explore data-driven methods for grounding navigation commands into high-level motion plans. While the dataset was collected with simulated environments, no structure was imposed on the navigation instructions while crowd-sourcing data. Thus, many instructions in our dataset are ambiguous. Moreover, the order of the behaviors in the instructions is not always the same. For instance, a person said “turn right and advance” to describe part of a route, while another person said “go straight after turning right” in a similar situation. The high variability present in the natural language descriptions of our dataset makes the problem of decoding instructions into behaviors not trivial. See Appendix A of the supplementary material for additional details on our data collection effort.
Output: Did the collection process use a WoZ method?
Thus we consider the task of answer generation for TweetQA and we use several standard metrics for natural language generation to evaluate QA systems on our dataset, namely we consider BLEU-1 BIBREF16 , Meteor BIBREF17 and Rouge-L BIBREF18 in this paper.
Output: What evaluation metrics do they use?
Unfortunately, $\mathit {PMI}(w,c)$ goes to negative infinity when the word-context pair $(w,c)$ does not appear in the training corpus. Due to unreliable statistics, this happens very frequently in finite corpora. 
Output: Why are statistics from finite corpora unreliable?
Most importantly, all three languages have error-corrected corpora for testing purposes, though work on their automatic grammatical error correction is extremely limited (see Section SECREF3 ).
Output: Do they introduce errors in the data or does the data already contain them?
We propose the following challenge to the community: We must develop formal definition and evaluation for faithfulness that allows us the freedom to say when a method is sufficiently faithful to be useful in practice.

We note two possible approaches to this end:

Across models and tasks: The degree (as grayscale) of faithfulness at the level of specific models and tasks. Perhaps some models or tasks allow sufficiently faithful interpretation, even if the same is not true for others.

For example, the method may not be faithful for some question-answering task, but faithful for movie review sentiment, perhaps based on various syntactic and semantic attributes of those tasks.

Across input space: The degree of faithfulness at the level of subspaces of the input space, such as neighborhoods of similar inputs, or singular inputs themselves. If we are able to say with some degree of confidence whether a specific decision's explanation is faithful to the model, even if the interpretation method is not considered universally faithful, it can be used with respect to those specific areas or instances only.
Output: What faithfulness criteria does they propose?
We propose the following challenge to the community: We must develop formal definition and evaluation for faithfulness that allows us the freedom to say when a method is sufficiently faithful to be useful in practice.

We note two possible approaches to this end:

Across models and tasks: The degree (as grayscale) of faithfulness at the level of specific models and tasks. Perhaps some models or tasks allow sufficiently faithful interpretation, even if the same is not true for others.

For example, the method may not be faithful for some question-answering task, but faithful for movie review sentiment, perhaps based on various syntactic and semantic attributes of those tasks.

Across input space: The degree of faithfulness at the level of subspaces of the input space, such as neighborhoods of similar inputs, or singular inputs themselves. If we are able to say with some degree of confidence whether a specific decision's explanation is faithful to the model, even if the interpretation method is not considered universally faithful, it can be used with respect to those specific areas or instances only.
Output: What approaches they propose?
In this paper we focus on the skipgram approach with random negative examples proposed in BIBREF0 . 
Output: Do they use skipgram version of word2vec?
The respondents in our MTurk survey had most difficulties recognizing reviews of category $(b=0.3, \lambda=-5)$, where true positive rate was $40.4\%$, while the true negative rate of the real class was $62.7\%$. The precision were $16\%$ and $86\%$, respectively. The class-averaged F-score is $47.6\%$, which is close to random. Detailed classification reports are shown in Table~\ref{table:MTurk_sub} in Appendix. Our MTurk-study shows that \emph{our NMT-Fake reviews pose a significant threat to review systems}, since \emph{ordinary native English-speakers have very big difficulties in separating real reviews from fake reviews}. We use the review category $(b=0.3, \lambda=-5)$ for future user tests in this paper, since MTurk participants had most difficulties detecting these reviews. We refer to this category as NMT-Fake* in this paper. The classifier is very effective in detecting reviews that humans have difficulties detecting. For example, the fake reviews MTurk users had most difficulty detecting ($b=0.3, \lambda=-5$) are detected with an excellent 97\% F-score.
Output: Does their detection tool work better than human detection?
Our data are collected from the yelp academic dataset, provided by Yelp.com, a popular restaurant review website. The data set contains three types of objects: business, user, and review, where business objects contain basic information about local businesses (i.e. restaurants), review objects contain review texts and star rating, and user objects contain aggregate information about a single user across all of Yelp. Table TABREF31 illustrates the general statistics of the dataset.
Output: Does they focus on any specific product/service domain?
We show that by determining and integrating heterogeneous set of features from different modalities – aesthetic features from posted images (colorfulness, hue variance, sharpness, brightness, blurriness, naturalness), choice of profile picture (for gender, age, and facial expression), the screen name, the language features from both textual content and profile's description (n-gram, emotion, sentiment), and finally sociability from ego-network, and user engagement – we can reliably detect likely depressed individuals in a data set of 8,770 human-annotated Twitter users.
Output: What is the source of the user interaction data?
We show that by determining and integrating heterogeneous set of features from different modalities – aesthetic features from posted images (colorfulness, hue variance, sharpness, brightness, blurriness, naturalness), choice of profile picture (for gender, age, and facial expression), the screen name, the language features from both textual content and profile's description (n-gram, emotion, sentiment), and finally sociability from ego-network, and user engagement – we can reliably detect likely depressed individuals in a data set of 8,770 human-annotated Twitter users.
Output: What is the source of the textual data?
We show that by determining and integrating heterogeneous set of features from different modalities – aesthetic features from posted images (colorfulness, hue variance, sharpness, brightness, blurriness, naturalness), choice of profile picture (for gender, age, and facial expression), the screen name, the language features from both textual content and profile's description (n-gram, emotion, sentiment), and finally sociability from ego-network, and user engagement – we can reliably detect likely depressed individuals in a data set of 8,770 human-annotated Twitter users.
Output: What is the source of the visual data?
Table TABREF24 shows that while the acceleration of trading decoder layers for encoding layers in training is small, in decoding is significant. Specifically, the Transformer with 10 encoder layers and 2 decoder layers is $2.32$ times as fast as the 6-layer Transformer while achieving a slightly higher BLEU.
Output: How much is decoding speed increased by increasing encoder and decreasing decoder depth?
We introduce our proposed diversity, density, and homogeneity metrics with their detailed formulations and key intuitions.
Output: Did they propose other metrics?
We use two datasets for experiments (Table TABREF29 ): (a) STAN INLINEFORM0 , created by BIBREF10 BansalBV15, which consists of 1,108 unique English hashtags from 1,268 randomly selected tweets in the Stanford Sentiment Analysis Dataset BIBREF36 along with their crowdsourced segmentations and our additional corrections; and (b) STAN INLINEFORM1 , our new expert curated dataset, which includes all 12,594 unique English hashtags and their associated tweets from the same Stanford dataset.
Output: Do the hashtag and SemEval datasets contain only English data?
We use two datasets for experiments (Table TABREF29 ): (a) STAN INLINEFORM0 , created by BIBREF10 BansalBV15, which consists of 1,108 unique English hashtags from 1,268 randomly selected tweets in the Stanford Sentiment Analysis Dataset BIBREF36 along with their crowdsourced segmentations and our additional corrections; and (b) STAN INLINEFORM1 , our new expert curated dataset, which includes all 12,594 unique English hashtags and their associated tweets from the same Stanford dataset.
Output: How is the dataset of hashtags sourced?
We focus on social media posts from the website Twitter, which are an excellent testing ground for character based models due to the noisy nature of text. Heavy use of slang and abundant misspellings means that there are many orthographically and semantically similar tokens, and special characters such as emojis are also immensely popular and carry useful semantic information. In our moderately sized training dataset of 2 million tweets, there were about 0.92 million unique word types.
Output: Does the paper clearly establish that the challenges listed here exist in this dataset and task?
Machine-machine Interaction A related line of work explores simulation-based dialogue generation, where the user and system roles are simulated to generate a complete conversation flow, which can then be converted to natural language using crowd workers BIBREF1. Such a framework may be cost-effective and error-resistant since the underlying crowd worker task is simpler, and semantic annotations are obtained automatically. It is often argued that simulation-based data collection does not yield natural dialogues or sufficient coverage, when compared to other approaches such as Wizard-of-Oz. We argue that simulation-based collection is a better alternative for collecting datasets like this owing to the factors below.
Output: How did they gather the data?
While BERT-QC achieves large gains over existing methods on the ARC dataset, here we demonstrate that BERT-QC also matches state-of-the-art performance on TREC BIBREF6 , while surpassing state-of-the-art performance on the GARD corpus of consumer health questions BIBREF3 and MLBioMedLAT corpus of biomedical questions BIBREF4 .
Output: Which datasets are used for evaluation?
We adopt a DNN-based acoustic model BIBREF0 with 11 hidden layers and the alignment used to train the model is derived from a HMM-GMM model trained with SAT criterion.
Output: What are the deep learning architectures used in the task?
The News Category Dataset BIBREF11 is a collection of headlines published by HuffPost BIBREF12 between 2012 and 2018, and was obtained online from Kaggle BIBREF13. The full dataset contains 200k news headlines with category labels, publication dates, and short text descriptions.
Output: What news dataset was used?
In this section, we evaluate the proposed method intrinsically in terms of whether the co-occurrence matrix after the low-rank approximation is able to capture similar concepts on student response data sets, and also extrinsically in terms of the end task of summarization on all corpora. In the following experiments, summary length is set to be the average number of words in human summaries or less. An alternative way to evaluate the hypothesis is to let humans judge whether two bigrams are similar or not, which we leave for future work.
Output: Do they quantitavely or qualitatively evalute the output of their low-rank approximation to verify the grouping of lexical items?
In the sentiment analysis experiments, we found that a tweet may contain multiple sentiments. The groundtruth labels contain 210 positive sentiments, 521 neutral sentiments, and 305 negative sentiments to the candidates.
Output: Is datasets for sentiment analysis balanced?
Our features consisted of direct similarity over ConceptNet Numberbatch embeddings, the relationships inferred over ConceptNet by SME, features that compose ConceptNet with other resources (WordNet and Wikipedia), and a purely corpus-based feature that looks up two-word phrases in the Google Books dataset.
Output: What features did they train on?
word embedding, input encoder, alignment, aggregation, and prediction
Output: what is the architecture of the baseline model?
On high level, we indirectly evaluate the content relevance between the candidate summary and the human summary using information retrieval. To accomplish this, we use the summaries as search queries and compare the overlaps of the retrieved results. 
Output: In the proposed metric, how is content relevance measured?
We include ten subreddits in the five domains of abuse, social, anxiety, PTSD, and financial, as detailed in tab:data-spread, and our analysis focuses on the domain level. 
Output: What categories does the dataset come from?
To further demonstrate the effectiveness of the additional task-specific BiRNN layers in our architecture, we conducted an ablation study using the CoNLL04 dataset. We trained and evaluated in the same manner described above, using the same hyperparameters, with the following exceptions:

We used either (i) zero NER-specific BiRNN layers, (ii) zero RE-specific BiRNN layers, or (iii) zero task-specific BiRNN layers of any kind.

We increased the number of shared BiRNN layers to keep the total number of model parameters consistent with the number of parameters in the baseline model.

We average the results for each set of hyperparameter across three trials with random weight initializations.
Output: What were the variables in the ablation study?
In particular, the best-performing systems at SemEval-2015 and SemEval-2016 used deep convolutional networks BIBREF53 , BIBREF54 
Output: What is the current SOTA for sentiment analysis on Twitter at the time of writing?
Crowdsourced annotators assigned similarity to word pairs during the word similarity task. 
Output: did they use a crowdsourcing platform for annotations?
Frege promoted what we could call sentence holism: “Only in the context of a sentence does a word have a meaning.” BIBREF10
Output: What does Frege's holistic and functional approach to meaning states?
Concretely, we selected gold standards that fit our problem definition and were published in the years 2016 to 2019, have at least $(2019 - publication\ year) \times 20$ citations, and bucket them according to the answer selection styles as described in Section SECREF4
Output: What modern MRC gold standards are analyzed?
Our analysis shows that attention models traditional alignment in some cases more closely while it captures information beyond alignment in others. For instance, attention agrees with traditional alignments to a high degree in the case of nouns. However, it captures other information rather than only the translational equivalent in the case of verbs. One can see that attention loss varies substantially across different POS tags. In particular, we focus on the cases of NOUN and VERB which are the most frequent POS tags in the dataset. As shown, the attention of NOUN is the closest to alignments on average. But the average attention loss for VERB is almost two times larger than the loss for NOUN.
Output: What useful information does attention capture?
The experiment settings from this paper and evaluation protocol for the Mboshi corpus (Boundary F-scores using the ZRC speech reference) are the same from BIBREF8. Table presents the results for bilingual UWS and multilingual leveraging. 
Output: Does the paper report the accuracy of the model?
VQA research began in earnest in late 2014 when the DAQUAR dataset was released BIBREF0
Output: From when are many VQA datasets collected?
In this section we describe how to evaluate and compare the outcomes of algorithms which assign relevance scores to words (such as LRP or SA) through intrinsic validation. Furthermore, we propose a measure of model explanatory power based on an extrinsic validation procedure.
Output: Are the document vectors that the authors introduce evaluated in any way other than the new way the authors propose?
The second series of tests consisted of using all the documents of the subcorpus “specialist” $E$, because the documents of the subcorpus of Annodis are not identical.  As far as system evaluations are concerned, we use the 78 $E$ documents as reference.  We calculate the precision $P$, the recall $R$ and the $F$-score on the text corpus used in our tests, as follow:
Output: How is segmentation quality evaluated?
Our model yields the best results on both UAS and LAS metrics of all languages except the Japanese. As for Japanese, our model gives unsatisfactory results because the original treebank was written in Roman phonetic characters instead of hiragana, which is used by both common Japanese writing and our pre-trained embeddings. Despite this, our model overall still gives 1.0% higher average UAS and LAS than the previous best parser, BIAF. Although both BIAF and STACKPTR parsers have achieved relatively high parsing accuracies on the 12 languages and have all UAS higher than 90%, our model achieves state-of-the-art results in all languages for both UAS and LAS. Overall, our model reports more than 1.0% higher average UAS than STACKPTR and 0.3% higher than BIAF.
Output: What are performance compared to former models?
This step was followed by a pre-processing stage where the texts were normalized by replacing zero-width-non-joiner (ZWNJ) BIBREF2 and manually verifying the orthography based on the reference orthography of the Kurdistan Region of Iraq.
Output: How is the corpus normalized?
S2-GORC is a large citation graph dataset which includes full texts of 8.1 million scientific documents. We select a subset of 154K computer science articles as our corpus. From these, we extract 622K citing sentences that link back to other documents in our corpus. We hold 2500 examples for each of the validation and test sets.
Output: What is the size of the corpus?
Data collected by BIBREF3 , which we term the Sexist/Racist (SR) data set, was collected using an initial Twitter search followed by analysis and filtering by the authors and their team who identified 17 common phrases, hashtags, and users that were indicative of abusive speech. BIBREF4 collected the HATE dataset by searching for tweets using a lexicon provided by Hatebase.org. The final data set we used, which we call HAR, was collected by BIBREF9 ; we removed all retweets reducing the dataset to 20,000 tweets.
Output: which datasets were used?
In terms of the future development of this technology, we emphasize the importance of answering four questions: 1) How to introduce unsupervised pre-training into NLG tasks with cross-modal context? 2) How to design a generic pre-training algorithm to fit a wide range of NLG tasks? 3) How to reduce the computing resources required for large-scale pre-training? 4) What aspect of knowledge do the pre-trained models provide for better language generation?
Output: Which future direction in NLG are discussed?
We evaluate our proposed model on a Twitter dataset obtained from the authors of BIBREF12. Our final dataset consists of 11,576 users (i.e, fact-checkers), 4,732 fact-checking URLs and 63,429 interactions. The dataset also contains each user's social network information. Note that each user's social relationship is restricted within available users in the dataset.
Output: What dataset is used?
Following previous studies BIBREF1, we collect event-related microposts from Twitter using 11 and 8 seed events (see Section SECREF2) for CyberAttack and PoliticianDeath, respectively.
Output: Which real-world datasets are used?
We extracted 200 sentence pairs from BIBREF3 's dataset and provided each pair with a document context consisting of a preceding and a following sentence, as in the following example.
Output: What document context was added?
We train a single layer LSTM with a 150-dimensional hidden state for hate / not hate classification. The input dimensionality is set to 100 and GloVe BIBREF26 embeddings are used as word input representations.
Output: What unimodal detection models were used?
Although MCDN doesn't obtain the highest precision, it increases F1-score by 10.2% and 3% compared with the existing best systems $LS \cup KLD \cup CONN$ and $KLD \cup LS \cup LS_{inter}$.
Output: What performance did proposed method achieve, how much better is than previous state-of-the-art?
The type and correctness of all the question answer pairs are verified by at least two annotators.
Output: What was the inter-annotator agreement?
We train Word2Vec with vector size $d_\mathrm {W2V} = d_\mathrm {LM} = 768$ on PubMed+PMC (see Appendix for details). Then, we follow the procedure described in Section SECREF3 to update the wordpiece embedding layer and tokenizer of general-domain BERT.
Output: What in-domain text did they use?
Our official scores (column Ens Test in Table TABREF19 ) have placed us second (EI-Reg, EI-Oc), fourth (V-Reg) and fifth (V-Oc) on the SemEval AIT-2018 leaderboard.
Output: What were the scores of their system?
We represent each instance of image, question, and user choice as a triplet consisting of image feature, question feature, and the label vector for the user's answer. In addition, collecting multiple choices from identical users enables us to represent any two instances by the same user as a pair of triplets, assuming source-target relation. With these pairs of triplets, we can train the system to predict a user's choice on a new image and a new question, given the same user's choice on the previous image and its associated question. As discussed earlier, we attempt to reflect user's interest by asking questions that provide visual context. In other words, a question whose answer is so obvious that it can be answered in an identical way would not be valid as an interactive question.   If the most likely candidate $c_i=\max p_{ans}$ , where $c_i \in C$ , has a probability of being answer over a certain threshold $\alpha $ , then the question is considered to have a single obvious answer, and is thus considered ineligible.  In our experiments, we set $\alpha $ as 0.33. We also excluded the yes/no type of questions. 
Output: What are the features of used to customize target user interaction?
In this task English tweets from conversation threads, each associated to a newsworthy event and the rumours around it, are provided as data. The goal is to determine whether a tweet in the thread is supporting, denying, querying, or commenting the original rumour which started the conversation. 
Output: Is this an English-language dataset?
This results in three additional vectors corresponding to INLINEFORM3 , INLINEFORM4 and INLINEFORM5 difference vectors.

Results
Output: How are the EAU text spans annotated?
We select a large-scale Chinese dataset BIBREF0 with millions of real comments and a human-annotated test set to evaluate our model. The dataset is collected from Tencent News, which is one of the most popular Chinese websites for news and opinion articles. The dataset consists of 198,112 news articles. Each piece of news contains a title, the content of the article, and a list of the users' comments.
Output: What news comment dataset was used?
In this approach the similarity between two words is not strictly based on their co–occurrence frequencies, but rather on the frequencies of the other words which occur with both of them (i.e., second order co–occurrences). This approach has been shown to be successful in quantifying semantic relatedness BIBREF12 , BIBREF13 .
Output: What is a second order co-ocurrence matrix?
We evaluate our model on two benchmark treebanks, English Penn Treebank (PTB) and Chinese Penn Treebank (CTB5.1) following standard data splitting BIBREF30, BIBREF31. POS tags are predicted by the Stanford Tagger BIBREF32. For constituent parsing, we use the standard evalb tool to evaluate the F1 score. For dependency parsing, we apply Stanford basic dependencies (SD) representation BIBREF4 converted by the Stanford parser. Following previous work BIBREF27, BIBREF33, we report the results without punctuations for both treebanks. Tables TABREF17, TABREF18 and TABREF19 compare our model to existing state-of-the-art, in which indicator Separate with our model shows the results of our model learning constituent or dependency parsing separately, (Sum) and (Concat) respectively represent the results with the indicated input token representation setting. On PTB, our model achieves 93.90 F1 score of constituent parsing and 95.91 UAS and 93.86 LAS of dependency parsing. On CTB, our model achieves a new state-of-the-art result on both constituent and dependency parsing. At last, our model is evaluated on two benchmark treebanks for both constituent and dependency parsing. The empirical results show that our parser reaches new state-of-the-art for all parsing tasks.
Output: How is dependency parsing empirically verified?
Several recent works have investigated jointly training the acoustic model with a masking speech enhancement model BIBREF11, BIBREF12, BIBREF13, but these works did not evaluate their system on speech enhancement metrics. Indeed, our internal experiments show that without access to the clean data, joint training severely harms performance on these metrics.
Output: Which frozen acoustic model do they use?
Experimental results on public datasets show that our CRU model could substantially outperform various systems by a large margin, and set up new state-of-the-art performances on related datasets.
Output: Are there some results better than state of the art on these tasks?
We employ precision, recall, F1 and accuracy for evaluation metrics.
Output: what evaluation metrics are reported?
As with INLINEFORM0 , Greens-EFA, S&D, and EPP exhibit the highest cohesion, even though their ranking is permuted when compared to the ranking obtained with INLINEFORM1 . At the other end of the scale, we observe the same situation as with INLINEFORM2 . The non-aligned members NI have the lowest cohesion, followed by EFDD and ENL. The only place where the two methods disagree is the level of cohesion of GUE-NGL. The Alpha attributes GUE-NGL a rather high level of cohesion, on a par with ALDE, whereas the ERGM attributes them a much lower cohesion.
Output: What insights does the analysis give about the cohesion of political groups in the European parliament?
We compare our method with the models trained using Adobe internal NLU tool, Pytext BIBREF18 and Rasa BIBREF19 NLU tools.
Output: What are the baselines?
Of the 144 schemas in the collection at WinogradSchemas there are 33 that can plausibly be translated this way.
Output: Did they collect their own datasets?
Following BIBREF11 , we experiment on the non-anonymized version of . When generating summaries, we follow standard practice of tuning the maximum output length and disallow repeating the same trigram BIBREF27 , BIBREF14 . For this task we train language model representations on the combination of newscrawl and the training data. Table TABREF16 shows that pre-trained embeddings can significantly improve on top of a strong baseline transformer. We also compare to BIBREF26 who use a task-specific architecture compared to our generic sequence to sequence baseline. Pre-trained representations are complementary to their method.
Output: What other models do they compare to?
Ablation Study
To further investigate the efficacy of the key components in our framework, namely, THA and STN, we perform ablation study as shown in the second block of Table TABREF39 . The results show that each of THA and STN is helpful for improving the performance, and the contribution of STN is slightly larger than THA. “OURS w/o THA & STN” only keeps the basic bi-linear attention. Although it performs not bad, it is still less competitive compared with the strongest baseline (i.e., CMLA), suggesting that only using attention mechanism to distill opinion summary is not enough. After inserting the STN component before the bi-linear attention, i.e. “OURS w/o THA”, we get about 1% absolute gains on each dataset, and then the performance is comparable to CMLA. By adding THA, i.e. “OURS”, the performance is further improved, and all state-of-the-art methods are surpassed.
Output: Do they explore how useful is the detection history and opinion summary?
a) Parallel Scan Inference b) Vectorized Parsing c) Semiring Matrix Operations Torch-Struct aims for computational and memory efficiency. Implemented naively, dynamic programming algorithms in Python are prohibitively slow. As such Torch-Struct provides key primitives to help batch and vectorize these algorithms to take advantage of GPU computation and to minimize the overhead of backpropagating through chart-based dynamic programmming.
Output: What general-purpose optimizations are included?
FarsNet [20] [21] is the first WordNet for Persian, developed by the NLP Lab at Shahid Beheshti University and it follows the same structure as the original WordNet.
Output: What is the WordNet counterpart for Persian?
We defined a sequence labeling task to extract custom entities from user input. We assumed seven (7) possible entities (see Table TABREF43) to be recognized by the model: topic, subtopic, examination mode and level, question number, intent, as well as the entity other for remaining words in the utterance.   We defined a classification problem to predict the system's next action according to the given user input. We assumed 13 custom actions (see Table TABREF42) that we considered being our labels. In the conversational dataset, each input was automatically labeled by the rule-based system with the corresponding next action and the dialogue-id. Thus, no additional post-labeling was required. 
Output: How does the IPA label data after interacting with users?
Figure FIGREF5 shows that E-BERT performs comparable to BERT and ERNIE on unfiltered LAMA.
Output: Which of the two ensembles yields the best performance?
We assess the optimal number of topics that need to be specified for the STM analysis. We follow the recommendations of the original STM paper and focus on exclusivity and semantic coherence measures. Highly frequent words in a given topic that do not appear very often in other topics are viewed as making that topic exclusive.  Following BIBREF8 we generate a set of candidate models ranging between 3 and 50 topics. We then plot the exclusivity and semantic coherence (numbers closer to 0 indicate higher coherence), with a linear regression overlaid (Figure FIGREF3 ). Models above the regression line have a “better” exclusivity-semantic coherence trade off. We select the 16-topic model, which has the largest positive residual in the regression fit, and provides higher exclusivity at the same level of semantic coherence. The topic quality is usually evaluated by highest probability words, which is presented in Figure FIGREF4 .
Output: How are the main international development topics that states raise identified?
The annotation process was a trial-and-error, with cycles composed of annotation, discussing confusing entities, updating the annotation guide schematic and going through the corpus section again to correct entities following guide changes.
Output: Did they experiment with the corpus?
In this section we detail the discussions we use to test our metric and how we determine the ground truth (i.e. if the discussion is controversial or not). We use thirty different discussions that took place between March 2015 and June 2019, half of them with controversy and half without it. We considered discussions in four different languages: English, Portuguese, Spanish and French, occurring in five regions over the world: South and North America, Western Europe, Central and Southern Asia. We also studied these discussions taking first 140 characters and then 280 from each tweet to analyze the difference in performance and computing time wrt the length of the posts.


Output: How many languages do they experiment with?
Our method is purely text-based, and ignores the publication date and the source of the article. It combines task-specific embeddings, produced by a two-level attention-based deep neural network model, with manually crafted features (stylometric, lexical, grammatical, and semantic), into a kernel-based SVM classifier.
Output: what types of features were used?
. A highly dynamic community constantly shifts interests from one time window to another, and these temporal variations are reflected in its use of volatile language. Formally, we define the dynamicity of a community INLINEFORM0 as the average volatility of all utterances in INLINEFORM1 . 
Output: How do the authors measure how temporally dynamic a community is?
All documents are segmented into paragraphs and processed at the paragraph level (both training and inference); this is acceptable because we observe that most paragraphs are less than 200 characters. The input sequences are segmented by the BERT tokenizer, with the special [CLS] token inserted at the beginning and the special [SEP] token added at the end.
Output: At what text unit/level were documents processed?
All documents are segmented into paragraphs and processed at the paragraph level (both training and inference); this is acceptable because we observe that most paragraphs are less than 200 characters. The input sequences are segmented by the BERT tokenizer, with the special [CLS] token inserted at the beginning and the special [SEP] token added at the end.
Output: Was the structure of regulatory filings exploited when training the model?
The quality class of a Wikipedia article is assigned by Wikipedia reviewers or any registered user, who can discuss through the article's talk page to reach consensus. The arXiv dataset BIBREF2 consists of three subsets of academic articles under the arXiv repository of Computer Science (cs), from the three subject areas of: Artificial Intelligence (cs.ai), Computation and Language (cs.cl), and Machine Learning (cs.lg). In line with the original dataset formulation BIBREF2 , a paper is considered to have been accepted (i.e. is positively labeled) if it matches a paper in the DBLP database or is otherwise accepted by any of the following conferences: ACL, EMNLP, NAACL, EACL, TACL, NIPS, ICML, ICLR, or AAAI. Failing this, it is considered to be rejected (noting that some of the papers may not have been submitted to one of these conferences).
Output: Where do they get their ground truth quality judgments?
In this paper, we have shown that both human and machine translation can alter superficial patterns in data, which requires reconsidering previous findings in cross-lingual transfer learning.
Output: Does the professional translation or the machine translation introduce the artifacts?
Adhering to the experimental settings of BIBREF1 , we concatenate sentence representations produced from our model with those obtained from the state-of-the-art unsupervised learning model (Layer Normalized Skip-Thoughts, ST-LN) BIBREF31 . Following the experimental design of BIBREF1 , we conduct experiments on three different learning objectives: Cap2All, Cap2Cap, Cap2Img.
Output: What baselines are the proposed method compared against?
Our study uses WikiHop BIBREF0, as it is an entity-based multi-hop QA dataset and has been actively used.
Output: What dataset was used in the experiment?
To tackle this issue, we investigate how to improve the generalization of BERT-based MCQA models with the constraint of limited training data using four representative MCQA datasets: DREAM, MCTest, TOEFL, and SemEval-2018 Task 11.
Output: What four representative datasets are used for bechmark?
One of the most important operations on a word is to obtain the set of words whose meaning is similar to the word, or whose usage in text is similar to the word. We call this set the neighbor of the word. We have observed that the neighbor of a polysemic word consists of words that resemble the primary sense of the polysemic word.  Even though a word may be a polysemic, it usually corresponds to a single vector in distributed representation. This vector is primarily determined by the major sense, which is most frequently used. The information about a word's minor sense is subtle, and the effect of a minor sense is difficult to distinguish from statistical fluctuation. To measure the effect of a minor sense, this paper proposes to use the concept of surrounding uniformity. The surrounding uniformity roughly corresponds to statistical fluctuation in the vectors that correspond to the words in the neighbor Surrounding Uniformity (SU) can be expressed as follows: $SU(\vec{w}) = \frac{|\vec{s}(\vec{w})|}{|\vec{w}| + \sum _{i}^{N}|\vec{a_i}(\vec{w})|}$

where $\vec{s}(\vec{w}) = \vec{w} + \sum _{i}^{N} \vec{a_i}(\vec{w}).$
Output: How is the fluctuation in the sense of the word and its neighbors measured?
Spoken-SQuAD is chosen as the target domain data for training and testing. Spoken-SQuAD BIBREF5 is an automatically generated corpus in which the document is in spoken form and the question is in text form. The reference transcriptions are from SQuAD BIBREF1 . There are 37,111 and 5,351 question answer pairs in the training and testing sets respectively, and the word error rate (WER) of both sets is around 22.7%.

The original SQuAD, Text-SQuAD, is chosen as the source domain data, where only question answering pairs appearing in Spoken-SQuAD are utilized. In our task setting, during training we train the proposed QA model on both Text-SQuAD and Spoken-SQuAD training sets. While in the testing stage, we evaluate the performance on Spoken-SQuAD testing set.
Output: Which datasets did they use for evaluation?