niryuu/sni-each-converted huggingface dataset:
We conducted experiments on the Amazon reviews dataset BIBREF9, which is a benchmark dataset in the cross-domain sentiment analysis field. This dataset contains Amazon product reviews of four different product domains: Books (B), DVD (D), Electronics (E), and Kitchen (K) appliances. Each review is originally associated with a rating of 1-5 stars and is encoded in 5,000 dimensional feature vectors of bag-of-words unigrams and bigrams.

Experiment ::: Dataset and Task Design ::: Binary-Class.
From this dataset, we constructed 12 binary-class cross-domain sentiment analysis tasks: B$\rightarrow $D, B$\rightarrow $E, B$\rightarrow $K, D$\rightarrow $B, D$\rightarrow $E, D$\rightarrow $K, E$\rightarrow $B, E$\rightarrow $D, E$\rightarrow $K, K$\rightarrow $B, K$\rightarrow $D, K$\rightarrow $E. Following the setting of previous works, we treated a reviews as class `1' if it was ranked up to 3 stars, and as class `2' if it was ranked 4 or 5 stars. For each task, $\mathcal {D}_S$ consisted of 1,000 examples of each class, and $\mathcal {D}_T$ consists of 1500 examples of class `1' and 500 examples of class `2'. In addition, since it is reasonable to assume that $\mathcal {D}_T$ can reveal the distribution of target domain data, we controlled the target domain testing dataset to have the same class ratio as $\mathcal {D}_T$. Using the same label assigning mechanism, we also studied model performance over different degrees of $\rm {P}(\rm {Y})$ shift, which was evaluated by the max value of $\rm {P}_S(\rm {Y}=i)/\rm {P}_T(\rm {Y}=i), \forall i=1, \cdots , L$. Please refer to Appendix C for more detail about the task design for this study.

Experiment ::: Dataset and Task Design ::: Multi-Class.
We additionally constructed 12 multi-class cross-domain sentiment classification tasks. Tasks were designed to distinguish reviews of 1 or 2 stars (class 1) from those of 4 stars (class 2) and those of 5 stars (class 3). For each task, $\mathcal {D}_S$ contained 1000 examples of each class, and $\mathcal {D}_T$ consisted of 500 examples of class 1, 1500 examples of class 2, and 1000 examples of class 3. Similarly, we also controlled the target domain testing dataset to have the same class ratio as $\mathcal {D}_T$. 
 Question: Which sentiment analysis tasks are addressed?
Output: 12 binary-class classification and multi-class classification of reviews based on rating
For example, it can be used to extract features for usage with other machine learning tools, or to evaluate given features with the existing classifiers or regressors. Figure FIGREF19 demonstrates a simple feature extractor that retrieves the sentence length. 
 Question: Do they show an example of usage for INFODENS?
Output: Yes
All cases exhibit high scores—in the vast majority of the cases substantially higher than reported in previous work. In particular, in BIBREF1 we assess the ability of LSTMs to learn subject-verb agreement patterns in English, and evaluate on naturally occurring wikipedia sentences.  BIBREF2 also consider subject-verb agreement, but in a “colorless green ideas” setting in which content words in naturally occurring sentences are replaced with random words with the same part-of-speech and inflection, thus ensuring a focus on syntax rather than on selectional-preferences based cues.  BIBREF3 consider a wider range of syntactic phenomena (subject-verb agreement, reflexive anaphora, negative polarity items) using manually constructed stimuli, allowing for greater coverage and control than in the naturally occurring setting. 
 Question: Were any of these tasks evaluated in any previous work?
Output: Yes
We evaluate sentence-level semantics using averaged bag of vectors (BoV) representations on the Semantic Textual Similarity (STSB) task BIBREF21 and Word Content (WC) probing task (identify from a list of words which is contained in the sentence representation) from SentEval BIBREF22. Syntax: Similarly, we use the Google Syntactic analogies (GSyn) BIBREF9 to evaluate word-level syntactic information, and Depth (Dep) and Top Constituent (TopC) (of the input sentence's constituent parse tree) probing tasks from SentEval BIBREF22 for sentence-level syntax. 
 Question: What semantic and syntactic tasks are used as probes?
Output: Word Content (WC) probing task Depth (Dep) and Top Constituent (TopC) (of the input sentence's constituent parse tree) probing tasks
We use data from Mandarin Chinese and Cantonese. For each language, the data consists of a list of spoken words, recorded by the same speaker. The Mandarin dataset is from a female speaker and is provided by Shtooka, and the Cantonese dataset is from a male speaker and is downloaded from Forvo, an online crowd-sourced pronunciation dictionary. 
 Question: What dataset is used for training?
Output: Mandarin dataset Cantonese dataset
For the English-German language pair we use the full WMT 2019 parallel dataset. For the English-French language pair we use a restricted dataset containing the full TED corpus from MUST-C BIBREF10 and sampled sentences from WMT 2019 dataset.  
 Question: Which datasets were used in the experiment?
Output: WMT 2019 parallel dataset a restricted dataset containing the full TED corpus from MUST-C BIBREF10 sampled sentences from WMT 2019 dataset
In this context, we decided to study the three main types of architectures which have demonstrated promising results over traditional systems: 1) Connectionist Temporal Classification (CTC) BIBREF5, BIBREF6 which uses Markov assumptions (i.e. conditional independence between predictions at each time step) to efficiently solve sequential problems by dynamic programming, 2) Attention-based methods BIBREF7, BIBREF8 which rely on an attention mechanism to perform non-monotonic alignment between acoustic frames and recognized acoustic units and 3) RNN-tranducer BIBREF0, BIBREF9, BIBREF10 which extends CTC by additionally modeling the dependencies between outputs at different steps using a prediction network analogous to a language model. 
 Question: What are the existing end-to-end ASR approaches for the French language?
Output: 1) Connectionist Temporal Classification (CTC) 2) Attention-based methods 3) RNN-tranducer
We learn that a number of factors can influence the performance of adversarial attacks, including architecture of the classifier, sentence length and input domain. 
 Question: What other factors affect the performance?
Output: architecture of the classifier sentence length  input domain
As a document classifier we employ a word-based CNN similar to Kim consisting of the following sequence of layers: $ \texttt {Conv} \xrightarrow{} \texttt {ReLU} \xrightarrow{} \texttt {1-Max-Pool} \xrightarrow{} \texttt {FC} \\ $ Future work would include applying LRP to other neural network architectures (e.g. character-based or recurrent models) on further NLP tasks, as well as exploring how relevance information could be taken into account to improve the classifier's training procedure or prediction performance. 
 Question: Do the experiments explore how various architectures and layers contribute towards certain decisions?
Output: No
Our architecture was designed to show that using a model that captures better information about the context and sequential nature of the text can outperform lexicon-based methods commonly used in the literature. 
 Question: What meaningful information does the GRU model capture, which traditional ML models do not?
Output:  the context and sequential nature of the text
Ensemble, which selects a sentence using 100 agents trained on clustered dialogues as described in section SECREF4 – the agent in focus is chosen using a regressor as predictor of dialogue reward INLINEFORM0 using a similar neural net as the ChatDQN agents except for the final layer having one node and using Batch Normalisation BIBREF44 between hidden layers as in BIBREF36 ; 
 Question: How many agents do they ensemble over?
Output: 100
We collect $37,263$ concept-sets as the inputs, each of which contains three to five common concepts. These concept-sets are sampled from several large corpora of image/video captions, such that the concepts inside them are more likely to co-occur in natural scenes. The expected concept-sets in our task are supposed to be likely co-occur in natural, daily-life scenes . The concepts in images/videos captions, which usually describe scenes in our daily life, thus possess the desired property. We therefore collect a large amount of caption sentences from a variety of datasets, including VATEX BIBREF4, LSMDC BIBREF12, ActivityNet BIBREF13, and SNLI BIBREF15, forming 1,040,330 sentences in total. 
 Question: Where do the concept sets come from?
Output: These concept-sets are sampled from several large corpora of image/video captions
The dataset needed to train the W2V was obtained using the information extracted from a dump of the Italian Wikipedia (dated 2019.04.01), from the main categories of Italian Google News (WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH) and from some anonymized chats between users and a customer care chatbot (Laila). 
 Question: What dataset is used for training Word2Vec in Italian language?
Output: extracted from a dump of the Italian Wikipedia (dated 2019.04.01), from the main categories of Italian Google News (WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH) and from some anonymized chats between users and a customer care chatbot (Laila)
We first introduce a method that detects bottlenecks in text-games using the overall reward gained and the knowledge graph state. This method freezes the policy used to reach the bottleneck and restarts the training from there on out, additionally conducting a backtracking search to ensure that a sub-optimal policy has not been frozen. The second contribution explore how to leverage knowledge graphs to improve existing exploration algorithms for dealing with combinatorial action-spaces such as Go-Explore BIBREF9.  
 Question: What are the two new strategies?
Output: a method that detects bottlenecks in text-games using the overall reward gained and the knowledge graph state to leverage knowledge graphs to improve existing exploration algorithms for dealing with combinatorial action-space
For a given transcribed utterance, it is firstly encoded with Byte Pair Encoding (BPE) BIBREF14, a compression algorithm splitting words to fundamental subword units (pairs of bytes or BPs) and reducing the embedded vocabulary size. Then we use a BiLSTM BIBREF15 encoder and the output state of the BiLSTM is regarded as a vector representation for this utterance. Finally, a fully connected Feed-forward Neural Network (FNN) followed by a softmax layer, labeled as a multilayer perceptron (MLP) module, is used to perform the domain/intent classification task based on the vector. We name it Oracle simply because we assume that hypotheses are noisy versions of transcription. 
 Question: Which ASR system(s) is used in this work?
Output: Oracle
As we can see that, all variants of our CRU model could give substantial improvements over the traditional GRU model, where a maximum gain of 2.7%, 1.0%, and 1.9% can be observed in three datasets, respectively. 
 Question: Do experiment results show consistent significant improvement of new approach over traditional CNN and RNN models?
Output: Yes
. In this work, we challenge current representation techniques and suggest to represent the state using natural language, similar to the way we, as humans, summarize and transfer information efficiently from one to the other BIBREF5. 
 Question: How is state to learn and complete tasks represented via natural language?
Output:  represent the state using natural language
We evaluate our model on both English and Chinese segmentation 
 Question: What language do they look at?
Output: English Chinese
To overcome the above-mentioned challenges, we propose a locally-linear meta-embedding learning method that (a) requires only the words in the vocabulary of each source embedding, without having to predict embeddings for missing words, (b) can meta-embed source embeddings with different dimensionalities, (c) is sensitive to the diversity of the neighbourhoods of the source embeddings.

Our proposed method comprises of two steps: a neighbourhood reconstruction step (Section "Nearest Neighbour Reconstruction" ), and a projection step (Section "Projection to Meta-Embedding Space" ). In the reconstruction step, we represent the embedding of a word by the linearly weighted combination of the embeddings of its nearest neighbours in each source embedding space. 
 Question: What is the introduced meta-embedding method introduced in this paper?
Output: proposed method comprises of two steps: a neighbourhood reconstruction step (Section "Nearest Neighbour Reconstruction" ), and a projection step (Section "Projection to Meta-Embedding Space" ). In the reconstruction step, we represent the embedding of a word by the linearly weighted combination of the embeddings of its nearest neighbours in each source embedding space.
Tests and Gold-Standard Data for Intrinsic Evaluation
Using the gold standard data (described below), we performed three types of tests:

Class Membership Tests: embeddings corresponding two member of the same semantic class (e.g. “Months of the Year", “Portuguese Cities", “Smileys") should be close, since they are supposed to be found in mostly the same contexts.

Class Distinction Test: this is the reciprocal of the previous Class Membership test. Embeddings of elements of different classes should be different, since words of different classes ere expected to be found in significantly different contexts.

Word Equivalence Test: embeddings corresponding to synonyms, antonyms, abbreviations (e.g. “porque" abbreviated by “pq") and partial references (e.g. “slb and benfica") should be almost equal, since both alternatives are supposed to be used be interchangeable in all contexts (either maintaining or inverting the meaning).

Therefore, in our tests, two words are considered:

distinct if the cosine of the corresponding embeddings is lower than 0.70 (or 0.80).

to belong to the same class if the cosine of their embeddings is higher than 0.70 (or 0.80).

equivalent if the cosine of the embeddings is higher that 0.85 (or 0.95). 
 Question: What intrinsic evaluation metrics are used?
Output: Class Membership Tests Class Distinction Test Word Equivalence Test
In the first scenario we equip the decoder with an additional morphology table including target-side affixes. 
 Question: What type of morphological information is contained in the "morphology table"?
Output: target-side affixes
To investigate whether our BERT-based model can transfer knowledge beyond language, we consider image features as simple visual tokens that can be presented to the model analogously to textual token embeddings. In order to make the $o_j$ vectors (of dimension $2048+4=2052$) comparable to BERT embeddings (of dimension 768), we use a simple linear cross-modal projection layer $W$ of dimensions $2052\hspace{-1.00006pt}\times \hspace{-1.00006pt}768$. The $N$ object regions detected in an image, are thus represented as $X_{img} = (W.o_1,...,W.o_N)$. Once mapped into the BERT embedding space with $W$, the image is seen by the rest of the model as a sequence of units with no explicit indication if it is a text or an image embedding. 
 Question: How are multimodal representations combined?
Output: The image feature vectors are mapped into BERT embedding dimensions and treated like a text sequence afterwards.
The SG model achieved a high average similarity score of 0.650 followed by CBoW with a 0.632 average similarity score. The GloVe also achieved a considerable average score of 0.591 respectively. However, the average similarity score of SdfastText is 0.388 and the word pair Microsoft-Bill Gates is not available in the vocabulary of SdfastText. Moreover, the average semantic relatedness similarity score between countries and their capitals is shown in Table TABREF78 with English translation, where SG also yields the best average score of 0.663 followed by CBoW with 0.611 similarity score. The GloVe also yields better semantic relatedness of 0.576 and the SdfastText yield an average score of 0.391. 
 Question: How does proposed word embeddings compare to Sindhi fastText word representations?
Output: Proposed SG model vs SINDHI FASTTEXT:
Average cosine similarity score: 0.650 vs 0.388
Average semantic relatedness similarity score between countries and their capitals: 0.663 vs 0.391
Moreover, to be able to make use of multiple answer styles within a single system, our model introduces an artificial token corresponding to the target style at the beginning of the answer sentence ( $y_1$ ), like BIBREF14 . At test time, the user can specify the first token to control the answer styles. 
 Question: Does their model also take the expected answer style as input?
Output: Yes
Our model improves the F1 score by almost 2%, which corresponds to a 12.3% error rate reduction. 
 Question: How better is proposed model compared to baselines?
Output:  improves the F1 score by almost 2%, which corresponds to a 12.3% error rate reduction
MTMSN BIBREF4 is the first, and only model so far, that specifically tried to tackle the multi-span questions of DROP. Their approach consisted of two parts. The first was to train a dedicated categorical variable to predict the number of spans to extract. The second was to generalize the single-span head method of extracting a span, by utilizing the non-maximum suppression (NMS) algorithm BIBREF7 to find the most probable set of non-overlapping spans. The number of spans to extract was determined by the aforementioned categorical variable 
 Question: What approach did previous models use for multi-span questions?
Output: Only MTMSM specifically tried to tackle the multi-span questions. Their approach consisted of two parts: first train a dedicated categorical variable to predict the number of spans to extract and the second was to generalize the single-span head method of extracting a span
We then ask a separate set of Turkers to rate the stories for overall quality and the three improvement areas. All ratings are on a five-point scale. We collect two ratings per story, and throw out ratings that disagree by more than 2 points. A total of 11% of ratings were thrown out, leaving four metrics across 241 stories for analysis. 
 Question: How do they evaluate generated stories?
Output: separate set of Turkers to rate the stories for overall quality and the three improvement areas
The first model, referred as `LSTM (PTB)' in the following sections, was trained on the sentences from Penn Treebank BIBREF12. The second model, referred as `LSTM (FTB)', was trained on the sentences from French Treebank BIBREF13. We incorporate two pretrained English language models: one trained on the Billion Word benchmark (referred as `LSTM (1B)') from BIBREF14, and the other trained on English Wikipedia (referred as `LSTM (enWiki)') from BIBREF3. For French, we trained a large LSTM language model (referred as `LSTM (frWaC)') on a random subset (about 4 million sentences, 138 million word tokens) of the frWaC dataset BIBREF15. 
 Question: What is the size of the datasets employed?
Output: (about 4 million sentences, 138 million word tokens) one trained on the Billion Word benchmark
The dataset MSParS is published by NLPCC 2019 evaluation task. The whole dataset consists of 81,826 samples annotated by native English speakers. 80% of them are used as training set. 10% of them are used as validation set while the rest is used as test set. 3000 hard samples are selected from the test set. Metric for this dataset is the exactly matching accuracy on both full test set and hard test subset. Each sample is composed of the question, the logical form, the parameters(entity/value/type) and question type as the Table TABREF3 demonstrates. 
 Question: Does the training dataset provide logical form supervision?
Output: Yes
In Figure FIGREF28, we show some examples of the annotation results in CORD-19-NER. We can see that our distantly- or weakly supervised methods achieve high quality recognizing the new entity types, requiring only several seed examples as the input. For example, we recognized “SARS-CoV-2" as the “CORONAVIRUS" type, “bat" and “pangolins" as the “WILDLIFE" type and “Van der Waals forces" as the “PHYSICAL_SCIENCE" type. This NER annotation results help downstream text mining tasks in discovering the origin and the physical nature of the virus. Our NER methods are domain-independent that can be applied to corpus in different domains. In addition, we show another example of NER annotation on New York Times with our system in Figure FIGREF29.

In Figure FIGREF30, we show the comparison of our annotation results with existing NER/BioNER systems. In Figure FIGREF30, we can see that only our method can identify “SARS-CoV-2" as a coronavirus. In Figure FIGREF30, we can see that our method can identify many more entities such as “pylogenetic" as a evolution term and “bat" as a wildlife. In Figure FIGREF30, we can also see that our method can identify many more entities such as “racism" as a social behavior. In summary, our distantly- and weakly-supervised NER methods are reliable for high-quality entity recognition without requiring human effort for training data annotation. 
 Question: Did they experiment with the dataset?
Output: Yes
We use two state-of-art neural coreference resolution models described by BIBREF2 and BIBREF1 . 
 Question: What is the state-of-the-art neural coreference resolution model?
Output: BIBREF2  BIBREF1
First, the GRU only allows sentences to have context from sentences before them, but not after them. This prevents information propagation from future sentences. Second, the supporting sentences may be too far away from each other on a word level to allow for these distant sentences to interact through the word level GRU. 
 Question: Why is supporting fact supervision necessary for DMN?
Output: First, the GRU only allows sentences to have context from sentences before them, but not after them. This prevents information propagation from future sentences. Second, the supporting sentences may be too far away from each other on a word level to allow for these distant sentences to interact through the word level GRU.
Our study uses WikiHop BIBREF0, as it is an entity-based multi-hop QA dataset and has been actively used. 
 Question: What dataset was used in the experiment?
Output: WikiHop
We have implemented the following interfaces for Macaw:

[leftmargin=*]

File IO: This interface is designed for experimental purposes, such as evaluating the performance of a conversational search technique on a dataset with multiple queries. This is not an interactive interface.

Standard IO: This interactive command line interface is designed for development purposes to interact with the system, see the logs, and debug or improve the system.

Telegram: This interactive interface is designed for interaction with real users (see FIGREF4). Telegram is a popular instant messaging service whose client-side code is open-source. We have implemented a Telegram bot that can be used with different devices (personal computers, tablets, and mobile phones) and different operating systems (Android, iOS, Linux, Mac OS, and Windows). This interface allows multi-modal interactions (text, speech, click, image). It can be also used for speech-only interactions. For speech recognition and generation, Macaw relies on online APIs, e.g., the services provided by Google Cloud and Microsoft Azure. In addition, there exist multiple popular groups and channels in Telegram, which allows further integration of social networks with conversational systems. For example, see the Naseri and Zamani's study on news popularity in Telegram BIBREF12. 
 Question: What interface does Macaw currently have?
Output: File IO Standard IO Telegram
We adopt the annotation machinery of BIBREF5 implemented using Amazon's Mechanical Turk, and annotate each predicate by 2 trained workers independently, while a third consolidates their annotations into a final set of roles and arguments. In this consolidation task, the worker validates questions, merges, splits or modifies answers for the same role according to guidelines, and removes redundant roles by picking the more naturally phrased questions.  
 Question: What is different in the improved annotation protocol?
Output: a trained worker consolidates existing annotations
While the plan generation stage is guaranteed to be faithful to the input, the translation process from plans to text is based on a neural seq2seq model and may suffer from known issues with such models: hallucinating facts that do not exist in the input, repeating facts, or dropping facts. While the clear mapping between plans and text helps to reduce these issues greatly, the system in BIBREF0 still has 2% errors of these kinds. Recent work in neural text generation and summarization attempt to address these issues by trying to map the textual outputs back to structured predicates, and comparing these predicates to the input data. 
 Question: What is the effectiveness plan generation?
Output: clear mapping between plans and text helps to reduce these issues greatly, the system in BIBREF0 still has 2% errors work in neural text generation and summarization attempt to address these issues
The main questions this paper attempts to answer are: Does compressing BERT impede it's ability to transfer to new tasks? And does fine-tuning make BERT more or less compressible?

To explore these questions, we compressed English BERT using magnitude weight pruning BIBREF8 and observed the results on transfer learning to the General Language Understanding Evaluation (GLUE) benchmark BIBREF9, a diverse set of natural language understanding tasks including sentiment analysis, NLI, and textual similarity evaluation. We chose magnitude weight pruning, which compresses models by removing weights close to 0, because it is one of the most fine-grained and effective compression methods and because there are many interesting ways to view pruning, which we explore in the next section. Our findings are as follows: Low levels of pruning (30-40%) do not increase pre-training loss or affect transfer to downstream tasks at all. Medium levels of pruning increase the pre-training loss and prevent useful pre-training information from being transferred to downstream tasks. 
 Question: How they observe that fine-tuning BERT on a specific task does not improve its prunability?
Output: we compressed English BERT using magnitude weight pruning BIBREF8 and observed the results on transfer learning to the General Language Understanding Evaluation (GLUE) benchmark BIBREF9, a diverse set of natural language understanding tasks including sentiment analysis, NLI, and textual similarity evaluation.
Specifically, we group the models based on the objective function it optimizes. We believe this work can aid the understanding of the existing literature.  We believe that the performance of these models is highly dependent on the objective function it optimizes – predicting adjacent word (within-tweet relationships), adjacent tweet (inter-tweet relationships), the tweet itself (autoencoder), modeling from structured resources like paraphrase databases and weak supervision.  
 Question: How do they encourage understanding of literature as part of their objective function?
Output: They group the existing works in terms of the objective function they optimize - within-tweet relationships, inter-tweet relationships, autoencoder, and weak supervision.
The number of fake news that have a duplicate in the training dataset are 1018 whereas, the number of articles with genuine content that have a duplicate article in the training set is 322. 
 Question: what is the size of the dataset?
Output: The number of fake news that have a duplicate in the training dataset are 1018 whereas, the number of articles with genuine content that have a duplicate article in the training set is 322.
We evaluate our method on two widely-studied datasets provided by Waseem and Hovey BIBREF5 and Davidson et al. BIBREF9. 
 Question: Which publicly available datasets are used?
Output: Waseem-dataset Davidson-dataset,
Methodology ::: US dataset
We collected tweets associated to a dozen US mainstream news websites, i.e. most trusted sources described in BIBREF18, with the Streaming API, and we referred to Hoaxy API BIBREF16 for what concerns tweets containing links to 100+ US disinformation outlets.  Methodology ::: Italian dataset
For what concerns the Italian scenario we first collected tweets with the Streaming API in a 3-week period (April 19th, 2019-May 5th, 2019), filtering those containing URLs pointing to Italian official newspapers websites as described in BIBREF22; these correspond to the list provided by the association for the verification of newspaper circulation in Italy (Accertamenti Diffusione Stampa). We instead referred to the dataset provided by BIBREF23 to obtain a set of tweets, collected continuously since January 2019 using the same Twitter endpoint, which contain URLs to 60+ Italian disinformation websites. 
 Question: What are the two large-scale datasets used?
Output: US dataset Italian dataset
This architecture builds upon sogaard2016deep, but is adapted in two ways: first, we add an unsupervised sequence labeling task (Language Modeling), second, we add a low-dimensional embedding layer between tasks in the hierarchy to learn dense representations of label tags. 
 Question: What is the unsupervised task in the final layer?
Output: Language Modeling
The primary feed for the analysis collected INLINEFORM0 million tweets containing the keywords `breast' AND `cancer'.  
 Question: How were breast cancer related posts compiled from the Twitter streaming API?
Output: By using  keywords `breast' AND `cancer' in tweet collecting process.
Experimental Setup 
 Question: which chinese datasets were used?
Output: Answer with content missing: (Data section) Chinese with version 5.1 of the Chinese Penn Treebank (CTB)
The model has around 836M parameters, of which only 66K are byte embeddings. 
 Question: How many parameters does the model have?
Output: model has around 836M parameters
Our baseline is a GRU network for each of the three tasks. 
 Question: Are the models compared to some baseline models?
Output: Yes
Specifically, we firstly extract multiple relations with an off-the-shelf Open Information Extraction (OpenIE) toolbox BIBREF7, then we select the relation that is most relevant to the answer with carefully designed heuristic rules. 
 Question: How they extract "structured answer-relevant relation"?
Output: Using the OpenIE toolbox and applying heuristic rules to select the most relevant relation.
They can answer each question with either `yes', `rather yes', `rather no', or `no'. They can supplement each answer with a comment of at most 500 characters. 
 Question: What annotations are present in dataset?
Output: answer each question with either `yes', `rather yes', `rather no', or `no'. can supplement each answer with a comment of at most 500 characters
To account for sequences of words and characters that might carry useful information, we extracted word unigrams, bigrams, and trigrams as features. 
 Question: Which features do they use to model Twitter messages?
Output: word unigrams, bigrams, and trigrams
To help solve the data problem we present Taskmaster-1, a dataset consisting of 13,215 dialogs, including 5,507 spoken and 7,708 written dialogs created with two distinct procedures. Each conversation falls into one of six domains: ordering pizza, creating auto repair appointments, setting up ride service, ordering movie tickets, ordering coffee drinks and making restaurant reservations.  
 Question: Which six domains are covered in the dataset?
Output: ordering pizza, creating auto repair appointments, setting up ride service, ordering movie tickets, ordering coffee drinks and making restaurant reservations
Without ELMo (the same setting as 4-th row in Table 3 ), our data settings is the same as qin-EtAl:2017:Long whose performance was state-of-the-art and will be compared directly. We see that even without the pre-trained ELMo encoder, our performance is better, which is mostly attributed to our better sentence pair representations. 
 Question: Why does their model do better than prior models?
Output: better sentence pair representations
For the competition, we provided 1,000 labeled dialogues from each dataset for training, and 240 unlabeled dialogues from each dataset for evaluation.  
 Question: What is the size of the second dataset?
Output: 1 000 labeled dialogues for training and 240 unlabeled dialogues for evaluation
Result for BLEU score for our model and Google's Neural Machine Translation is compared in table TABREF19 
 Question: What is their baseline?
Output: Google's Neural Machine Translation
BIBREF0 (BIBREF0) developed Moral Choice Machine computes the cosine similarity in a sentence embedding space of an arbitrary action embedded in question/answer pairs. 
 Question: What is the Moral Choice Machine?
Output: Moral Choice Machine computes the cosine similarity in a sentence embedding space of an arbitrary action embedded in question/answer pairs
In recent years, researchers have proposed various filter based feature selection methods to raise the performance of document text classification BIBREF19.  Furthermore, in order to rank the terms based on their discriminative power among the classes, we use filter based feature selection method named as Normalized Difference Measure (NDM)BIBREF5. Considering the features contour plot, Rehman et al. BIBREF5 suggested that all those features which exist in top left, and bottom right corners of the contour are extremely significant as compared to those features which exist around diagonals. State-of-the-art filter based feature selection algorithms such as ACC2 treat all those features in the same fashion which exist around the diagonals BIBREF5. 
 Question: Is the filter based feature selection (FSE) a form of regularization?
Output: No
In the multiple-choice setting, which is the variety of question-answering (QA) that we focus on in this paper, there is also pragmatic reasoning involved in selecting optimal answer choices (e.g., while greenhouse effect might in some other context be a reasonable answer to the second question in Figure FIGREF1, global warming is a preferable candidate). 
 Question: Do they focus on Reading Comprehension or multiple choice question answering?
Output: MULTIPLE CHOICE QUESTION ANSWERING
We propose a simple and practical human evaluation for evaluating text summarization, where the summary is evaluated against the source content instead of the reference. It handles both of the problems of paraphrasing and lack of high-quality reference.  To avoid the deficiencies, we propose a simple human evaluation method to assess the semantic consistency. Each summary candidate is evaluated against the text rather than the reference. If the candidate is irrelevant or incorrect to the text, or the candidate is not understandable, the candidate is labeled bad. Otherwise, the candidate is labeled good. Then, we can get an accuracy of the good summaries. The proposed evaluation is very simple and straight-forward. It focuses on the relevance between the summary and the text. The semantic consistency should be the major consideration when putting the text summarization methods into practice, but the current automatic methods cannot judge properly. For detailed guidelines in human evaluation, please refer to Appendix SECREF6 .  
 Question: What human evaluation method is proposed?
Output: comparing the summary with the text instead of the reference and labeling the candidate bad if it is incorrect or irrelevant
To address this drawback in ROUGE, we propose a new evaluation metric: Critical Information Completeness (CIC). Formally, CIC is a recall of semantic slot information between a candidate summary and a reference summary. CIC is defined as follows:

where $V$ stands for a set of delexicalized values in the reference summary, $Count_{match}(v)$ is the number of values co-occurring in the candidate summary and reference summary, and $m$ is the number of values in set $V$. In our experiments, CIC is computed as the arithmetic mean over all the dialog domains to retain the overall performance.

CIC is a suitable complementary metric to ROUGE because it accounts for the most important information within each dialog domain. CIC can be applied to any summarization task with predefined essential entities. 
 Question: How does new evaluation metric considers critical informative entities?
Output: Answer with content missing: (formula for CIC) it accounts for the most important information within each dialog domain. CIC can be applied to any summarization task with predefined essential entities
Then we define the similarity between INLINEFORM7 and INLINEFORM8 by, DISPLAYFORM0 We note that the similarity metric defined above for sentences is equally applicable for n-gram retrieval. 
 Question: Which similarity measure do they use in their n-gram retrieval approach?
Output: we define the similarity between INLINEFORM7 and INLINEFORM8 by, DISPLAYFORM0
Besides, we report the sentiment accuracy (Senti ACC) which measures whether the output sentence has the same sentiment polarity as the input sentence based on our standardized sentiment classifiers. The BLEU score BIBREF25 between the input sentences and the output sentences is calculated to evaluate the content preservation performance. In order to evaluate the overall performance of different models, we also report the geometric mean (G2) and harmonic mean (H2) of the sentiment accuracy and the BLEU score. As for the irony accuracy, we only report it in human evaluation results because it is more accurate for the human to evaluate the quality of irony as it is very complicated. 
 Question: Who judged the irony accuracy, sentiment preservation and content preservation?
Output: Irony accuracy is judged only by human ; senriment preservation and content preservation are judged  both by human and using automatic metrics (ACC and BLEU).
We pit our model against the following baselines: 1) SVM with unigram, bigram, and trigram features, which is a standard yet rather strong classifier for text features; 2) SVM with average word embedding, where a document is represented as a continuous representation by averaging the embeddings of the composite words; 3) SVM with average transformed word embeddings (the INLINEFORM0 in equation EQREF6 ), where a document is represented as a continuous representation by averaging the transformed embeddings of the composite words; 4) two mature deep learning models on text classification, CNN BIBREF3 and Recurrent Convolutional Neural Networks (RCNN) BIBREF0 , where the hyperparameters are based on their work; 5) the above SVM and deep learning models with comment information;  
 Question: What are the baselines?
Output: SVM with unigram, bigram, and trigram features SVM with average word embedding SVM with average transformed word embeddings CNN ecurrent Convolutional Neural Networks SVM and deep learning models with comment information
In our setup, our starting point is a base model, trained on NLI data. Rather than employing automated adversarial methods, here the model's “adversary” is a human annotator. Given a context (also often called a “premise” in NLI), and a desired target label, we ask the human writer to provide a hypothesis that fools the model into misclassifying the label. One can think of the writer as a “white hat” hacker, trying to identify vulnerabilities in the system. For each human-generated example that is misclassified, we also ask the writer to provide a reason why they believe it was misclassified. 
 Question: Do they use active learning to create their dataset?
Output: Yes
For the first coarse-tuning stage with NLI tasks, we use MultiNLI BIBREF15 and SNLI BIBREF16 as the out-of-domain source datasets.  
 Question: What out of domain datasets authors used for coarse-tuning stage?
Output: MultiNLI BIBREF15 and SNLI BIBREF16
Therefore, in order to create fair systems it is necessary to take into account the representation problems in society that are going to be encapsulated in the data. 
 Question: What is the goal of investigating NLP gender bias specifically in the news broadcast domain and Anchor role?
Output: create fair systems
The end-to-end system (prefix E2E) uses the DNN topology depicted in Figure FIGREF8 . We present results with 3 distinct size configurations (infixes 700K, 318K, and 40K) each representing the number of approximate parameters, and 2 types of training recipes (suffixes 1stage and 2stage) corresponding to end-to-end and encoder+decoder respectively, as described in UID7 . 
 Question: How many parameters does the presented model have?
Output: (infixes 700K, 318K, and 40K) each representing the number of approximate parameters
Table TABREF24 shows that while the acceleration of trading decoder layers for encoding layers in training is small, in decoding is significant. Specifically, the Transformer with 10 encoder layers and 2 decoder layers is $2.32$ times as fast as the 6-layer Transformer while achieving a slightly higher BLEU. 
 Question: How much is decoding speed increased by increasing encoder and decreasing decoder depth?
Output: the Transformer with 10 encoder layers and 2 decoder layers is $2.32$ times as fast as the 6-layer Transformer
We conduct our experiments using the evaluation methodology of SemEval 2010 Task 14: Word Sense Induction & Disambiguation BIBREF5 .  
 Question: What evaluation is conducted?
Output: Word Sense Induction & Disambiguation
The Business Language Testing Service (BULATS) test of Cambridge Assessment English BIBREF27 is a multi-level computer-based English test. It consists of read speech and free-speaking components, with the candidate responding to prompts. The BULATS spoken test has five sections, all with materials appropriate to business scenarios.  In this work, non-native speech from the BULATS test is used as both training and test data for the speaker verification systems. To investigate how the systems generalise, data for testing is also taken from the Cambridge Assessment English Linguaskill online test. Like BULATS, this is also a multi-level test and has a similar format composed of the same five sections as described before but assesses general English ability. 
 Question: What standard large speaker verification corpora is used for evaluation?
Output: non-native speech from the BULATS test
Among the GLUE tasks, these state-of-the-art systems make their greatest gains on the acceptability task with the Corpus of Linguistic Acceptability BIBREF0 . CoLA contains example sentences from linguistics publications labeled by experts for grammatical acceptability, and written to show subtle grammatical features. Because minimal syntactic differences can separate acceptable sentences from unacceptable ones (What did Bo write a book about? / *What was a book about written by Bo?), and acceptability classifiers are more reliable when trained on GPT and BERT than on recurrent models, it stands to reason that GPT and BERT have better implicit knowledge of syntactic features relevant to acceptability. 
 Question: Where does the data in CoLA come from?
Output:  CoLA contains example sentences from linguistics publications labeled by experts
The same 36 questions were answered using four QALD tools: WDAqua BIBREF0 , QAKiS BIBREF7 , gAnswer BIBREF6 and Platypus BIBREF8 . 
 Question: Which four QA systems do they use?
Output: WDAqua BIBREF0 , QAKiS BIBREF7 , gAnswer BIBREF6 and Platypus BIBREF8
In future work we also intend to add these types of studies to the ERP predictions.

Discussion 
 Question: What datasets are used?
Output: Answer with content missing: (Whole Method and Results sections) The primary dataset we use is the ERP data collected and computed by Frank et al. (2015), and we also use behavioral data (eye-tracking data and self-paced reading times) from Frank et al. (2013) which were collected on the same set of 205 sentences.
Select:
- ERP data collected and computed by Frank et al. (2015)
- behavioral data (eye-tracking data and self-paced reading times) from Frank et al. (2013)
Our unsupervised ranking model outperforms the supervised IMS system by 1.02% on the CoNLL F1 score, and achieves competitive performance with the latent tree model. Moreover, our approach considerably narrows the gap to other supervised systems listed in Table 3 . 
 Question: Is the model presented in the paper state of the art?
Output: No, supervised models perform better for this task.
Motivated by this, we introduce resolution mode variables $\Pi = \lbrace \pi _1, \ldots , \pi _n\rbrace $ , where for each mention $j$ the variable $\pi _j \in \lbrace str, prec, attr\rbrace $ indicates in which mode the mention should be resolved. In our model, we define three resolution modes — string-matching (str), precise-construct (prec), and attribute-matching (attr) — and $\Pi $ is deterministic when $D$ is given (i.e. $P(\Pi |D)$ is a point distribution). We determine $\pi _j$ for each mention $m_j$ in the following way:

$\pi _j = str$ , if there exists a mention $m_i, i < j$ such that the two mentions satisfy the String Match sieve, the Relaxed String Match sieve, or the Strict Head Match A sieve in the Stanford multi-sieve system BIBREF1 .

$\pi _j = prec$ , if there exists a mention $m_i, i < j$ such that the two mentions satisfy the Speaker Identification sieve, or the Precise Constructs sieve.

$\pi _j = attr$ , if there is no mention $m_i, i < j$ satisfies the above two conditions. 
 Question: Are resolution mode variables hand crafted?
Output: No
We first measure the utility of various components in Logician to select the optimal model, and then compare this model to the state-of-the-art methods in four types of information extraction tasks: verb/preposition-based relation, nominal attribute, descriptive phrase and hyponymy relation. 
 Question: What open relation extraction tasks did they experiment on?
Output: verb/preposition-based relation, nominal attribute, descriptive phrase and hyponymy relation.
Frege promoted what we could call sentence holism: “Only in the context of a sentence does a word have a meaning.” BIBREF10 
 Question: What does Frege's holistic and functional approach to meaning states?
Output: Only in the context of a sentence does a word have a meaning.
We compare Blse (Sections UID23 – UID30 ) to VecMap, Muse, and Barista (Section "Previous Work" ) as baselines, which have similar data requirements and to machine translation (MT) and monolingual (Mono) upper bounds which request more resources. 
 Question: what baseline do they compare to?
Output: VecMap Muse Barista
We pair 11'248 standard German written words with their phonetical representations in six different Swiss dialects: Zürich, St. Gallen, Basel, Bern, Visp, and Stans (Figure FIGREF1). 
 Question: How many words are coded in the dictionary?
Output: 11'248
Results are shown in Table TABREF11 . Results show that incorporating self-attention mechanism in the encoder is beneficial for most tasks. However, original models were better in some tasks (CR, MPQA, MRPC), suggesting that self-attention mechanism could sometimes introduce noise in sentence features. Overall, utilizing self-attentive sentence representation further improves performances in 5 out of 8 tasks. 
 Question: How much better performing is the proposed method over the baselines?
Output: original models were better in some tasks (CR, MPQA, MRPC) utilizing self-attentive sentence representation further improves performances in 5 out of 8 tasks
Firstly, LastStateRNN is the classic RNN model, where the last state passes through an MLP and then the LR Layer estimates the corresponding probability. In contrast, in the AvgRNN model we consider the average vector of all states that come out of the cells. The AttentionRNN model is the one that it has been presented in BIBREF9. 
 Question: What was the baseline?
Output: LastStateRNN AvgRNN AttentionRNN
We evaluated our attention transformations on three language pairs. We use the IWSLT 2014 corpus for De-En, the KFTT corpus for Ja-En BIBREF19 , and the WMT 2016 dataset for Ro-En. 
 Question: What are the language pairs explored in this paper?
Output: De-En Ja-En Ro-En