DinoStackAI/qasper-rag huggingface dataset:
11.03% and 14.7% for en–ta and en–hi pairs respectively
attentional encoder–decoder
the OpenSubtitles dataset BIBREF18
datasets generated by two methods (translated by Google Translation service and annotated by human)
the original MS-COCO English dataset
Neural Image Captioning (NIC) model BIBREF14 Image Captioning model from the Pytorch-tutorial BIBREF15 by Yunjey
human translation and Google Translation service
encoder-decoder architecture of CNN for encoding and LSTM for decoding
The two models are trained with three mentioned datasets, then validated on  subset for each dataset and evaluated using BLEU, ROUGE and CIDEr measures.
Yes, they also evaluate on the ROCStories
(Spring 2016) dataset which collects 50k five sentence commonsense stories.
CNNs in combination with LSTMs create word embeddings from domain specific materials Tree–Structured LSTMs
scored tweets by assigning them a probability based on each model higher probability according to the funny tweet model are considered funnier since they are more like the humorous training data
bigrams and trigrams as features KenLM BIBREF8 with modified Kneser-Ney smoothing and a back-off technique
Attention neural networks can represent more languages than other networks. Simple recurring networks can describe regular languages. CNNs can describe only strictly local languages.
Counting Counting with Noise Reversing
memory module could be applied to other domains such as summary generation future approach might combine memory module architectures with pointer softmax networks
This is a 0.68 perplexity improvement over the vanilla language model without the NTM augmentation.
NTM-LM
Bilingual Evaluation Understudy (BLEU) BIBREF22 accuracy
Best accuracy result of proposed model is 82.73, 79.35 compared to best baseline result of 80.75, 78.68 on MultiWoz and DailyDialogue datasets respectively.
Bidirectional Gated Recurrent Units (Bi-GRUs) BIBREF23, TextCNNs BIBREF12 and BERT BIBREF14
human-to-human conversations
PETRARCH PETRARCH2 scraping of news content from the web geolocation of the coded events a comprehensive pipeline
Theorem 1.11. For any context-free language INLINEFORM0 , suppose we relabel and write INLINEFORM1 for some regular language INLINEFORM2 , whose corresponding minimum-size DFA has INLINEFORM3 states. Then there exists a simple RNN with a hidden layer of size INLINEFORM4 whose INLINEFORM5 -language is INLINEFORM6 .
With post-processing
Turning this score into the prediction is usually performed by setting a threshold, such as 0 and 0.5, so labels which have a score assigned greater than that are assigned to the sample
accuracy improves with increasing power of the enhancement on the training data, however, only as long as the enhancement on the training data is not stronger than on the test data
in terms of WER for the DEV (EVAL) set, the single proposed model (GSS1) has higher WER than the multiple proposed model  (GSS6) by 7.4% (4.1%).
Previous single system state of the art had WER of  58.3 (53.1).
In case of singe model the WER was better by 10.%  (6.4%) and in case of multi model it was 3.5% ( 4.1%)
we use MultiCCA word embeddings published by BIBREF3 joint multilingual sentence representations
Veridicality  class,  log-linear model for  measuring  distribution over a tweet's veridicality, Twitter NER system  to  to identify named entities,  five feature templates: context words, distance between entities, presence of punctuation, dependency paths, and negated keyword.
neutral (“Uncertain about the outcome")
debate topics such as healthcare, military programs, administration processes, worker safety, energy projects, gun control.
We use pre-trained GloVe BIBREF10 word embeddings to compute vector representation of each VPD by averaging the GloVe vectors for all words in the document. A logistic regression classifier is then trained on the vector representations thus obtained.
Convote dataset BIBREF3
At each iteration of the training process, we first generate INLINEFORM6 samples from the generator G then we draw INLINEFORM7 samples from the real dataset. We then perform gradient descent on the AC-BLSTM and generative net G and finally update the parameters of both nets.
Model is evaluated on six tasks: TREC, MR, SST-1, SST-2, SUBJ and YELP13.
word similarity POS tagging
POS is one of the biggest factors in stability
We define stability as the percent overlap between nearest neighbors in an embedding space. 0% stability indicates complete disagreement
word2vec, GloVe, and PPMI
Infrequent N-gram Recovery (INR) and Feature Decay Algorithms (FDA)
they can be considered as candidate sentences for a data-selection algorithm to decide which sentence-pairs should be used to fine-tune the NMT model
WMT 2017 biomedical translation WMT 2015 News Translation
nouns have case, number and gender verbs have person, number, tense and aspect features may be underspecified
weighted sum of the annotations
Tags categories ranging from anaphoric (definite, possessive pronoun), attribute (colour, concrete quantity, intersective, relation), unnamed entity (concept), logical (alternative, disjunction), discourse (subordinate relation), events (present simple, past simple), etc.
For Document- level comparison, the model achieves highest CS precision and  F1 score and it achieves higher BLEU score that TMTE, Coatt, CCDT, and HEDT. 
In terms of Human Evaluation, the model had the highest average score, the highest Fluency score, and the second highest Content Fidelity. 
In terms of Sentence-level comparison the model had the highest Recall and F1 scores for Content Fidelity.
Content Fidelity (CF) Style Preservation BLEU score Content selection
Rule-based Slot Filling Method (Rule-SF) Copy-based Slot Filling Method (Copy-SF) Conditional Copy based Data-To-Text (CCDT) Hierarchical Encoder for Data-To-Text (HEDT) Text Manipulation with Table Encoder (TMTE) Co-attention-based Method (Coatt) attention-based Seq2Seq method with copy mechanism rule-based method MAST AdvST S-SOTA
Document-level dataset has total of 4821 instances. 
Sentence-level dataset has total of 45583 instances.
The data set obtained from Pypestream
the bias score of a word $x$ considering its word embedding $h^{fair}(x)$ and two gender indicators (words man and woman)
considers the notion of a Fair Region to update a subset of the trainable parameters of a Memory Network
employ CRF as a learning model. In the feature engineering process they use morphosyntactic features, distributional ones as well as word clusters based on these learned representations.
the number of entities, unique entities in the training and test sets
Average Cohen’s Kappa score of inter-annotator agreement was 0.655
Identify the entities occurring in the dataset and annotate them with one of the 13 possible types.
NER systems are usually trained using texts that follow particular morpho-syntactic rules. The tweets have a different style and don't follow these rules.
The tweets were gathered using Twitter API plus tweets provided by the French National Railway Corporation; the tweets were split into training and test sets, and then annotated.
we explore strategies to reduce forgetting for comprehension systems during domain adaption. Our goal is to preserve the source domain's performance as much as possible, while keeping target domain's performance optimal and assuming no access to the source data.
Factorization Machines (FM) BIBREF5, SVD BIBREF0, Probabilistic Matrix Factorization (PMF) BIBREF24, Nonnegative Matrix Factorization (NMF) BIBREF25, DeepCoNN BIBREF1, D-ATT BIBREF3, MPCN BIBREF17, and HUITA BIBREF16
9 Amazon product review datasets for 9 different domains, and the large-scale Yelp challenge dataset on restaurant reviews
Semeval 2014 for ASC has total of  2951 and 4722 sentiments for Laptop and Restaurnant respectively, while SemEval 2016 for AE has total of 3857 and 5041 sentences on Laptop and Resaurant respectively.
By using a white-box method using perturbation calculated based on the gradient of the loss function.
adversarial examples from BERT embeddings using the gradient of the loss we feed the perturbed examples to the BERT encoder
To construct a database for information retrieval, we collected human-human utterances from massive online forums, microblogs, and question-answering communities the database contains 7 million query-reply pairs for retrieval For the generation part, we constructed another dataset from various resources in public websites comprising 1,606,741 query-reply pairs
BIBREF9
BIBREF9 took approximately 4.5 months even after applying optimization of trimming sentences, while the training process of our FOFE-based model took around 3 days
BIBREF4
different from language model in terms of that the sense prediction of a target word depends on its surrounding sequence rather than only preceding sequence
GRURNTN reduced the BPC from 1.39 to 1.33 (0.06 absolute / 4.32% relative BPC) from the baseline GRURNN, and LSTMRNTN reduced the BPC from 1.37 to 1.34 (0.03 absolute / 2.22% relative BPC) from the baseline LSTMRNN. From our experiment on the PennTreeBank corpus, our proposed models outperformed the baseline models with a similar number of parameters in character-level language modeling and word-level language modeling tasks. In a character-level language modeling task, GRURNTN obtained 0.06 absolute (4.32% relative) BPC reduction over GRURNN, and LSTMRNTN obtained 0.03 absolute (2.22% relative) BPC reduction over LSTMRNN. In a word-level language modeling task, GRURNTN obtained 10.4 absolute (10.63% relative) PPL reduction over GRURNN, and LSTMRNTN obtained 11.29 absolute (10.42% relative PPL) reduction over LSTMRNN.
For the former limitation, the RecNN performance can be improved by adding more interaction between the two input vectors. Therefore, a new architecture called a Recursive Neural Tensor Network (RecNTN) tried to overcome the previous problem by adding interaction between two vectors using a tensor product, which is connected by tensor weight parameters. Each slice of the tensor weight can be used to capture the specific pattern between the left and right child vectors. For RecNTN, value $p_1$ from Eq. 13 and is defined by:

$$p_1 &=& f\left( \begin{bmatrix} x_1 & x_2 \end{bmatrix} W_{tsr}^{[1:d]} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} + \begin{bmatrix} x_1 & x_2 \end{bmatrix} W + b \right) \\ p_2 &=& f\left( \begin{bmatrix} p_1 & x_3 \end{bmatrix} W_{tsr}^{[1:d]} \begin{bmatrix} p_1 \\ x_3 \end{bmatrix} + \begin{bmatrix} p_1 & x_3 \end{bmatrix} W + b \right)$$ (Eq. 15)

where $W_{tsr}^{[1:d]} \in \mathbb {R}^{2d \times 2d \times d}$ is the tensor weight to map the tensor product between two children vectors. Each slice $W_{tsr}^{[i]}$ is a matrix $\mathbb {R}^{2d \times 2d}$ .
we report our experiment results on PTB character-level language modeling using our baseline models GRURNN and LSTMRNN as well as our proposed models GRURNTN and LSTMRNTN. In this experiment, GRURNN made faster progress than LSTMRNN, but eventually LSTMRNN converged into a better BPC based on the development set. Our proposed model GRURNTN made faster and quicker progress than LSTMRNTN and converged into a similar BPC in the last epoch. Both proposed models produced lower BPC than our baseline models from the first epoch to the last epoch. Our proposed model GRURNTN and LSTMRNTN outperformed both baseline models. GRURNTN reduced the BPC from 1.39 to 1.33 (0.06 absolute / 4.32% relative BPC) from the baseline GRURNN, and LSTMRNTN reduced the BPC from 1.37 to 1.34 (0.03 absolute / 2.22% relative BPC) from the baseline LSTMRNN. Overall, GRURNTN slightly outperformed LSTMRNTN, and both proposed models outperformed all of the baseline models on the character-level language modeling task. we report our experiment results on PTB word-level language modeling using our baseline models GRURNN and LSTMRNN and our proposed models GRURNTN and LSTMRNTN. In this experiment, GRURNN made faster progress than LSTMRNN. Our proposed GRURNTN's progress was also better than LSTMRNTN. The best model in this task was GRURNTN, which had a consistently lower PPL than the other models. GRURNTN reduced the perplexity from 97.78 to 87.38 (10.4 absolute / 10.63% relative PPL) over the baseline GRURNN and LSTMRNTN reduced the perplexity from 108.26 to 96.97 (11.29 absolute / 10.42% relative PPL) over the baseline LSTMRNN. Overall, LSTMRNTN improved the LSTMRNN model and its performance closely resembles the baseline GRURNN. However, GRURNTN outperformed all the baseline models as well as the other models by a large margin. In a character-level language modeling task, GRURNTN obtained 0.06 absolute (4.32% relative) BPC reduction over GRURNN, and LSTMRNTN obtained 0.03 absolute (2.22% relative) BPC reduction over LSTMRNN. In a word-level language modeling task, GRURNTN obtained 10.4 absolute (10.63% relative) PPL reduction over GRURNN, and LSTMRNTN obtained 11.29 absolute (10.42% relative PPL) reduction over LSTMRNN.
a Bidirectional Encoding model BIBREF2
They collect data using the AYLIEN News API,  which provides search capabilities for news articles enriched with extracted entities and other metadata and take a step to compile a curated list of topics. The final dataset consists of 32,227 pairs of news articles and topics annotated with their stance.
English corpus has a dictionary of length 106.848 German version has a dictionary of length 163.788
The accuracy of addressee selection is improved by 11.025 percent points on average, the accuracy of response selection is improved by 3.09 percent points on average.
Dynamic-RNN model from BIBREF4
BIBREF3, BIBREF4, BIBREF5
each iteration they can align two words only if they share the same morphosyntactic category
They find relations that connect questions to the answer-options.
Application I: Learning a Knowledge Graph Embedding Space for Context Understanding in Automotive Driving Scenes Application II: Neural Question-Answering using Commonsense Knowledge Bases
time and the linguistic content of posts by the users
We used a third party social media site (i.e., Google Plus), one that was not used in our analysis to compile our ground truth We discarded all users who did not link to an account for both Twitter and Facebook
from January 2014 to December 2015
A word-level sentiment analysis was made, using Sentilex-PT BIBREF7 - a sentiment lexicon for the portuguese language, which can be used to determine the sentiment polarity of each word, i.e. a value of -1 for negative words, 0 for neutral words and 1 for positive words
portuguese and english
abundance or lack of the elements of urban lifestyle
Semantic Features : Opinion Words, Vulgar Words, Emoticons,  Speech Act Verbs,  N-grams.
Syntactic Features: Punctuations, Twitter-specific Characters, Abbreviations, Dependency Sub-trees,  Part-of-speech.
Assertion, Recommendation Expression, Question, Request, and Miscellaneous
topic-specific classifier
three undergraduate annotators
Binary features indicating opinion words, vulgar words, emoticons, speech act verbs and unigram, bigram and trigram that appear at least five times in the dataset
Binary features indicating appeance of punctuations, twitter-specific characters - @, #, and RT, abbreviations, length one and two sub-trees extracted from dependency sub-tree and Part-of-speech - adjectives and interjections.
A dataset they annotated, "Harvard General Inquirer" Lexicon for opinion words, a collection of vulgar words, an online collection of text-based emoticons, and Wierzbicka's collection of English speech act verbs
Model A, which was trained on LibriSpeech data with SAT, together with the language model also trained with LibriSpeech.
110 pieces of music-removed version of commercial English popular songs
The performance is significantly degraded without two special tags (0,025 in MRR)
ESIM
Conversations that are typical for a social networking service.
300-dimensional GloVe vectors
Precision and recall for 2-way classification and F1 for 4-way classification.
In 2-way classification they used LUN-train for training, LUN-test for development and the entire SLN dataset for testing. In 4-way classification they used LUN-train for training and development and LUN-test for testing.
In 2-way classification precision score was 88% and recall 82%. In 4-way classification on LUN-dev F1-score was 91% and on LUN-test F1-score was 65%.
Satirical and Legitimate News Database BIBREF2 RPN: Random Political News Dataset BIBREF10 LUN: Labeled Unreliable News Dataset BIBREF0
CNN LSTM BERT
dropout embedding dropout DropBlock
Accuracy, precision, recall and F1 score.
almost doubles the number of commits in the training split to 1493 validation, and test splits containing 808, 265, and 264 commits
manually-curated dataset of publicly disclosed vulnerabilities in 205 distinct open-source Java projects mapped to commits fixing them
randomly sampling them rather than enumerate them all simple max-pooling to extract span representation because those features are implicitly included in self-attention layers of transformers
In ELMo model, SCIIE achieves almost 3.0% F1 higher than BiLSTM in SciBERT, the performance becomes similar, which is only a 0.5% gap
14,769 actions
1,268
Concreteness Feature-based Classifier LSTM and ELMo Yolo Object Detection
Amazon Mechanical Turk (AMT)
length of our collected videos varies from two minutes to twenty minutes
More precisely, the experts were instructed to re-consider each problem and be especially wary of missing hypotheses. If they considered the entailment to hold, we still gave the instruction to optionally mention any additional implicit hypothesis that they would be using.
fluency relevance diversity originality
Chinese dataset containing human-human context response pairs collected from Douban Group
a GRU language model
similar context INLINEFORM1 and its associated response INLINEFORM2
CoQA BIBREF17 DuoRC BIBREF18 HotpotQA (distractor) BIBREF1 SQuAD v1.1 BIBREF0 SQuAD v2.0 BIBREF20 ARC (Challenge) BIBREF21 MCTest BIBREF22 MultiRC BIBREF23 RACE BIBREF24 SWAG BIBREF25
BERT-large BIBREF3
It is meant that only most contributive k elements are reserved, while other elements are removed.
For En-De translation the newstest 2014 set from WMT 2014 En-De translation dataset, for En-Vi translation the tst2013 from IWSLT 2015 dataset. and for De-En translation the teste set from IWSLT 2014.
6,138 pieces of logical reasoning questions
They identify biases as lexical choice and sentence length for right and wrong answer options in an isolated context, without the question and paragraph context that typically precedes answer options. Lexical choice was identified by calculating per-token correlation scores with "right" and "wrong labels. They calculated the average sentence length for "right" and "wrong" sentences.
data expansion using oversampling add loss control in the model
six different variations of their multi-head attention model
normalize unreadable numbers, symbols or characters
These 980 PLOs were annotated within a total of 670 tweets.
PERSON, LOCATION, and ORGANIZATION
Word-Level Knowledge Distillation Sequence-Level Knowledge Distillation Sequence-Level Interpolation
Prune %x of the parameters by removing the weights with the lowest absolute values.
WMT 2014 IWSLT 2015
Berant et al. (2013), Yao and Van Durme (2014), Xu et al. (2014), Berant and Liang (2014), Bao et al. (2014), Border et al. (2014), Dong et al. (2015), Yao (2015), Bast and Haussmann (2015), Berant and Liang (2015), Reddy et al. (2016), Yih et al. (2015)
by applying an answer refinement model which takes the Wikipedia page of the topic entity into consideration to filter out the wrong answers and pick the correct ones
F1 score of 39.9 for semantic-based parsing methods. For information extraction methods, 49.4 using relation extraction, 40.8 using distributed representations, and 52.5 using neural networks models
Document-level context encoder, entity and sentence-level context encoders with common attention, then logistic regression, followed by adaptive thresholds.
e.g., lexical and syntactic features
an uncased base BERT QA model BIBREF9 trained on SQuAD 1.1 BIBREF0
359 samples
SARI compares the predicted simplification with both the source and the target references
PBMT-R, Hybrid, SBMT+PPDB+SARI, DRESS-LS, Pointer+Ent+Par, NTS+SARI, NSELSTM-S and DMASS+DCSS
PC-GITA corpus 88 PD patients and 88 HC speakers from Germany 100 native Czech speakers (50 PD, 50 HC)
SemEval-2017 task 8 dataset is split into train, development and test sets. Two events go into test set and eight events go to train and development sets for every thread in the dataset. PHEME dataset is split as leave-one-event-out cross-validation. One event goes to test and the rest of events go to training set for each conversation. Nine folds are created
PKD-Skip
Sentiment Classification Paraphrase Similarity Matching Natural Language Inference Machine Reading Comprehension
deletion model again increases the BLEU score over just the insertion model, by around 2 BLEU points
maximum entropy neural network tagging model
find a set of relevant activations of the listener INLINEFORM0 and summarize them into a context
forbids emitting the EOS token beam search criterion can be extended to promote long transcripts coverage criterion prevents looping over the utterance ground-truth label distribution is smoothed
40% speed-up rate
experiments on two widely used tasks: the IWSLT14 German-to-English (De-En) task and the WMT14 English-to-German (En-De) task we train the Pre-LN BERT without the warm-up stage and compare it with the Post-LN BERT
2008 (NIST08) datasets Baidu Speech Translation Corpus (BSTC)
It depends on the model used.
we also evaluate the original ill-formed question using the automatic metrics
Is the question grammatically correct? Is the spelling correct? Is the question an explicit question, rather than a search query, a command, or a statement?
Is the question grammatically correct? Is the spelling correct? Is the question an explicit question annotators were asked to annotate each aspect with a binary (0/1) answer
The domains represent different subfields related to the topic of the questions.
Data used has total of 23315 sentences.
BLEU score
A supervised model with byte pair encoding was the best for English to Pidgin, while a supervised model with word-level encoding was the best for Pidgin to English.
word-level Byte Pair Encoding (BPE) subword-level
Transformer architecture of BIBREF7
5,415 sentences
SVM classifier with an RBF kernel deep feed-forward neural network (FNN) with two hidden layers (with 200 and 50 neurons, respectively) and a softmax output unit for the binary classification
ClaimBuster
four transcripts of the 2016 US election: one vice-presidential and three presidential debates
Answer with content missing: (Table 4) Mean reciprocal rank of proposed model is 0.76 compared to 0.64 of bag-of-ngrams.
entity dataset released by xie16entitydesc2
a model-driven demographic annotation pipeline for apparent age and gender, analysis of said annotation models and the presentation of annotations for each image in the training set of the ILSVRC 2012 subset of ImageNet
Females and males with age 75+
100 sentences
CoNLL 2003 English NER BIBREF8
BIBREF2 BIBREF3 BIBREF4 BIBREF5 BIBREF6 BIBREF7
Wine Ontology m-piro ontology Disease Ontology
7.3% on RACE-M and 1.5% on RACE-H
separate algorithm for pruning out spurious logical forms using fictitious tables
trained Binary PV-DBOW with bigrams on the English Wikipedia, and then inferred binary codes for the test parts of the 20 Newsgroups and RCV1 datasets
20 Newsgroups Reuters Corpus Volume English Wikipedia
125383 death certificates
International Classification of Diseases, 10th revision (ICD-10) BIBREF1
Swedish online data from 2018 crawled by Trendiction manually scraped party platforms from the eight parties in parliament and Feministiskt Initiativ (Feminist Initiative)
the average agreement is 18% higher with INLINEFORM7 than with INLINEFORM8
around 100,000 annotations
Krippendorff's Alpha-reliability ( INLINEFORM0 ) BIBREF6 F score ( INLINEFORM0 ) Accuracy ( INLINEFORM0 ) Accuracy within 1 ( INLINEFORM0 )
Friedman-Nemenyi test BIBREF14 , BIBREF15