DinoStackAI/qasper-rag huggingface dataset:
Text Similarity to Source Tweet Text Similarity to Replied Tweet Tweet Depth
average accuracy over each single-language model (Avg), and accuracy obtained when training on the concatenation of all languages but the target one (All)
Language Modeling (LM) PTB BIBREF25 , WikiText-103 BIBREF26 and One-Billion Word benchmark BIBREF27 datasets neural machine translation (NMT) WMT 2016 English-German dataset
Data augmentation (es)  improved Adv es by 20% comparing to baseline 
Data augmentation (cs) improved Adv cs by 16.5% comparing to baseline
Data augmentation (cs+es) improved both Adv cs and Adv es by at least 10% comparing to baseline 
All models show improvements over adversarial sets
we devise a test set consisting of ‘adversarial’ examples, i.e, perturbed examples that can potentially change the base model's prediction. We use two approaches described in literature: back-translation and noisy sequence autoencoder.
Clusters of Twitter user ids from accounts of American or German political actors, musicians, media websites or sports club
inappropriate discriminating
On subtask 3 best proposed model has F1 score of 92.18 compared to best previous F1 score of 88.58.
On subtask 4 best proposed model has 85.9, 89.9 and 95.6 compared to best previous results of 82.9, 84.0 and 89.9 on 4-way, 3-way and binary aspect polarity.
The sentence we want to generate from the target-aspect pair is a question, and the format needs to be the same. For the NLI task, the conditions we set when generating sentences are less strict, and the form is much simpler. For QA-B, we add the label information and temporarily convert TABSA into a binary classification problem ( INLINEFORM0 ) to obtain the probability distribution auxiliary sentence changes from a question to a pseudo-sentence
from 469 posts to 17 million
hatespeechdata.com
7,507
1448 sentences more than the dataset from Bhat et al., 2017
weighted F1-score
The dataset from a joint ADAPT-Microsoft project
includes 1,941 citation instances from 186 papers
Annotators went through various phases to make sure their annotations did not deviate from the mean.
D-IMDB (derived from large scale IMDB data snapshot) D-FB (derived from large scale Freebase data snapshot)
the model proposed in BIBREF3
$0.3$ million records
The sequence model architectures which this method is transferred to are: LSTM and Transformer-based models
Across 4 datasets, the best performing proposed model (CNN) achieved an average of 363% improvement over the state of the art method (LR-CNN)
error rate in a minimal pair ABX discrimination task
a standard beam search decoder BIBREF5 with several straightforward performance optimizations
BLEU scores
The unsupervised tasks include five tasks from SemEval Semantic Textual Similarity (STS) in 2012-2016 BIBREF30 , BIBREF31 , BIBREF32 , BIBREF33 , BIBREF34 and the SemEval2014 Semantic Relatedness task (SICK-R) BIBREF35 .

The cosine similarity between vector representations of two sentences determines the textual similarity of two sentences, and the performance is reported in Pearson's correlation score between human-annotated labels and the model predictions on each dataset. Supervised Evaluation
It includes Semantic relatedness (SICK) BIBREF35 , SemEval (STS-B) BIBREF36 , paraphrase detection (MRPC) BIBREF37 , question-type classification (TREC) BIBREF38 , movie review sentiment (MR) BIBREF39 , Stanford Sentiment Treebank (SST) BIBREF40 , customer product reviews (CR) BIBREF41 , subjectivity/objectivity classification (SUBJ) BIBREF42 , opinion polarity (MPQA) BIBREF43 .
a linear projection and a bijective function with continuous transformation though  ‘affine coupling layer’ of (Dinh et al.,2016).
perplexity of the models
WN18 and FB15k
simulation-based dialogue generation, where the user and system roles are simulated to generate a complete conversation flow, which can then be converted to natural language using crowd workers
Alarm
Bank
Bus
Calendar
Event
Flight
Home
Hotel
Media
Movie
Music
RentalCar
Restaurant
RideShare
Service
Travel
Weather
Google's Neural Machine Translation
They pre-train the models using 600000 articles as an unsupervised dataset and then fine-tune the models on small training set.
645, 600000
The negative effects were insignificant.
the WMT'14 English-French (En-Fr) and English-German (En-De) datasets.
English to French and English to German
Transformer models in their base configuration BIBREF11, using 6 encoder and decoder layers, with model and hidden dimensions of 512 and 2048 respectively, and 8 heads for all attention layers
abstracts sentences
Sequential (Denoising) Autoencoder TF-IDF BOW SkipThought FastSent Siamese C-BOW C-BOW C-PHRASE ParagraphVector
Accuracy and F1 score for supervised tasks, Pearson's and Spearman's correlation for unsupervised tasks
by also learning source embeddings for not only unigrams but also n-grams present in each sentence, and averaging the n-gram embeddings along with the words
AMI IHM meeting corpus
Zipf's law describes change of word frequency rate, while Heaps-Herdan describes different word number in large texts (assumed that Hepas-Herdan is consequence of Zipf's)
Lead-3
SumBasic Lsa LexRank TextRank Bayes Hmm MaxEnt NeuralSum Lead-N
Random perturbation of Wikipedia sentences using mask-filling with BERT, backtranslation and randomly drop out
Proposed method achieves 94.5 UAS and 92.4 LAS  compared to 94.3 and 92.2 of best state-of-the -art greedy based parser. Best state-of-the art parser overall achieves 95.8 UAS and 94.6 LAS.
root mean square error between the actual and the predicted price of Bitcoin for every minute
By using Apache Spark which stores all executions in a lineage graph and recovers to the previous steady state from any fault
handling large volume incoming data, sentiment analysis on tweets and predictive online learning
Google Images Reddit Memes Dataset
weakly labeled into hate or non-hate memes, depending on their source
Penn Treebank (PTB) end-to-end (E2E) text generation corpus
Loss analysis. To conduct a more thorough evaluation, we further investigate model behaviours in terms of both reconstruction loss and KL loss, as shown in Figure FIGREF14. These plots were obtained based on the E2E training set using the inputless setting.
BLUE utilizes different metrics for each of the tasks: Pearson correlation coefficient, F-1 scores, micro-averaging, and accuracy
bilingual dictionary induction, monolingual and cross-lingual word similarity, and cross-lingual hypernym discovery
because word pair similarity increases if the two words translate to similar parts of the cross-lingual embedding space
Word alignments are generated for parallel text, and aligned words are assumed to also share AMR node alignments.
number of dialogs that resulted in launching a skill divided by total number of dialogs
Answer with content missing: (Table 2) CONCAT ensemble
mean-pooling their outputs (AVG) concatenating the entity and its name with a slash symbol (CONCAT)
filter deletes all KB triples where the correct answer (e.g., Apple) is a case-insensitive substring of the subject entity name (e.g., Apple Watch) person name filter uses cloze-style questions to elicit name associations inherent in BERT, and deletes KB triples that correlate with them
At last, our model is evaluated on two benchmark treebanks for both constituent and dependency parsing. The empirical results show that our parser reaches new state-of-the-art for all parsing tasks.
For different numbers of shared layers, the results are in Table TABREF14. We respectively disable the constituent and the dependency parser to obtain a separate learning setting for both parsers in our model.
. On PTB, our model achieves 93.90 F1 score of constituent parsing and 95.91 UAS and 93.86 LAS of dependency parsing. On CTB, our model achieves a new state-of-the-art result on both constituent and dependency parsing.
token representation self-attention encoder, Constituent Parsing Decoder Dependency Parsing Decoder
They initialize their word and entity embeddings with vectors pre-trained over a large corpus of unlabeled data.
The authors believe that the Wikilinks corpus  contains ground truth annotations while being noisy. They discard mentions that cannot have ground-truth verified by comparison with Wikipedia.
No, it is a probabilistic model trained by finding feature weights through gradient ascent
image feature question feature label vector for the user's answer
Through human evaluation where they are asked to evaluate the generated output on a likert scale.
BLEU Self-BLEU n-gram based score probability score
annotators who were not security experts researchers in either NLP or computer security
Stanford CoreNLP BIBREF11
RNN encoders
improvements of up to 1.5 BLEU over the seq2seq baseline
transformer model achieves higher BLEU score than both Attention encoder-decoder and sequence-sequence model
word embeddings to generate a new feature, i.e., summarizing a local context
The most simple one is to directly apply a CNN layer after the embedding layer to obtain blended contextual representations. Then a GRU layer is applied afterward.
CBT NE/CN MR Movie reviews IMDB Movie reviews SUBJ
BERT generates sentence embeddings that represent words in context. These sentence embeddings are merged into a single conversational-context vector that is used to calculate a gated embedding and is later combined with the output of the decoder h to provide the gated activations for the next hidden layer.
the training dataset is large while the target dataset is usually much smaller
in the OpenBookQA setup the open book part is much larger the open book part is much larger (than a small paragraph) and is not complete as additional common knowledge may be required
documents are segmented into paragraphs and processed at the paragraph level
F$_1$, precision, and recall
Variety of formats supported (PDF, Word...), user can define content elements of document
To the best of our knowledge, our method achieves state-of-the-art results in weighted-accuracy and standard accuracy on the dataset
Stanford Sentiment Treebank
optimize single task with no synthetic data
networks where nodes represent causes and effects, and directed edges represent cause-effect relationships proposed by humans
if it includes  negative utterances, negative generalizations and insults concerning ethnicity, nationality, religion and culture.
The baseline model BIBREF5 we compare with regards the output space of the model as a subset INLINEFORM2 where INLINEFORM3 is the set of all tag sets seen in this training data.
Danish/Swedish (da/sv), Russian/Bulgarian (ru/bg), Finnish/Hungarian (fi/hu), Spanish/Portuguese (es/pt)
By treating logical forms as a latent variable and training a discriminative log-linear model over logical form y given x.
GIZA++ BIBREF3 or fast_align BIBREF4
BTEC corpus the CSTAR03 and IWSLT04 held out sets the NIST2008 Open Machine Translation Campaign
ISOT dataset: LLVM
Liar dataset: Hybrid CNN and LSTM with attention
(i) Uniform (ii) SVR+W (iii) SVR+O (iv) C4.5SSL (v) GLM
BIBREF2 BIBREF1
8757 news records
Inconsistency in Noun Phrase Structures Inconsistency Between Clauses Inconsistency Between Named Entities and Noun Phrases Word Level Feature Using TF-IDF
unweighted average recall (UAR) metric
75 meetings and about 70 hours of real-time audio duration
8 levels and degrees, ranging from `not hot' to `luke warm' to `hot +'. Every utterance was labeled with one of these discrete labels by a single annotator
fragments are interchangeable if they occur in at least one lexical environment that is exactly the same
Answer with content missing: (Applications section) We use Wikipedia articles
in five languages
(Kinyarwanda, Lao, Pashto, Tok Pisin, and a subset of English) as well as the Na dataset of Adams
et al. (2017).
Select:
Kinyarwanda, Lao, Pashto, Tok Pisin, and a subset of English
deciding publisher partisanship, risk annotator bias because of short description text provided to annotators
partisan news detector
concatenation consistently outperforms the gated-attention mechanism for both training and testing instructions
No feature is given, only discussion that semantic features are use in practice and yet to be discovered how to embed that knowledge into statistical decision theory framework.
Outputs from models that are better in the sense of cross-entropy or perplexity are harder to distinguish from authentic text.
It is not completely valid for natural languages because of diversity of language - this is called smoothing requirement.
type of recurrent unit type of attention choice of sequential vs. tree-based model structure
proposed method comprises of two steps: a neighbourhood reconstruction step (Section "Nearest Neighbour Reconstruction" ), and a projection step (Section "Projection to Meta-Embedding Space" ). In the reconstruction step, we represent the embedding of a word by the linearly weighted combination of the embeddings of its nearest neighbours in each source embedding space.
average 12.8 min per recording
national dialects of English
automatic translator with Moses
2-layer LSTM model with 500 hidden units in both encoder and decoder
46 documents makes up our base corpus
exact string matching inflectional string matching
Wiktionary Oxford Dictionary of English Idioms UsingEnglish.com (UE) Sporleder corpus VNC dataset SemEval-2013 Task 5 dataset
Direct source$\rightarrow $target: A standard NMT model trained on given source$\rightarrow $target Multilingual: A single, shared NMT model for multiple translation directions Many-to-many: Trained for all possible directions among source, target, and pivot languages Many-to-one: Trained for only the directions to target language
the CIDEr-D BIBREF22 , SPICE BIBREF23 , BLEU BIBREF24 , METEOR BIBREF25 , and ROUGE-L BIBREF26 metrics
Conll, Weblogs, Newsgroups, Reviews, Answers
a new metric to reveal a model's robustness against exposure bias
The relation R(x,y) is mapped onto a question q whose answer is y
The resolution of the low-frequency feature maps is reduced by an octave – height and width dimensions are divided by 2. In this work, we explore spatial reduction by up to 3 octaves – dividing by $2^t$, where $t=1,2,3$ – and for up to 4 groups. We refer to such a layer as a multi-octave convolutional (MultiOctConv) layer,
Couples Therapy Corpus (CoupTher) BIBREF21
pre-trained to identify the presence of behavior from a sequence of word using the Couples Therapy Corpus
The model does not add new relations to the knowledge graph.
6,127 scientific entities, including 2,112 Process, 258 Method, 2,099 Material, and 1,658 Data entities
avoids the need for expensive cross-validation for hyperparameter selection
the personalized bAbI dialog dataset
The dataset contains about 590 tweets about DDos attacks.
Tweets related to a Bank of America DDos attack were used as training data. The test datasets contain tweets related to attacks to Bank of America, PNC and Wells Fargo.
inspired by the OntoNotes5 corpus BIBREF7 as well as the ACE (Automatic Content Extraction) English Annotation Guidelines for Entities Version 6.6 2008.06.13 BIBREF8
conduct experiments using artificially constructed unnormalized text by corrupting words in the normal dev set
We propose a listening comprehension model for the task defined above, the Attention-based Multi-hop Recurrent Neural Network (AMRNN) framework, and show that this model is able to perform reasonably well for the task. In the proposed approach, the audio of the stories is first transcribed into text by ASR, and the proposed model is developed to process the transcriptions for selecting the correct answer out of 4 choices given the question.
listening comprehension task
BBC and CNN
high-order representation of a relation, loss gradient of relation meta
NELL-One, Wiki-One
ontology-based knowledge tree heuristics-based n-grams model
To test our proposed category induction model, we consider all BabelNet categories with fewer than 50 known instances. This is motivated by the view that conceptual neighborhood is mostly useful in cases where the number of known instances is small. For each of these categories, we split the set of known instances into 90% for training and 10% for testing.
Once this classifier has been trained, we can then use it to predict conceptual neighborhood for categories for which only few instances are known.
Exp. 5.1
comparing to the results from reducing the number of layers in the decoder, the BLEU score was 69.93 which is less than 1% in case of test2016 and in case of test2017 it was less by 0.2 %. In terms of TER it had higher score by 0.7 in case of test2016 and 0.1 in case of test2017.
pal-EtAl:2018:WMT proposed an APE model that uses three self-attention-based encoders tebbifakhr-EtAl:2018:WMT, the NMT-subtask winner of WMT 2018 ($wmt18^{nmt}_{best}$), employ sequence-level loss functions in order to avoid exposure bias during training and to be consistent with the automatic evaluation metrics. shin-lee:2018:WMT propose that each encoder has its own self-attention and feed-forward layer to process each input separately. The APE PBSMT-subtask winner of WMT 2018 ($wmt18^{smt}_{best}$) BIBREF11 also presented another transformer-based multi-source APE which uses two encoders and stacks an additional cross-attention component for $src \rightarrow pe$ above the previous cross-attention for $mt \rightarrow pe$.
Multilingual Neural Machine Translation Models
we obtain 52,053 dialogues and 460,358 utterances
30M utterances
We collected Japanese fictional stories from the Web
the dropout technique of Gal & Ghahramani gal
At the moment defender can draw on methods from image area to text for improving the robustness of DNNs, e.g. adversarial training BIBREF107 , adding extra layer BIBREF113 , optimizing cross-entropy function BIBREF114 , BIBREF115 or weakening the transferability of adversarial examples.
Wav2vec BIBREF22 a fully-supervised system using all labeled data
words length distribution, the frequency of use of the different forms and a measure for the combinatoriality
Automatic transcription of 5000 tokens through sequential neural models trained on the annotated part of the corpus
For example, in QuasarT, it improves 16.8% in EM score and 20.4% in F1 score. For example, in QuasarT, it improves 4.6% in EM score and 3.5% in F1 score.
The framework jointly learns parametrized QA and QG models subject to the constraint in equation 2. In more detail, they minimize QA and QG loss functions, with a third dual loss for regularization.
represent each word with an expressive multimodal distribution, for multiple distinct meanings, entailment, heavy tailed uncertainty, and enhanced interpretability. For example, one mode of the word `bank' could overlap with distributions for words such as `finance' and `money', and another mode could overlap with the distributions for `river' and `creek'.
holistic TraAtt RegAtt ConAtt ConAtt iBOWIMG VQA VQA WTL NMN SAN AMA FDA D-NMN DMN+
Queensland flood which provided 96% accuracy Alberta flood with the same configuration of train-test split which provided 95% accuracy
disaster data from BIBREF5 Queensland flood in Queensland, Australia and Alberta flood in Alberta, Canada
CNN model can be trained in a purely online setting. We first initialize the model parameters $\theta _0$ (line 1), which can be a trained model from other disaster events or it can be initialized randomly to start from scratch.

As a new batch of labeled tweets $B_t= \lbrace \mathbf {s}_1 \ldots \mathbf {s}_n \rbrace $ arrives, we first compute the log-loss (cross entropy) in Equation 11 for $B_t$ with respect to the current parameters $\theta _t$ (line 2a). Then, we use backpropagation to compute the gradients $f^{\prime }(\theta _{t})$ of the loss with respect to the current parameters (line 2b). Finally, we update the parameters with the learning rate $\eta _t$ and the mean of the gradients (line 2c). We take the mean of the gradients to deal with minibatches of different sizes. Notice that we take only the current minibatch into account to get an updated model.
Hindi-English
we extract sentences from Wikipedia in languages for which public multilingual is pretrained. For each sentence, we use the open-source BERT wordpiece tokenizer BIBREF4 , BIBREF1 and compute cross-entropy loss for each wordpiece: INLINEFORM0
the Meta-LSTM BIBREF0
beyond localized features and have access to the entire sequence
improves the DAR accuracy over Bi-LSTM-CRF by 2.1% and 0.8% on SwDA and MRDA respectively
word2vec 200 as the dimension of the obtained word vectors
Turkish news-web corpus TS TweetS by Sezer-2013 and 20M Turkish Tweets by Bolat and Amasyalı
For each source article, Retrieve aims to return a few candidate templates from the training corpus. Then, the Fast Rerank module quickly identifies a best template from the candidates.
efficiency task aimed  at reducing the number of parameters while minimizing drop in performance
ncorporating NNGLM and NNJM both independently and jointly into baseline system
CoNLL 2014
Support Vector Machine (SVM) Logistic Regression (LR) Random Forest (RF)
It is observed some variability - but not significant. Bert does not seem to gain much more syntax information than with type level information.
GENIA corpus
3000 hard samples are selected from the test set
the best permutation is decided by $\mathcal {J}_{\text{SEQ}}(\mathbf {L}_{un}^{(s^{\prime })},\mathbf {L}_{un}^{(r)})$ , which is the sequence discriminative criterion of taking the $s^{\prime }$ -th permutation in $n$ -th output inference stream at utterance $u$
we sort the speech segments by length we take segments in pairs, zero-padding the shorter segment so both have the same length These pairs are then mixed together
Our lexical features include 1-, 2-, and 3-grams in both word and character levels. number of characters and the number of words POS tags 300-dimensional pre-trained word embeddings from GloVe latent semantic indexing tweet representation by applying the Brown clustering algorithm positive words (e.g., love), negative words (e.g., awful), positive emoji icon and negative emoji icon boolean features that check whether or not a negation word is in a tweet
selects the next summary sentence based not only on properties of the source text, but also on the previously selected sentences in the summary
There is no reason to think that this approach wouldn't also be successful for other technical domains. Technical terms are replaced with tokens, therefore so as long as there is a corresponding process for identifying and replacing technical terms in the new domain this approach could be viable.
44,000 tweets
The same 2K set from Gigaword used in BIBREF7
The Online Retail Data Set consists of a clean list of 25873 invoices, totaling 541909 rows and 8 columns.
BERT RoBERTa DistilBERT GPT GPT2 Transformer-XL XLNet XLM
they use ROC curves and cross-validation
recurrent neural network (RNN)-based sequence-to-sequence (Seq2Seq) models for NATS
hundreds of thousands of legal agreements
They use self-play learning , optimize the model for specific metrics, train separate models per user, use model  and response classification predictors, and filter the dataset to obtain higher quality training data.
Training datasets: TTS System dataset and embedding selection dataset. Evaluation datasets: Common Prosody Errors dataset and LFR dataset.
The mixed objective improves EM by 2.5% and F1 by 2.2%
5 domains: software, stuff, african wildlife, healthcare, datatypes
using a mixture of manual and semi-automatic techniques
Improve existing NLP methods. Improve linguistic analysis. Measure impact of word normalization tools.
Two neural networks: an extractor based on an encoder (BERT) and a decoder (LSTM Pointer Network BIBREF22) and an abstractor identical to the one proposed in BIBREF8.
AE-HCN outperforms by 17%, AE-HCN-CNN outperforms by 20% on average
Similar to standard convolutional networks but instead they skip some input values effectively operating on a broader scale.
mean rank (MR), mean reciprocal rank (MRR), as well as Hits@1, Hits@3, and Hits@10
Kaggle
Subversive Kaggle
Wikipedia
Subversive Wikipedia
Reddit
Subversive Reddit