DinoStackAI/qasper-rag huggingface dataset:
By choosing English (En) as the pivot language, we perform pivot alignments for identical English segments on Europarl Fr-En and En-De parallel corpora BIBREF18 , constructing a multi-parallel corpus of Fr-En-De. Then each of the Fr*-De and Fr-De* pseudo parallel corpora is established from the multi-parallel data by applying the pivot language-based translation described in the previous section.
They exclude slot-specific parameters and incorporate better feature representation of user utterance and dialogue states using syntactic information and convolutional neural networks (CNN).
convolutional neural networks (CNN)
By the number of parameters.
LSTM-based RNN-T
970 hours of audio data with corresponding text transcripts (around 10M word tokens) and an additional 800M word token text only dataset
standard benchmarks BIBREF36 , BIBREF37 to use our learned representations as features for a low complexity classifier (typically linear) on a novel supervised task/domain unseen during training without updating the parameters transfer learning evaluation in an artificially constructed low-resource setting
Answer with content missing: (Skip-thought vectors-Natural Language Inference paragraphs) The encoder for the current sentence and the decoders for the previous (STP) and next sentence (STN) are typically parameterized as separate RNNs
- RNN
multi-lingual NMT natural language inference constituency parsing skip-thought vectors
German English Chinese
IWSLT16 WMT15 NIST
By using  keywords `breast' AND `cancer' in tweet collecting process.
ML  logistic regression classifier combined with a Convolutional Neural Network (CNN) to identify self-reported diagnostic tweets.
NLP methods:  tweet conversion to numeric word vector,  removing tweets containing hyperlinks, removing "retweets", removing all tweets containing horoscope indicators,  lowercasing and  removing punctuation.
TB-Dense MATRES
PPL: SVT
Diversity: GVT
Embeddings Similarity: SVT
Human Evaluation: SVT
attention-based sequence-to-sequence model CVAE
MojiTalk PersonaChat Empathetic-Dialogues
Table TABREF44 Table TABREF44 Table TABREF47 Table TABREF47
large raw Czech corpora available from the LINDAT/CLARIN repository Czech Wikipedia
a total of 2,560 pseudo-tweets in three different languages: Japanese (ja), English (en) and Chinese (zh)
translations that were reasonable but not consistent with the labels
Attention-based translation model with convolution sequence to sequence model
German Spanish Chinese
Bidirectional LSTM based NER model of Flair
Frequency masking Time masking Additive noise Speed and volume perturbation
Little overlap except common basic Latin alphabet and that Hindi and Marathi languages use same script.
macro F1 score of 0.62
Hindi, English and German (German task won)
IBM-UB-1 dataset BIBREF25 IAM-OnDB dataset BIBREF42 The ICDAR-2013 Competition for Online Handwriting Chinese Character Recognition BIBREF45 ICFHR2018 Competition on Vietnamese Online Handwritten Text Recognition using VNOnDB BIBREF50
Moral Choice Machine computes the cosine similarity in a sentence embedding space of an arbitrary action embedded in question/answer pairs
Answer with content missing: (formula 1) bias(q, a, b) = cos(a, q) − cos(b, q)
Bias is calculated as substraction of cosine similarities of question and some answer for two opposite answers.
These ask which choices are morally required, forbidden, or permitted norms are understood as universal rules of what to do and what not to do
The multi-curricula learning scheme is scheduled according to the model's performance on the validation set, where the scheduling mechanism acts as the policy $\pi $ interacting with the dialogue model to acquire the learning status $s$. The reward of the multi-curricula learning mechanism $m_t$ indicates how well the current dialogue model performs.
coherence, logical consistency, fluency and diversity
BLEU embedding-based metrics (Average, Extrema, Greedy and Coherence) , entropy-based metrics (Ent-{1,2}) distinct metrics (Dist-{1,2,3} and Intra-{1,2,3})
SEQ2SEQ CVAE Transformer HRED DialogWAE
Model Confidence Continuity Query-relatedness Repetitiveness Specificity
PersonaChat BIBREF12 DailyDialog BIBREF13 OpenSubtitles BIBREF7
Wikipedia toxic comments
word error rate
163,110,000 utterances
from YouTube videos, with associated transcripts obtained from semi-supervised caption filtering from a Voice Search service
Single-Turn Multi-Turn
F1-score BLEU score
PDP-60 WSC-273
Their model does not differ from BERT.
interdependence between rate and distortion impact of KL on the sharpness of the approximated posteriors demonstrate how certain generative behaviours could be imposed on VAEs via a range of maximum channel capacities some experiments to find if any form of syntactic information is encoded in the latent space
by setting a non-zero positive constraint ($C\ge 0$) on the KL term ($|D_{KL}\big (q_\phi ({z}|{x}) || p({z})\big )-C|$)
Answer with content missing: (Formula 2) Formula 2 is an answer: 
\big \langle\! \log p_\theta({x}|{z}) \big \rangle_{q_\phi({z}|{x})}  -  \beta |D_{KL}\big(q_\phi({z}|{x}) || p({z})\big)-C|
tweetLID workshop shared task
We use Junto (mad) BIBREF5 to propagate labels from labelled to unlabelled nodes.
the graph, composed of three types of nodes: tweets (T), users (U) and the “world” (W). Edges are created between nodes and weighted as follows: T-T the unigram cosine similarity between tweets, T-U weighted 100 between a tweet and its author, U-U weighted 1 between two users in a “follows” relationship and U-W weighted 0.001 to ensure a connected graph for the mad algorithm.
five subtasks which involve standard classification, ordinal classification and distributional estimation. For a more detailed description see BIBREF0
10 CNNs and 10 LSTMs
There is no baseline.
precision recall Hamming loss micro averaged precision and recall
the public financial news dataset released by BIBREF4
It is an order of magnitude more efficient in terms of training time. his model requires pre-training and mutual-learning and requires days of training time, whereas the simple architecture we propose requires on the order of an hour
MC-CNN
MVCNN
CNN
In terms of Subj the Average MGNC-CNN is better than the average score of baselines by 0.5.  Similarly, Scores of SST-1, SST-2, and TREC where MGNC-CNN has similar improvements. 
In case of Irony the difference is about 2.0.
SST-1 SST-2 Subj TREC Irony
standard CNN C-CNN MVCNN
pure summarization model NHG
Humor in headlines (TitleStylist vs Multitask baseline):
Relevance: +6.53% (5.87 vs 5.51)
Attraction: +3.72% (8.93 vs 8.61)
Fluency: 1,98% (9.29 vs 9.11)
annotators are asked how attractive the headlines are Likert scale from 1 to 10 (integer values)
human evaluation task about the style strength
fine-tuned the GPT-2 medium model BIBREF51 on our collected headlines and then used it to measure the perplexity (PPL) on the generated outputs
test-set perplexity, likelihood convergence and clustering measures visualizing the topic summaries, authors' topic distributions and by performing an automatic labeling task
The languages considered were English, Chinese, German, Russian, Arabic, Spanish, French
They experimented with sentiment analysis and natural language inference task
See Figure FIGREF3
2) Naïve Bayes with SVM (NBSVM) 3) Extreme Gradient Boosting (XGBoost) 4) FastText algorithm with Bidirectional LSTM (FastText-BiLSTM)
Accuracy on each dataset and the average accuracy on all datasets.
16 different datasets from several popular review corpora used in BIBREF20 CoNLL 2000 BIBREF22
Sentiment Classification Transferability of Shared Sentence Representation Introducing Sequence Labeling as Auxiliary Task
MultiUN BIBREF20 OpenSubtitles BIBREF21
our method outperforms the baseline in both relevance and fluency significantly.
youtube video transcripts on news covering different topics like technology, human rights, terrorism and politics
Convolutional Neural Network bidirectional Recurrent Neural Network model with attention mechanism
It takes into account the agreement between different systems
Answer with content missing: (Table 1) The performance of all the target models raises significantly, while that on the original
examples remain comparable (e.g. the overall accuracy of BERT on modified examples raises from 24.1% to 66.0% on Quora)
BERT on Quora drops from 94.6% to 24.1%
current model
Quora MRPC
the number of speakers of each gender category their speech duration
Answer with content missing: (Formula) Formula is the answer.
model points out plausible signals which were passed over by an annotator it also picks up on a recurring tendency in how-to guides in which the second person pronoun referring to the reader is often the benefactee of some action
influence of each word on the score of the correct relation, that impact should and does still correlate with human judgments
IR methods perform better than the best neural models
GPT2 SciBERT model of BIBREF11
8.1 million scientific documents 154K computer science articles 622K citing sentences
two surveys by two groups - school students and meteorologists to draw on a map a polygon representing a given geographical descriptor
Two different BERT models were developed
1 000 labeled dialogues for training and 240 unlabeled dialogues for evaluation
document categorization regression tasks
mean reciprocal rank
skip-gram LDA
Training embeddings from small-corpora can increase the performance of some tasks
Visualization of State of the union addresses
BIBREF35 for VQA dataset BIBREF5 BIBREF36
Decoder that generates question using an LSTM-based language model
The proposed Multimodal Differential Network (MDN) consists of a representation module and a joint mixture module.
Exemplars aim to provide appropriate context. joint image-caption embedding for the supporting exemplar are closer to that of the target image-caption
Northeast U.S, South U.S., West U.S. and Midwest U.S.
0.9098 correlation
Using Latent Dirichlet Allocation on TF-IDF transformed from the corpus
Northeast U.S., West U.S. and South U.S.
51,104
evidence extraction and answer synthesis
extraction-then-synthesis framework
there are several related passages for each question in the MS-MARCO dataset. MS-MARCO also annotates which passage is correct
an official F1-score of 0.2905 on the test set
a balanced dataset of 2,396 ironic and 2,396 non-ironic tweets is provided
a non-parameter optimized linear-kernel SVM that uses TF-IDF bag-of-word vectors as inputs
bert-large-wwm bert-base bert-large
LJSpeech
Replacing attention mechanism to query-key attention, and adding a loss to make the attention mask as diagonal as possible
Direct comparison of model parameters
Majority ESA Word2Vec Binary-BERT
how long it takes the system to lemmatize a set number of words
MADAMIRA BIBREF6 system
Lemmatization is done by an expert Arabic linguist where spelling corrections are marked, and lemmas are provided with full diacritization
Articles contain 18,300 words, and they are evenly distributed among these 7 genres with 10 articles per each
from Arabic WikiNews site https://ar.wikinews.org/wiki
train the parser on six other languages in the Google universal dependency treebanks version 2.0 (de, en, es, fr, it, pt, sv, excluding whichever is the target language), and we use gold coarse POS tags
E2E NLG challenge Dataset The Wikipedia Company Dataset
Seq2Seq HLSTM HLSTM+Copy HLSTM+Graph Attention HLSTM+Contextual Attention
cloze-style reading comprehension and user query reading comprehension questions
the parser first learns to parse simple sentences, then proceeds to learn more complex ones. The induction method is iterative, semi-automatic and based on frequent patterns
1.1 million sentences 119 different relation types (unique predicates)
2 accuracy points
continuous relaxation to top-k-argmax
To achieve this purpose, we introduce a trainable class weight $\mathbf {w}$ to reweigh source domain examples by class when performing DIRL, with $\mathbf {w}_i > 0$
Through the experiments, we empirically studied our analysis on DIRL and the effectiveness of our proposed solution in dealing with the problem it suffered from.
12 binary-class classification and multi-class classification of reviews based on rating
sentiment analysis, information extraction, document summarization, spoken dialogue, cross lingual (research), dialogue, systems, language generation
size, demographics, areas of research, impact, and correlation of citations with demographic attributes (age and gender)
44,896 articles
choosing the answer from the network that had the highest probability and choosing no answer if any of the networks predicted no answer
number of coattention blocks, the batch size, and the number of epochs trained and ensembled our three best networks
Simple Skip improves F1 from 74.34 to 74.81
Transformer Skip improes F1 from 74.34 to 74.95
represent every sentence by their reduced n-gram set
standard Transformer Base model
we define the similarity between INLINEFORM7 and INLINEFORM8 by, DISPLAYFORM0
on the unlabeled data of each task
The system benefits from filters of each size. features of multigranular phrases are extracted with variable-size convolution filters.
each embedding version is crucial for good performance
MVCNN, a novel CNN architecture for sentence classification. It combines multichannel initialization – diverse versions of pretrained word embeddings are used – and variable-size filters – features of multigranular phrases are extracted with variable-size convolution filters.
we manually label an in-house dataset of 1,100 users with gender tags we randomly sample 20,000 tweets for each class from an in-house dataset gold labeled with the same 15 classes as the shared task
Data released for APDA shared task contains 3 datasets.
translated the responses in multiple languages into English using machine translation words without functional meaning (e.g. `I'), rare words that occurred in only one narrative, numbers, and punctuation were all removed remaining words were stemmed to remove plural forms of nouns or conjugations of verbs
We collected the historical 52 week stock prices prior to this event and calculated the daily stock price change. The distribution of the daily price change of the previous 52 weeks is Figure FIGREF13 with a mean INLINEFORM1 and standard deviation INLINEFORM2 .
For each cluster, its overall sentiment score is quantified by the mean of the sentiment scores among all tweets
the punchline of the joke
It had the highest accuracy comparing to all datasets 0.986% and It had the highest improvement comparing to previous methods on the same dataset by 8%
a subset of the population where we can quantitatively measure reactions: the popular Reddit r/Jokes thread These Reddit posts consist of the body of the joke, the punchline, and the number of reactions or upvotes.
The distribution of joke scores varies wildly, ranging from 0 to 136,354 upvotes. We found that there is a major jump between the 0-200 upvote range and the 200 range and onwards, with only 6% of jokes scoring between 200-20,000. We used this natural divide as the cutoff to decide what qualified as a funny joke, giving us 13884 not-funny jokes and 2025 funny jokes.
These models are likely to be deficient in encoding morphological features is that they are word level models, and do not have direct access sub-word information like inflectional endings, which indicates that these features are difficult to learn effectively purely from lexical distributions.
the transformer models
CoLA contains example sentences from linguistics publications labeled by experts
labeled by experts
Human, FastQA, BiDAF, Coref-GRU, MHPGM, Weaver / Jenga, MHQA-GRN
Assign a value to the relation based on whether mentions occur in the same document, if mentions are identical, or if mentions are in the same coreference chain.
Exact matches to the entity string and predictions from a coreference resolution system
Support Vector Machine (SVM) Logistic regression (Log.Reg) Random Forest (RF) gradient boosting (XGB)
Support Vector Machine (SVM), Logistic regression (Log.Reg), Random Forest (RF) and gradient boosting (XGB)
Amazon reviews
Language Vision Acoustic
Long-short Term Hybrid Memory (LSTHM) is an extension of the Long-short Term Memory (LSTM)
adding context causes speakers to focus on broader semantic and pragmatic issues of discourse coherence
Preceding and following sentence of each metaphor and paraphrase are added as document context
Best performance achieved is 0.72 F1 score
contains thousands of XML files, each of which are constructed by several records
Answer with content missing: (Table II) Proposed model has F1 score of  0.7220 compared to 0.7148 best state-state-of-the-art result.
Answer with content missing: (Table II) Proposed model has F1 score of  0.7220.
Chowdhury BIBREF14 and Thomas et al. BIBREF11 FBK-irst BIBREF10 Liu et al. BIBREF9 Sahu et al. BIBREF12
Glove Twitter word2vec
CNN-C CNN-W CNN-Lex-C CNN-Lex-W Bi-LSTM-C Bi-LSTM-W Lex-rule BOW
Travel dataset contains 4100 raw samples, 11291 clauses, Hotel dataset contains 3825 raw samples, 11264 clauses, and the Mobile dataset contains 3483 raw samples and 8118 clauses
User reviews written in Chinese collected online for hotel, mobile phone, and travel domains
They use a two-stage labeling strategy where in the first stage single annotators label a large number of short texts with relatively pure sentiment orientations and in the second stage multiple annotators label few text samples with mixed sentiment orientations
1) How to introduce unsupervised pre-training into NLG tasks with cross-modal context? 2) How to design a generic pre-training algorithm to fit a wide range of NLG tasks? 3) How to reduce the computing resources required for large-scale pre-training? 4) What aspect of knowledge do the pre-trained models provide for better language generation?
The advantage of pre-training gradually diminishes with the increase of labeled data Fixed representations yield better results than fine-tuning in some cases pre-training the Seq2Seq encoder outperforms pre-training the decoder
fine-tuning schedules that elaborately design the control of learning rates for optimization proxy tasks that leverage labeled data to help the pre-trained model better fit the target data distribution knowledge distillation approaches that ditch the paradigm of initialization with pre-trained parameters by adopting the pre-trained model as a teacher network
task-specific architecture during pre-training (task-specific methods) aim at building a general pre-training architecture to fit all downstream tasks (task-agnostic methods)
The changes are evaluated based on accuracy of intent and entity recognition on SNIPS dataset
MNCut spectral clustering algorithm BIBREF58
Answer with content missing: (3 Experimental Setup) We experiment with six target languages: French (FR), Brazilian Portuguese (PT), Italian (IT), Polish (PL), Croatian (HR), and Finnish (FI).
Support Vector Machines (SVM) classifier
F1 accuracy
stylometric, lexical, grammatical, and semantic
TF.IDF-based features
The number of fake news that have a duplicate in the training dataset are 1018 whereas, the number of articles with genuine content that have a duplicate article in the training set is 322.
training dataset contains 2,815 examples 761 testing examples
the latest CLWE developments almost exclusively focus on fully unsupervised approaches BIBREF23 , BIBREF24 , BIBREF25 , BIBREF26 , BIBREF27 , BIBREF28 , BIBREF29 , BIBREF30 : they fully abandon any source of (even weak) supervision and extract the initial seed dictionary by exploiting topological similarities between pre-trained monolingual embedding spaces
we use mean average precision (MAP) as the main evaluation metric
Unsupervised CLWEs. These methods first induce a seed dictionary $D^{(1)}$ leveraging only two unaligned monolingual spaces (C1). While the algorithms for unsupervised seed dictionary induction differ, they all strongly rely on the assumption of similar topological structure between the two pretrained monolingual spaces. Once the seed dictionary is obtained, the two-step iterative self-learning procedure (C2) takes place: 1) a dictionary $D^{(k)}$ is first used to learn the joint space $\mathbf {Y}^{(k)} = \mathbf {X{W}}^{(k)}_x \cup \mathbf {Z{W}}^{(k)}_z$ ; 2) the nearest neighbours in $\mathbf {Y}^{(k)}$ then form the new dictionary $D^{(k+1)}$ . We illustrate the general structure in Figure 1 .
440 sentences, 2247 triples extracted from those sentences, and 11262 judgements on those triples.
all annotators that a triple extraction was incorrect
OpenIE4 and MiniIE
Crowd workers were asked to mark whether a triple was correct, namely, did the triple reflect the consequence of the sentence.
Meta vertex candidate identification. Edit distance and word lengths distance are used to determine whether two words should be merged into a meta vertex (only if length distance threshold is met, the more expensive edit distance is computed). The meta vertex creation. As common identifiers, we use the stemmed version of the original vertices and if there is more than one resulting stem, we select the vertex from the identified candidates that has the highest centrality value in the graph and its stemmed version is introduced as a novel vertex (meta vertex).
The given corpus is traversed, and for each element INLINEFORM6 , its successor INLINEFORM7 , together with a given element, forms a directed edge INLINEFORM8 . Finally, such edges are weighted according to the number of times they appear in a given corpus. Thus the graph, constructed after traversing a given corpus, consists of all local neighborhoods (order one), merged into a single joint structure. Global contextual information is potentially kept intact (via weights), even though it needs to be detected via network analysis
They pre-train forward and backward LMs separately, remove top layer softmax, and concatenate to obtain the bidirectional LMs.
micro-averaged F1
91.93% F1 score on CoNLL 2003 NER task and 96.37% F1 score on CoNLL 2000 Chunking task
Chiu and Nichols (2016), Lample et al. (2016), Ma and Hovy (2016), Yang et al. (2017), Hashimoto et al. (2016), Søgaard and Goldberg (2016)
CoNLL 2003 CoNLL 2000
Both supervised and unsupervised, depending on the task that needs to be solved.
For evaluating the interpretability, we use $Coherence@k$ (Equation 6 ) , automated and manual word intrusion tests.
GloVE; SGNS
It is a network used to encode speech lattices to maintain a rich hypothesis space.
simplified set of input data, in a variety of different formats that occur frequently in a healthcare setting
RDF was designed as an abstract information model and FHIR was designed for operational use in a healthcare setting RDF makes statements of fact, whereas FHIR makes records of events RDF is intended to have the property of monotonicity, meaning that previous facts cannot be invalidated by new facts
One of the several formats into which FHIR can be serialized is RDF there is the potential for a slight mismatch between the models
Health Level Seven Fast Healthcare Interoperability Resources (HL7 FHIR) Resource Description Framework (RDF)
Such a system would benefit educators by saving time to generate quizzes and tests.
WER can reflect our model's effectiveness in generating questions that are similar to those of SQuAD WER can be used for initial analyses
For sentence-level prediction they used tolerance accuracy, for segment retrieval accuracy and MRR and for the pipeline approach they used overall accuracy
tutorial videos for a photo-editing software
a sentence-level prediction algorithm, a segment retrieval algorithm and a pipeline segment retrieval algorithm
a tutorial website about an image editing program
significantly improves the accuracy and F1 score of aspect polarity classification
BERT-ADA BERT-PT, AEN-BERT, SDGCN-BERT
GANN obtained the state-of-the-art APC performance on the Chinese review datasets
Car, Phone, Notebook, Camera
Evaluation Dataset of Japanese Lexical Simplification kodaira