DinoStackAI/qasper-rag huggingface dataset:
Multitask learning is used for the task of predicting relevance of a comment on a different question to a given question, where the supplemental tasks are predicting relevance between the questions, and between the comment and the corresponding question
Proposed model has 1.16 million parameters and 11.04 MB.
overall architecture of the proposed Attentional Encoder Network (AEN), which mainly consists of an embedding layer, an attentional encoder layer, a target-specific attention layer, and an output layer.
datasets given on the shared task, without using any additional external data
second on Subtask A with an F1 score of 77.78% among 33 other team submissions performs well on Subtask B with an F1 score of 79.59%
Depeche + SVM
TF-IDF + SVM Depeche + SVM NRC + SVM TF-NRC + SVM Doc2Vec + SVM Hierarchical RNN BiRNN + Self-Attention ELMo + BiRNN Fine-tuned BERT
9710 passages, with an average of 6.24 sentences per passage, 16.16 words per sentence, and an average length of 86 words
our method enables to control the morphological realization of first and second-person pronouns, together with verbs and adjectives related to them
Speaker's Gender Effects Interlocutors' Gender and Number Effects
correct information substantially improves it - we see an increase of up to 2.3 BLEU over the baseline Finally, the “She said” prefixes substantially increase the number of feminine-marked verbs, bringing the proportion much closer to that of the reference
Google's machine translation system (GMT)
supply an NMT system with knowledge regarding the speaker and interlocutor of first-person sentences
FBanks with cepstral mean normalization (CMN) variance with mean normalization (CMVN)
40 mel-scaled log filterbank enegries (FBanks) computed every 10 ms with 25 ms window deltas and delta-deltas (120 features in vector) spectrogram
6-layer bLSTM with 1024 hidden units
Only MTMSM specifically tried to tackle the multi-span questions. Their approach consisted of two parts: first train a dedicated categorical variable to predict the number of spans to extract and the second was to generalize the single-span head method of extracting a span
To model an answer which is a collection of spans, the multi-span head uses the $\mathtt {BIO}$ tagging format BIBREF8: $\mathtt {B}$ is used to mark the beginning of a span, $\mathtt {I}$ is used to mark the inside of a span and $\mathtt {O}$ is used to mark tokens not included in a span
The proposed model achieves  EM 77,63 and F1 80,73  on the test and EM  76,95 and  F1 80,25 on the dev
MTMSN BIBREF4
DISPLAYFORM0
The Lemming model in BIBREF17
ROMULUS dataset NLU-Benchmark dataset
We also report the metrics in BIBREF7 for consistency we report the span F1 Exact Match (EM) accuracy of the entire sequence of labels metric that combines intent and entities
They evaluate F1 score and agent's test performance on their own built interactive datasets (iSQuAD and iNewsQA)
Thus, our task requires models to `feed themselves' rather than spoon-feeding them with information. This casts MRC as a sequential decision-making problem amenable to reinforcement learning (RL).
previous next Ctrl+F $<$query$>$ stop
Feature Concatenation Model (FCM) Spatial Concatenation Model (SCM) Textual Kernels Model (TKM)
$150,000$ tweets
Unimodal LSTM vs Best Multimodal (FCM)
- F score: 0.703 vs 0.704
- AUC: 0.732 vs 0.734 
- Mean Accuracy: 68.3 vs 68.4
Noisy data Complexity and diversity of multimodal relations Small set of multimodal examples
F-score Area Under the ROC Curve (AUC) mean accuracy (ACC) Precision vs Recall plot ROC curve (which plots the True Positive Rate vs the False Positive Rate)
single layer LSTM with a 150-dimensional hidden state for hate / not hate classification
No attacks to any community racist sexist homophobic religion based attacks attacks to other communities
accuracy normalized mutual information
K-means, Skip-thought Vectors, Recursive Neural Network and Paragraph Vector based clustering methods
SearchSnippets StackOverflow Biomedical
tried many configurations of our network models, but report results with only three configurations Transformer Type 1 Transformer Type 2 Transformer Type 3
BLEU-2 average accuracies over 3 test trials on different randomly sampled test sets
AI2 BIBREF2 CC BIBREF19 IL BIBREF4 MAWPS BIBREF20
AUC-ROC
jokes of different categories (sci-fi, sports, etc) and types (puns, limerick, etc)
five-minute reuse and one-day return
finite sequence of terms
14 TDs BIBREF15
SRCC
unshuffled version of the French OSCAR corpus
its performance still lags behind models trained on the original English training set in the TRANSLATE-TEST setting, 81.2 vs. 82.91 for RoBERTa
POS and DP task: CONLL 2018
NER task: (no extensive work) Strong baselines CRF and BiLSTM-CRF
NLI task: mBERT or XLM (not clear from text)
2.36 point increase in the F1 score with respect to the best SEM architecture on the TRANSLATE-TRAIN setting (81.2 vs. 80.2 for XLM) lags behind models trained on the original English training set in the TRANSLATE-TEST setting, 81.2 vs. 82.91 for RoBERTa For POS tagging, we observe error reductions of respectively 0.71% for GSD, 0.81% for Sequoia, 0.7% for Spoken and 0.28% for ParTUT For parsing, we observe error reductions in LAS of 2.96% for GSD, 3.33% for Sequoia, 1.70% for Spoken and 1.65% for ParTUT
Randomwalk Walktrap Louvain clustering
political events such as elections, corruption cases or justice decisions
BIBREF32, BIBREF23, BIBREF33 discussions in four different languages: English, Portuguese, Spanish and French, occurring in five regions over the world: South and North America, Western Europe, Central and Southern Asia.
four different languages: English, Portuguese, Spanish and French
deep convolutional networks BIBREF53 , BIBREF54
Tweets noisy nature, use of creative spelling and punctuation, misspellings, slang, new words, URLs, and genre-specific terminology and abbreviations, short (length limited) text
27.41 transformation on average of single seed sentence is available in dataset.
For each source sentence, transformation sentences that are transformed according to some criteria (paraphrase, minimal change etc.)
Yes, as new sentences.
- paraphrase 1
- paraphrase 2
- different meaning
- opposite meaning
- nonsense
- minimal change
- generalization
- gossip
- formal sentence
- non-standard sentence
- simple sentence
- possibility
- ban
- future
- past
we were looking for original and uncommon sentence change suggestions
we achieve better results than GCN+PADG but without any use of domain-specific hand-crafted features RegSum achieves a similar ROUGE-2 score
We compare our model performance with the approaches of BIBREF0 BIBREF5 on STS Corpus. BIBREF0 reported the results of Maximum Entropy (MaxEnt), NB, SVM on STS Corpus having good performance in previous time. The model of BIBREF5 is a state-of-the-art so far by using a CharSCNN. we compare results with the model of BIBREF14 that used a ensemble of multiple base classifiers (ENS) such as NB, Random Forest (RF), SVM and Logistic Regression (LR). The ENS model is combined with bag-of-words (BoW), feature hashing (FH) and lexicons. The model of BIBREF14 is a state-of-the-art on Sanders and HCR datasets.
Stanford - Twitter Sentiment Corpus (STS Corpus) Sanders - Twitter Sentiment Corpus Health Care Reform (HCR)
rules that compute polarity of words after POS tagging or parsing steps
link prediction triplet classification
movie sentence polarity dataset from BIBREF19 laptop and restaurant datasets collected from SemEval-201 we collected 2,000 reviews for each domain from the same review source
Experiment 1: ACC around 0.5 with 50% noise rate in worst case - clearly higher than baselines for all noise rates
Experiment 2: ACC on real noisy datasets: 0.7 on Movie, 0.79 on Laptop, 0.86 on Restaurant (clearly higher than baselines in almost all cases)
Noun WordNet Semantic Text Exchange Model (NWN-STEM) General WordNet Semantic Text Exchange Model (GWN-STEM) Word2Vec Semantic Text Exchange Model (W2V-STEM)
entertainment, interview, singing, play, movie, vlog, live broadcast, speech, drama, recitation and advertisement
SST (Stanford Sentiment Treebank) Subj (Subjectivity dataset) MPQA Opinion Corpus RT is another movie review sentiment dataset TREC is a dataset for classification of the six question types
Considering "What" and "How" separately versus jointly optimizing for both.
Textual inputs, knowledge bases, and images.
cross-lingual NE recognition
Naive Bayes SVM Maximum Entropy classifiers
We define the Effective Word Score of score x as

EFWS(x) = N(+x) - N(-x),

where N(x) is the number of words in the tweet with polarity score x.
First, the GRU only allows sentences to have context from sentences before them, but not after them. This prevents information propagation from future sentences. Second, the supporting sentences may be too far away from each other on a word level to allow for these distant sentences to interact through the word level GRU.
the facts that are relevant for answering a particular question) are labeled during training.
For the DMN+, we propose replacing this single GRU with two different components. The first component is a sentence reader The second component is the input fusion layer
the new DMN+ model does not require that supporting facts (i.e. the facts that are relevant for answering a particular question) are labeled during training. In addition, we introduce a new input module to represent images.
the input fusion layer to allow interactions between input facts and a novel attention based GRU that allows for logical reasoning over ordered inputs.
classification accuracy BLEU scores model perplexities of the reconstruction
words found in the control word lists are then removed The remaining words, which represent the content
style of a sentence is represented as a vector of counts of closed word classes (like personal pronouns) as well as counts of syntactic features like the number of SBAR non-terminals in its constituency parse, since clause structure has been shown to be indicative of style
either likely depressed-user population is younger, or depressed youngsters are more likely to disclose their age more women than men were given a diagnosis of depression
Demographic information is predicted using weighted lexicon of terms.
facial presence Facial Expression General Image Features textual content analytical thinking clout authenticity emotional tone Sixltr informal language markers 1st person singular pronouns
The data are self-reported by Twitter users and then verified by two human experts.
From Twitter profile descriptions of the users.
Sociability from ego-network on Twitter
Users' tweets
Profile pictures from the Twitter users' profiles.
Sememes are minimum semantic units of word meanings, and the meaning of each word sense is typically composed of several sememes
the Dutch section of the OSCAR corpus
BERTje BIBREF8 an ULMFiT model (Universal Language Model Fine-tuning for Text Classification model) BIBREF19. mBERT
sentiment analysis the disambiguation of demonstrative pronouns,
Average reward across 5 seeds show that NLP representations are robust to changes in the environment as well task-nuisances
a basic scenario, a health gathering scenario, a scenario in which the agent must take cover from fireballs, a scenario in which the agent must defend itself from charging enemies, and a super scenario, where a mixture of the above scenarios
represent the state using natural language
Moreover, our TL-TranSum method also outperforms other approaches such as MaxCover ( $5\%$ ) and MRMR ( $7\%$ )
Google N-grams
COHA
Moral Foundations Dictionary (MFD)
A Centroid model summarizes each set of seed words by its expected vector in embedding space, and classifies concepts into the class of closest expected embedding in Euclidean distance following a softmax rule; A Naïve Bayes model considers both mean and variance, under the assumption of independence among embedding dimensions, by fitting a normal distribution with mean vector and diagonal covariance matrix to the set of seed words of each class;
By complementing morally relevant seed words with a set of morally irrelevant seed words based on the notion of valence
Care / Harm, Fairness / Cheating, Loyalty / Betrayal, Authority / Subversion, and Sanctity / Degradation
the neural approach is generally preferred by a greater percentage of participants than the rules or random human-made game outperforms them all
neural question-answering technique to extract relations from a story text OpenIE5, a commonly used rule-based information extraction technique
if a vernacular paragraph contains more poetic images used in classical literature, its generated poem usually achieves higher score poems generated from descriptive paragraphs achieve higher scores than from logical or philosophical paragraphs
Perplexity of the best model is 65.58 compared to best baseline 105.79.
Bleu of the best model is 6.57 compared to best baseline 5.50.
We collected a corpus of poems and a corpus of vernacular literature from online resources
BLEU Micro Entity F1 quality of the responses according to correctness, fluency, and humanlikeness on a scale from 1 to 5
Attn seq2seq Ptr-UNK KV Net Mem2Seq DSR
Camrest InCar Assistant
Answer with content missing: (list missing) 
Scheduled sampling: In our experiments, we found that models trained with scheduled sampling performed better (about 0.004 BLEU-4 on validation set) than the ones trained using teacher-forcing for the AVSD dataset. Hence, we use scheduled sampling for all the results we report in this paper.

Yes.
multi-class Naive Bayes
BiMPM ESIM Decomposable Attention Model KIM BERT
Upward reasoning is defined as going from one specific concept to a more general one. Downward reasoning is defined as the opposite, going from a general concept to one that is more specific.
a type of reasoning based on word replacement, requires the ability to capture the interaction between lexical and syntactic structures
Quotation (⌃q) dialogue acts, on the other hand, are mostly used with `Anger' and `Frustration' Action Directive (ad) dialogue act utterances, which are usually orders, frequently occur with `Anger' or `Frustration' although many with `Happy' emotion in case of the MELD dataset Acknowledgements (b) are mostly with positive or neutral Appreciation (ba) and Rhetorical (bh) backchannels often occur with a greater number in `Surprise', `Joy' and/or with `Excited' (in case of IEMOCAP) Questions (qh, qw, qy and qy⌃d) are mostly asked with emotions `Surprise', `Excited', `Frustration' or `Disgust' (in case of MELD) and many are neutral No-answers (nn) are mostly `Sad' or `Frustrated' as compared to yes-answers (ny). Forward-functions such as Apology (fa) are mostly with `Sadness' whereas Thanking (ft) and Conventional-closing or -opening (fc or fp) are usually with `Joy' or `Excited'
First preference is given to the labels that are perfectly matching in all the neural annotators. In case two out of three context models are correct, then it is being checked if that label is also produced by at least one of the non-context models. When we see that none of the context models is producing the same results, then we rank the labels with their respective confidence values produced as a probability distribution using the $softmax$ function. The labels are sorted in descending order according to confidence values. Then we check if the first three (case when one context model and both non-context models produce the same label) or at least two labels are matching, then we allow to pick that one. Finally, when none the above conditions are fulfilled, we leave out the label with an unknown category.
Dialogue Act Markup in Several Layers (DAMSL) tag set
Use an existing one
apply an ensemble of deep learning and linguistics t
MUC, CoNLL, ACE, OntoNotes, MSM, Ritter, UMBC
1000 hours of WSJ audio data
wav2vec has 12 convolutional layers
shared character embeddings for taggers in both languages together through optimization of a joint loss function
We use the morphological tagging datasets provided by the Universal Dependencies (UD) treebanks (the concatenation of the $4^\text{th}$ and $6^\text{th}$ columns of the file format) BIBREF13 .
In-house dataset consists of  3716 documents 
ACE05 dataset consists of  1635 documents
English, German, Spanish, Italian, Japanese and Portuguese English, Arabic and Chinese
in-house dataset ACE05 dataset
16k images and 740k corresponding region descriptions
we replace user embeddings with a low-dimensional image representation
increasing accuracy by 35% and 25% in comparison to the Baseline and Ablation models, respectively over INLINEFORM0 increase in EM and GM between our model and the next best two models
the baseline where path generation uses a standard sequence-to-sequence model augmented with attention mechanism and path verification uses depth-first search
exact match, f1 score, edit distance and goal match
using Amazon Mechanical Turk using simulated environments with topological maps
distinguishing between clinically positive and negative phenomena within each risk factor domain and accounting for structured data collected on the target cohort
a corpus of discharge summaries, admission notes, individual encounter notes, and other clinical notes from 220 patients in the OnTrackTM program at McLean Hospital an additional data set for training our vector space model, comprised of EHR texts queried from the Research Patient Data Registry (RPDR)
A BPE model is applied to the stem after morpheme segmentation.
morpheme segmentation BIBREF4 and Byte Pair Encoding (BPE) BIBREF5 Zemberek BIBREF12
Comparison of test accuracies of neural network models on an inflection task and qualitative analysis of the errors
English, Spanish and Zulu
The experiment settings from this paper and evaluation protocol for the Mboshi corpus (Boundary F-scores using the ZRC speech reference) are the same from BIBREF8. For the multilingual selection experiments, we experimented combining the languages from top to bottom as they appear Table (ranked by performance; e.g. 1-3 means the combination of FR(1), EN(2) and PT(3)). Lastly, following the methodology from BIBREF8, we extract the most confident alignments (in terms of ANE) discovered by the bilingual models.
Neural Machine Translation (NMT) models are trained between language pairs, using as source language the translation (word-level) and as target
Results show that similar languages rank better in terms of segmentation performance, and that by combining the information learned by different models, segmentation is further improved.
4-layer encoder, 4-layer decoder, residual-connected model, with embedding and hidden size set as 256
German-English Turkish-English English-German
IWSLT14 German-English, IWSLT14 Turkish-English, WMT14 English-German
draw our data from news publications, wine reviews, and Reddit develop new metrics for the agreement of binomial orderings across communities and the movement of binomial orderings over time develop a null model to determine how much variation in binomial orderings we might expect across communities and across time
news publications, wine reviews, and Reddit
Trinomials are likely to appear in exactly one order
null model
close to random,
word length, number of phonemes, number of syllables, alphabetical order, and frequency
We experiment with three different pretrained representations: ELMo BIBREF5 , BERT BIBREF6 , and GloVe BIBREF18 . To produce a single city embedding, we compute the TF-IDF weighted element-wise mean of the token-level representations. For all pretrained methods, we additionally reduce the dimensionality of the city embeddings to 40 using PCA for increased compatibility with our clustering algorithm.
Using crowdsourcing
We devise a simple clustering algorithm to approximate this process. First, we initialize with random cluster assignments and define “cluster strength” to be the relative difference between “intra-group” Euclidean distance and “inter-group” Euclidean distance. Then, we iteratively propose random exchanges of memberships, only accepting these proposals when the cluster strength increases, until convergence.
joint goal accuracy
the single domain dataset, WoZ2.0 the multi-domain dataset, MultiWoZ
documents from the CommonCrawl dataset that has the most overlapping n-grams with the question
Because, unlike other languages, English does not mark grammatical genders
by calculating log ratio of grammatical phrase over ungrammatical phrase
Markov random field with an optional neural parameterization
(about 4 million sentences, 138 million word tokens) one trained on the Billion Word benchmark
Recurrent Neural Network (RNN) ActionLSTM Generative Recurrent Neural Network Grammars (RNNG)
the evaluation metrics include BLEU and ROUGE (1, 2, L) scores
The E2E NLG challenge dataset BIBREF21
better sentence pair representations
English wikipedia dataset has more than 18 million a dump of 15 million English news articles
the BIBREF5 sieve system from the rule-based paradigm (referred to as RULE), BIBREF6 from the statistical paradigm (STAT), and the BIBREF11 deep reinforcement system from the neural paradigm (NEURAL).
Compared with the previous SOTA without BERT on SParC, our model improves Ques.Match and Int.Match by $10.6$ and $5.4$ points, respectively.
SParC BIBREF2 and CoSQL BIBREF6
Concat
Turn
Gate
Action Copy
Tree Copy
SQL Attn
Concat + Action Copy
Concat + Tree Copy
Concat + SQL Attn
Turn + Action Copy
Turn + Tree Copy
Turn + SQL Attn
Turn + SQL Attn + Action Copy
AGDT improves the performance by 2.4% and 1.6% in the “DS” part of the two dataset Our AGDT surpasses GCAE by a very large margin (+11.4% and +4.9% respectively) on both datasets In the “HDS” part, the AGDT model obtains +3.6% higher accuracy than GCAE on the restaurant domain and +4.2% higher accuracy on the laptop domain
There were hierarchical and non-hierarchical baselines; BERT was one of those baselines
agreement rates Kappa value
agreement of 0.85 and Kappa value of 0.83
removing AltLexC and adding Progression into our sense hierarchy
F1 score of best authors' model is 55.98 compared to BiLSTM and FastText that have F1 score slighlty higher than 46.61.
SVM with RBF kernel
While evaluating the impact of a claim, users have access to the full argument context and therefore, they can assess how impactful a claim is in the given context of an argument.
5 possible impact labels for a particular claim: no impact, low impact, medium impact, high impact and very high impact
4,261  days for France and 4,748 for the UK
Winter and summer words formed two separate clusters. Week day and week-end day words also formed separate clusters.
Relative error is less than 5%
F1 score of 66.66%
F1 score of 82.10%
For sentiment analysis UWB, INF-UFRGS-OPINION-MINING, LitisMind, pkudblab and SVM + n-grams + sentiment and for emotion analysis MaxEnt, SVM, LSTM, BiLSTM and CNN
The proposed system consists of a Bi-directional Long Short-Term Memory (BiLSTM) BIBREF16, a two-level attention mechanism BIBREF29, BIBREF30 and a shared representation for emotion and sentiment analysis tasks. Each of the shared representations is then fed to the primary attention mechanism
SemEval 2016 Task 6 BIBREF7 Stance Sentiment Emotion Corpus (SSEC) BIBREF15
BIBREF7 BIBREF39 BIBREF37 LitisMind Maximum entropy, SVM, LSTM, Bi-LSTM, and CNN
quality of the classifier predictions is too low to be integrated into the network analysis right away, the classifier drastically facilitates the annotation process for human annotators compared to annotating unfiltered tweets
applying reasoning BIBREF36 or irony detection methods BIBREF37
correct class can be directly inferred from the text content easily, even without background knowledge correct class can be inferred from the text content, given that event-specific knowledge is provided orrect class can be inferred from the text content if the text is interpreted correctly
convolutional neural network (CNN) BIBREF29
MH17 Twitter dataset
widely used method for classifying misleading content is to use distant annotations, for example to classify a tweet based on the domain of a URL that is shared by the tweet, or a hashtag that is contained in the tweet Natural Language Processing (NLP) models can be used to automatically label text content