DinoStackAI/qasper-rag huggingface dataset:
Set/Change Destination Set/Change Route Go Faster Go Slower Stop Park Pull Over Drop Off Open Door Other
3347 unique utterances
A continuous emission HMM uses the hidden states of a 2-layer LSTM as features and a discrete emission HMM uses data as features. 
The interpretability of the model is shown in Figure 2.
The HMM can identify punctuation or pick up on vowels.
decision trees to predict individual hidden state dimensions apply k-means clustering to the LSTM state vectors, and color-code the training data with the clusters
BIBREF3 BIBREF4 BIBREF9
300 Dimensional Glove
our model requires 100k parameters , while BIBREF8 requires 250k parameters
Excluding the embedding weights, our model requires 100k parameters
Sexist/Racist (SR) data set HATE dataset HAR
Proposed model achieves 0.86, 0.924, 0.71 F1 score on SR, HATE, HAR datasets respectively.
KFTT BIBREF12 and BTEC BIBREF13
English-Japanese
278 more annotations
QA pairs per predicate
Inter-annotator agreement, comparison against expert annotation, agreement with PropBank Data annotations.
trained annotators BIBREF4 crowdsourcing BIBREF5
extensive personal feedback
a trained worker consolidates existing annotations
the annotation machinery of BIBREF5
1593 annotations
WMT 2014 En-Fr parallel corpus
late 2014
96-97.6% using the objects color or shape and 79% using shape alone
a simulated binning task in which the robot is tasked to place a cube into a bowl as outlined by the verbal command
supervised learning
These features are derived directly from the word and capture the general tendency of a word being echoed in explanations.
random method LSTM
Non-contextual properties of a word Word usage in an OP or PC (two groups) How a word connects an OP and PC. General OP/PC properties
all of our models outperform the random baseline by a wide margin he best F1 score in content words more than doubles that of the random baseline (0.286 vs. 0.116)
F1 score
Non-contextual properties of a word Word usage in an OP or PC (two groups) How a word connects an OP and PC General OP/PC properties
using the BLEU score as a quantitative metric and human evaluation for quality
5 questions per image
For the question generation model 15,000 images with 75,000 questions. For the chatbot model, around 460k utterances over 230k dialogues.
By considering words as vertices and generating directed edges between neighboring words within a sentence
Precision@1 Mean Average Precision Mean Reciprocal Rank
DBQA KBRE
synchronized acoustic and ultrasound data from 58 typically developing children, aged 5-12 years old (31 female, 27 male) data was aligned at the phone-level 121fps with a 135 field of view single ultrasound frame consists of 412 echo returns from each of the 63 scan lines (63x412 raw frames)
feedforward neural networks (DNNs) convolutional neural networks (CNNs)
feedforward neural networks convolutional neural networks
Perplexity of proposed MEED model is 19.795 vs 19.913 of next best result on test set.
we extract the emotion information from the utterances in $\mathbf {X}$ by leveraging an external text analysis program, and use an RNN to encode it into an emotion context vector $\mathbf {e}$, which is combined with $\mathbf {c}_t$ to produce the distribution
(1) grammatical correctness (2) contextual coherence (3) emotional appropriateness
sequence-to-sequence model (denoted as S2S) HRAN
PCFGLA-based parser, viz. Berkeley parser BIBREF5 minimal span-based neural parser BIBREF6
syntax-based system may generate correct syntactic analyses for partial grammatical fragments
the rank-correlation for MFH model increases by 36.4% when is evaluated in VQA-HAT dataset and 7.7% when is evaluated in VQA-X
rank-correlation BIBREF25
they are available in the Visual Genome dataset
BiLSTM-max, HBMP, ESIM, KIM, ESIM + ELMo, and BERT
SNLI, MultiNLI and SICK
Stacking method LSTMCNN SARNN simple LSTM bidirectional model TextCNN
Private dashboard is leaderboard where competitors can see results after competition is finished - on hidden part of test set (private test set).
Public dashboard where competitors can see their results during competition, on part of the test set (public test set).
They used Wiki Vietnamese language and Vietnamese newspapers to pretrain embeddings and dataset provided in HSD task to train model (details not mentioned in paper).
Women-Yoga
RNN model CNN model RNN-CNN model attn1511 model Deep Averaging Network model avg mean of word embeddings in the sentence with projection matrix
ensemble of hand-crafted syntactic and frame-semantic features BIBREF16
Argus Dataset AI2-8grade/CK12 Dataset MCTest Dataset
The BLEU metric
WMT14 En-Fr and En-De datasets IWSLT De-En and En-Vi datasets
De-En, En-Fr and En-Vi translation tasks
2.2 BLEU gains
the weighted sum of the new opinion representations, according to their associations with the current aspect representation
INLINEFORM0 (SemEval 2014) contains reviews of the laptop domain and those of INLINEFORM1 (SemEval 2014), INLINEFORM2 (SemEval 2015) and INLINEFORM3 (SemEval 2016) are for the restaurant domain.
Compared with the winning systems of SemEval ABSA, our framework achieves 5.0%, 1.6%, 1.4%, 1.3% absolute gains on INLINEFORM0 , INLINEFORM1 , INLINEFORM2 and INLINEFORM3 respectively.
The average number of utterances per dialog is about 23
3-gram and 4-gram conditional language model Convolution LSTM models BIBREF27 with and without attention BIBREF28 Transformer GPT-2
ordering pizza, creating auto repair appointments, setting up ride service, ordering movie tickets, ordering coffee drinks and making restaurant reservations
general classification tasks use of the methodology in other networked systems a network could be enriched with embeddings obtained from graph embeddings techniques
long-range syntactical links, though less frequent than adjacent syntactical relationships, might be disregarded from a simple word adjacency approach
They use it as addition to previous model - they add new edge between words if word embeddings are similar.
in a co-occurrence network each different word becomes a node and edges are established via co-occurrence in a desired window connects only adjacent words in the so called word adjacency networks
balanced accuracy, i.e., the average of the three accuracies on each class
2,060 workers
Totally 6980 validation and test image-sentence pairs have been corrected.
73.02% on the uncorrected SNLI-VE test set, achieves 73.18% balanced accuracy when tested on the corrected test set
neutral class
Italian Wikipedia and Google News extraction producing final vocabulary of 618224 words
$421\,829\,960$ words divided into $17\,305\,401$ sentences
number of epochs is an important parameter and its increase leads to results that rank our two worst models almost equal, or even better than others
extracted from a dump of the Italian Wikipedia (dated 2019.04.01), from the main categories of Italian Google News (WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH) and from some anonymized chats between users and a customer care chatbot (Laila)
an additional morphology table including target-side affixes. We inject the decoder with morphological properties of the target language.
target-side affixes
RNNs CNNs Naive Bayes with Laplace Smoothing k-clustering SVM with linear kernel
Influential tweeters ( who they define as individuals certain to have a classifiable sentiment regarding the topic at hand) is used to label tweets in bulk in the absence of manually-labeled tweets.
the East Coast Bomb Cyclone the Mendocino, California wildfires Hurricane Florence Hurricane Michael the California Camp Fires
BIBREF8 a refined collection of tweets gathered from twitter
9,892 stories of sexual harassment incidents
joint learning NLP models that use convolutional neural network (CNN) BIBREF8 and bi-directional long short-term memory (BiLSTM)
we demonstrate that harassment occurred more frequently during the night time than the day time it shows that besides unspecified strangers (not shown in the figure), conductors and drivers are top the list of identified types of harassers, followed by friends and relatives we uncovered that there exist strong correlations between the age of perpetrators and the location of harassment, between the single/multiple harasser(s) and location, and between age and single/multiple harasser(s) We also found that the majority of young perpetrators engaged in harassment behaviors on the streets we found that adult perpetrators of sexual harassment are more likely to act alone we also found that the correlations between the forms of harassment with the age, single/multiple harasser, type of harasser, and location commenting happened more frequently when harassers were in groups. Last but not least, public transportation is where people got indecently touched most frequently both by fellow passengers and by conductors and drivers.
The multi-task model outperforms the single-task model at all data sizes but none have an overall benefit from the open vocabulary system
Slot filling we consider the actions that a user might perform via apps on their phone The corresponding actions are booking a flight, renting a home, buying bus tickets, and making a reservation at a restaurant
AMS method.
They identify documents that contain the unigrams 'caused', 'causing', or 'causes'
Randomly selected from a Twitter dump, temporally matched to causal documents
Presence of only the exact unigrams 'caused', 'causing', or 'causes'
Only automatic methods
Randomly from a Twitter dump
Randomly from Twitter
MemN2N BIBREF12 Attentive and Impatient Readers BIBREF6
correctness of all the question answer pairs are verified by at least two annotators
Variables in the set {str, prec, attr} indicating in which mode the mention should be resolved.
No, supervised models perform better for this task.
no gold standard for automatically evaluating two (or more) dialogue systems when considering the satisfaction of the human and the fluency of the generated dialogue
For task 1 best F1 score was 0.9391 on closed and 0.9414 on open test.
For task2 best result had: Ratio 0.3175 , Satisfaction 64.53, Fluency 0, Turns -1 and Guide 2
For task 1, we use F1-score Task completion ratio User satisfaction degree Response fluency Number of dialogue turns Guidance ability for out of scope input
Rouge-L Bleu-1
well-formed sentences vs concise answers
WAS LipCH-Net-seq CSSMCM-w/o video
video sequence is first fed into the VGG model BIBREF9 to extract visual feature
performances of a purely content-based model naturally stays stable
BIBREF13 majority baseline
Answer with content missing: (Data and pre-processing section) The data is suited for our experiments because the annotators were explicitly asked to provide annotations on a clausal level.
They collected tweets in Russian language using a heuristic query specific to Russian
Proposed SG model vs SINDHI FASTTEXT:
Average cosine similarity score: 0.650 vs 0.388
Average semantic relatedness similarity score between countries and their capitals: 0.663 vs 0.391
908456 unique words are available in collected corpus.
daily Kawish and Awami Awaz Sindhi newspapers Wikipedia dumps short stories and sports news from Wichaar social blog news from Focus Word press blog historical writings, novels, stories, books from Sindh Salamat literary website novels, history and religious books from Sindhi Adabi Board tweets regarding news and sports are collected from twitter
audiences wanted products more and more contemporary, intense and a little bit novel or sophisticated, but less and less mellow and (surprisingly) unpretentious
between 1900s and 2010s
77 genres
the results obtained on development and test set (F1 = 89.60, F1 = 87.82) and the results on the supplemental test set (F1 = 71.49)
the task of detecting anglicisms can be approached as a sequence labeling problem where only certain spans of texts will be labeled as anglicism (in a similar way to an NER task). The chosen model was conditional random field model (CRF), which was also the most popular model in both Shared Tasks on Language Identification for Code-Switched Data
Bias feature Token feature Uppercase feature (y/n) Titlecase feature (y/n) Character trigram feature Quotation feature (y/n) Word suffix feature (last three characters) POS tag (provided by spaCy utilities) Word shape (provided by spaCy utilities) Word embedding (see Table TABREF26)
special dedicated discriminator is added to the model to control that the latent representation does not contain stylistic information shifted autoencoder or SAE combination of both approaches
accuracy can change up to 5 percentage points, whereas BLEU can vary up to 8 points
standard parametrized attention and a non-attention baseline
Sequence Copy Task and WMT'17
an adversarial loss ($\ell _{adv}$) for each model as in the baseline a cycle consistency loss ($\ell _{cycle}$) on each side
New best results of accuracy (P@1) on Vecmap:
Ours-GeoMMsemi: EN-IT 50.00 IT-EN 42.67 EN-DE 51.60 DE-EN 47.22 FI-EN 39.62 EN-ES 39.47 ES-EN 36.43
Proposed method vs best baseline result on Vecmap (Accuracy P@1):
EN-IT: 50 vs 50
IT-EN: 42.67 vs 42.67
EN-DE: 51.6 vs 51.47
DE-EN: 47.22 vs 46.96
EN-FI: 35.88 vs 36.24
FI-EN: 39.62 vs 39.57
EN-ES: 39.47 vs 39.30
ES-EN: 36.43 vs 36.06
EN<->ES
EN<->DE
EN<->IT
EN<->EO
EN<->MS
EN<->FI
Procrustes GPA GeoMM GeoMM$_{semi}$ Adv-C-Procrustes Unsup-SL Sinkhorn-BT
For the entailment classifier we compare Decomposable Attention BIBREF2 , BIBREF3 as implemented in the official baseline, ESIM BIBREF4 , and a transformer network with pre-trained weights BIBREF5 .
BIBREF5
tests a combination of retrieval and textual entailment capabilities. To verify a claim in the dataset as supported, refuted, or undecided, a system must retrieve relevant articles and sentences from Wikipedia. Then it must decide whether each of those sentences, or some combination of them, entails or refutes the claim, which is an entailment problem
empirically compare automatic differentiation (AD, our implementation based on Clad) and numerical differentiation (ND, based on finite difference method)
English$\rightarrow $Italian/German portions of the MuST-C corpus As additional data, we use a mix of public and proprietary data for about 16 million sentence pairs for English-Italian (En-It) and $4.4$ million WMT14 sentence pairs for the English-German (En-De)
They introduce new trigonometric encoding which besides information about position uses additional length information (abs or relative).
They use three groups short/normal/long translation classes to learn length token, which is in inference used to bias network to generate desired length group.
two translation directions (En-It and En-De)
a deep neural network (DNN) architecture proposed in BIBREF24 maximum entropy (MaxEnt) proposed in BIBREF23 type of discriminative model
It uses deep learning framework (pytorch)
Typical implementations of dynamic programming algorithms are serial in the length of the sequence Computational complexity is even more of an issue for parsing algorithms, which cannot be as easily parallelized Unfortunately for other semirings, such as log and max, these operations are either slow or very memory inefficient
Parallel Scan Inference Vectorized Parsing Semiring Matrix Operations
VecMap Muse Barista
No reliability diagrams are provided and no explicit comparison is made between confidence scores or methods.
word embeddings
In the overall available data there are 40,071 training, 4,988 validation, and 5,050 usable testing stories.
a form of wordplay jokes in which one sign (e.g. a word or a phrase) suggests two or more meanings by exploiting polysemy, homonymy, or phonological similarity to another sign, for an intended humorous or rhetorical effect
intra-sequential and intra-word
pairing crowdworkers and having half of them acting as Wizards by limiting their dialogue options only to relevant and plausible ones, at any one point in the interaction
The Wizard can select one of several predefined messages to send, or type their own message if needed. Free text messages do not change the dialogue state in the FSM, so it is important to minimise their use by providing enough dialogue options to the Wizard.
Yes, CRWIZ has been used for data collection and its initial use resulted in 145 dialogues. The average time taken for the task was close to the estimate of 10 minutes, 14 dialogues (9.66%) resolved the emergency in the scenario, and these dialogues rated consistently higher in subjective and objective ratings than those which did not resolve the emergency. Qualitative results showed that participants believed that they were interacting with an automated assistant.
The Wizard can select one of several predefined messages to send, or type their own message if needed. Free text messages do not change the dialogue state in the FSM, so it is important to minimise their use by providing enough dialogue options to the Wizard. Predefined messages can also trigger other associated events such as pop-ups or follow-up non-verbal actions.
maximum of two scores assigned by the two separate models average score
Logistic regression model with character-level n-gram features
title of the news article screen name of the user
language which explicitly or implicitly threatens or demeans a person or a group based upon a facet of their identity such as gender, ethnicity, or sexual orientation
three parallel LSTM BIBREF21 layers
displays three different versions of a story written by three distinct models for a human to compare human can select the model to interact with (potentially after having chosen it via cross-model), and can collaborate at all stages
separate set of Turkers to rate the stories for overall quality and the three improvement areas
Title-to-Story system
As the baseline, we simply judge the input token as IOCs on the basis of the spelling features described in BIBREF12
The words that can indicate the characteristics of the neighbor words as contextual keywords and generate it from the automatically extracted contextual keywords.
from a collection of advanced persistent threats (APT) reports which are published from 2008 to 2018
Using a set of annotation tools such as Morfeusz, PANTERA, Spejd, NERF and Liner
only the first word sense (usually the most common) is taken into account
Our method performs a statistical test to determine whether a given word is used polysemously in the text, according to the following steps:
1) Setting N, the size of the neighbor.
2) Choosing N neighboring words ai in the order whose angle with the vector of the given word w is the smallest.
3) Computing the surrounding uniformity for ai(0 < i ≤ N) and w.
4) Computing the mean m and the sample variance σ for the uniformities of ai .
5) Checking whether the uniformity of w is less than m − 3σ. If the value is less than m − 3σ, we may regard w as a polysemic word.
BIBREF5 to train neural sequence-to-sequence NMF topic model with scikit-learn BIBREF14
LDC corpus
LDC corpus NIST 2003(MT03) NIST 2004(MT04), NIST 2005(MT05), NIST 2006(MT06) NIST 2008(MT08)
average INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 values
WDAqua BIBREF0 , QAKiS BIBREF7 , gAnswer BIBREF6 and Platypus BIBREF8
The Dutch section consists of 2,333,816 sentences and 53,487,257 words. The SONAR500 corpus consists of more than 500 million words obtained from different domains.
Table TABREF10 displays the twenty resulting topics
SPNet vs best baseline:
ROUGE-1: 90.97 vs 90.68
CIC: 70.45 vs 70.25
ROUGE and CIC relevance, conciseness and readability on a 1 to 5 scale, and rank the summary pair
Not at the moment, but summaries can be additionaly extended with this annotations.
Our encoder-decoder framework employs separate encoding for different speakers in the dialog. We integrate semantic slot scaffold by performing delexicalization on original dialogs. We integrate dialog domain scaffold through a multi-task framework.
Pointer-Generator Transformer
Answer with content missing: (formula for CIC) it accounts for the most important information within each dialog domain. CIC can be applied to any summarization task with predefined essential entities
To have an estimation about human performance in each metric, we iteratively treat every reference sentence in dev/test data as the prediction to be compared with all references (including itself).
BLEU-3/4 ROUGE-2/L CIDEr SPICE BERTScore
These concept-sets are sampled from several large corpora of image/video captions
test accuracy of 88.9%, which exceeds the previous best by 16.9%
MultiNLI BIBREF15 and SNLI BIBREF16
DREAM, MCTest, TOEFL, and SemEval-2018 Task 11
LSTM SCIBERT
Some sentences are associated to ambiguous dimensions in the hidden state output
the memory slots correspond to entities and the edges correspond to relationships between entities, each represented as a vector.
entity memory and relational memory.
The RelNet model achieves a mean error of 0.285% across tasks which is better than the results of the EntNet model BIBREF17 . The RelNet model is able to achieve 0% test error on 11 of the tasks, whereas the EntNet model achieves 0% error on 7 of the tasks.
extends memory-augmented neural networks with a relational memory to reason about relationships between multiple entities present within the text. The model is sequential in nature, consisting of the following steps: read text, process it into a dynamic relational memory and then attention conditioned on the question generates the answer. We model the dynamic memory in a fashion similar to Recurrent Entity Networks BIBREF17 and then equip it with an additional relational memory.
We compare the performance with the Recurrent Entity Networks model (EntNet) BIBREF17
by number of distinct n-grams
ON Event2Mind, the accuracy of proposed method is improved by  absolute BLUE  2.9,  10.87, 1.79 for xIntent, xReact and oReact respectively.
On Atomic dataset, the accuracy of proposed method is improved by  absolute BLUE 3.95.   4.11, 4.49 for xIntent, xReact and oReact.respectively.
RNN-based Seq2Seq Variational Seq2Seq VRNMT CWVAE-Unpretrained
CWVAE is trained on an auxiliary dataset to learn the event background information by using the context-aware latent variable.  Then, in finetute stage, CWVAE is trained on the task-specific dataset to adapt the event background information to each specific aspect of If-Then inferential target.
pipeline that is used at Microsoft for production data
Appending the domain tag “<2domain>" to the source sentences of the respective corpora
do not follow a particular plan or pursue a particular fixed information need integrating content found via search with content from structured data at each system turn, there are a large number of conversational moves that are possible most other domains do not have such high quality structured data available live search may not be able to achieve the required speed and efficiency
time elapsed between the moment the link or content was posted/tweeted and the moment that the reaction comment or tweet occurred
we combine a text sequence sub-network with a vector representation sub-network as shown in Figure FIGREF5 . The text sequence sub-network consists of an embedding layer initialized with 200-dimensional GloVe embeddings BIBREF15 followed by two 1-dimensional convolution layers, then a max-pooling layer followed by a dense layer. The vector representation sub-network consists of two dense layers. We incorporate information from both sub-networks through concatenated padded text sequences and vector representations of normalized Linguistic Inquiry and Word Count (LIWC) features BIBREF16 for the text of each post and its parent.
agreement answer appreciation disagreement elaboration humor negative reaction question other
a vector of frame-level acoustic features
Switchboard conversational English corpus
interpretation of Frege's work are examples of holistic approaches to meaning
Only in the context of a sentence does a word have a meaning.
10*n paraphrases, where n depends on the number of paraphrases that contain the entity mention spans
syntactic information semantic and topical information
GraphParser without paraphrases monolingual machine translation based model for paraphrase generation
weak correlation with p-value of 0.08
using topic modeling model Latent Dirichlet Allocation (LDA)
Swissmetro dataset