z-uo/qasper-squad huggingface dataset:
Paper: Evaluation of basic modules for isolated spelling error correction in Polish texts
The methods that we evaluated are baselines are the ones we consider to be basic and with moderate potential of yielding particularly good results. Probably the most straightforward approach to error correction is selecting known words from a dictionary that are within the smallest edit distance from the error. We used the Levenshtein distance metric BIBREF8 implemented in Apache Lucene library BIBREF9 . It is a version of edit distance that treats deletions, insertions and replacements as adding one unit distance, without giving a special treatment to character swaps. The SGJP – Grammatical Dictionary of Polish BIBREF10 was used as the reference vocabulary.
Question: How is PIEWi annotated?
Paper: Evaluation of basic modules for isolated spelling error correction in Polish texts
A promising method, adapted from work on correcting texts by English language learners BIBREF11 , expands on the concept of selecting a correction nearest to the spelling error according to some notion of distance. Here, the Levenshtein distance is used in a weighted sum to cosine distance between word vectors. This is based on the observation that trained vectors models of distributional semantics contain also representations of spelling errors, if they were not pruned. Their representations tend to be similar to those of their correct counterparts. For example, the token enginir will appear in similar contexts as engineer, and therefore will be assigned a similar vector embedding.
Question: What methods are tested in PIEWi?
Paper: Evaluation of basic modules for isolated spelling error correction in Polish texts
Another powerful approach, if conceptually simple in linguistic terms, is using a character-based recurrent neural network. Here, we test uni- and bidirectional Long Short-Term Memory networks BIBREF14 that are fed characters of the error as their input and are expected to output its correct form, character after character. This is similar to traditional solutions conceptualizing the spelling error as a chain of characters, which are used as evidence to predict the most likely chain of replacements (original characters). This was done with n-gram methods, Markov chains and other probabilistic models BIBREF15 . Since nowadays neural networks enjoy a large awareness as an element of software infrastructure, with actively maintained packages readily available, their evaluation seems to be the most practically useful. We used the PyTorch BIBREF16 implementation of LSTM in particular.
Question: Which specific error correction solutions have been proposed for specialized corpora in the past?
Paper: Few-shot Natural Language Generation for Task-Oriented Dialog
Task-oriented dialog systems are becoming increasingly popular, as they can assist users in various daily activities such as ticket booking and restaurant reservations. In a typical task-oriented dialog system, the Natural Language Generation (NLG) module plays a crucial role: it converts a system action (often specified in a semantic form selected by a dialog policy) into a final response in natural language. Hence, the response should be adequate to represent semantic dialog actions, and fluent to engage users' attention. As the ultimate interface to interacts with users, NLG plays a significant impact on the users' experience.
Question: What was the criteria for human evaluation?
Paper: Few-shot Natural Language Generation for Task-Oriented Dialog
A typical task-oriented spoken dialog system uses a pipeline architecture, as shown in Figure FIGREF2 (a), where each dialog turn is processed using a four-step procedure. $({1})$ Transcriptions of user’s input are first passed to the natural language understanding (NLU) module, where the user’s intention and other key information are extracted. $({2})$ This information is then formatted as the input to dialog state tracking (DST), which maintains the current state of the dialog. $({3})$ Outputs of DST are passed to the dialog policy module, which produces a dialog act based on the facts or entities retrieved from external resources (such as a database or a knowledge base). $({4})$ The dialog act emitted by the dialog policy module serves as the input to the NLG, through which a system response in natural language is generated. In this paper, we focus on the NLG component of task-oriented dialog systems, how to produce natural language responses conditioned on dialog acts.
Question: What automatic metrics are used to measure performance of the system?
Paper: Few-shot Natural Language Generation for Task-Oriented Dialog
We tackle this generation problem using conditional neural language models. Given training data of $N$ samples $=\lbrace (_n, _n)\rbrace _{n=1}^{N}$, our goal is to build a statistical model parameterized by $$ to characterize $p_{}(| )$. To leverage the sequential structure of response, one may further decompose the joint probability of $$ using the chain rule, casting an auto-regressive generation process as follows:
Question: What existing methods is SC-GPT compared to?
Paper: Using Whole Document Context in Neural Machine Translation
Neural machine translation (NMT) has grown rapidly in the past years BIBREF0, BIBREF1. It usually takes the form of an encoder-decoder neural network architecture in which source sentences are summarized into a vector representation by the encoder and are then decoded into target sentences by the decoder. NMT has outperformed conventional statistical machine translation (SMT) by a significant margin over the past years, benefiting from gating and attention techniques. Various models have been proposed based on different architectures such as RNN BIBREF0, CNN BIBREF2 and Transformer BIBREF1, the latter having achieved state-of-the-art performances while significantly reducing training time.
Question: Which language-pair had the better performance?
Answer: French-English
Paper: Using Whole Document Context in Neural Machine Translation
Interest in considering the whole document instead of a set of sentences preceding the current pair lies in the necessity for a human translator to account for broader context in order to keep a coherent translation. The idea of representing and using documents for a model is interesting, since the model could benefit from information located before or after the current processed sentence.
Question: Which datasets were used in the experiment?
Paper: Using Whole Document Context in Neural Machine Translation
We propose to use the simplest method to estimate document embeddings. The approach is called SWEM-aver (Simple Word Embedding Model – average) BIBREF12. The embedding of a document $k$ is computed by taking the average of all its $N$ word vectors (see Eq. DISPLAY_FORM2) and therefore has the same dimension. Out of vocabulary words are ignored.
Question: What evaluation metrics did they use?
Paper: Finding Street Gang Members on Twitter
The crime and violence street gangs introduce into neighborhoods is a growing epidemic in cities around the world. Today, over 1.23 million people in the United States are members of a street gang BIBREF0 , BIBREF1 , which is a coalition of peers, united by mutual interests, with identifiable leadership and internal organization, who act collectively to conduct illegal activity and to control a territory, facility, or enterprise BIBREF2 . They promote criminal activities such as drug trafficking, assault, robbery, and threatening or intimidating a neighborhood BIBREF1 . Moreover, data from the Centers for Disease Control in the United States suggests that the victims of at least 1.3% of all gang-related homicides are merely innocent bystanders who live in gang occupied neighborhoods BIBREF3 .
Question: Do they evaluate only on English datasets?
Paper: Finding Street Gang Members on Twitter
Gang violence is a well studied social science topic dating back to 1927 BIBREF17 . However, the notions of “Cyber-” or “Internet banging”, which is defined as “the phenomenon of gang affiliates using social media sites to trade insults or make violent threats that lead to homicide or victimization” BIBREF7 , was only recently introduced BIBREF18 , BIBREF10 . Patton et al. introduced the concept of “Internet banging” and studied how social media is now being used as a tool for gang self-promotion and as a way for gang members to gain and maintain street credibility BIBREF7 . They also discussed the relationship between gang-related crime and hip-hop culture, giving examples on how hip-hop music shared on social media websites targeted at harassing rival gang members often ended up in real-world collisions among those gangs. Decker et al. and Patton et al. have also reported that street gangs perform Internet banging with social media posts of videos depicting their illegal behaviors, threats to rival gangs, and firearms BIBREF19 , BIBREF20 .
Question: What are the differences in the use of emojis between gang member and the rest of the Twitter population?
Paper: Finding Street Gang Members on Twitter
This section discusses the methodology we followed to study and classify the Twitter profiles of gang members automatically. It includes a semi-automatic data collection process to discover a large set of verifiable gang member profiles, an evaluation of the tweets of gang and non-gang member posts to identify promising features, and the deployment of multiple supervised learning algorithms to perform the classification.
Question: What are the differences in the use of YouTube links between gang member and the rest of the Twitter population?
Paper: Finding Street Gang Members on Twitter
Discovering gang member profiles on Twitter to build training and testing datasets is a challenging task. Past strategies to find these profiles were to search for keywords, phrases, and events that are known to be related to gang activity in a particular city a priori BIBREF10 , BIBREF20 . However, such approaches are unlikely to yield adequate data to train an automatic classifier since gang members from different geographic locations and cultures use local languages, location-specific hashtags, and share information related to activities in a local region BIBREF10 . Such region-specific tweets and profiles may be used to train a classifier to find gang members within a small region but not across the Twitterverse. To overcome these limitations, we adopted a semi-automatic workflow, illustrated in Figure FIGREF7 , to build a dataset of gang member profiles suitable for training a classifier. The steps of the workflow are:
Question: What are the differences in the use of images between gang member and the rest of the Twitter population?
Paper: Finding Street Gang Members on Twitter
We next explore differences between gang and non-gang member Twitter usage to find promising features for classifying profiles. For this purpose, profiles of non-gang members were collected from the Twitter Streaming API. We collected a random sample of tweets and the profiles of the users who authored the tweets in the random sample. We manually verified that all Twitter profiles collected in this approach belong to non-gang members. The profiles selected were then filtered by location to remove non-U.S. profiles by reverse geo-coding the location stated in their profile description by the Google Maps API. Profiles with location descriptions that were unspecified or did not relate to a location in the U.S. were discarded. We collected 2,000 non-gang member profiles in this manner. In addition, we added 865 manually verified non-gang member profiles collected using the location neutral keywords discussed in Section SECREF3 . Introducing these profiles, which have some characteristics of gang members (such as cursing frequently or cursing at law enforcement) but are not, captures local languages used by family/friends of gang members and ordinary people in a neighborhood where gangs operate.
Question: What are the differences in language use between gang member and the rest of the Twitter population?
Paper: Finding Street Gang Members on Twitter
The unigrams of tweets, profile text, and linked YouTube video descriptions and comments, along with the distribution of emoji symbols and the profile image tags were used to train four different classification models: a Naive Bayes net, a Logistic Regression, a Random Forest, and a Support Vector Machine (SVM). These four models were chosen because they are known to perform well over text features, which is the dominant type of feature considered. The performance of the models are empirically compared to determine the most suitable classification technique for this problem. Data for the models are represented as a vector of term frequencies where the terms were collected from one or more feature sets described above.
Question: How is gang membership verified?
Paper: Finding Street Gang Members on Twitter
We next evaluate the performance of classifiers that use the above features to discover gang member profiles on Twitter. For this purpose, we use the training set discussed in Section SECREF3 with 400 gang member profiles (the `positive'/`gang' class) and 2,865 non-gang member profiles (the `negative'/`non-gang' class). We trained and evaluated the performance of the classifiers mentioned in Section SECREF31 under a 10-fold cross validation scheme. For each of the four learning algorithms, we consider variations involving only tweet text, emoji, profile, image, or music interest (YouTube comments and video description) features, and a final variant that considers all types of features together. The classifiers that use a single feature type were intended to help us study the quality of their predictive power by itself. When building these single-feature classifiers, we filtered the training dataset based on the availability of the single feature type in the training data. For example, we only used the twitter profiles that had at least a single emoji in their tweets to train classifiers that consider emoji features. We found 3,085 such profiles out of the 3,265 profiles in the training set. When all feature types were considered, we developed two different models:
Question: Do the authors provide evidence that 'most' street gang members use Twitter to intimidate others?
Paper: A Unified System for Aggression Identification in English Code-Mixed and Uni-Lingual Texts
The exponential increase of interactions on the various social media platforms has generated the huge amount of data on social media platforms like Facebook and Twitter, etc. These interactions resulted not only positive effect but also negative effect over billions of people owing to the fact that there are lots of aggressive comments (like hate, anger, and bullying). These cause not only mental and psychological stress but also account deactivation and even suicideBIBREF1. In this paper we concentrate on problems related to aggressiveness.
Question: What is English mixed with in the TRAC dataset?
Paper: A Unified System for Aggression Identification in English Code-Mixed and Uni-Lingual Texts
There are several works for aggression identification submitted at TRAC 2018 among them some approaches use the ensemble of multiple statistical modelsBIBREF6, BIBREF7, BIBREF8, BIBREF9. Similarly, some of the models likeBIBREF10, BIBREF11, BIBREF12 have used ensemble of statistical and deep learning models. In these models the statistical part of the model uses additional features from text analysis like parts-of-speech tags, punctuation, emotion, emoticon etc. Model like: BIBREF13 has used the ensemble of deep learning models based on majority voting.
Question: Which psycholinguistic and basic linguistic features are used?
Answer: Emotion Sensor Feature, Part of Speech, Punctuation, Sentiment Analysis, Empath, TF-IDF Emoticon features
Paper: A Unified System for Aggression Identification in English Code-Mixed and Uni-Lingual Texts
In this section, we describe our system architecture for aggressiveness classifier. In section SECREF23 we describe data preprocessing applied on the input text before feeding it to each of the classification models. Section SECREF26 describes the computation of NLP features. In Sections SECREF30, SECREF34 and SECREF45 we have described the architecture of different deep learning models like Deep Pyramid CNN, Disconnected RNN and Pooled BiLSTM respectively. Finally, in Section SECREF49, we describe model averaging based classification model which combines the prediction probabilities from three deep learninig architectures discussed above. (see Figure FIGREF22. for block diagram of system architecture).
Question: How have the differences in communication styles between Twitter and Facebook increase the complexity of the problem?
Answer: Systems do not perform well both in Facebook and Twitter texts
Paper: A Unified System for Aggression Identification in English Code-Mixed and Uni-Lingual Texts
We consider the text to be well formatted before applying the text to the embedding layer. First, we detect non-English text(which are few) and translate all of them to English using Google Translate. Still, there is some code mixed words like "mc", "bc" and other English abbreviations and spelling errors like "nd" in place of "and", "u" in place of "you" causes deep learning model to confuse with sentences of the same meaning. We follow the strategy of preprocessor as inBIBREF17 to normalize the abbreviations and remove spelling errors, URLs and punctuation marks, converting emojis to their description.
Question: What are the key differences in communication styles between Twitter and Facebook?
Paper: A Unified System for Aggression Identification in English Code-Mixed and Uni-Lingual Texts
We have identified a novel combination of features which are highly effective in aggression classification when applied in addition to the features obtained from the deep learning classifier at the classification layer. We have introduced two new features in addition to the previously available features. The first one is the Emotion Sensor Feature which use a statistical model to classify the words into 7 different classes based on the sentences obtained from twitter and blogs which contain total 1,185,540 words. The second one is the collection of selected topical signal from text collected using Empath (see Table 1.).
Question: What data/studies do the authors provide to support the assertion that the majority of aggressive conversations contain code-mixed languages?
Answer: None
Paper: An Emotional Analysis of False Information in Social Media and News Articles
With the complicated political and economic situations in many countries, some agendas are publishing suspicious news to affect public opinions regarding specific issues BIBREF0. The spreading of this phenomenon is increasing recently with the large usage of social media and online news sources. Many anonymous accounts in social media platforms start to appear, as well as new online news agencies without presenting a clear identity of the owner. Twitter has recently detected a campaign organized by agencies from two different countries to affect the results of the last U.S. presidential elections of 2016. The initial disclosures by Twitter have included 3,841 accounts. A similar attempt was done by Facebook, as they detected coordinated efforts to influence U.S. politics ahead of the 2018 midterm elections.
Question: What is the baseline?
Paper: An Emotional Analysis of False Information in Social Media and News Articles
Trusted news is recounting its content in a naturalistic way without attempting to affect the opinion of the reader. On the other hand, false news is taking advantage of the presented issue sensitivity to affect the readers' emotions which sequentially may affect their opinions as well. A set of works has been done previously to investigate the language of false information. The authors in BIBREF3 have studied rumours in Twitter. They have investigated a corpus of true and false tweets rumours from different aspects. From an emotional point of view, they found that false rumours inspired fear, disgust, and surprise in their replies while the true ones inspired joy and anticipation. Some kinds of false information are similar to other language phenomena. For example, satire by its definition showed similarity with irony language. The work in BIBREF4 showed that affective features work well in the detection of irony. In addition, they confirmed that positive words are more relevant for identifying sarcasm and negative words for irony BIBREF5. The results of these works motivate us to investigate the impact of emotions on false news types. These are the research questions we aim to answer:
Question: What datasets did they use?
Paper: STransE: a novel embedding model of entities and relationships in knowledge bases
Knowledge bases (KBs), such as WordNet BIBREF0 , YAGO BIBREF1 , Freebase BIBREF2 and DBpedia BIBREF3 , represent relationships between entities as triples $(\mathrm {head\ entity, relation, tail\ entity})$ . Even very large knowledge bases are still far from complete BIBREF4 , BIBREF5 . Link prediction or knowledge base completion systems BIBREF6 predict which triples not in a knowledge base are likely to be true BIBREF7 , BIBREF8 . A variety of different kinds of information is potentially useful here, including information extracted from external corpora BIBREF9 , BIBREF10 and the other relationships that hold between the entities BIBREF11 , BIBREF12 . For example, toutanova-EtAl:2015:EMNLP used information from the external ClueWeb-12 corpus to significantly enhance performance.
Question: What scoring function does the model use to score triples?
Paper: STransE: a novel embedding model of entities and relationships in knowledge bases
Let $\mathcal {E}$ denote the set of entities and $\mathcal {R}$ the set of relation types. For each triple $(h, r, t)$ , where $h, t \in \mathcal {E}$ and $r \in \mathcal {R}$ , the STransE model defines a score function $f_r(h, t)$ of its implausibility. Our goal is to choose $f$ such that the score $f_r(h,t)$ of a plausible triple $(h,r,t)$ is smaller than the score $f_{r^{\prime }}(h^{\prime },t^{\prime })$ of an implausible triple $\mathcal {R}$0 . We define the STransE score function $\mathcal {R}$1 as follows:
Question: What datasets are used to evaluate the model?
Answer: WN18, FB15k
Paper: Doc2Vec on the PubMed corpus: study of a new approach to generate related articles
Background PubMed is the biggest and most used bibliographic database worldwide, hosting more than 26M biomedical publications. One of its useful features is the “similar articles” section, allowing the end-user to find scientific articles linked to the consulted document in term of context. The aim of this study is to analyze whether it is possible to replace the statistic model PubMed Related Articles (pmra) with a document embedding method.
Question: How long it took for each Doc2Vec model to be trained?
Paper: Doc2Vec on the PubMed corpus: study of a new approach to generate related articles
PubMed is the largest database of bio-medical articles worldwide with more than 29,000,000 freely available abstracts. Each article is identified by an unique PubMed IDentifier (PMID) and is indexed with the Medical Subject Headings (MeSH) terminology. In order to facilitate the Information Retrieval (IR) process for the end-user, PubMed launched in 2007 a service of related articles search, available both through its Graphical User Interface (GUI) and its Application Programming Interface (API). Regarding the GUI, while the user is reading a publication, a panel presents title of articles that may be linked to the current reading. For the API, the user must query eLink with a given PMID BIBREF0. The output will be a list of others PMIDs, each associated with the similarity score computed by the pmra (pubmed related article) model BIBREF1.
Question: How better are results for pmra algorithm  than Doc2Vec in human evaluation?