z-uo/qasper-squad huggingface dataset:
Paper: A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media
We find mentions of users, numbers, hashtags, URLs and common emoticons and replace them with the tokens <user>,<number>,<hashtag>,<url>,<emoticon>. We also find elongated words and convert them into short and standard format; for example, converting yeeeessss to yes. With hashtags that include some tokens without any with space between them, we replace them by their textual counterparts; for example, we convert hashtag “#notsexist" to “not sexist". All punctuation marks, unknown uni-codes and extra delimiting characters are removed, but we keep all stop words because our model trains the sequence of words in a text directly. We also convert all tweets to lower case.
Question: What biases does their model capture?
Answer: Data annotation biases where tweet containing disrespectful words are annotated as hate or offensive without any presumption about the social context of tweeters
Paper: A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media
For the implementation of our neural network, we used pytorch-pretrained-bert library containing the pre-trained BERT model, text tokenizer, and pre-trained WordPiece. As the implementation environment, we use Google Colaboratory tool which is a free research tool with a Tesla K80 GPU and 12G RAM. Based on our experiments, we trained our classifier with a batch size of 32 for 3 epochs. The dropout probability is set to 0.1 for all layers. Adam optimizer is used with a learning rate of 2e-5. As an input, we tokenized each tweet with the BERT tokenizer. It contains invalid characters removal, punctuation splitting, and lowercasing the words. Based on the original BERT BIBREF11, we split words to subword units using WordPiece tokenization. As tweets are short texts, we set the maximum sequence length to 64 and in any shorter or longer length case it will be padded with zero values or truncated to the maximum length.
Question: What existing approaches do they compare to?
Paper: Learning Concept Embeddings for Efficient Bag-of-Concepts Densification
Vector-based semantic mapping models are used to represent textual structures (words, phrases, and documents) as high-dimensional meaning vectors. Typically, these models utilize textual corpora and/or Knowledge Bases (KBs) to acquire world knowledge, which is then used to generate a vector representation for the given text in the semantic space. The goal is thus to accurately place semantically similar structures close to each other in that semantic space. On the other hand, dissimilar structures should be far apart.
Question: What is the benchmark dataset?
Paper: Learning Concept Embeddings for Efficient Bag-of-Concepts Densification
Concept/Entity Embeddings: neural embedding models have been proposed to learn distributed representations of concepts/entities. songunsupervised proposed using the popular Word2Vec model BIBREF12 to obtain the embeddings of each concept by averaging the vectors of the concept's individual words. For example, the embeddings of Microsoft Office would be obtained by averaging the embeddings of Microsoft and Office obtained from the Word2Vec model. Clearly, this method disregards the fact that the semantics of multi-word concepts is different from the semantics of their individual words. More robust concept embeddings can be learned from the concept's corresponding article and/or from the structure of the employed KB (e.g., its link graph). Such concept embedding models were proposed by hu2015entity,li2016joint,yamada2016joint who all utilize the skip-gram model BIBREF11 , but differ in how they define the context of the target concept.
Question: What are the two neural embedding models?
Paper: Learning Concept Embeddings for Efficient Bag-of-Concepts Densification
A main objective of learning concept embeddings is to overcome the inherent problem of data sparsity associated with the BOC representation. Here we try to learn continuous concept vectors by building upon the skip-gram embedding model BIBREF11 . In the conventional skip-gram model, a set of contexts are generated by sliding a context window of predefined size over sentences of a given text corpus. Vector representation of a target word is learned with the objective to maximize the ability of predicting surrounding words of that target word.
Question: which neural embedding model works better?
Answer: the CRX model
Paper: Learning Concept Embeddings for Efficient Bag-of-Concepts Densification
In this model, we jointly learn the embeddings of both words and concepts. First, all concept mentions are identified in the given corpus. Second, contexts are generated for both words and concepts from both other surrounding words and other surrounding concepts as well. After generating all the contexts, we use the skip-gram model to jointly learn words and concepts embeddings. Formally, given a training corpus of INLINEFORM0 words INLINEFORM1 . We iterate over the corpus identifying words and concept mentions and thus generating a sequence of INLINEFORM2 tokens INLINEFORM3 where INLINEFORM4 (as multi-word concepts will be counted as one token). Afterwards we train the a skip-gram model aiming to maximize: DISPLAYFORM0 
Question: What is the degree of dimension reduction of the efficient aggregation method?
Answer: The number of dimensions can be reduced by up to 212 times.
Paper: Incorporating Subword Information into Matrix Factorization Word Embeddings
Low dimensional word representations (embeddings) have become a key component in modern NLP systems for language modeling, parsing, sentiment classification, and many others. These embeddings are usually derived by employing the distributional hypothesis: that similar words appear in similar contexts BIBREF0 .
Question: For which languages do they build word embeddings for?
Answer: English
Paper: Incorporating Subword Information into Matrix Factorization Word Embeddings
Word embeddings that leverage subword information were first introduced by BIBREF14 which represented a word of as the sum of four-gram vectors obtained running an SVD of a four-gram to four-gram co-occurrence matrix. Our model differs by learning the subword vectors and resulting representation jointly as weighted factorization of a word-context co-occurrence matrix is performed.
Question: How do they evaluate their resulting word embeddings?
Paper: Incorporating Subword Information into Matrix Factorization Word Embeddings
The LexVec BIBREF7 model factorizes the PPMI-weighted word-context co-occurrence matrix using stochastic gradient descent. 
Question: What types of subwords do they incorporate in their model?
Paper: Incorporating Subword Information into Matrix Factorization Word Embeddings
Our experiments aim to measure if the incorporation of subword information into LexVec results in similar improvements as observed in moving from Skip-gram to fastText, and whether unsupervised morphemes offer any advantage over n-grams. For IV words, we perform intrinsic evaluation via word similarity and word analogy tasks, as well as downstream tasks. OOV word representation is tested through qualitative nearest-neighbor analysis.
Question: Which matrix factorization methods do they use?
Paper: Imparting Interpretability to Word Embeddings while Preserving Semantic Structure
Distributed word representations, commonly referred to as word embeddings BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , serve as elementary building blocks in the course of algorithm design for an expanding range of applications in natural language processing (NLP), including named entity recognition BIBREF4 , BIBREF5 , parsing BIBREF6 , sentiment analysis BIBREF7 , BIBREF8 , and word-sense disambiguation BIBREF9 . Although the empirical utility of word embeddings as an unsupervised method for capturing the semantic or syntactic features of a certain word as it is used in a given lexical resource is well-established BIBREF10 , BIBREF11 , BIBREF12 , an understanding of what these features mean remains an open problem BIBREF13 , BIBREF14 and as such word embeddings mostly remain a black box. It is desirable to be able to develop insight into this black box and be able to interpret what it means, while retaining the utility of word embeddings as semantically-rich intermediate representations. Other than the intrinsic value of this insight, this would not only allow us to explain and understand how algorithms work BIBREF15 , but also set a ground that would facilitate the design of new algorithms in a more deliberate way.
Question: Do they report results only on English data?
Paper: Imparting Interpretability to Word Embeddings while Preserving Semantic Structure
Methodologically, our work is related to prior studies that aim to obtain “improved” word embeddings using external lexical resources, under some performance metric. Previous work in this area can be divided into two main categories: works that i) modify the word embedding learning algorithm to incorporate lexical information, ii) operate on pre-trained embeddings with a post-processing step.
Question: What experiments do they use to quantify the extent of interpretability?
Answer: Human evaluation for interpretability using the word intrusion test and automated evaluation for interpretability using a semantic category-based approach based on the method and category dataset (SEMCAT).
Paper: Imparting Interpretability to Word Embeddings while Preserving Semantic Structure
For the task of unsupervised word embedding extraction, we operate on a discrete collection of lexical units (words) INLINEFORM0 that is part of an input corpus INLINEFORM1 , with number of tokens INLINEFORM2 , sourced from a vocabulary INLINEFORM3 of size INLINEFORM4 . In the setting of distributional semantics, the objective of a word embedding algorithm is to maximize some aggregate utility over the entire corpus so that some measure of “closeness” is maximized for pairs of vector representations INLINEFORM14 for words which, on the average, appear in proximity to one another. In the GloVe algorithm BIBREF2 , which we base our improvements upon, the following objective function is considered: DISPLAYFORM0 
Question: Along which dimension do the semantically related words take larger values?
Paper: Imparting Interpretability to Word Embeddings while Preserving Semantic Structure
Our approach falls into a joint-learning framework where the distributional information extracted from the corpus is allowed to fuse with the external lexicon-based information. Word-groups extracted from Roget's Thesaurus are directly mapped to individual dimensions of word embeddings. Specifically, the vector representations of words that belong to a particular group are encouraged to have deliberately increased values in a particular dimension that corresponds to the word-group under consideration. This can be achieved by modifying the objective function of the embedding algorithm to partially influence vector representation distributions across their dimensions over an input vocabulary. To do this, we propose the following modification to the GloVe objective in ( EQREF6 ): rCl J = i,j=1V f(Xij)[ (wiTwj + bi + bj -Xij)2
Question: What is the additive modification to the objective function?
Answer: The cost function for any one of the words of concept word-groups is modified by the introduction of an additive term to the cost function. . Each embedding vector dimension is first associated with a concept. For a word belonging to any one of the word-groups representing these concepts, the modified cost term favors an increase for the value of this word's embedding vector dimension corresponding to the concept that the particular word belongs to,
An additive term added to the cost function for any one of the words of concept word-groups
Paper: Improving Distributed Representations of Tweets - Present and Future
Twitter is a widely used microblogging platform, where users post and interact with messages, “tweets”. Understanding the semantic representation of tweets can benefit a plethora of applications such as sentiment analysis BIBREF0 , BIBREF1 , hashtag prediction BIBREF2 , paraphrase detection BIBREF3 and microblog ranking BIBREF4 , BIBREF5 . However, tweets are difficult to model as they pose several challenges such as short length, informal words, unusual grammar and misspellings. Recently, researchers are focusing on leveraging unsupervised representation learning methods based on neural networks to solve this problem. Once these representations are learned, we can use off-the-shelf predictors taking the representation as input to solve the downstream task BIBREF6 , BIBREF7 . These methods enjoy several advantages: (1) they are cheaper to train, as they work with unlabelled data, (2) they reduce the dependence on domain level experts, and (3) they are highly effective across multiple applications, in practice.
Question: Which dataset do they use?
Paper: Improving Distributed Representations of Tweets - Present and Future
There are various models spanning across different model architectures and objective functions in the literature to compute tweet representation in an unsupervised fashion. These models work in a semi-supervised way - the representations generated by the model is fed to an off-the-shelf predictor like Support Vector Machines (SVM) to solve a particular downstream task. These models span across a wide variety of neural network based architectures including average of word vectors, convolutional-based, recurrent-based and so on. We believe that the performance of these models is highly dependent on the objective function it optimizes – predicting adjacent word (within-tweet relationships), adjacent tweet (inter-tweet relationships), the tweet itself (autoencoder), modeling from structured resources like paraphrase databases and weak supervision. In this section, we provide the first of its kind survey of the recent tweet-specific unsupervised models in an organized fashion to understand the literature. Specifically, we categorize each model based on the optimized objective function as shown in Figure FIGREF1 . Next, we study each category one by one.
Question: Do they evaluate their learned representations on downstream tasks?
Paper: Improving Distributed Representations of Tweets - Present and Future
Motivation: Every tweet is assumed to have a latent topic vector, which influences the distribution of the words in the tweet. For example, though the appearance of the phrase catch the ball is frequent in the corpus, if we know that the topic of a tweet is about “technology”, we can expect words such as bug or exception after the word catch (ignoring the) instead of the word ball since catch the bug/exception is more plausible under the topic “technology”. On the other hand, if the topic of the tweet is about “sports”, then we can expect ball after catch. These intuitions indicate that the prediction of neighboring words for a given word strongly relies on the tweet also.
Question: Which representation learning architecture do they adopt?
Paper: Improving Distributed Representations of Tweets - Present and Future
Motivation: To capture rich tweet semantics, researchers are attempting to exploit a type of sentence-level Distributional Hypothesis BIBREF10 , BIBREF13 . The idea is to infer the tweet representation from the content of adjacent tweets in a related stream like users' Twitter timeline, topical, retweet and conversational stream. This approach significantly alleviates the context insufficiency problem caused due to the ambiguous and short nature of tweets BIBREF0 , BIBREF14 .
Question: How do they encourage understanding of literature as part of their objective function?
Answer: They group the existing works in terms of the objective function they optimize - within-tweet relationships, inter-tweet relationships, autoencoder, and weak supervision.
Paper: State-of-the-Art Vietnamese Word Segmentation
Lexical analysis, syntactic analysis, semantic analysis, disclosure analysis and pragmatic analysis are five main steps in natural language processing BIBREF0 , BIBREF1 . While morphology is a basic task in lexical analysis of English, word segmentation is considered a basic task in lexical analysis of Vietnamese and other East Asian languages processing. This task is to determine borders between words in a sentence. In other words, it is segmenting a list of tokens into a list of words such that words are meaningful.
Question: What are the limitations of existing Vietnamese word segmentation systems?
Paper: State-of-the-Art Vietnamese Word Segmentation
Vietnamese, like many languages in continental East Asia, is an isolating language and one branch of Mon-Khmer language group. The most basic linguistic unit in Vietnamese is morpheme, similar with syllable or token in English and “hình vị” (phoneme) or “tiếng” (syllable) in Vietnamese. According to the structured rule of its, Vietnamese can have about 20,000 different syllables (tokens). However, there are about 8,000 syllables used the Vietnamese dictionaries. There are three methods to identify morphemes in Vietnamese text BIBREF10 .
Question: Why challenges does word segmentation in Vietnamese pose?
Answer: Acquire very large Vietnamese corpus and build a classifier with it, design a develop a big data warehouse and analytic framework, build a system to incrementally learn new corpora and interactively process feedback.
Paper: State-of-the-Art Vietnamese Word Segmentation
In Vietnamese, not all of meaningful proper names are in the dictionary. Identifying proper names in input text are also important issue in word segmentation. This issue is sometimes included into unknown word issue to be solved. In addition, named entity recognition has to classify it into several types such as person, location, organization, time, money, number, and so on.
Question: How successful are the approaches used to solve word segmentation in Vietnamese?
Answer: Their accuracy in word segmentation is about 94%-97%.
Paper: State-of-the-Art Vietnamese Word Segmentation
In general, building corpus is carried out through four stages: (1) choose target of corpus and source of raw data; (2) building a guideline based on linguistics knowledge for annotation; (3) annotating or tagging corpus based on rule set in the guideline; and (4) reviewing corpus to check the consistency issue.
Question: Which approaches have been applied to solve word segmentation in Vietnamese?
Paper: A multi-layer approach to disinformation detection on Twitter
In recent years there has been increasing interest on the issue of disinformation spreading on online social media. Global concern over false (or "fake") news as a threat to modern democracies has been frequently raised–ever since 2016 US Presidential elections–in correspondence of events of political relevance, where the proliferation of manipulated and low-credibility content attempts to drive and influence people opinions BIBREF0BIBREF1BIBREF2BIBREF3.
Question: Which two news domains are country-independent?
Answer: mainstream news and disinformation
Paper: A multi-layer approach to disinformation detection on Twitter
In this work we formulate our classification problem as follows: given two classes of news articles, respectively $D$ (disinformation) and $M$ (mainstream), a set of news articles $A_i$ and associated class labels $C_i \in \lbrace D,M\rbrace $, and a set of tweets $\Pi _i=\lbrace T_i^1, T_i^2, ...\rbrace $ each of which contains an Uniform Resource Locator (URL) pointing explicitly to article $A_i$, predict the class $C_i$ of each article $A_i$. There is huge debate and controversy on a proper taxonomy of malicious and deceptive information BIBREF1BIBREF2BIBREF15BIBREF16BIBREF17BIBREF3BIBREF11. In this work we prefer the term disinformation to the more specific fake news to refer to a variety of misleading and harmful information. Therefore, we follow a source-based approach, a consolidated strategy also adopted by BIBREF6BIBREF16BIBREF2BIBREF1, in order to obtain relevant data for our analysis. We collected:
Question: How is the political bias of different sources included in the model?
Answer: By assigning a political bias label to each news article and training only on left-biased or right-biased outlets of both disinformation and mainstream domains
Paper: A multi-layer approach to disinformation detection on Twitter
We collected tweets associated to a dozen US mainstream news websites, i.e. most trusted sources described in BIBREF18, with the Streaming API, and we referred to Hoaxy API BIBREF16 for what concerns tweets containing links to 100+ US disinformation outlets. We filtered out articles associated to less than 50 tweets. The resulting dataset contains overall $\sim $1.7 million tweets for mainstream news, collected in a period of three weeks (February 25th, 2019-March 18th, 2019), which are associated to 6,978 news articles, and $\sim $1.6 million tweets for disinformation, collected in a period of three months (January 1st, 2019-March 18th, 2019) for sake of balance of the two classes, which hold 5,775 distinct articles. Diffusion censoring effects BIBREF14 were correctly taken into account in both collection procedures. We provide in Figure FIGREF4 the distribution of articles by source and political bias for both news domains.
Question: What are the two large-scale datasets used?
Paper: A multi-layer approach to disinformation detection on Twitter
For what concerns the Italian scenario we first collected tweets with the Streaming API in a 3-week period (April 19th, 2019-May 5th, 2019), filtering those containing URLs pointing to Italian official newspapers websites as described in BIBREF22; these correspond to the list provided by the association for the verification of newspaper circulation in Italy (Accertamenti Diffusione Stampa). We instead referred to the dataset provided by BIBREF23 to obtain a set of tweets, collected continuously since January 2019 using the same Twitter endpoint, which contain URLs to 60+ Italian disinformation websites. In order to get balanced classes (April 5th, 2019-May 5th, 2019), we retained data collected in a longer period w.r.t to mainstream news. In both cases we filtered out articles with less than 50 tweets; overall this dataset contains $\sim $160k mainstream tweets, corresponding to 227 news articles, and $\sim $100k disinformation tweets, corresponding to 237 news articles. We provide in Figure FIGREF5 the distribution of articles according to distinct sources for both news domains. As in the US dataset, we took into account censoring effects BIBREF14 by excluding tweets published before (left-censoring) or after two weeks (right-censoring) from the beginning of the collection process.
Question: What are the global network features which quantify different aspects of the sharing process?
Paper: Multi-class Hierarchical Question Classification for Multiple Choice Science Exams
Understanding what a question is asking is one of the first steps that humans use to work towards an answer. In the context of question answering, question classification allows automated systems to intelligently target their inference systems to domain-specific solvers capable of addressing specific kinds of questions and problem solving methods with high confidence and answer accuracy BIBREF0 , BIBREF1 .
Question: Which datasets are used for evaluation?