z-uo/qasper-squad huggingface dataset:
Paper: RaKUn: Rank-based Keyword extraction via Unsupervised learning and Meta vertex aggregation
In this work we consider directed graphs. Let INLINEFORM0 represent a graph comprised of a set of vertices INLINEFORM1 and a set of edges ( INLINEFORM2 ), which are ordered pairs. Further, each edge can have a real-valued weight assigned. Let INLINEFORM3 represent a document comprised of tokens INLINEFORM4 . The order in which tokens in text appear is known, thus INLINEFORM5 is a totally ordered set. A potential way of constructing a graph from a document is by simply observing word co-occurrences. When two words co-occur, they are used as an edge. However, such approaches do not take into account the sequence nature of the words, meaning that the order is lost. We attempt to take this aspect into account as follows. The given corpus is traversed, and for each element INLINEFORM6 , its successor INLINEFORM7 , together with a given element, forms a directed edge INLINEFORM8 . Finally, such edges are weighted according to the number of times they appear in a given corpus. Thus the graph, constructed after traversing a given corpus, consists of all local neighborhoods (order one), merged into a single joint structure. Global contextual information is potentially kept intact (via weights), even though it needs to be detected via network analysis as proposed next.
Question: In what sense if the proposed method interpretable?
Paper: Semi-supervised sequence tagging with bidirectional language models
Due to their simplicity and efficacy, pre-trained word embedding have become ubiquitous in NLP systems. Many prior studies have shown that they capture useful semantic and syntactic information BIBREF0 , BIBREF1 and including them in NLP systems has been shown to be enormously helpful for a variety of downstream tasks BIBREF2 .
Question: how are the bidirectional lms obtained?
Answer: They pre-train forward and backward LMs separately, remove top layer softmax, and concatenate to obtain the bidirectional LMs.
Paper: Semi-supervised sequence tagging with bidirectional language models
The main components in our language-model-augmented sequence tagger (TagLM) are illustrated in Fig. FIGREF4 . After pre-training word embeddings and a neural LM on large, unlabeled corpora (Step 1), we extract the word and LM embeddings for every token in a given input sequence (Step 2) and use them in the supervised sequence tagging model (Step 3).
Question: what metrics are used in evaluation?
Answer: micro-averaged F1
Paper: Semi-supervised sequence tagging with bidirectional language models
Our baseline sequence tagging model is a hierarchical neural tagging model, closely following a number of recent studies BIBREF4 , BIBREF5 , BIBREF3 , BIBREF8 (left side of Figure FIGREF5 ).
Question: what results do they achieve?
Answer: 91.93% F1 score on CoNLL 2003 NER task and 96.37% F1 score on CoNLL 2000 Chunking task
Paper: Semi-supervised sequence tagging with bidirectional language models
A language model computes the probability of a token sequence INLINEFORM0 INLINEFORM1 
Question: what previous systems were compared to?
Answer: Chiu and Nichols (2016), Lample et al. (2016), Ma and Hovy (2016), Yang et al. (2017), Hashimoto et al. (2016), Søgaard and Goldberg (2016)
Paper: Semi-supervised sequence tagging with bidirectional language models
Our combined system, TagLM, uses the LM embeddings as additional inputs to the sequence tagging model. In particular, we concatenate the LM embeddings INLINEFORM0 with the output from one of the bidirectional RNN layers in the sequence model. In our experiments, we found that introducing the LM embeddings at the output of the first layer performed the best. More formally, we simply replace ( EQREF6 ) with DISPLAYFORM0 
Question: what are the evaluation datasets?
Paper: Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
The design and discovery of novel drugs for protein targets is powered by an understanding of the underlying principles of protein-compound interaction. Biochemical methods that measure affinity and biophysical methods that describe the interaction in atomistic level detail have provided valuable information toward a mechanistic explanation for bimolecular recognition BIBREF0. However, more often than not, compounds with drug potential are discovered serendipitously or by phenotypic drug discovery BIBREF1 since this highly specific interaction is still difficult to predict BIBREF2. Protein structure based computational strategies such as docking BIBREF3, ultra-large library docking for discovering new chemotypes BIBREF4, and molecular dynamics simulations BIBREF3 or ligand based strategies such as quantitative structure-activity relationship (QSAR) BIBREF5, BIBREF6, and molecular similarity BIBREF7 have been powerful at narrowing down the list of compounds to be tested experimentally. With the increase in available data, machine learning and deep learning architectures are also starting to play a significant role in cheminformatics and drug discovery BIBREF8. These approaches often require extensive computational resources or they are limited by the availability of 3D information. On the other hand, text based representations of biochemical entities are more readily available as evidenced by the 19,588 biomolecular complexes (3D structures) in PDB-Bind BIBREF9 (accessed on Nov 13, 2019) compared with 561,356 (manually annotated and reviewed) protein sequences in Uniprot BIBREF10 (accessed on Nov 13, 2019) or 97 million compounds in Pubchem BIBREF11 (accessed on Nov 13, 2019). The advances in natural language processing (NLP) methodologies make processing of text based representations of biomolecules an area of intense research interest.
Question: Are datasets publicly available?
Paper: Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
BIBREF20 describes NLP on three levels: (i) the word level in which the smallest meaningful unit is extracted to define the morphological structure, (ii) the sentence level where grammar and syntactic validity are determined, and (iii) the domain or context level in which the sentences have global meaning. Similarly, our review is organized in three parts in which bio-chemical data is investigated at: (i) word level, (ii) sentence (text) level, and (iii) understanding text and generating meaningful sequences. Table TABREF37 summarizes important NLP concepts related to the processing of biochemical data. We refer to these concepts and explain their applications in the following sections.
Question: Are this models usually semi/supervised or unsupervised?
Answer: Both supervised and unsupervised, depending on the task that needs to be solved.
Paper: Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
The language-like properties of text-based representations of chemicals were recognized more than 50 years ago by Garfield BIBREF21. He proposed a “chemico-linguistic" approach to representing chemical nomenclature with the aim of instructing the computer to draw chemical diagrams. Protein sequence has been an important source of information about protein structure and function since Anfinsen's experiment BIBREF22. Alignment algorithms, such as Needleman-Wunsh BIBREF23 and Smith-Waterman BIBREF24, rely on sequence information to identify functionally or structurally critical elements of proteins (or genes).
Question: Is there any concrete example in the paper that shows that this approach had huge impact on drug discovery?
Paper: Inducing Interpretability in Knowledge Graph Embeddings
Knowledge Graphs such as Freebase, WordNet etc. have become important resources for supporting many AI applications like web search, Q&A etc. They store a collection of facts in the form of a graph. The nodes in the graph represent real world entities such as Roger Federer, Tennis, United States etc while the edges represent relationships between them.
Question: Do the authors analyze what kinds of cases their new embeddings fail in where the original, less-interpretable embeddings didn't?
Paper: Inducing Interpretability in Knowledge Graph Embeddings
Several methods have been proposed for learning KG embeddings. They differ on the modeling of entities and relations, usage of text data and interpretability of the learned embeddings. We summarize some of these methods in following sections.
Question: When they say "comparable performance", how much of a performance drop do these new embeddings result in?
Answer: Performance was comparable, with the proposed method quite close and sometimes exceeding performance of baseline method.
Paper: Inducing Interpretability in Knowledge Graph Embeddings
A very effective and powerful set of models are based on translation vectors. These models represent entities as vectors in $d$ -dimensional space, $\mathbb {R}^d$ and relations as translation vectors from head entity to tail entity, in either same or a projected space. TransE BIBREF2 is one of the initial works, which was later improved by many works [ BIBREF3 , BIBREF4 , BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 ]. Also, there are methods which are able to incorporate text data while learning KG embeddings. BIBREF0 is one such method, which assumes a combined universal schema of relations from KG as well as text. BIBREF1 further improves the performance by sharing parameters among similar textual relations.
Question: How do they evaluate interpretability?
Paper: CA-EHN: Commonsense Word Analogy from E-HowNet
Commonsense reasoning is fundamental for natural language agents to generalize inference beyond their training corpora. Although the natural language inference (NLI) task BIBREF0 , BIBREF1 has proved a good pre-training objective for sentence representations BIBREF2 , commonsense coverage is limited and most models are still end-to-end, relying heavily on word representations to provide background world knowledge.
Question: What types of word representations are they evaluating?
Answer: GloVE; SGNS
Paper: Encoding Word Confusion Networks with Recurrent Neural Networks for Dialog State Tracking
Spoken dialog systems (SDSs) allow users to naturally interact with machines through speech and are nowadays an important research direction, especially with the great success of automatic speech recognition (ASR) systems BIBREF0 , BIBREF1 . SDSs can be designed for generic purposes, e.g. smalltalk BIBREF2 , BIBREF3 ) or a specific task such as finding restaurants or booking flights BIBREF4 , BIBREF5 . Here, we focus on task-oriented dialog systems, which assist the users to reach a certain goal.
Question: What type of recurrent layers does the model use?
Answer: GRU
Paper: Encoding Word Confusion Networks with Recurrent Neural Networks for Dialog State Tracking
Our model depicted in Figure FIGREF3 is based on an incremental DST system BIBREF12 . It consists of an embedding layer for the words in the system and user utterances, followed by a fully connected layer composed of Rectified Linear Units (ReLUs) BIBREF17 , which yields the input to a recurrent layer to encode the system and user outputs in each turn with a softmax classifier on top. INLINEFORM0 denotes a weighted sum INLINEFORM1 of the system dialog act INLINEFORM2 and the user utterance INLINEFORM3 , where INLINEFORM4 , and INLINEFORM5 are learned parameters: DISPLAYFORM0 
Question: What is a word confusion network?
Answer: It is a network used to encode speech lattices to maintain a rich hypothesis space.
Paper: Semantic Enrichment of Streaming Healthcare Data
Since the early 1970s, healthcare information technology has moved toward comprehensive electronic medical records (EMR) in which almost every aspect of the patient's healthcare has been digitized and retained indefinitelyBIBREF0, which has vastly improved the efficiency with which patient information can be retained, communicated, and analyzed. At the same time, the healthcare industry has moved from a fee-for-service model to a value-based model, facilitated in part by the existence of such a record and in part by public policy, such as the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009 BIBREF1, which provided financial incentives for the "meaningful use" of electronic medical records.
Question: What type of simulations of real-time data feeds are used for validaton?
Paper: Semantic Enrichment of Streaming Healthcare Data
HL7v2 is a healthcare messaging standard developed by the standards organization Health Level Seven International. It first emerged in 1988 and today is the most widely used such standard, having been adopted by over ninety-five percent of health systems in the United States and thirty-five countries worldwide BIBREF4. As such, it is something of a universal medium in the field of healthcare interoperability, yet it is terse and, without specialized training and access to the standard reference, cryptic.
Question: How are FHIR and RDF combined?
Paper: Semantic Enrichment of Streaming Healthcare Data
FHIR BIBREF5 is a new open standard for healthcare data developed by the same company that developed HL7v2. However, whereas HL7v2 uses an idiosyncratic data exchange format, FHIR uses data exchange formats based on those already in wide use on the World-Wide Web such as Extensible Markup Language (XML) and JavaScript Object Notation (JSON) BIBREF6, as well as the web's familiar transfer control protocols such as HyperText Transfer Protocol Secure (HTTPS) and Representational State Transfer (REST) BIBREF6 and system of contextual hyperlinks implemented with Uniform Resource Locators / Identifiers (URL/URI) BIBREF7. This design choice simplifies interoperability and discoverability and enables applications to be built rapidly on top of FHIR by the large number of engineers already familiar with web application design without a steep learning curve.
Question: What are the differences between FHIR and RDF?
Paper: Semantic Enrichment of Streaming Healthcare Data
The term Semantic Web BIBREF8 denotes an interconnected machine-readable network of information. In some ways it is analogous to the World-Wide Web, but with some crucial differences. The most important similarity is in the vision for the two technologies: Like the World-Wide Web, the Semantic Web was envisioned as a way for users from different institutions, countries, disciplines, etc. to exchange information openly and in doing so to add to the sum of human knowledge. The difference, however, is in the different emphases put on human readability versus machine readability: Whereas the World-Wide Web was intended to be visually rendered by one of any number of web browsers before being read by humans and therefore prioritizes fault tolerance and general compatibility over precision, the semantic web prioritizes precision and logical rigor in order for the information contained in it to be machine readable and used for logical inference.
Question: What do FHIR and RDF stand for?
Paper: Question Generation by Transformers
Existing question generating systems reported in the literature involve human-generated templates, including cloze type BIBREF0, rule-based BIBREF1, BIBREF2, or semi-automatic questions BIBREF3, BIBREF4, BIBREF5. On the other hand, machine learned models developed recently have used recurrent neural networks (RNNs) to perform sequence transduction, i.e. sequence-to-sequence BIBREF6, BIBREF7. In this work, we investigated an automatic question generation system based on a machine learning model that uses transformers instead of RNNs BIBREF8, BIBREF9. Our goal was to generate questions without templates and with minimal human involvement using machine learning transformers that have been demonstrated to train faster and better than RNNs. Such a system would benefit educators by saving time to generate quizzes and tests.
Question: What is the motivation behind the work? Why question generation is an important task?
Paper: Question Generation by Transformers
A relatively simple method for question generation is the fill-in-the-blank approach, which is also known as cloze tasks. Such a method typically involves the sentence first being tokenized and tagged for part-of-speech with the named entity or noun part of the sentence masked out. These generated questions are an exact match to the one in the reading passage except for the missing word or phrase. Although fill-in-the-blank questions are often used for reading comprehension, answering such questions correctly may not necessarily indicate comprehension if it is too easy to match the question to the relevant sentence in the passage. To improve fill in the blank type questions, a prior study used a supervised machine learning model to generate fill-in-the-blank type questions. The model paraphrases the sentence from the passage with the missing word by anonymizing entity markers BIBREF0.
Question: Why did they choose WER as evaluation metric?
Paper: TutorialVQA: Question Answering Dataset for Tutorial Videos
Video is the fastest growing medium to create and deliver information today. Consequentially, videos have been increasingly used as main data sources in many question answering problems BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4, BIBREF2, BIBREF5. These previous studies have mostly focused on factoid questions, each of which can be answered in a few words or phrases generated by understanding multimodal contents in a short video clip.
Question: What evaluation metrics were used in the experiment?
Answer: For sentence-level prediction they used tolerance accuracy, for segment retrieval accuracy and MRR and for the pipeline approach they used overall accuracy
Paper: TutorialVQA: Question Answering Dataset for Tutorial Videos
Most relevant to our proposed work is the reading comprehension task, which is a question answering task involving a piece of text such as a paragraph or article. Such datasets for the reading comprehension task, such as SQuAD BIBREF6 based on Wikipedia, TriviaQA BIBREF7 constructed from trivia questions with answer evidence from Wikipedia, or those from Hermann et al. based on CNN and Daily Mail articles BIBREF8 are factoid-based, meaning the answers typically involve a single entity. Differing from video transcripts, the structures of these data sources, namely paragraphs from Wikipedia and news sources, are typically straightforward since they are meant to be read. In contrast, video transcripts originate from spoken dialogue, which can verbose, unstructured, and disconnected. Furthermore, the answers in instructional video transcripts can be longer, spanning multiple sentences if the process is multi-step or even fragmented into multiple segments throughout the video.
Question: What kind of instructional videos are in the dataset?
Answer: tutorial videos for a photo-editing software
Paper: TutorialVQA: Question Answering Dataset for Tutorial Videos
In this section, we introduce the TutorialVQA dataset and describe the data collection process .
Question: What baseline algorithms were presented?
Answer: a sentence-level prediction algorithm, a segment retrieval algorithm and a pipeline segment retrieval algorithm
Paper: TutorialVQA: Question Answering Dataset for Tutorial Videos
Our dataset consists of 76 tutorial videos pertaining to an image editing software. All of the videos include spoken instructions which are transcribed and manually segmented into multiple segments. Specifically, we asked the annotators to manually divide each video into multiple segments such that each of the segments can serve as an answer to any question. For example, Fig. FIGREF1 shows example segments marked in red (each which are a complete unit as an answer span). Each sentence is associated with the starting and ending time-stamps, which can be used to access the relevant visual information.
Question: What is the source of the triples?
Paper: A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction
Aspect-based sentiment analysis BIBREF0, BIBREF1, BIBREF2 (ABSA) is a fine-grained task compared with traditional sentiment analysis, which requires the model to be able to automatic extract the aspects and predict the polarities of all the aspects. For example, given a restaurant review: "The dessert at this restaurant is delicious but the service is poor," the full-designed model for ABSA needs to extract the aspects "dessert" and "service" and correctly reason about their polarity. In this review, the consumers' opinions on "dessert" and "service" are not consistent, with positive and negative sentiment polarity respectively.
Question: How much better is performance of the proposed model compared to the state of the art in these various experiments?
Paper: A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction
Most ABSA-oriented methodologies regard the ATE and the APC as independent tasks and major in one of them. Accordingly, this section will introduce the related works of ATE and APC in two parts.
Question: What was state of the art on SemEval-2014 task4 Restaurant and Laptop dataset?
Paper: A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction
The approaches to ATE tasks are classified into two categories: the early dictionary-based or rule-based approaches, and methodologies based on machine-learning or deep learning. BIBREF17 proposed a new rule-based approach to extracting aspects from product reviews using common sense and sentence dependency trees to detect explicit and implicit aspects. BIBREF18 adopts an unsupervised and domain-independent aspect extraction method that relies on syntactic dependency rules and can selects rules automatically.
Question: What was previous state-of-the-art on four Chinese reviews datasets?
Paper: A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction
Aspect polarity classification is another important subtask of ABSA. The approaches designed for the APC task can be categorized into traditional machine learning and recent deep learning methods.The APC task has been comprehensively turned to the the deep neural networks. Therefore, this section mainly introduces approaches based on deep learning techniques.
Question: In what four Chinese review datasets does LCF-ATEPC achieves state of the art?
Paper: A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction
Aspect-based sentiment analysis relies on the targeted aspects, and most existing studies focus on the classification of aspect polarity, leaving the problem of aspect term extraction. To propose an effective aspect-based sentiment analysis model based on multi-task learning, we adopted domain-adapted BERT model from BERT-ADA and integrated the local context focus mechanism into the proposed model. This section introduces the architecture and methodology of LCF-ATEPC.
Question: Why authors think that researches do not pay attention to the research of the Chinese-oriented ABSA task?
Paper: A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction
Similar to name entity recognition (NER) task, the ATE task is a kind of sequence labeling task, and prepare the input based on IOB labels. We design the IOB labels as $B_{asp}, I_{asp}, O$, and the labels indicate the beginning, inside and outside of the aspect terms, respectively. For ATE task, the input of the example review “The price is reasonable although the service is poor.” will be prepared as $S=\lbrace w_1,w_2 \cdots w_n\rbrace $, and $w$ stands for a token after tokenization, $n=10$ is the total number of tokens. The example will be labeled in $Y=\lbrace O, B_{asp}, O, O, O, O, B_{asp}, O, O, O\rbrace $.
Question: What is specific to Chinese-oriented ABSA task, how is it different from other languages?