z-uo/qasper-squad huggingface dataset:
Paper: Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks
Machine Learning, in general, and affective computing, in particular, rely on good data representations or features that have a good discriminatory faculty in classification and regression experiments, such as emotion recognition from speech. To derive efficient representations of data, researchers have adopted two main strategies: (1) carefully crafted and tailored feature extractors designed for a particular task BIBREF0 and (2) algorithms that learn representations automatically from the data itself BIBREF1 . The latter approach is called Representation Learning (RL), and has received growing attention in the past few years and is highly reliant on large quantities of data. Most approaches for emotion recognition from speech still rely on the extraction of standard acoustic features such as pitch, shimmer, jitter and MFCCs (Mel-Frequency Cepstral Coefficients), with a few notable exceptions BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 . In this work we leverage RL strategies and automatically learn representations of emotional speech from the spectrogram directly using a deep convolutional neural network (CNN) architecture.
Question: What model achieves state of the art performance on this task?
Paper: Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks
The proposed model builds upon previous results in the field of emotion recognition, and leverages prior work in representation learning.
Question: Which multitask annotated corpus is used?
Paper: Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks
The investigated multitask model is based upon the DCGAN architecture described in Section SECREF2 and is implemented in TensorFlow. For emotion classification a fully connected layer is attached to the final convolutional layer of the DCGAN's discriminator. The output of this layer is then fed to two separate fully connected layers, one of which outputs a valence label and the other of which outputs an activation label. This setup is shown visually in Figure FIGREF4 . Through this setup, the model is able to take advantage of unlabeled data during training by feeding it through the DCGAN layers in the model, and is also able to take advantage of multitask learning and train the valence and activation outputs simultaneously.
Question: What are the tasks in the multitask learning setup?
Paper: Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks
Due to the semi-supervised nature of the proposed Multitask DCGAN model, we utilize both labeled and unlabeled data. For the unlabeled data, we use audio from the AMI BIBREF8 and IEMOCAP BIBREF7 datasets. For the labeled data, we use audio from the IEMOCAP dataset, which comes with labels for activation and valence, both measured on a 5-point Likert scale from three distinct annotators. Although IEMOCAP provides per-word activation and valence labels, in practice these labels do not generally change over time in a given audio file, and so for simplicity we label each audio clip with the average valence and activation. Since valence and activation are both measured on a 5-point scale, the labels are encoded in 5-element one-hot vectors. For instance, a valence of 5 is represented with the vector INLINEFORM0 . The one-hot encoding can be thought of as a probability distribution representing the likelihood of the correct label being some particular value. Thus, in cases where the annotators disagree on the valence or activation label, this can be represented by assigning probabilities to multiple positions in the label vector. For instance, a label of 4.5 conceptually means that the “correct” valence is either 4 or 5 with equal probability, so the corresponding vector would be INLINEFORM1 . These “fuzzy labels” have been shown to improve classification performance in a number of applications BIBREF14 , BIBREF15 . It should be noted here that we had generally greater success with this fuzzy label method than training the neural network model on the valence label directly, i.e. classification task vs. regression.
Question: What are the subtle changes in voice which have been previously overshadowed?
Paper: Subword-augmented Embedding for Cloze Reading Comprehension
This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http://creativecommons.org/licenses/by/4.0/
Question: how are rare words defined?
Paper: Subword-augmented Embedding for Cloze Reading Comprehension
The concerned reading comprehension task can be roughly categorized as user-query type and cloze-style according to the answer form. Answers in the former are usually a span of texts while in the cloze-style task, the answers are words or phrases which lets the latter be the harder-hit area of OOV issues, inspiring us to select the cloze-style as our testbed for SAW strategies. Our preliminary study shows even for the advanced word-character based GA reader, OOV answers still account for nearly 1/5 in the error results. This also motivates us to explore better representations to further performance improvement.
Question: which public datasets were used?
Paper: Subword-augmented Embedding for Cloze Reading Comprehension
Word in most languages usually can be split into meaningful subword units despite of the writing form. For example, “indispensable" could be split into the following subwords: INLINEFORM0 .
Question: what are the baselines?
Answer: AS Reader, GA Reader, CAS Reader
Paper: Kurdish (Sorani) Speech to Text: Presenting an Experimental Dataset
Kurdish language processing requires endeavor by interested researchers and scholars to overcome with a large gap which it has regarding the resource scarcity. The areas that need attention and the efforts required have been addressed in BIBREF0.
Question: What are the results of the experiment?
Answer: They were able to create a language model from the dataset, but did not test.
Paper: Kurdish (Sorani) Speech to Text: Presenting an Experimental Dataset
The work on Automatic Speech Recognition (ASR) has a long history, but we could not retrieve any literature on Kurdish ASR at the time of compiling this article. However, the literature on ASR for different languages is resourceful. Also, researchers have widely used CMUSphinx for ASR though other technologies have been emerging in recent years BIBREF1.
Question: How was the dataset collected?
Answer: extracted text from Sorani Kurdish books of primary school and randomly created sentences
Paper: Kurdish (Sorani) Speech to Text: Presenting an Experimental Dataset
To develop the dataset, we extracted 200 sentences from Sorani Kurdish books of grades one to three of the primary school in the Kurdistan Region of Iraq. We randomly created 2000 sentences from the extracted sentences.
Question: What is the size of the dataset?
Paper: Kurdish (Sorani) Speech to Text: Presenting an Experimental Dataset
The phoneset includes 34 phones for Sorani Kurdish. A sample of the file content is given below.
Question: How many different subjects does the dataset contain?
Paper: Kurdish (Sorani) Speech to Text: Presenting an Experimental Dataset
The filler phone file usually contains fillers in spoken sentences. In our basic sentences, we have only considered silence. Therefore it only includes three lines to indicate the possible pauses at the beginning and end of the sentences and also after each word.
Question: How many annotators participated?
Answer: 1
Paper: Kurdish (Sorani) Speech to Text: Presenting an Experimental Dataset
This file includes the list of files in which the narrated sentences have been recorded. The recorded files are in wav formats. However, in the file IDs, the extension is omitted. A sample of the file content is given below. The test directory is the directory in which the files are located.
Question: How long is the dataset?
Paper: Cohesion and Coalition Formation in the European Parliament: Roll-Call Votes and Twitter Activities
We study the cohesion within and the coalitions between political groups in the Eighth European Parliament (2014–2019) by analyzing two entirely different aspects of the behavior of the Members of the European Parliament (MEPs) in the policy-making processes. On one hand, we analyze their co-voting patterns and, on the other, their retweeting behavior. We make use of two diverse datasets in the analysis. The first one is the roll-call vote dataset, where cohesion is regarded as the tendency to co-vote within a group, and a coalition is formed when the members of several groups exhibit a high degree of co-voting agreement on a subject. The second dataset comes from Twitter; it captures the retweeting (i.e., endorsing) behavior of the MEPs and implies cohesion (retweets within the same group) and coalitions (retweets between groups) from a completely different perspective.
Question: Do the authors mention any possible confounds in their study?
Paper: Cohesion and Coalition Formation in the European Parliament: Roll-Call Votes and Twitter Activities
Social-media activities often reflect phenomena that occur in other complex systems. By observing social networks and the content propagated through these networks, we can describe or even predict the interplay between the observed social-media activities and another complex system that is more difficult, if not impossible, to monitor. There are numerous studies reported in the literature that successfully correlate social-media activities to phenomena like election outcomes BIBREF0 , BIBREF1 or stock-price movements BIBREF2 , BIBREF3 .
Question: What is the relationship between the co-voting and retweeting patterns?
Paper: Cohesion and Coalition Formation in the European Parliament: Roll-Call Votes and Twitter Activities
In this paper we study and relate two very different aspects of how MEPs behave in policy-making processes. First, we look at their co-voting behavior, and second, we examine their retweeting patterns. Thus, we draw related work from two different fields of science. On one hand, we look at how co-voting behavior is analyzed in the political-science literature and, on the other, we explore how Twitter is used to better understand political and policy-making processes. The latter has been more thoroughly explored in the field of data mining (specifically, text mining and network analysis).
Question: Does the analysis find that coalitions are formed in the same way for different policy areas?
Paper: Cohesion and Coalition Formation in the European Parliament: Roll-Call Votes and Twitter Activities
In this section we present the methods to quantify cohesion and coalitions from the roll-call votes and Twitter activities.
Question: What insights does the analysis give about the cohesion of political groups in the European parliament?
Paper: Cohesion and Coalition Formation in the European Parliament: Roll-Call Votes and Twitter Activities
We first show how the co-voting behaviour of MEPs can be quantified by a measure of the agreement between them. We treat individual RCVs as observations, and MEPs as independent observers or raters. When they cast the same vote, there is a high level of agreement, and when they vote differently, there is a high level of disagreement. We define cohesion as the level of agreement within a political group, a coalition as a voting agreement between political groups, and opposition as a disagreement between different groups.
Question: Do they authors account for differences in usage of Twitter amongst MPs into their model?
Paper: Cohesion and Coalition Formation in the European Parliament: Roll-Call Votes and Twitter Activities
In this section we describe a network-based approach to analyzing the co-voting behavior of MEPs. For each roll-call vote we form a network, where the nodes in the network are MEPs, and an undirected edge between two MEPs is formed when they cast the same vote.
Question: Did the authors examine if any of the MEPs used the disclaimer that retweeting does not imply endorsement on their twitter profile?
Paper: Neural Language Modeling by Jointly Learning Syntax and Lexicon
Linguistic theories generally regard natural language as consisting of two part: a lexicon, the complete set of all possible words in a language; and a syntax, the set of rules, principles, and processes that govern the structure of sentences BIBREF0 . To generate a proper sentence, tokens are put together with a specific syntactic structure. Understanding a sentence also requires lexical information to provide meanings, and syntactical knowledge to correctly combine meanings. Current neural language models can provide meaningful word represent BIBREF1 , BIBREF2 , BIBREF3 . However, standard recurrent neural networks only implicitly model syntax, thus fail to efficiently use structure information BIBREF4 .
Question: How do they show their model discovers underlying syntactic structure?
Answer: By visualizing syntactic distance estimated by the parsing network
Paper: Neural Language Modeling by Jointly Learning Syntax and Lexicon
The idea of introducing some structures, especially trees, into language understanding to help a downstream task has been explored in various ways. For example, BIBREF5 , BIBREF4 learn a bottom-up encoder, taking as an input a parse tree supplied from an external parser. There are models that are able to infer a tree during test time, while still need supervised signal on tree structure during training. For example, BIBREF18 , BIBREF19 , BIBREF20 , BIBREF21 , etc. Moreover, BIBREF22 did an in-depth analysis of recursive models that are able to learn tree structure without being exposed to any grammar trees. Our model is also able to infer tree structure in an unsupervised setting, but different from theirs, it is a recurrent network that implicitly models tree structure through attention.
Question: Which dataset do they experiment with?
Paper: Neural Language Modeling by Jointly Learning Syntax and Lexicon
Suppose we have a sequence of tokens INLINEFORM0 governed by the tree structure showed in Figure FIGREF4 . The leafs INLINEFORM1 are observed tokens. Node INLINEFORM2 represents the meaning of the constituent formed by its leaves INLINEFORM3 , where INLINEFORM4 and INLINEFORM5 stands for the leftmost child and right most child. Root INLINEFORM6 represents the meaning of the whole sequence. Arrows represent the dependency relations between nodes. The underlying assumption is that each node depends only on its parent and its left siblings.
Question: How do they measure performance of language model tasks?
Answer: BPC, Perplexity
Paper: Extracting information from free text through unsupervised graph-based clustering: an application to patient incident records
Question: How are content clusters used to improve the prediction of incident severity?
Answer: they are used as additional features in a supervised classification task
Paper: Extracting information from free text through unsupervised graph-based clustering: an application to patient incident records
The full dataset includes more than 13 million confidential reports of patient safety incidents reported to the National Reporting and Learning System (NRLS) between 2004 and 2016 from NHS trusts and hospitals in England and Wales. Each record has more than 170 features, including organisational details (e.g., time, trust code and location), anonymised patient information, medication and medical devices, among many other details. In most records, there is also a detailed description of the incident in free text, although the quality of the text is highly variable.
Question: What cluster identification method is used in this paper?
Answer: A combination of Minimum spanning trees, K-Nearest Neighbors and Markov Stability BIBREF15, BIBREF16, BIBREF17, BIBREF18
Paper: Question Answering from Unstructured Text by Retrieval and Comprehension
Natural language based consumer products, such as Apple Siri and Amazon Alexa, have found wide spread use in the last few years. A key requirement for these conversational systems is the ability to answer factual questions from the users, such as those about movies, music, and artists.
Question: How can a neural model be used for a retrieval if the input is the entire Wikipedia?
Paper: UDS--DFKI Submission to the WMT2019 Similar Language Translation Shared Task
The shared tasks organized annually at WMT provide important benchmarks used in the MT community. Most of these shared tasks include English data, which contributes to make English the most resource-rich language in MT and NLP. In the most popular WMT shared task for example, the News task, MT systems have been trained to translate texts from and to English BIBREF0, BIBREF1.
Question: Which algorithm is used in the UDS-DFKI system?
Paper: UDS--DFKI Submission to the WMT2019 Similar Language Translation Shared Task
With the widespread use of MT technology and the commercial and academic success of NMT, there has been more interest in training systems to translate between languages other than English BIBREF5. One reason for this is the growing need of direct translation between pairs of similar languages, and to a lesser extent language varieties, without the use of English as a pivot language. The main challenge is to overcome the limitation of available parallel data taking advantage of the similarity between languages. Studies have been published on translating between similar languages (e.g. Catalan - Spanish BIBREF5) and language varieties such as European and Brazilian Portuguese BIBREF6, BIBREF7. The study by lakew2018neural tackles both training MT systems to translate between European–Brazilian Portuguese and European–Canadian French, and two pairs of similar languages Croatian–Serbian and Indonesian–Malay.
Question: Does the use of out-of-domain data improve the performance of the method?
Paper: Exploration on Generating Traditional Chinese Medicine Prescriptions from Symptoms with an End-to-End Approach
Traditional Chinese Medicine (TCM) is one of the most important forms of medical treatment in China and the surrounding areas. TCM has accumulated large quantities of documentation and therapy records in the long history of development. Prescriptions consisting of herbal medication are the most important form of TCM treatment. TCM practitioners prescribe according to a patient's symptoms that are observed and analyzed by the practitioners themselves instead of using medical equipment, e.g., the CT. The patient takes the decoction made out of the herbal medication in the prescription. A complete prescription includes the composition of herbs, the proportion of herbs, the preparation method and the doses of the decoction. In this work, we focus on the composition part of the prescription, which is the most essential part of the prescription.
Question: Do they impose any grammatical constraints over the generated output?
Paper: Exploration on Generating Traditional Chinese Medicine Prescriptions from Symptoms with an End-to-End Approach
There has not been much work concerning computational TCM. zhou2010development attempted to build a TCM clinical data warehouse so that the TCM knowledge can be analyzed and used. This is a typical way of collecting data, since the number of prescriptions given by the practitioners in the clinics is very large. However, in reality, most of the TCM doctors do not refer to the constructed digital systems, because the quality of the input data tends to be poor. Therefore, we choose prescriptions in the classics (books or documentation) of TCM. Although the available data can be fewer than the clinical data, it guarantees the quality of the prescriptions.
Question: Why did they think this was a good idea?
Answer: They think it will help human TCM practitioners make prescriptions.
Paper: Grounding the Semantics of Part-of-Day Nouns Worldwide using Twitter
Human languages are intertwined with their cultures and societies, having evolved together, reflecting them and in turn shaping them BIBREF0 , BIBREF1 . Part-of-day nouns (e.g. ‘morning’ or ‘night’) are an example of this, as their meaning depends on how each language's speakers organize their daily schedule. For example, while the morning in English-speaking countries is assumed to end at noon, the Spanish term (‘mañana’) is understood to span until lunch time, which normally takes place between 13:00 and 15:00 in Spain. It is fair to relate this difference to cultural (lunch being the main meal of the day in Spain, as opposed to countries like the uk, and therefore being a milestone in the daily timetable) and sociopolitical factors (the late lunch time being influenced by work schedules and the displacement of the Spanish time zones with respect to solar time). Similar differences have been noted for different pairs of languages BIBREF2 and for cultures using the same language BIBREF3 , based on manual study, field research and interviews with natives. Work on automatically extracting the semantics of part-of-day nouns is scarce, as classic corpora are not timestamped. Reiter2003a,Reiter2003b overcome it by analyzing weather forecasts and aligning them to timestamped simulations, giving approximate groundings for time-of-day nouns and showing idiolectal variation on the term ‘evening’, but the work is limited to English.
Question: How many languages are included in the tweets?
Paper: Grounding the Semantics of Part-of-Day Nouns Worldwide using Twitter
To ground the semantics of greetings we used 5 terms as seeds: ‘good morning’, ‘good afternoon’, ‘good evening’, ‘good night’ and ‘hello’ (a time-unspecific greeting used for comparison). We translated them to 53 languages and variants using Bing translator. We use italics to refer to greetings irrespective of the language. 172,802,620 tweets were collected from Sept. 2 to Dec. 7 2016.
Question: What languages are explored?
Paper: Grounding the Semantics of Part-of-Day Nouns Worldwide using Twitter
Given a country, some of the tweets are written in foreign languages for reasons like tourism or immigration. This paper refers to tweets written in official or de facto languages, unless otherwise specified. Also, analyzing differences according to criteria such as gender or solar time can be relevant. As determining the impact of all those is a challenge on its own, we focus on the primary research question: can we learn semantics of the part-of-day nouns from simple analysis of tweets? To verify data quality, good morning tweets were revised: out of 1 000 random tweets from the usa, 97.9% were legitimate greetings and among the rest, some reflected somehow that the user just started the day (e.g ‘Didn't get any good morning sms’). We did the same for Spain (98,1% legitimate), Brazil (97.8%) and India (99.6%).
Question: Which countries did they look at?
Paper: QA4IE: A Question Answering based Framework for Information Extraction
Information Extraction (IE), which refers to extracting structured information (i.e., relation tuples) from unstructured text, is the key problem in making use of large-scale texts. High quality extracted relation tuples can be used in various downstream applications such as Knowledge Base Population BIBREF0 , Knowledge Graph Acquisition BIBREF1 , and Natural Language Understanding. However, existing IE systems still cannot produce high-quality relation tuples to effectively support downstream applications.
Question: What QA models were used?
Answer: A pointer network decodes the answer from a bidirectional LSTM with attention flow layer and self-matching layer, whose inputs come from word and character embeddings of the query and input text fed through a highway layer.
Paper: QA4IE: A Question Answering based Framework for Information Extraction
Most of previous IE systems can be divided into Relation Extraction (RE) based systems BIBREF2 , BIBREF3 and Open IE systems BIBREF4 , BIBREF5 , BIBREF6 .
Question: Can this approach model n-ary relations?