z-uo/qasper-squad huggingface dataset:
Paper: Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation
This section describes our evaluation of the proposed approach for interpreting navigation commands in natural language. We provide both quantitative and qualitative results.
Question: How were the navigation instructions collected?
Answer: using Amazon Mechanical Turk using simulated environments with topological maps
Paper: Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation
While computing evaluation metrics, we only consider the behaviors present in the route because they are sufficient to recover the high-level navigation plan from the graph. Our metrics treat each behavior as a single token. For example, the sample plan “R-1 oor C-1 cf C-1 lt C-0 cf C-0 iol O-3" is considered to have 5 tokens, each corresponding to one of its behaviors (“oor", “cf", “lt", “cf", “iol"). In this plan, “R-1",“C-1", “C-0", and “O-3" are symbols for locations (nodes) in the graph.
Question: What language is the experiment done in?
Answer: english language
Paper: Analysis of Risk Factor Domains in Psychosis Patient Health Records
Psychotic disorders typically emerge in late adolescence or early adulthood BIBREF0 , BIBREF1 and affect approximately 2.5-4% of the population BIBREF2 , BIBREF3 , making them one of the leading causes of disability worldwide BIBREF4 . A substantial proportion of psychiatric inpatients are readmitted after discharge BIBREF5 . Readmissions are disruptive for patients and families, and are a key driver of rising healthcare costs BIBREF6 , BIBREF7 . Reducing readmission risk is therefore a major unmet need of psychiatric care. Developing clinically implementable machine learning tools to enable accurate assessment of risk factors associated with readmission offers opportunities to inform the selection of treatment interventions and implement appropriate preventive measures.
Question: What additional features are proposed for future work?
Answer: distinguishing between clinically positive and negative phenomena within each risk factor domain and accounting for structured data collected on the target cohort
Paper: Analysis of Risk Factor Domains in Psychosis Patient Health Records
McCoy et al. mccoy2015clinical constructed a corpus of web data based on the Research Domain Criteria (RDoC) BIBREF15 , and used this corpus to create a vector space document similarity model for topic extraction. They found that the `negative valence' and `social' RDoC domains were associated with readmission. Using web data (in this case data retrieved from the Bing API) to train a similarity model for EHR texts is problematic since it differs from the target data in both structure and content. Based on reconstruction of the procedure, we conclude that many of the informative MWEs critical to understanding the topics of paragraphs in EHRs are not captured in the web data. Additionally, RDoC is by design a generalized research construct to describe the entire spectrum of mental disorders and does not include domains that are based on observation or causes of symptoms. Important indicators within EHRs of patient health, like appearance or occupation, are not included in the RDoC constructs.
Question: What are their initial results on this task?
Answer: Achieved the highest per-domain scores on Substance (F1 ≈ 0.8) and the lowest scores on Interpersonal and Mood (F1 ≈ 0.5), and show consistency in per-domain performance rankings between MLP and RBF models.
Paper: Analysis of Risk Factor Domains in Psychosis Patient Health Records
[2]The vast majority of patients in our target cohort are
Question: What datasets did the authors use?
Paper: Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation
Neural machine translation (NMT) has achieved impressive performance on machine translation task in recent years for many language pairs BIBREF0, BIBREF1, BIBREF2. However, in consideration of time cost and space capacity, the NMT model generally employs a limited-size vocabulary that only contains the top-N highest frequency words (commonly in the range of 30K to 80K) BIBREF3, which leads to the Out-of-Vocabulary (OOV) problem following with inaccurate and terrible translation results. Research indicated that sentences with too many unknown words tend to be translated much more poorly than sentences with mainly frequent words. For the low-resource and source-side morphologically-rich machine translation tasks, such as Turkish-English and Uyghur-Chinese, all the above issues are more serious due to the fact that the NMT model cannot effectively identify the complex morpheme structure or capture the linguistic and semantic information with too many rare and unknown words in the training corpus.
Question: How many linguistic and semantic features are learned?
Paper: Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation
We will elaborate two popular word segmentation methods and our newly proposed segmentation strategies in this section. The two popular segmentation methods are morpheme segmentation BIBREF4 and Byte Pair Encoding (BPE) BIBREF5. After word segmentation, we additionally add an specific symbol behind each separated subword unit, which aims to assist the NMT model to identify the morpheme boundaries and capture the semantic information effectively. The sentence examples with different segmentation strategies for Turkish-English machine translation task are shown in Table 1.
Question: How is morphology knowledge implemented in the method?
Answer: A BPE model is applied to the stem after morpheme segmentation.
Paper: Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation
The words of Turkish and Uyghur are formed by a stem followed with unlimited number of suffixes. Both of the stem and suffix are called morphemes, and they are the smallest functional unit in agglutinative languages. Study indicated that modeling language based on the morpheme units can provide better performance BIBREF6. Morpheme segmentation can segment the complex word into morpheme units of stem and suffix. This representation maintains a full description of the morphological properties of subwords while minimizing the data sparseness caused by inflection and allomorphy phenomenon in highly-inflected languages.
Question: How does the word segmentation method work?
Paper: Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation
In this segmentation strategy, each word is segmented into a stem unit and a combined suffix unit. We add “##” behind the stem unit and add “$$” behind the combined suffix unit. We denote this method as SCS. The segmented word can be denoted as two parts of “stem##” and “suffix1suffix2...suffixN$$”. If the original word has no suffix unit, the word is treated as its stem unit. All the following segmentation strategies will follow this rule.
Question: Is the word segmentation method independently evaluated?
Paper: Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
In this paper, we propose the processing of features not only in the input layer of a deep network, but in the intermediate layers as well. We are motivated by a desire to enable a neural network acoustic model to adaptively process the features depending on partial hypotheses and noise conditions. Many previous methods for adaptation have operated by linearly transforming either input features or intermediate layers in a two pass process where the transform is learned to maximize the likelihood of some adaptation data BIBREF0, BIBREF1, BIBREF2. Other methods have involved characterizing the input via factor analysis or i-vectors BIBREF3, BIBREF4. Here, we suggest an alternative approach in which adaptation can be achieved by re-presenting the feature stream at an intermediate layer of the network that is constructed to be correlated with the ultimate graphemic or phonetic output of the system.
Question: Do they normalize the calculated intermediate output hypotheses to compensate for the incompleteness?
Paper: Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
A transformer network BIBREF5 is a powerful approach to learning and modeling sequential data. A transformer network is itself constructed with a series of transformer modules that each perform some processing. Each module has a self-attention mechanism and several feed-forward layers, enabling easy parallelization over time-steps compared to recurrent models such as RNNs or LSTMs BIBREF10. We use the architecture defined in BIBREF5, and provide only a brief summary below.
Question: How many layers do they use in their best performing network?
Paper: Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
In this section, we present our proposal for allowing the network to (re)-consider the input features in the light of intermediate processing. We do this by again deploying a self-attention mechanism to combine the information present in the original features with the information available in the activations of an intermediate layer. As described earlier, we calculate the output posteriors and auxiliary loss at the intermediate layer as well. The overall architecture is illustrated in Figure FIGREF6. Here, we have used a 24 layer network, with feature re-presentation after the 12th layer.
Question: Do they just sum up all the loses the calculate to end up with one single loss?
Paper: Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
We process the features in the intermediate later by concatenating a projection of the original features with a projection of previous hidden layer activations, and then applying self-attention.
Question: Does their model take more time to train than regular transformer models?
Paper: Acquisition of Inflectional Morphology in Artificial Neural Networks With Prior Knowledge
A widely agreed-on fact in language acquisition research is that learning of a second language (L2) is influenced by a learner's native language (L1) BIBREF0, BIBREF1. A language's morphosyntax seems to be no exception to this rule BIBREF2, but the exact nature of this influence remains unknown. For instance, it is unclear whether it is constraints imposed by the phonological or by the morphosyntactic attributes of the L1 that are more important during the process of learning an L2's morphosyntax.
Question: Are agglutinative languages used in the prediction of both prefixing and suffixing languages?
Paper: Acquisition of Inflectional Morphology in Artificial Neural Networks With Prior Knowledge
Many of the world's languages exhibit rich inflectional morphology: the surface form of an individual lexical entry changes in order to express properties such as person, grammatical gender, or case. The citation form of a lexical entry is referred to as the lemma. The set of all possible surface forms or inflections of a lemma is called its paradigm. Each inflection within a paradigm can be associated with a tag, i.e., 3rdSgPres is the morphological tag associated with the inflection dances of the English lemma dance. We display the paradigms of dance and eat in Table TABREF1.
Question: What is an example of a prefixing language?
Paper: Acquisition of Inflectional Morphology in Artificial Neural Networks With Prior Knowledge
Let ${\cal M}$ be the paradigm slots which are being expressed in a language, and $w$ a lemma in that language. We then define the paradigm $\pi $ of $w$ as:
Question: How is the performance on the task evaluated?
Answer: Comparison of test accuracies of neural network models on an inflection task and qualitative analysis of the errors
Paper: Acquisition of Inflectional Morphology in Artificial Neural Networks With Prior Knowledge
The models we experiment with are based on a pointer–generator network architecture BIBREF10, BIBREF11, i.e., a recurrent neural network (RNN)-based sequence-to-sequence network with attention and a copy mechanism. A standard sequence-to-sequence model BIBREF12 has been shown to perform well for morphological inflection BIBREF13 and has, thus, been subject to cognitively motivated experiments BIBREF14 before. Here, however, we choose the pointer–generator variant of sharma-katrapati-sharma:2018:K18-30, since it performs better in low-resource settings, which we will assume for our target languages. We will explain the model shortly in the following and refer the reader to the original paper for more details.
Question: What are the tree target languages studied in the paper?
Paper: How Does Language Influence Documentation Workflow? Unsupervised Word Discovery Using Translations in Multiple Languages
The Cambridge Handbook of Endangered Languages BIBREF3 estimates that at least half of the 7,000 languages currently spoken worldwide will no longer exist by the end of this century. For these endangered languages, data collection campaigns have to accommodate the challenge that many of them are from oral tradition, and producing transcriptions is costly. This transcription bottleneck problem can be handled by translating into a widely spoken language to ensure subsequent interpretability of the collected recordings, and such parallel corpora have been recently created by aligning the collected audio with translations in a well-resourced language BIBREF1, BIBREF2, BIBREF4. Moreover, some linguists suggested that more than one translation should be collected to capture deeper layers of meaning BIBREF5.
Question: Is the model evaluated against any baseline?
Paper: How Does Language Influence Documentation Workflow? Unsupervised Word Discovery Using Translations in Multiple Languages
In this work we extend the bilingual Mboshi-French parallel corpus BIBREF2, fruit of the documentation process of Mboshi (Bantu C25), an endangered language spoken in Congo-Brazzaville. The corpus contains 5,130 utterances, for which it provides audio, transcriptions and translations in French. We translate the French into four other well-resourced languages through the use of the $DeepL$ translator. The languages added to the dataset are: English, German, Portuguese and Spanish. Table shows some statistics for the produced Multilingual Mboshi parallel corpus.
Question: Does the paper report the accuracy of the model?
Paper: How Does Language Influence Documentation Workflow? Unsupervised Word Discovery Using Translations in Multiple Languages
We use the bilingual neural-based Unsupervised Word Segmentation (UWS) approach from BIBREF6 to discover words in Mboshi. In this approach, Neural Machine Translation (NMT) models are trained between language pairs, using as source language the translation (word-level) and as target, the language to document (unsegmented phonemic sequence). Due to the attention mechanism present in these networks BIBREF7, posterior to training, it is possible to retrieve soft-alignment probability matrices between source and target sequences. These matrices give us sentence-level source-to-target alignment information, and by using it for clustering neighbor phonemes aligned to the same translation word, we are able to create segmentation in the target side. The product of this approach is a set of (discovered-units, translation words) pairs.
Question: How is the performance of the model evaluated?
Paper: How Does Language Influence Documentation Workflow? Unsupervised Word Discovery Using Translations in Multiple Languages
In this work we apply two simple methods for including multilingual information into the bilingual models from BIBREF6. The first one, Multilingual Voting, consists of merging the information learned by models trained with different language pairs by performing a voting over the final discovered boundaries. The voting is performed by applying an agreement threshold $T$ over the output boundaries. This threshold balances between accepting all boundaries from all the bilingual models (zero agreement) and accepting only input boundaries discovered by all these models (total agreement). The second method is ANE Selection. For every language pair and aligned sentence in the dataset, a soft-alignment probability matrix is generated. We use Average Normalized Entropy (ANE) BIBREF8 computed over these matrices for selecting the most confident one for segmenting each phoneme sequence. This exploits the idea that models trained on different language pairs will have language-related behavior, thus differing on the resulting alignment and segmentation over the same phoneme sequence.
Question: What are the different bilingual models employed?
Paper: How Does Language Influence Documentation Workflow? Unsupervised Word Discovery Using Translations in Multiple Languages
The experiment settings from this paper and evaluation protocol for the Mboshi corpus (Boundary F-scores using the ZRC speech reference) are the same from BIBREF8. Table presents the results for bilingual UWS and multilingual leveraging. For the former, we reach our best result by using as aligned information the French, the original aligned language for this dataset. Languages closely related to French (Spanish and Portuguese) ranked better, while our worst result used German. English also performs notably well in our experiments. We believe this is due to the statistics features of the resulting text. We observe in Table that the English portion of the dataset contains the smallest vocabulary among all languages. Since we train our systems in very low-resource settings, vocabulary-related features can impact greatly the system's capacity to language-model, and consequently the final quality of the produced alignments. Even in high-resource settings, it was already attested that some languages are more difficult to model than others BIBREF9.
Question: How does the well-resourced language impact the quality of the output?
Paper: Dense Information Flow for Neural Machine Translation
Neural machine translation (NMT) is a challenging task that attracts lots of attention in recent years. Starting from the encoder-decoder framework BIBREF0 , NMT starts to show promising results in many language pairs. The evolving structures of NMT models in recent years have made them achieve higher scores and become more favorable. The attention mechanism BIBREF1 added on top of encoder-decoder framework is shown to be very useful to automatically find alignment structure, and single-layer RNN-based structure has evolved into deeper models with more efficient transformation functions BIBREF2 , BIBREF3 , BIBREF4 .
Question: what are the baselines?
Paper: Dense Information Flow for Neural Machine Translation
In this section, we introduce our DenseNMT architecture. In general, compared with residual connected NMT models, DenseNMT allows each layer to provide its information to all subsequent layers directly. Figure FIGREF9 - FIGREF15 show the design of our model structure by parts.
Question: did they outperform previous methods?
Paper: Dense Information Flow for Neural Machine Translation
Different from residual connections, later layers in the dense encoder are able to use features from all previous layers by concatenating them: DISPLAYFORM0 
Question: what language pairs are explored?
Paper: Dense Information Flow for Neural Machine Translation
Prior works show a trend of designing more expressive attention mechanisms (as discussed in Section 2). However, most of them only use the last encoder layer. In order to pass more abundant information from the encoder side to the decoder side, the attention block needs to be more expressive. Following the recent development of designing attention architectures, we propose DenseAtt as the dense attention block, which serves for the dense connection between the encoder and the decoder side. More specifically, two options are proposed accordingly. For each decoding step in the corresponding decoder layer, the two options both calculate attention using multiple encoder layers. The first option is more compressed, while the second option is more expressive and flexible. We name them as DenseAtt-1 and DenseAtt-2 respectively. Figure FIGREF15 shows the architecture of (a) multi-step attention BIBREF2 , (b) DenseAtt-1, and (c) DenseAtt-2 in order. In general, a popular multiplicative attention module can be written as: DISPLAYFORM0 
Question: what datasets were used?
Answer: IWSLT14 German-English, IWSLT14 Turkish-English, WMT14 English-German
Paper: Frozen Binomials on the Web: Word Ordering and Language Conventions in Online Text
Lists are extremely common in text and speech, and the ordering of items in a list can often reveal information. For instance, orderings can denote relative importance, such as on a to-do list, or signal status, as is the case for author lists of scholarly publications. In other cases, orderings might come from cultural or historical conventions. For example, `red, white, and blue' is a specific ordering of colors that is recognizable to those familiar with American culture.
Question: How is order of binomials tracked across time?
Paper: Frozen Binomials on the Web: Word Ordering and Language Conventions in Online Text
Interest in list orderings spans the last century BIBREF10, BIBREF1, with a focus almost exclusively on binomials. This research has primarily investigated frozen binomials, also called irreversible binomials, fixed coordinates, and fixed conjuncts BIBREF11, although some work has also looked at non-coordinate freezes where the individual words are nonsensical by themselves (e.g., `dribs and drabs') BIBREF11. One study has directly addressed mostly frozen binomials BIBREF5, and we expand the scope of this paper by exploring the general question of how frequently binomials appear in a particular order. Early research investigated languages other than English BIBREF1, BIBREF10, but most recent research has worked almost exclusively with English. Overall, this prior research can be separated into three basic categories — phonological rules, semantic rules, and metadata rules.
Question: What types of various community texts have been investigated for exploring global structure of binomials?
Paper: Frozen Binomials on the Web: Word Ordering and Language Conventions in Online Text
We take our data mostly from Reddit, a large social media website divided into subcommunities called `subreddits' or `subs'. Each subreddit has a theme (usually clearly expressed in its name), and we have focused our study on subreddits primarily in sports and politics, in part because of the richness of proper names in these domains: r/nba, r/nfl, r/politics, r/Conservative, r/Libertarian, r/The_Donald, r/food, along with a variety of NBA team subreddits (e.g., r/rockets for the Houston Rockets). Apart from the team-specific and food subreddits, these are among the largest and most heavily used subreddits BIBREF23. We gather text data from comments made by users in discussion threads. In all cases, we have data from when the subreddit started until mid-2018. (Data was contributed by Cristian Danescu-Niculescu-Mizil.) Reddit in general, and the subreddits we examined in particular, are rapidly growing, both in terms of number of users and number of comments.
Question: Are there any new finding in analasys of trinomials that was not present binomials?
Paper: Frozen Binomials on the Web: Word Ordering and Language Conventions in Online Text
In this paper we introduce a new framework to interpret binomials, based on three properties: asymmetry (how frozen a binomial is), movement (how binomial orderings change over time), and agreement (how consistent binomial orderings are between communities), which we will visualize as a cube with three dimensions. Again, prior work has focused essentially entirely on asymmetry, and we argue that this can only really be understood in the context of the other two dimensions.
Question: What new model is proposed for binomial lists?