z-uo/qasper-squad huggingface dataset:
Paper: A New Corpus for Low-Resourced Sindhi Language with Word Embeddings
Sindhi is a rich morphological, mutltiscript, and multidilectal language. It belongs to the Indo-Aryan language family BIBREF0, with significant cultural and historical background. Presently, it is recognized as is an official language BIBREF1 in Sindh province of Pakistan, also being taught as a compulsory subject in Schools and colleges. Sindhi is also recognized as one of the national languages in India. Ulhasnagar, Rajasthan, Gujarat, and Maharashtra are the largest Indian regions of Sindhi native speakers. It is also spoken in other countries except for Pakistan and India, where native Sindhi speakers have migrated, such as America, Canada, Hong Kong, British, Singapore, Tanzania, Philippines, Kenya, Uganda, and South, and East Africa. Sindhi has rich morphological structure BIBREF2 due to a large number of homogeneous words. Historically, it was written in multiple writing systems, which differ from each other in terms of orthography and morphology. The Persian-Arabic is the standard script of Sindhi, which was officially accepted in 1852 by the British government. However, the Sindhi-Devanagari is also a popular writing system in India being written in left to right direction like the Hindi language. Formerly, Khudabadi, Gujrati, Landa, Khojki, and Gurumukhi were also adopted as its writing systems. Even though, Sindhi has great historical and literal background, presently spoken by nearly 75 million people BIBREF1. The research on SNLP was coined in 2002, however, IT grabbed research attention after the development of its Unicode system BIBREF3. But still, Sindhi stands among the low-resourced languages due to the scarcity of core language processing resources of the raw and annotated corpus, which can be utilized for training robust word embeddings or the use of machine learning algorithms. Since the development of annotated datasets requires time and human resources.
Question: How does proposed word embeddings compare to Sindhi fastText word representations?
Answer: Proposed SG model vs SINDHI FASTTEXT:
Average cosine similarity score: 0.650 vs 0.388
Average semantic relatedness similarity score between countries and their capitals: 0.663 vs 0.391
Paper: A New Corpus for Low-Resourced Sindhi Language with Word Embeddings
The natural language resources refer to a set of language data and descriptions BIBREF31 in machine readable form, used for building, improving, and evaluating NLP algorithms or softwares. Such resources include written or spoken corpora, lexicons, and annotated corpora for specific computational purposes. Many world languages are rich in such language processing resources integrated in the software tools including NLTK for English BIBREF5, Stanford CoreNLP BIBREF6, LTP for Chinese BIBREF7, TectoMT for German, Russian, Arabic BIBREF8 and multilingual toolkit BIBREF9. But Sindhi language is at an early stage for the development of such resources and software tools.
Question: Are trained word embeddings used for any other NLP task?
Paper: A New Corpus for Low-Resourced Sindhi Language with Word Embeddings
This section presents the employed methodology in detail for corpus acquisition, preprocessing, statistical analysis, and generating Sindhi word embeddings.
Question: How many uniue words are in the dataset?
Answer: 908456 unique words are available in collected corpus.
Paper: A New Corpus for Low-Resourced Sindhi Language with Word Embeddings
We initiate this work from scratch by collecting large corpus from multiple web resources. After preprocessing and statistical analysis of the corpus, we generate Sindhi word embeddings with state-of-the-art CBoW, SG, and GloVe algorithms. The generated word embeddings are evaluated using the intrinsic evaluation approaches of cosine similarity between nearest neighbors, word pairs, and WordSim-353 for distributional semantic similarity. Moreover, we use t-SNE with PCA for the comparison of the distance between similar words via visualization.
Question: How is the data collected, which web resources were used?
Paper: The Wiki Music dataset: A tool for computational analysis of popular music
Until recent times, the research in popular music was mostly bound to a non-computational approach BIBREF0 but the availability of new data, models and algorithms helped the rise of new research trends. Computational analysis of music structure BIBREF1 is focused on parsing and annotate patters in music files; computational music generation BIBREF2 trains systems able to generate songs with specific music styles; computational sociology of music analyzes databases annotated with metadata such as tempo, key, BPMs and similar (generally referred to as sonic features); even psychology of music use data to find new models.
Question: What trends are found in musical preferences?
Paper: The Wiki Music dataset: A tool for computational analysis of popular music
We define ”popular music” as the music which finds appeal out of culturally closed music groups, also thanks to its commercial nature. Non-popular music can be divided into three broad groups: classical music (produced and performed by experts with a specific education), folk/world music (produced and performed by traditional cultures), and utility music (such as hymns and military marches, not primarily intended for commercial purposes). Popular music is a great mean for spreading culture, and a perfect ground where cultural practices and industry processes combine. In particular the cultural processes select novelties, broadly represented by means of underground music genres, and the industry tries to monetize, making them commercially successful. In the following description we include almost all the genres that reach commercial success and few of the underground genres that are related to them.
Question: Which decades did they look at?
Paper: The Wiki Music dataset: A tool for computational analysis of popular music
From the description of music genres provided above emerges that there is a limited number of super-genres and derivation lines BIBREF19, BIBREF20, as shown in figure FIGREF1.
Question: How many genres did they collect from?
Paper: An Annotated Corpus of Emerging Anglicisms in Spanish Newspaper Headlines
The study of English influence in the Spanish language has been a hot topic in Hispanic linguistics for decades, particularly concerning lexical borrowing or anglicisms BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4, BIBREF5, BIBREF6.
Question: Does the paper mention other works proposing methods to detect anglicisms in Spanish?
Paper: An Annotated Corpus of Emerging Anglicisms in Spanish Newspaper Headlines
Corpus-based studies of English borrowings in Spanish media have traditionally relied on manual evaluation of either previously compiled general corpora such as CREA BIBREF10, BIBREF11, BIBREF12, BIBREF13, either new tailor-made corpora designed to analyze specific genres, varieties or phenomena BIBREF14, BIBREF15, BIBREF16, BIBREF17, BIBREF18, BIBREF19, BIBREF20.
Question: What is the performance of the CRF model on the task described?
Paper: An Annotated Corpus of Emerging Anglicisms in Spanish Newspaper Headlines
Linguistic borrowing can be defined as the transference of linguistic elements between two languages. Borrowing and code-switching have frequently been described as a continuum BIBREF33, with a fuzzy frontier between the two. As a result, a precise definition of what borrowing is remains elusive BIBREF34 and some authors prefer to talk about code-mixing in general BIBREF35 or “lone other-language incorporations" BIBREF36.
Question: Does the paper motivate the use of CRF as the baseline model?
Paper: An Annotated Corpus of Emerging Anglicisms in Spanish Newspaper Headlines
In this subsection we describe the characteristics of the corpus. We first introduce the main corpus, with the usual train/development/test split that was used to train, tune and evaluate the model. We then present an additional test set that was designed to assess the performance of the model on more naturalistic data.
Question: What are the handcrafted features used?
Paper: Style Transfer for Texts: to Err is Human, but Error Margins Matter
Deep generative models attract a lot of attention in recent years BIBREF0. Such methods as variational autoencoders BIBREF1 or generative adversarial networks BIBREF2 are successfully applied to a variety of machine vision problems including image generation BIBREF3, learning interpretable image representations BIBREF4 and style transfer for images BIBREF5. However, natural language generation is more challenging due to many reasons, such as the discrete nature of textual information BIBREF6, the absence of local information continuity and non-smooth disentangled representations BIBREF7. Due to these difficulties, text generation is mostly limited to specific narrow applications and is usually working in supervised settings.
Question: What is state of the art method?
Paper: Style Transfer for Texts: to Err is Human, but Error Margins Matter
Style of a text is a very general notion that is hard to define in rigorous terms BIBREF15. However, the style of a text can be characterized quantitatively BIBREF16; stylized texts could be generated if a system is trained on a dataset of stylistically similar texts BIBREF17; and author-style could be learned end-to-end BIBREF18, BIBREF19, BIBREF20. A majority of recent works on style transfer focus on the sentiment of text and use it as a target attribute. For example, in BIBREF21, BIBREF22, BIBREF23 estimate the quality of the style transfer with binary sentiment classifier trained on the corpora further used for the training of the style-transfer system. BIBREF24 and especially BIBREF9 generalize this ad-hoc approach defining a style as a set of arbitrary quantitively measurable categorial or continuous parameters. Such parameters could include the 'style of the time' BIBREF16, author-specific attributes (see BIBREF25 or BIBREF26 on 'shakespearization'), politeness BIBREF27, formality of speech BIBREF28, and gender or even political slant BIBREF29.
Question: By how much do proposed architectures autperform state-of-the-art?
Paper: Style Transfer for Texts: to Err is Human, but Error Margins Matter
In this work we experiment with extensions of a model, described in BIBREF6, using Texar BIBREF40 framework. To generate plausible sentences with specific semantic and stylistic features every sentence is conditioned on a representation vector $z$ which is concatenated with a particular code $c$ that specifies desired attribute, see Figure FIGREF8. Under notation introduced in BIBREF6 the base autoencoder (AE) includes a conditional probabilistic encoder $E$ defined with parameters $\theta _E$ to infer the latent representation $z$ given input $x$
Question: What are three new proposed architectures?
Paper: Style Transfer for Texts: to Err is Human, but Error Margins Matter
We have found that the baseline, as well as the proposed extensions, have noisy outcomes, when retrained from scratch, see Figure FIGREF1. Most of the papers mentioned in Section SECREF2 measure the performance of the methods proposed for the sentiment transfer with two metrics: accuracy of the external sentiment classifier measured on test data, and BLEU between the input and output that is regarded as a coarse metric for semantic similarity.
Question: How much does the standard metrics for style accuracy vary on different re-runs?
Paper: Efficient Attention using a Fixed-Size Memory Representation
Sequence-to-sequence models BIBREF0 , BIBREF1 have achieved state of the art results across a wide variety of tasks, including Neural Machine Translation (NMT) BIBREF2 , BIBREF3 , text summarization BIBREF4 , BIBREF5 , speech recognition BIBREF6 , BIBREF7 , image captioning BIBREF8 , and conversational modeling BIBREF9 , BIBREF10 .
Question: Which baseline methods are used?
Answer: standard parametrized attention and a non-attention baseline
Paper: Efficient Attention using a Fixed-Size Memory Representation
Our models are based on an encoder-decoder architecture with attention mechanism BIBREF2 , BIBREF11 . An encoder function takes as input a sequence of source tokens $\mathbf {x} = (x_1, ..., x_m)$ and produces a sequence of states $\mathbf {s} = (s_1, ..., s_m)$ .The decoder is an RNN that predicts the probability of a target sequence $\mathbf {y} = (y_1, ..., y_T \mid \mathbf {s})$ . The probability of each target token $y_i \in \lbrace 1, ... ,|V|\rbrace $ is predicted based on the recurrent state in the decoder RNN, $h_i$ , the previous words, $y_{<i}$ , and a context vector $c_i$ . The context vector $c_i$ , also referred to as the attention vector, is calculated as a weighted average of the source states. 
Question: How much is the BLEU score?
Answer: Ranges from 44.22 to 100.00 depending on K and the sequence length.
Paper: Efficient Attention using a Fixed-Size Memory Representation
Our proposed model is shown in Figure 1 . During encoding, we compute an attention matrix $C \in \mathbb {R}^{K \times D}$ , where $K$ is the number of attention vectors and a hyperparameter of our method, and $D$ is the dimensionality of the top-most encoder state. This matrix is computed by predicting a score vector $\alpha _t \in \mathbb {R}^K$ at each encoding time step $t$ . $C$ is then a linear combination of the encoder states, weighted by $\alpha _t$ : 
Question: Which datasets are used in experiments?
Answer: Sequence Copy Task and WMT'17
Paper: Duality Regularization for Unsupervised Bilingual Lexicon Induction
Unsupervised bilingual lexicon induction (UBLI) has been shown to benefit NLP tasks for low resource languages, including unsupervised NMT BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4, information retrieval BIBREF5, BIBREF6, dependency parsing BIBREF7, and named entity recognition BIBREF8, BIBREF9.
Question: What regularizers were used to encourage consistency in back translation cycles?
Paper: Duality Regularization for Unsupervised Bilingual Lexicon Induction
UBLI. A typical line of work uses adversarial training BIBREF17, BIBREF10, BIBREF18, BIBREF11, matching the distributions of source and target word embeddings through generative adversarial networks BIBREF19. Non-adversarial approaches have also been explored. For instance, BIBREF15 Mukherjee18EMNLP use squared-loss mutual information to search for optimal cross-lingual word pairing. BIBREF13 and BIBREF20 exploit the structural similarity of word embedding spaces to learn word mappings. In this paper, we choose BIBREF11 Conneau18a as our baseline as it is theoretically attractive and gives strong results on large-scale datasets.
Question: What are new best results on standard benchmark?
Answer: New best results of accuracy (P@1) on Vecmap:
Ours-GeoMMsemi: EN-IT 50.00 IT-EN 42.67 EN-DE 51.60 DE-EN 47.22 FI-EN 39.62 EN-ES 39.47 ES-EN 36.43
Paper: Duality Regularization for Unsupervised Bilingual Lexicon Induction
We take BIBREF11 as our baseline, introducing a novel regularizer to enforce cycle consistency. Let $X=\lbrace x_1,...,x_n\rbrace $ and $Y=\lbrace y_1,...,y_m\rbrace $ be two sets of $n$ and $m$ word embeddings for a source and a target language, respectively. The primal UBLI task aims to learn a linear mapping $\mathcal {F}:X\rightarrow Y$ such that for each $x_i$, $\mathcal {F}(x_i)$ corresponds to its translation in $Y$. Similarly, a linear mapping $\mathcal {G}:Y\rightarrow X$ is defined for the dual task. In addition, we introduce two language discriminators $D_x$ and $D_y$, which are trained to discriminate between the mapped word embeddings and the original word embeddings.
Question: How better is performance compared to competitive baselines?
Answer: Proposed method vs best baseline result on Vecmap (Accuracy P@1):
EN-IT: 50 vs 50
IT-EN: 42.67 vs 42.67
EN-DE: 51.6 vs 51.47
DE-EN: 47.22 vs 46.96
EN-FI: 35.88 vs 36.24
FI-EN: 39.62 vs 39.57
EN-ES: 39.47 vs 39.30
ES-EN: 36.43 vs 36.06
Paper: Duality Regularization for Unsupervised Bilingual Lexicon Induction
BIBREF11 align two word embedding spaces through generative adversarial networks, in which two networks are trained simultaneously. Specifically, take the primal UBLI task as an example, the linear mapping $\mathcal {F}$ tries to generate “fake” word embeddings $\mathcal {F}(x)$ that look similar to word embeddings from $Y$, while the discriminator $D_y$ aims to distinguish between “fake” and real word embeddings from $Y$. Formally, this idea can be expressed as the minmax game min$_{\mathcal {F}}$max$_{D{_y}}\ell _{adv}(\mathcal {F},D_y,X,Y)$, where
Question: How big is data used in experiments?
Paper: Duality Regularization for Unsupervised Bilingual Lexicon Induction
We train $\mathcal {F}$ and $\mathcal {G}$ jointly and introduce two regularizers. Formally, we hope that $\mathcal {G}(\mathcal {F}(X))$ is similar to $X$ and $\mathcal {F}(\mathcal {G}(Y))$ is similar to $Y$. We implement this constraint as a cycle consistency loss. As a result, the proposed model has two learning objectives: i) an adversarial loss ($\ell _{adv}$) for each model as in the baseline. ii) a cycle consistency loss ($\ell _{cycle}$) on each side to avoid $\mathcal {F}$ and $\mathcal {G}$ from contradicting each other. The overall architecture of our model is illustrated in Figure FIGREF4.
Question: What 6 language pairs is experimented on?
Answer: EN<->ES
EN<->DE
EN<->IT
EN<->EO
EN<->MS
EN<->FI
Paper: Duality Regularization for Unsupervised Bilingual Lexicon Induction
We follow BIBREF11, using an unsupervised criterion to perform model selection. In preliminary experiments, we find in adversarial training that the single-direction criterion $S(\mathcal {F}, X, Y)$ by BIBREF11 does not always work well. To address this, we make a simple extension by calculating the weighted average of forward and backward scores:
Question: What are current state-of-the-art methods that consider the two tasks independently?
Paper: Team Papelo: Transformer Networks at FEVER
The release of the FEVER fact extraction and verification dataset BIBREF0 provides a large-scale challenge that tests a combination of retrieval and textual entailment capabilities. To verify a claim in the dataset as supported, refuted, or undecided, a system must retrieve relevant articles and sentences from Wikipedia. Then it must decide whether each of those sentences, or some combination of them, entails or refutes the claim, which is an entailment problem. Systems are evaluated on the accuracy of the claim predictions, with credit only given when correct evidence is submitted.
Question: How big is their training set?
Paper: Team Papelo: Transformer Networks at FEVER
The core of our system is an entailment module based on a transformer network. Transformer networks BIBREF6 are deep networks applied to sequential input data, with each layer implementing multiple heads of scaled dot product attention. This attention mechanism allows deep features to be compared across positions in the input.
Question: What baseline do they compare to?
Paper: Team Papelo: Transformer Networks at FEVER
The baseline FEVER system BIBREF0 ran the AllenNLP BIBREF3 implementation of Decomposable Attention BIBREF2 to classify a group of five premise statements concatenated together against the claim. These five premise statements were fixed by the retrieval module and not considered individually. In our system, premise statements are individually evaluated.
Question: Which pre-trained transformer do they use?
Paper: Team Papelo: Transformer Networks at FEVER
Regardless of how strong the entailment classifier is, FEVER score is limited by whether the document and sentence retrieval modules, which produce the input to the entailment classifier, find the right evidence. In Table 3 , we examine the percentage of claims for which correct evidence is retrieved, before filtering with the entailment classifier. For this calculation, we skip any claim with an evidence group with multiple statements, and count a claim as succesfully retrieved if it is not verifiable or if the statement in one of the evidence groups is retrieved. The baseline system retrieves the five articles with the highest TFIDF score, and then extracts the five sentences from that collection with the highest TFIDF score against the claim. It achieves 66.1% evidence retrieval.
Question: What is the FEVER task?
Paper: Automatic Differentiation in ROOT
Accurate and efficient computation of derivatives is vital for a wide variety of computing applications, including numerical optimization, solution of nonlinear equations, sensitivity analysis, and nonlinear inverse problems. Virtually every process could be described with a mathematical function, which can be thought of as an association between elements from different sets. Derivatives track how a varying quantity depends on another quantity, for example how the position of a planet varies as time varies.
Question: How is correctness of automatic derivation proved?
Paper: Automatic Differentiation in ROOT
Here, we briefly discuss main algorithmic and implementation principles behind AD. An in-depth overview and more formal description can be found in BIBREF1 and BIBREF2, respectively.
Question: Is this AD implementation used in any deep learning framework?
Paper: Controlling the Output Length of Neural Machine Translation
The sequence to sequence BIBREF0, BIBREF1 approach to Neural Machine Translation (NMT) has shown to improve quality in various translation tasks BIBREF2, BIBREF3, BIBREF4. While translation quality is normally measured in terms of correct transfer of meaning and of fluency, there are several applications of NMT that would benefit from optimizing the output length, such as the translation of document elements that have to fit a given layout – e.g. entries of tables or bullet points of a presentation – or subtitles, which have to fit visual constraints and readability goals, as well as speech dubbing, for which the length of the translation should be as close as possible to the length of the original sentence.
Question: Do they conduct any human evaluation?
Paper: Controlling the Output Length of Neural Machine Translation
Our proposal is based on the transformer architecture and a recently proposed extension of its positional encoding aimed to control the length of generated sentences in text summarization.
Question: What dataset do they use for experiments?
Paper: Controlling the Output Length of Neural Machine Translation
Transformer BIBREF12 is a sequence-to-sequence architecture that processes sequences using only attention and feed forward layers. Its core component is the so-called multi-head attention, which computes attention BIBREF0, BIBREF13 between two sequences in a multi-branch fashion BIBREF14. Within the encoder or the decoder, each layer first computes attention between two copies of the same sequence (self-attention). In the decoder, this step is followed by an attention over the encoder output sequence. The last step in each layer is a two-layered time-distributed feed-forward network, with a hidden size larger than its input and output. Attention and feed-forward layers are characterized by a position-invariant processing of their input. Thus, in order to enrich input embeddings in source and target with positional information, they are summed with positional vectors of the same dimension $d$, which are computed with the following trigonometric encoding ($\text{PE}$):
Question: How do they enrich the positional embedding with length information
Answer: They introduce new trigonometric encoding which besides information about position uses additional length information (abs or relative).