In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

Example Input: Before designing this NLP toolkit, we conducted a survey among engineers and identified a spectrum of three typical personas.
Example Output: How do the authors evidence the claim that many engineers find it a big overhead to choose from multiple frameworks, models and optimization techniques?

Example Input: In this paper, we use three data sets from the literature to train and evaluate our own classifier. Data collected by BIBREF3 , which we term the Sexist/Racist (SR) data set, was collected using an initial Twitter search followed by analysis and filtering by the authors and their team who identified 17 common phrases, hashtags, and users that were indicative of abusive speech. BIBREF4 collected the HATE dataset by searching for tweets using a lexicon provided by Hatebase.org. The final data set we used, which we call HAR, was collected by BIBREF9 ; we removed all retweets reducing the dataset to 20,000 tweets. Tweets were labeled as “Harrassing” or “Non-Harrassing”; hate speech was not explicitly labeled, but treated as an unlabeled subset of the broader “Harrassing” category BIBREF9 .
Example Output: Which publicly available datasets are used?

Example Input: TextCat is the most well-known Perl implementation of the out-of-place method, it lists models for 76 languages in its off-the-shelf configuration; the program is not actively maintained. TextCat is the most well-known Perl implementation of the out-of-place method, it lists models for 76 languages in its off-the-shelf configuration; the program is not actively maintained. TextCat is not the only example of an off-the-shelf implementation of the out-of-place method: other implementations include libtextcat with 76 language models, JTCL with 15 languages, and mguesser with 104 models for different language-encoding pairs. The main issue addressed by later implementations is classification speed: TextCat is implemented in Perl and is not optimized for speed, whereas implementations such as libtextcat and mguesser have been specifically written to be fast and efficient. whatlang-rs uses an algorithm based on character trigrams and refers the user to the BIBREF7 article. It comes pre-trained with 83 languages.

is the language identifier embedded in the Google Chrome web browser. It uses a NB classifier, and script-specific classification strategies. assumes that all the input is in UTF-8, and assigns the responsibility of encoding detection and transcoding to the user. uses Unicode information to determine the script of the input. also implements a number of pre-processing heuristics to help boost performance on its target domain (web pages), such as stripping character sequences like .jpg. The standard implementation supports 83 languages, and an extended model is also available, that supports 160 languages.

is a Java library that implements a language identifier based on a NB classifier trained over character . The software comes with pre-trained models for 53 languages, using data from Wikipedia. makes use of a range of normalization heuristics to improve the performance on particular languages, including: (1) clustering of Chinese/Japanese/Korean characters to reduce sparseness; (2) removal of “language-independent” characters, and other text normalization; and (3) normalization of Arabic characters.

is a Python implementation of the method described by BIBREF150 , which exploits training data for the same language across multiple different sources of text to identify sequences of characters that are strongly predictive of a given language, regardless of the source of the text. This feature set is combined with a NB classifier, and is distributed with a pre-trained model for 97 languages prepared using data from 5 different text sources. BIBREF151 provide an empirical comparison of to , and and find that it compares favorably both in terms of accuracy and classification speed. There are also implementations of the classifier component (but not the training portion) of in Java, C, and JavaScript.

BIBREF153 uses a vector-space model with per-feature weighting on character sequences. One particular feature of is that it uses discriminative training in selecting features, i.e. it specifically makes use of features that are strong evidence against a particular language, which is generally not captured by NB models. Another feature of is that it uses inter-string smoothing to exploit sentence-level locality in making language predictions, under the assumption that adjacent sentences are likely to be in the same language. BIBREF153 reports that this substantially improves the accuracy of the identifier. Another distinguishing feature of is that it comes pre-trained with data for 1400 languages, which is the highest number by a large margin of any off-the-shelf system.

whatthelang is a recent language identifier written in Python, which utilizes the FastText NN-based text classification algorithm. It supports 176 languages.

implements an off-the-shelf classifier trained using Wikipedia data, covering 122 languages. Although not described as such, the actual classification algorithm used is a linear model, and is thus closely related to both NB and a cosine-based vector space model.

In addition to the above-mentioned general-purpose language identifiers, there have also been efforts to produce pre-trained language identifiers targeted specifically at Twitter messages. is a Twitter-specific tool with built-in models for 19 languages. It uses a document representation based on tries BIBREF401 . The algorithm is a LR classifier using all possible substrings of the data, which is important to maximize the available information from the relatively short Twitter messages.

BIBREF152 provides a comparison of 8 off-the-shelf language identifiers applied without re-training to Twitter messages. One issue they report is that comparing the accuracy of off-the-shelf systems is difficult because of the different subset of languages supported by each system, which may also not fully cover the languages present in the target data. The authors choose to compare accuracy over the full set of languages, arguing that this best reflects the likely use-case of applying an off-the-shelf system to new data. They find that the best individual systems are , and , but that slightly higher accuracy can be attained by a simple voting-based ensemble classifier involving these three systems.

In addition to this, commercial or other closed-source language identifiers and language identifier services exist, of which we name a few. The Polyglot 3000 and Lextek Language Identifier are standalone language identifiers for Windows. Open Xerox Language Identifier is a web service with available REST and SOAP APIs.
Example Output:
what are the off-the-shelf systems discussed in the paper?