Miracle-dz/UniFAR-FTUs huggingface dataset:
HardEval: Focusing on Challenging Tokens to Assess Robustness of NER

To assess the robustness of NER systems, we propose an evaluation method that focuses on subsets of tokens that represent specific sources of errors: unknown words and label shift or ambiguity. These subsets provide a system-agnostic basis for evaluating specific sources of NER errors and assessing room for improvement in terms of robustness. We analyze these subsets of challenging tokens in two widely-used NER benchmarks, then exploit them to evaluate NER systems in both in-domain and out-of-domain settings. Results show that these challenging tokens explain the majority of errors made by modern NER systems, although they represent only a small fraction of test tokens. They also indicate that label shift is harder to deal with than unknown words, and that there is much more room for improvement than the standard NER evaluation procedure would suggest. We hope this work will encourage NLP researchers to adopt rigorous and meaningful evaluation methods, and will help them develop more robust models.
Typical Sentences as a Resource for Valence

Verb valence information can be derived from corpora by using subcorpora of typical sentences that are constructed in a language independent manner based on frequent POS structures. The inspection of typical sentences with a fixed verb in a certain position can show the valence information directly. Using verb fingerprints, consisting of the most typical sentence patterns the verb appears in, we are able to identify standard valence patterns and compare them against a language’s valence profile. With a very limited number of training data per language, valence information for other verbs can be derived as well. Based on the Norwegian valence patterns we are able to find comparative patterns in German where typical sentences are able to express the same situation in an equivalent way and can so construct verb valence pairs for a bilingual PolyVal dictionary. This contribution discusses this application with a focus on the Norwegian valence dictionary NorVal.
Dry mass allocation, water use efficiency and δ13C in clones of Eucalyptus grandis, E. grandis × camaldulensis and E. grandis × nitens grown under two irrigation regimes

Clonal variation in water use efficiency (WUE), dry mass accumulation and allocation, and stable carbon isotope ratio (delta(13)C) of crude leaf fiber extracts was determined in six clones of Eucalyptus grandis W. Hill ex Maiden. grown for 16 months in field lysimeters in two soil water regimes. The relationships between delta(13)C and WUE calculated on the basis of leaf, harvestable stem, shoot and whole-plant dry mass accumulation were investigated. There was no clonal variation in dry mass accumulation but clonal allocation to roots, harvestable stems, branches and leaves varied. Water use efficiencies (mass of plant or plant part/water used over 16 months) differed significantly between clones. The clonal ranking of WUE varied depending on the units of dry mass accumulation used. Significant relationships between delta(13)C values and instantaneous water use efficiencies and ratios of internal leaf to ambient CO(2) concentrations were found only in the high soil water treatment. There were no relationships between delta(13)C values and whole-plant, shoot and harvestable stem water use efficiencies and soil water availability. Values of delta(13)C were negatively correlated with dry mass accumulation in the low soil water treatment. At the whole-plant level, WUE was positively correlated with dry mass accumulation in the high soil water treatment. We found significant differences in delta(13)C values between clones and the clonal rankings in delta(13)C and WUE were maintained in both soil water treatments.
An In-Depth Comparison of 14 Spelling Correction Tools on a Common Benchmark

Determining and correcting spelling and grammar errors in text is an important but surprisingly difficult task. There are several reasons why this remains challenging. Errors may consist of simple typing errors like deleted, substituted, or wrongly inserted letters, but may also consist of word confusions where a word was replaced by another one. In addition, words may be erroneously split into two parts or get concatenated. Some words can contain hyphens, because they were split at the end of a line or are compound words with a mandatory hyphen. In this paper, we provide an extensive evaluation of 14 spelling correction tools on a common benchmark. In particular, the evaluation provides a detailed comparison with respect to 12 error categories. The benchmark consists of sentences from the English Wikipedia, which were distorted using a realistic error model. Measuring the quality of an algorithm with respect to these error categories requires an alignment of the original text, the distorted text and the corrected text provided by the tool. We make our benchmark generation and evaluation tools publicly available.
A Comparison of Explicit and Implicit Proactive Dialogue Strategies for Conversational Recommendation

Recommendation systems aim at facilitating information retrieval for users by taking into account their preferences. Based on previous user behaviour, such a system suggests items or provides information that a user might like or find useful. Nonetheless, how to provide suggestions is still an open question. Depending on the way a recommendation is communicated influences the user’s perception of the system. This paper presents an empirical study on the effects of proactive dialogue strategies on user acceptance. Therefore, an explicit strategy based on user preferences provided directly by the user, and an implicit proactive strategy, using autonomously gathered information, are compared. The results show that proactive dialogue systems significantly affect the perception of human-computer interaction. Although no significant differences are found between implicit and explicit strategies, proactivity significantly influences the user experience compared to reactive system behaviour. The study contributes new insights to the human-agent interaction and the voice user interface design. Furthermore, we discover interesting tendencies that motivate futurework.
Cross-domain Author Gender Classification in Brazilian Portuguese

Author profiling models predict demographic characteristics of a target author based on the text that they have written. Systems of this kind will often follow a single-domain approach, in which the model is trained from a corpus of labelled texts in a given domain, and it is subsequently validated against a test corpus built from precisely the same domain. Although single-domain settings are arguably ideal, this strategy gives rise to the question of how to proceed when no suitable training corpus (i.e., a corpus that matches the test domain) is available. To shed light on this issue, this paper discusses a cross-domain gender classification task based on four domains (Facebook, crowd sourced opinions, Blogs and E-gov requests) in the Brazilian Portuguese language. A number of simple gender classification models using word- and psycholinguistics-based features alike are introduced, and their results are compared in two kinds of cross-domain setting: first, by making use of a single text source as training data for each task, and subsequently by combining multiple sources. Results confirm previous findings related to the effects of corpus size and domain similarity in English, and pave the way for further studies in the field.
Creating a Dataset for Named Entity Recognition in the Archaeology Domain

In this paper, we present the development of a training dataset for Dutch Named Entity Recognition (NER) in the archaeology domain. This dataset was created as there is a dire need for semantic search within archaeology, in order to allow archaeologists to find structured information in collections of Dutch excavation reports, currently totalling around 60,000 (658 million words) and growing rapidly. To guide this search task, NER is needed. We created rigorous annotation guidelines in an iterative process, then instructed five archaeology students to annotate a number of documents. The resulting dataset contains ~31k annotations between six entity types (artefact, time period, place, context, species & material). The inter-annotator agreement is 0.95, and when we used this data for machine learning, we observed an increase in F1 score from 0.51 to 0.70 in comparison to a machine learning model trained on a dataset created in prior work. This indicates that the data is of high quality, and can confidently be used to train NER classifiers.
Optimizing Annotation Effort Using Active Learning Strategies: A Sentiment Analysis Case Study in Persian

Deep learning models are the current State-of-the-art methodologies towards many real-world problems. However, they need a substantial amount of labeled data to be trained appropriately. Acquiring labeled data can be challenging in some particular domains or less-resourced languages. There are some practical solutions regarding these issues, such as Active Learning and Transfer Learning. Active learning’s idea is simple: let the model choose the samples for annotation instead of labeling the whole dataset. This method leads to a more efficient annotation process. Active Learning models can achieve the baseline performance (the accuracy of the model trained on the whole dataset), with a considerably lower amount of labeled data. Several active learning approaches are tested in this work, and their compatibility with Persian is examined using a brand-new sentiment analysis dataset that is also introduced in this work. MirasOpinion, which to our knowledge is the largest Persian sentiment analysis dataset, is crawled from a Persian e-commerce website and annotated using a crowd-sourcing policy. LDA sampling, which is an efficient Active Learning strategy using Topic Modeling, is proposed in this research. Active Learning Strategies have shown promising results in the Persian language, and LDA sampling showed a competitive performance compared to other approaches.
EMPAC: an English–Spanish Corpus of Institutional Subtitles

The EuroparlTV Multimedia Parallel Corpus (EMPAC) is a collection of subtitles in English and Spanish for videos from the EuropeanParliament’s Multimedia Centre. The corpus has been compiled with the EMPAC toolkit. The aim of this corpus is to provide a resource to study institutional subtitling on the one hand, and, on the other hand, facilitate the analysis of web accessibility to institutional multimedia content. The corpus covers a time span from 2009 to 2017, it is made up of 4,000 texts amounting to two and half millions of tokens for every language, corresponding to approximately 280 hours of video. This paper provides 1) a review of related corpora; 2) a revision of typical compilation methodologies of subtitle corpora; 3) a detailed account of the corpus compilation methodology followed; and, 4) a description of the corpus. In the conclusion, the key findings are summarised regarding formal aspects of the subtitles conditioning the accessibility to the multimedia content of the EuroparlTV.
Some Issues with Building a Multilingual Wordnet

In this paper we discuss the experience of bringing together over 40 different wordnets. We introduce some extensions to the GWA wordnet LMF format proposed in Vossen et al. (2016) and look at how this new information can be displayed. Notable extensions include: confidence, corpus frequency, orthographic variants, lexicalized and non-lexicalized synsets and lemmas, new parts of speech, and more. Many of these extensions already exist in multiple wordnets – the challenge was to find a compatible representation. To this end, we introduce a new version of the Open Multilingual Wordnet (Bond and Foster, 2013), that integrates a new set of tools that tests the extensions introduced by this new format, while also ensuring the integrity of the Collaborative Interlingual Index (CILI: Bond et al., 2016), avoiding the same new concept to be introduced through multiple projects.
Attenuation of visceral nociception by α- and β-amyrin, a triterpenoid mixture isolated from the resin of Protium heptaphyllum, in mice

In the search for novel natural compounds effective against visceral nociception, the triterpenoid mixture alpha- and beta-amyrin, isolated from Protium heptaphyllum resin, was assessed in two established mouse models of visceral nociception. Mice were pretreated orally with alpha- and beta-amyrin (3, 10, 30, and 100 mg/kg) or vehicle, and the pain-related behavioral responses to intraperitoneal cyclophosphamide or to intracolonic mustard oil were analyzed. The triterpenoid mixture showed a dose-related significant antinociception against the cyclophosphamide-induced bladder pain, and at 100 mg/kg, the nociceptive behavioral expression was almost completely suppressed. Intracolonic mustard oil-induced nociceptive behaviors were maximally inhibited by 10 mg/kg alpha- and beta-amyrin mixture in a naloxone-reversible manner. While pretreatment with ruthenium red (3 mg/kg, s. c.), a non-specific transient receptor potential cation channel V1 (TRPV1) antagonist, also caused significant inhibition, the alpha (2)-adrenoceptor antagonist, yohimbine (2 mg/kg, s. c.), showed no significant effect. The triterpene mixture (10 mg/kg, p. o.) neither altered significantly the pentobarbital sleeping time, nor impaired the ambulation or motor coordination in open-field and rotarod tests, respectively, indicating the absence of sedative or motor abnormalities that could account for its antinociception. These results indicate that the antinociceptive potential of alpha- and beta-amyrin possibly involves the opioid and vanilloid (TRPV1) receptor mechanisms and further suggests that it could be useful to treat visceral pain of intestinal and pelvic origins.
Linking the TUFS Basic Vocabulary to the Open Multilingual Wordnet

We describe the linking of the TUFS Basic Vocabulary Modules, created for online language learning, with the Open Multilingual Wordnet. The TUFS modules have roughly 500 lexical entries in 30 languages, each with the lemma, a link across the languages, an example sentence, usage notes and sound files. The Open Multilingual Wordnet has 34 languages (11 shared with TUFS) organized into synsets linked by semantic relations, with examples and definitions for some languages. The links can be used to (i) evaluate existing wordnets, (ii) add data to these wordnets and (iii) create new open wordnets for Khmer, Korean, Lao, Mongolian, Russian, Tagalog, Urdua nd Vietnamese
A Real-World Data Resource of Complex Sensitive Sentences Based on Documents from the Monsanto Trial

In this work we present a corpus for the evaluation of sensitive information detection approaches that addresses the need for real world sensitive information for empirical studies. Our sentence corpus contains different notions of complex sensitive information that correspond to different aspects of concern in a current trial of the Monsanto company. This paper describes the annotations process, where we both employ human annotators and furthermore create automatically inferred labels regarding technical, legal and informal communication within and with employees of Monsanto, drawing on a classification of documents by lawyers involved in the Monsanto court case. We release corpus of high quality sentences and parse trees with these two types of labels on sentence level. We characterize the sensitive information via several representative sensitive information detection models, in particular both keyword-based (n-gram) approaches and recent deep learning models, namely, recurrent neural networks (LSTM) and recursive neural networks (RecNN). Data and code are made publicly available.
Are White Ravens Ever White? - Non-Literal Adjective-Noun Phrases in Polish

In the paper we describe two resources of Polish data focused on literal and metaphorical meanings of adjective-noun phrases. The first one is FigAN and consists of isolated phrases which are divided into three types: phrases with only literal meaning, with only metaphorical meaning, and phrases which can be interpreted as literal or metaphorical ones depending on a context of use. The second data is the FigSen corpus which consists of 1833 short fragments of texts containing at least one phrase from the FigAN data which may have both meanings. The corpus is annotated in two ways. One approach concerns annotation of all adjective-noun phrases. In the second approach, literal or metaphorical senses are assigned to all adjectives and nouns in the data. The paper addresses statistics of data and compares two types of annotation. The corpora were used in experiments of automatic recognition of Polish non-literal adjective noun phrases.
Development and Validation of a Corpus for Machine Humor Comprehension

This work developed a Chinese humor corpus containing 3,365 jokes collected from over 40 sources. Each joke was labeled with five levels of funniness, eight skill sets of humor, and six dimensions of intent by only one annotator. To validate the manual labels, we trained SVM (Support Vector Machine) and BERT (Bidirectional Encoder Representations from Transformers) with half of the corpus (labeled by one annotator) to predict the skill and intent labels of the other half (labeled by the other annotator). Based on two assumptions that a valid manually labeled corpus should follow, our results showed the validity for the skill and intent labels. As to the funniness label, the validation results showed that the correlation between the corpus label and user feedback rating is marginal, which implies that the funniness level is a harder annotation problem to be solved. The contribution of this work is two folds: 1) a Chinese humor corpus is developed with labels of humor skills, intents, and funniness, which allows machines to learn more intricate humor framing, effect, and amusing level to predict and respond in proper context (https://github.com/SamTseng/Chinese_Humor_MultiLabeled). 2) An approach to verify whether a minimum human labeled corpus is valid or not, which facilitates the validation of low-resource corpora.
Sense-Annotated Corpora for Word Sense Disambiguation in Multiple Languages and Domains

The knowledge acquisition bottleneck problem dramatically hampers the creation of sense-annotated data for Word Sense Disambiguation (WSD). Sense-annotated data are scarce for English and almost absent for other languages. This limits the range of action of deep-learning approaches, which today are at the base of any NLP task and are hungry for data. We mitigate this issue and encourage further research in multilingual WSD by releasing to the NLP community five large datasets annotated with word-senses in five different languages, namely, English, French, Italian, German and Spanish, and 5 distinct datasets in English, each for a different semantic domain. We show that supervised WSD models trained on our data attain higher performance than when trained on other automatically-created corpora. We release all our data containing more than 15 million annotated instances in 5 different languages at http://trainomatic.org/onesec.
Perfil de ácidos graxos do leite de vacas alimentadas com óleo de soja e monensina no início da lactação

ABSTRACT - The objective of this trial was to evaluate the effects of dietary monensin and soybean oil on milk fatty acid(FA) profile in the 5 th and 15 th week of lactation of dairy cows. Sixteen multiparous crossbred dairy cows averaging 30 days inmilk were assigned to a completely randomized block design in a 2 x 2 factorial arrangement (presence or absence of monensinand soybean oil). The following diets were used: control not supplemented with monensin or soybean oil (CT), 33 ppm of monensin(MN), 3.9% of soybean oil (OL) or a combination of soybean oil plus monensin (OM). Cows were confined and fed diets with52% of corn silage and 48% of concentrate. No significant interaction between soybean oil and monensin was observed for anymeasured FA. Monensin increased unsaturated, monounsaturated, and polyunsaturated FA by 9.0, 8.8 and 10.7%, respectively,while supplementation with soybean oil resulted in greater responses: 39.2, 39.3, and 24.2% for the same FA. Soybean oil alsoreduced short chain FA (43.7%) and medium chain FA (49.1%) and increased long chain FA (55.3%) in this study. The isomers
A mixed finite element scheme for biharmonic equation with variable coefficient and von Kármán equations

In this paper, a new mixed finite element scheme using element-wise stabilization is introduced for the biharmonic equation with variable coefficient on Lipschitz polyhedral domains. The proposed scheme doesn't involve any integration along mesh interfaces. The gradient of the solution is approximated by $H({\rm div})$-conforming $BDM_{k+1}$ element or vector valued Lagrange element with order $k+1$, while the solution is approximated by Lagrange element with order $k+2$ for any $k\geq 0$. This scheme can be easily implemented and produces positive definite linear system. We provide a new discrete $H^{2}$-norm stability, which is useful not only in analysis of this scheme but also in $C^{0}$ interior penalty methods and DG methods. Optimal convergences in both discrete $H^{2}$-norm and $L^{2}$-norm are derived. This scheme with its analysis is further generalized to the von Karman equations. Finally, numerical results verifying the theoretical estimates of the proposed algorithms are also presented.
Fine-grained Morphosyntactic Analysis and Generation Tools for More Than One Thousand Languages

Exploiting the broad translation of the Bible into the world’s languages, we train and distribute morphosyntactic tools for approximately one thousand languages, vastly outstripping previous distributions of tools devoted to the processing of inflectional morphology. Evaluation of the tools on a subset of available inflectional dictionaries demonstrates strong initial models, supplemented and improved through ensembling and dictionary-based reranking. Likewise, a novel type-to-token based evaluation metric allows us to confirm that models generalize well across rare and common forms alike
Fine-grained Named Entity Annotations for German Biographic Interviews

We present a fine-grained NER annotations with 30 labels and apply it to German data. Building on the OntoNotes 5.0 NER inventory, our scheme is adapted for a corpus of transcripts of biographic interviews by adding categories for AGE and LAN(guage) and also features extended numeric and temporal categories. Applying the scheme to the spoken data as well as a collection of teaser tweets from newspaper sites, we can confirm its generality for both domains, also achieving good inter-annotator agreement. We also show empirically how our inventory relates to the well-established 4-category NER inventory by re-annotating a subset of the GermEval 2014 NER coarse-grained dataset with our fine label inventory. Finally, we use a BERT-based system to establish some baseline models for NER tagging on our two new datasets. Global results in in-domain testing are quite high on the two datasets, near what was achieved for the coarse inventory on the CoNLLL2003 data. Cross-domain testing produces much lower results due to the severe domain differences.