dataesr/scientific-paragraphs-categorization huggingface dataset:
Our model The CWN model described above has several drawbacks: (i) each tag prediction is made by considering only local information, i. e. , no dependencies between the output tags are taken into account; (ii) publicly available annotated datasets for NER are usually too small to train neural networks thus often leading to overfitting. We address both problems by proposing: (i) a novel recurrent context window network (RCWN) architecture; (ii) a network pre-training technique using weakly labeled data; and (iii) we also experiment with a set of recent techniques to improve the generalization of our DNN to avoid overfitting, i. e. , we use early stopping  (Prechelt, 1998) , weight decay  (Krogh and Hertz, 1992) , and Dropout  (Hinton, 2014).
Experimental setup Dataset. We evaluate our models on the Evalita 2009 Italian dataset for NERs  (Speranza, 2009)  summarized in Tab. 1. There are four types of NEs: person (PER), location (LOC), organization (ORG) and geo-political entity (GPE), (see Tab. 2). Data is annotated using the IOB tagging schema, i. e. , for inside, outside and beginning of a entity, respectively. Training and testing the network. We use (i) the Negative Log Likelihood cost function,  i. e. , -log(p c ), where c is the correct label for the target word, (ii) stochastic gradient descent (SGD) to learn the parameters of the network and (iii) the backpropogation algorithm to compute the updates. At test time, the tag c, associated with the highest class conditional probability p c , is selected, i. e. , c = argmax c∈C p c.
Features. In addition to words, all our models also use 4 basic morphological features: all lowercase, all uppercase, capitalized and it contains uppercase character. These can reduce the size of the word embedding dictionary as showed by  (Collobert et al. , 2011)   (Bengio, 2009) , word embeddings  (Collobert et al. , 2011)  or by relying on distant supervised datasets  (Severyn and Moschitti, 2015). Here, we propose a pretraining technique using an off-the-shelf NER to generate noisily annotated data, e. g. , a sort of distance/weakly supervision or self-training. Our Weakly Labeled Dataset (WLD) is built by automatically annotating articles from the local newspaper "L'Adige", which is the same source of the training and test sets of Evalita challenge. We split the articles in sentences and tokenized them.
Results Our models are evaluated on the Evalita 2009 dataset. We applied 10-fold cross-validation to the training set of the challenge  4  for performing parameter tuning and picking the best models. Table  3  reports performance of our models averaged over 10-folds. We note that (i) modeling the output dependencies with RCWN leads to a considerable improvement in F1 over the CWN model of  Collobert et al. (2011)  (our baseline); (ii) adding the gazetteer features leads to an improvement both in Precision and Recall, and therefore in F1; and (iii) pre-training the network on the weakly labeled training set produces improvement (although small), which is due to a better initialization of the network weights. Table  4  shows the comparative results between our models and the current state of the art for Italian NER on the Evalita 2009 official test set.
Conclusion In this paper, we propose a new DNN for designing NERs in Italian. Its main characteristics are: (i) the RCWN feedback method, which can model dependencies of the output label sequence and (ii) a pre-training technique involving a weakly supervised dataset. Our system is rather simple and efficient as it involves only one model at test time. Additionally, it does not require time-consuming feature engineering or extensive data processing for their extraction. In the future, we would like to apply rerankers to our methods and explore combinations of DNNs with structural kernels.
The Open Multilingual WordNet (OMW) 4 is an open-source multilingual database that connects all open WNs linked to the English WN, including Italian  (Pianta et al. , 2002)  among the 28 languages supported  (Bond and Paik, 2012; Bond and Foster, 2013). Another valid option for the multilingual sense inventory would be BabelNet, created from the automatic integration of WN 3. 0, OMW, Wikipedia and many other resources  (Navigli and Ponzetto, 2012) , with an estimated accuracy of 91% for the WN-Wikipedia mapping  (Navigli et al. , 2013). However, we chose to use OMW since we wanted to test our hypothesis on resources that were purposely built to be mapped to one another. The Romanian WordNet (RW) was created within the BalkaNet project  (Stamou et al. , 2002). The current version has 59,348 synsets in its latest release  (Barbu Mititelu et al.
Introduction The use of naturalistic data to investigate child language features and development has a well-established tradition in L1 acquisition research. The most notable example is the CHILDES database  (MacWhinney, 2000) , which contains transcripts of spoken interactions involving children of different ages for over 25 languages, Italian included. Yet, CHILDES data refer especially to preschool children, with only a minor section dedicated to their older mates, thus making this resource less adequate for studying how language skills evolve during early schooling. The rapid and remarkable changes children's language undergoes before age five justify the amount of research for the earliest stages of acquisition. However, over the last two decades also "later language acquisition" has gained increasing interest  (Tolchinsky, 2004) , prompted by the awareness that "becoming a native speaker is a rapid and highly efficient process, but becoming a proficient speaker takes a long time"  (Berman, 2004).
We define therefore a new framework involving entailment graphs, where pairs of textual fragments connected by semantic relations are also part of a graph that provides an overall view of the statements' interactions (bipolar entailment graphs). This opens new challenges for TE, that originally considers the pairs as "self-contained" (i. e. , the meaning of one text has to be derived from the meaning of the other). One challenge consists in checking BEGs to identify possible inconsistencies due to wrong relation assignments by the TE system. Figure  1  shows the architecture of the BE-Gincs framework to support human annotators in detecting inconsistencies in TE graphs. Annotators provide the dataset to be checked as input of the BEGincs framework, which consists of two main modules: (1) a TE module, takes as input the dataset of text fragments, and returns the pairs annotated with the entailment or contradiction relations; and (2) a BEG-AF Inconsistencies Detection module, which translates the received BEGs into an argumentation framework such that argumentation semantics can be applied to retrieve consistent sets of arguments.
Experimental setting Data set. We added 60 pairs to the Debatepedia dataset 2  (extracted from a sample of Debatepedia 3  debates  (Cabrio and Villata, 2012) ), resulting in 160 pairs as training set, and 100 pairs as test set (balanced wrt to entailment/contradiction). Evaluation. First step: we assess the performances of the TE system to correctly assign the TE relations to the pairs of arguments in the dataset. Second step: we evaluate how much such performances impact on the flattening of the BEG-AF, i. e. , how much a wrong assignment of a relation to a pair of arguments is propagated in the AF. It is actually to detect such wrong assignments that the BEGincs framework has been conceived. To recognize TE, we tested several algorithms from the EOP 4  , i. e. BIUTEE  (Stern and Dagan, 2011) , TIE 5  and EDITS  (Kouylekov and Negri, 2010).
g. when mixing messages about different topics, sentiment detection can be difficult. We take inspiration from the contexts defined in  (Vanzo et al. , 2014b). In  (Speriosu et al. , 2011)  no explicit relation between messages is considered. We, instead, build a graph where messages in the same context are linked each other and to the words appearing in them. Moreover, we inject prior polarity of words as available in a polarity lexicon  (Castellucci et al. , 2015). Experiments are carried out over a subset of the Evalita 2014 Sentipolc  (Basile et al. , 2014)  dataset, showing improvements in the polarity classification with respect to not using networked information. In the remaining, Section 2 presents our graphbased approach. In Section 3 we evaluate the pro-posed method with respect to a dataset in Italian and we derive the conclusions in Section 4.
Experimental Evaluation In order to evaluate the Contextual Graph and the MAD algorithm, we adopted a subset of the Evalita 2014 Sentipolc dataset  (Basile et al. , 2014). It consists of short messages annotated with the subjectivity, polarity and irony classes. We selected those messages annotated with polarity and that were not expressing any ironic content to focus our investigations on less ambiguous messages. Thus, the datasets used for our evaluations consist of a training set T r of 2,449 messages and a testing set T s of 1,129 messages. (Vanzo et al. , 2014b) , we downloaded the conversation and hashtag contexts that were available at the time of downloading  4. In Table  1  the number of messages involved in the different contexts are shown. In the experiments, messages are classified with respect to the positive, negative and neutral polarity classes.
The SVM classifier realizes the function f that assign the initial label distribution, reflecting the classifier confidence in assigning a polarity to each message, i. e. belonging to both train and test datasets, as well as belonging to contexts. Words seeding. Seeds words are also considered when building the Contextual Graph. In particular, we adopt the Distributional Polarity Lexicon (DPL)  (Castellucci et al. , 2015)  that associates each word to the prior information about the positivity, negativity and neutrality. The lexicon is built as follows: a classifier d is acquired from a dataset of generic messages gathered by Twitter considering the occurrence of noisy labels, i. e. emoticons expressing positivity, negativity or neutrality. In a nutshell, given the properties of the vector space WS, we project sentences and words in the same space, in order to transfer the polarity from sentences to words via the classifier d and obtaining the polarity scores of the DPL.
Experimental Results A first measure is given by the SVM classifier that is used to assign a polarity distribution to seeds belonging to the test dataset. We measure the mean between the F1 scores of the positive and negative classes (F1-Pn), and the mean between the F1 scores of all the three classes (F1-Pnn). Different feature representations are exploited in the SVM evaluation, as pointed out in Table  3. When adopting also the WS features, the performance increases in both the performance measures, with respect to the setting where only BOW features are considered. In Tables  4  and 5  the MAD algorithm results 7  are reported w. r. t. the Conversation and Hashtag contexts, as well to different context sizes, e. g. at size 3 we consider a maximum of 2 messages preceding a target one. The MAD algorithm is able to consistently increase the F1-Pn performance measure, while it is equally performing in the F1-Pnn.
), psychological processes (anxiety, anger, feeling, cognitive mechanisms, etc. ), and personal concerns (money, religion, leisure, TV, achievement, home, sleep. etc. ). In the next section we report the results of the analysis of the data collected. work to the analysis of message level, leaving to future work the possibility to extend this analysis to the characteristics of the media or the participants. Experiments First of all we analyzed the topics in Facebook and meeting transcriptions. We removed the stopwords and we generated two word clouds with the 70 most frequent words in each dataset with 5 as minimum term frequency. We report the word clouds in Figure  2. The comparison of the two clouds reveal that participants to the experiments in Facebook discussed and planned actions ("dormire", "andare") places ("rimini", "copenhagen") and times ("sera", stasera", "domani") while in meetings they told and discussed mainly about places ("bologna", "rimini") and people ("tipo", "gente").
To this purpose, our team has inspected the panorama of LR descriptions available in the aforementioned catalogues, and in particular the LRE Map which allows us to monitor how communities build around LR use. Our hypothesis is that many of the resources that the Italian community uses and produces are not as well documented as they should be. As a consequence, many researchers may not be aware of the existence of resources that could be of use for them, and limit themselves to those they know best. In order to verify this, we carried out a cross-analysis of Italian LRs and their uses by Italian researchers, exploiting the data found in the LRE Map from the LREC2014 dataset, which is currently available in LOD format  (Del Gratta et al. , 2014a). Such data is compared with similar evidence gathered from the proceedings of the CLiC-it 2014 conference, which are available online.
Metadata description The set of metadata used for documenting language resources can vary from repository to repository. Some harmonization initiatives are currently being carried out in order to make diverse datasets interoperable, e. g. (McCrae et al. , 2015). Nevertheless a common core has been broadly agreed upon by all; this includes type of resource (corpus, lexicon, tool), modality, language(s), use, availability. To this core set of metadata, the LRE Map adds other metadata that are linked not to the resource itself, but to its use in the paper that is being submitted: thus information about the conference, the paper, the authors and their affiliations is available for each entry in the LRE Map. This also means that any given resource can have more than just one entry in the LRE Map, one for each paper that has used it.
Sometimes the resource is marked as new, and in that case we can assume that the authors of the paper are also the producers of this new resource; in most cases the resource is a well known one. So for instance some of the most used resources according to the LRE Map are Princeton WordNet and the British National Corpus. For the purposes of this paper we only took into consideration the following metadata for each entry in the LREC2014 LRE Map: resource name, language, authors and affiliations. We extracted all used LRs with Italian as one of the languages and authors with an Italian affiliation. We then analysed the proceedings of CLiC-it 2014 and manually extracted the same type of information for each paper 2. We thus obtained two datasets: 2 One of the most interesting features of the LRE Map is the fact that it provides a user's perspective on language resources.
People and Resources: visualising networks Data visualisation is a method that enables the exploration, filtering and searching of data, skipping the interaction with databases. Data can be mainly visualised for presentation or exploration but in well designed projects there is a continuum between these two modalities  (Cairo, 2013). In this paper we propose two visualisation modalities to discover the interrelations between authors from different institutions and the convergence of authors on the usage of the same resource. In comparing these two conferences the aim was to portray the Italian NLP community highlighting collaborations between people through resources used. The implementation of the visualisation is based on a well known tool, D3. js, a JavaScript library designed to display digital data in a dynamic graphical form.
The first phenomenon is that at the LREC2014 there are more international collaborations between Italian and foreign groups. The first edition of CLiC-it instead presents less involvement of foreign co-authors and more collaborations between different Italian institutions. This is clearly due to the fact that CLiC-it is a national conference, while LREC an international one. The second fact is that at LREC2014 we find a smaller number of Italian LRs, as typically papers use the best known ones. CLiC-it instead presents us with a broader panorama: in addition to the best known resources we find a plethora of minor resources -in particular corpora -that are not mentioned in the LREC2014 dataset and are mostly used in a single paper. In many cases the user of the resource is also its creator: these resources need documentations to foster future collaborations.
Conclusions and future works In this work we use visualisations to show how the Italian NLP community uses LRs in the works presented at two recent conferences of the sector (LREC2014 and CLiC-it 2014). We highlight how collaborations cluster around the use of major resources, and how networks are created by users of the same resource. From the comparison of the two datasets we can infer that the Italian panorama of language resources is rich and varied. We also confirm the prior hypothesis that Italian LRs are rather under-documented and that some positive action is needed in the direction of enhancing their visibility. As a consequence the creation of an observatory of Italian language resources, which is meant to be the nucleus of a newly established CLARIN-IT center, is more than justified.
Data Acquisition The first step required by this proposal is to aggregate linguistic and semantic resources such as ConceptNet, FrameNet, WordNet, BabelNet, etc. The result will be a semantic database (SDB) of lexical and semantic information. This will require integration of data from different sources with problems such as alignment, conflict resolution, and granularity mismatch. The second step regards the expansion of an input corpus (result of a selection from existing available corpora) with the semantic information contained in SDB for each of its words. Let us assume a word w i in SDB can be associated with a set σ i = {< rel a , c 1 >, < rel b , c 2 >,. , < rel k , c n >} of semantic features of the type < rel, c j > to mean that word w i has a relation rel with concept c j in some semantic resource (e.
Evaluation We compared the classification results between the baseline and the new approach which includes the extended feature set and the additional data extracted via active learning. All the experiments were conducted using the Weka package  (Hall et al. , 2009). Table  2  presents the results using the J48 classifier, an implementation of the C4. 5 algorithm for decision trees  (Quinlan, 1993) , while Table  3  presents the results using Weka's SMO classifier, a type of SVM trained using sequential minimal optimization  (Platt, 1998). In all experiments, we follow  Fernández et al. (2007)  and remove from the classification task the NSUs whose antecedents are not the preceding utterance, thus leaving a total of 1123 utterances. All empirical results were computed with 10fold cross validation over the full dataset.
Thus, for every media member the attribute value sex is always u, namely, unknown, and both role and who attribute values are always journalist. Conversely, since information about Press Secretaries and members related to the presidential administration staff was available in the transcripts, attribute values contain information about the role, gender and name of the speaker. This operation had to be made manually, but one of the main objectives of this work is to make it semi-automatic querying an existent political database that will make the process less burdensome. As many are the White House members involved in the press conferences, we decided to categorize them by role. Thus, Press Secretaries are the only ones identified as podium, due to their function of conducting the briefing. Administration officials and presidential staff members can be instead recognized by the role value podium plus the position held by them (e.
Conclusions At the time of writing, the CompWHoB Corpus is probably one of the largest political corpora mainly based on spontaneous spoken language. This feature represents one of its strongest points, as the linguistic analysis performed by employing the TRI has proved. As for the near future, two are our main goals: the first one is to make the process of structural annotation as much computational as possible by retrieving information from available political databases; the second one is to provide the corpus with syntactic parsing and improve the overall performance of the linguistic annotation process. In terms of accessibility, we intend to make the CompWHoB Corpus available via the CPQ web interface  (Hardie, 2012)  by the end of next year. For now, the fully annotated corpus is accessible and available on request.
This does not give good results with languages like Arabic which has its own syntactic and morphological features. Secondly, the corpus of evaluation is an Islamic corpus which contains a lot of Islamic terminologies which do not have a translation in other languages, but just transliteration. Regarding the domain terminology extraction from parallel corpora including the Italian language, we can mention the CLE project  (Streiter et al. , 2004) , where a trilingual corpus with legal texts in Ladin, German and Italian has been created. CLE is stored in a relational database and is accessible via the Internet through BISTRO 2 , the Juridical Terminology Information System of Bolzano. Furthermore, there is the LexALP project  (Lyding et al. , 2006) , where sophisticated tools have been developed for the collection, description and harmonization of the legal terminology of spatial planning and sustainable development in four languages, namely French, German, Italian and Slovene.
Introduction Collecting and manually annotating linguistic data (typically referred to as gold standard) is a very expensive activity, both in terms of time and effort  (Tomanek et al. , 2007). For this reason, in the last years the question of whether we can train good Natural Language Processing (NLP) models by using just automatically annotated data (called silver standard) is arising interest  (Hahn et al. , 2010; Chowdhury and Lavelli, 2011). In this case, human annotations are replaced by those generated by pre-existing state-of-the-art systems. The annotations are then merged by using a committee approach specifically tailored on the data  (Rebholz-Schuhmann et al. , 2010a). The key advantage of such approach is the possibility to drastically reduce both time and effort, therefore generating considerably larger data sets in a fraction of the time.
This is particularly true for text data in different fields such as temporal information extraction  (Filannino et al. , 2013) , text chunking  (Kang et al. , 2012)  and named entity recognition  (Rebholz-Schuhmann et al. , 2010b; Nothman et al. , 2013)  to cite just a few, and for non-textual data like in in medical imaging recognition  (Langs et al. , 2013). In this paper we focus on the case of dependency parsing for the Italian language. Dependency parsers are systems that automatically generate the linguistic dependency structure of a given sentence  (Nivre, 2005). An example is given in Figure  1  for the sentence "Essenziale per l'innesco delle reazioni è la presenza di radiazione solare. " (The presence of solar radiation is essential for triggering the reactions). We investigate whether the use of very large silver standard corpora leads to train good dependency parsers, in order to address the following question: Which characteristic is more important for a training set: quantity or quality? The paper is organised as follows: Section 2 presents some background works on dependency parsers for Italian; Section 3 presents the silver standard corpus used for the experiments and its linguistic features, with Section 4 describing the experimental settings and Section 5 describing the results of the comparison between the trained parsers (considering different sizes of data) and two test sets: gold and silver.