dataesr/scientific-paragraphs-categorization huggingface dataset:
Related work A number of semantically annotated corpora are available for English, whereas most other languages are under-resourced. As far as Italian is concerned, WItaC is the first corpus offering annotations of entities, events, and event factuality, together with semantic role labeling and crossdocument coreference annotation. For entities and entity coreference, the reference Italian corpus is I-CAB  (Magnini et al. , 2006) , which is annotated with entities, time expressions (following the TIMEX2 standard), and intradocument entity coreference; for cross-document person entity coreference, we refer to CRIPCO  (Bentivogli et al. , 2008). Regarding temporal information and event factuality, two annotated corpora are available: respectively, the EVENTI corpus  (Caselli et al. , 2014) , used as the evaluation dataset for the EVENTI task at Evalita 2014 and annotated with events, time expressions (TIMEX3), temporal signals, and temporal relations, and Fact-Ita Bank  (Minard et al.
Dataset Description WItaC is composed of 120 articles. In Table  1  we give the size of the whole corpus and the size of the "first 5 sentences" section, i. e. the subsection annotated with markables, relations, intra-document coreference and cross-document entity coreference. In total 6,127 markables have been annotated in Italian; of these, 5,580 are aligned to English markables while 547 have no English correspondent. As a result of the projection of event and entity cross-document coreference chains from English, WItaC contains 740 entity instances and 887 event instances annotated at the corpus level. Annotation by projection enables us to also have crosslingual annotation, which means that the instances are shared between English and Italian.
We also plan to use the corpus as a dataset for a shared evaluation task and afterwards we will make it freely available from the website of the HLT-NLP group at FBK 6  and from the website of the NewsReader project. Parsing Events: a New Perspective on Old Challenges Rachele Sprugnoli 1  , Felice Dell'Orletta 2  , Tommaso Caselli 3  , Simonetta Montemagni 2 , Cristina Bosco 4  1 Fondazione Bruno Kessler -Università di Trento 2 Istituto di Linguistica Computazionale "Antonio Zampolli" -CNR, Pisa 3 VU Amsterdam, 4 Dipartimento di Informatica -Università di Torino sprugnoli@fbk. eu, felice. dellorletta@ilc. cnr. it, t. caselli@vu. nl simonetta. montemagni@ilc. cnr. it, bosco@di. unito. it Abstract English. The paper proposes a new evaluation exercise, meant to shed light on the syntax-semantics interface for the analysis of written Italian and resulting from the combination of the EVALITA 2014 dependency parsing and event extraction tasks.
Task Description Parsing Events will qualify as a new evaluation exercise for promoting research in Information Extraction and access to the text meaning for Italian. The exercise, which will start from previous research and datasets for Dependency Parsing and Temporal Processing of Italian, aims at opening a new perspective for what concerns the evaluation of systems to be carried out both at a high level, targeting complex Information Extraction architectures, and at a low level, as single components. The Parsing Events exercise will be thus articulated as follows: a main task, joint dependency parsing and event extraction, and two subtasks, dependency parsing and event extraction, respectively. Main task -Joint Dependency Parsing and Event Extraction: The main task will test systems for Dependency Parsing and Event Extraction.
Annotation and Data Format In the spirit of re-using available datasets, the annotation efforts for the Parsing Events task will be mainly devoted to the creation of a new test set, called Platinum data, which will contain manual annotation for both dependency parsing and events. The size of the Platinum data will be around 10k-20k tokens. The annotation of the dataset will be conducted by applying the ISDT guidelines for the dependency parsing information and the EVENTI guidelines for events. An innovative aspect of the Platinum data concerns the text genres. To provide a more reliable evaluation, the Platinum data will consist of newspaper articles and biographies from Wikipedia 7. The training data (Gold data) will be based on the EVENTI and the Dependency Parsing data. A subset of 27,597 tokens between the two datasets perfectly overlaps, thus making already available Gold annotations.
Given that the focus of the evaluation exercise is on the reciprocal influence of the two basic tasks, we will provide the missing annotations on the remaining parts (i. e. 102,682 tokens for the EVENTI dataset and 160,398 tokens for the Dependency Parsing dataset) by means of automatically generated annotation, i. e. Silver data. Silver data have already been successfully used to extend the size of training data in previous evaluation exercises (e. g. TempEval-3). Furthermore, we plan to extend the set of overlapping Gold data by manual revision. Training data will be distributed in a unified representation format compliant with the CoNLL-X specifications  (Buchholz and Marsi, 2006)  and extended for the encoding of event information which will be annotated in terms of standard IOB representation as exemplified in Figure  1  (the example is taken from the overlapping portion of the training data of the two task at EVALITA 2014).
Event annotation (last seven columns) is concerned with the following information types: event extent, class, tense, aspect, vform, mood and polarity. The test set for the main task will be distributed in the same format of the training dataset providing participants with pre-tokenized, POS-tagged and lemmatized data. This distribution format will be adopted also for the two subtasks. In addition to the information regarding tokens, POS tags and lemmas, Gold data for events will be available for the dependency parsing subtask, while Gold data for dependency parsing will be available for the event extraction subtask. Systems will be required to produce a tabdelimited file. Systems participating to the main task will provide in output the extended CoNLL-X format including the information for the event an- notation as shown in Figure  1.
Introduction Native Language Identification (NLI) is the task of recognizing an author's native language (L1) from text written in a second language (L2). NLI is important for applications such as the detection of phishing attacks  (Estival et al. , 2007)  or data collection for the study of L2 acquisition  (Odlin, 1989). State-of-the-art methods couch NLI as a classification task, where the classes are the L1 of the author and the features are supposed to model the effects of the author's L1 on L2 (language transfer). Such features may be of varying linguistic sophistication, from function words and structural features  (Tetreault et al. , 2012)  on one side to N-grams over characters, words and POS tags  (Brooke and Hirst, 2011; Bykh and Meurers, 2012)  on the other side. Like in many NLP tasks, there are few large datasets for NLI.
This experiment establishes an upper bound of performance for when a lot of target domain data is available. Simple CONCATentation does not perform well, with degradation compared to TGTonly for ICLE and, with a notably large slump, ACL. Feature Augmentation works to some extent, but mSDA improves the results much more substantially, to almost 90% accuracy and above, and yielding the largest improvements over TGT-only (ICLE: +2. 8%, Lang-8: +12. 9%, ACL:+2. 7%). We surmise that FA is handicapped by the relatively small sizes of Lang-8, and the very small size of the ACL corpus, which are "overpowered" by the large TOEFL-11 dataset. SRC plus some TGT data. The final group of results (-small) shows the results of combining the SRC data with only half the available TGT data. In comparison to -big, the performance drops substantially for CONCAT and FA, but only somewhat for mSDA (ICLE: -0.
3%, Lang-8: -3. 2%, ACL: -2. 4%; difference statistically significant only for Lang-8). This indicates that mSDA can take advantage of relatively small target domain datasets. Summary. Domain adaptation, in particular mSDA, can construct highly accurate NLI models (85%+) by combining large source datasets with relatively small target datasets. Contrary to Table  4 : Accuracies on reduced feature set expectations, we do not see a clear division between learner and scientific datasets: rather, the less well controlled Lang-8 behaves much more like the ACL dataset than like ICLE, which in turns clusters together with the TOEFL-11 dataset, the other "classical" learner corpus. This explains the good generalization results found by  Bykh and Meurers (2012)  but indicates that they may be restricted to "classical" learner corpora.
Abstract English. The paper describes a research about the socio-political debate on the reform of the education sector in Italy. It includes the development of an Italian dataset for sentiment analysis from two different comparable sources: Twitter and the online institutional platform implemented for supporting the debate. We describe the collection methodology, which is based on theoretical hypotheses about the communicative behavior of actors in the debate, the annotation scheme and the results of its application to the collected dataset. Finally, a comparative analysis of data is presented. Italiano. L'articolo descrive un progetto di ricerca sul dibattito socio-politico sulla riforma della scuola in Italia, che include lo sviluppo di un dataset per la sentiment analysis della lingua italiana estratto da due differenti fonti tra loro confrontabili: Twitter e la piattaforma istituzionale online implementata per supportare il dibattito.
The data selected to create the corpus have been chosen by analyzing in Twitter and other contexts the diffusion of three hashtags, i. e. #labuonascuola, #italicum, #jobsact. In particular, we focus on #labuonascuola (the good school), which was coined to communicate the school reform proposed by the actual government. A side effect of our work is the development of a new lexical resource for sentiment analysis in Italian, a currently under-resourced language. Among the existing resources let us mention Senti-TUT  (Bosco et al. , 2013) , which has been exploited together with the TWITA corpus  (Basile and Nissim, 2013)  for building the training and testing datasets in the SENTIment POLarity Classification shared task  (Basile et al. , 2014)  recently proposed during the last edition of the evaluation campaign for Italian NLP tools and resources  (Attardi et al.
, 2015). The Sentipolc's dataset includes tweets collected during the alternation between Berlusconi and Monti on the chair of Prime Minister of the Italian government. The current proposal aims at expanding the available Italian Twitter data annotated with sentiment labels on the topic of politics, and it is compatible with the existing datasets w. r. t. the annotation scheme and other features. The paper is organized as follows. The next section describes the dataset mainly focusing on collection. In the section 3 we describe the annotation applied to the collected data and the annotation process. Section 4 concludes the paper with a discussion of the analysis applied to the dataset.
Data collection: criteria and subcorpora In this section, we describe the methodology applied in collection, which depends on some assumption about the dynamics of the debate, and the features of the resulting dataset, which is organized in two different subcorpora: the Twitter dataset (TW-BS) and the dataset including the textual comments extracted from the online consultation about the reform (WEB-BS). In order to describe the communicative behavior of the government, we assume, as a theoretical hypothesis, that the communication strategy acting in the debate can be usefully modeled by exploiting frames. In political communication, this cognitive strategy led to impose a narration to opponents  (Conoscenti, 2011). Following this hypothesis we can see that the Prime Minister and his staff coined two categories of frames by hashtagging in order to impose a narration to the public opinion: the first one aimed at legitimating the new born government and its novelty in the political arena (#lavoltabuona; #passodopopasso); the other one in order to create a general agreement on some proposal (#labuonascuola, #italicum, #jobsact).
gov. it. The first corpus we collected, WEB-BS henceforth, includes therefore texts from this online consultation 1. We collected 4,129 messages composed by short texts posted in the consultation platform. All contents were manually tagged by authors with one among the 53 sub-topics labels made available, and organized by themselves in four categories: 'what I liked' (642), 'what I didn't like'(892), 'what is missing' (675) and 'new integration' (1,920). So, the label which conveys a positive opinion represented the 15. 55% of the total. Otherwise, the negative label has been used the 21,60% of times. This manual classification in sub-topic and polarity categories of the messages, makes the WEB-BS dataset especially interesting, since the explicit tagging applied by the users can be in principle compared with the results of some automatic sentiment or topic detection engine.
In addition to #labuonascuola, we used also keywords like 'la buona scuola', 'buona scuola', 'riforma scuola', 'riforma istruzione'. The resulting dataset is composed of 35,148 tweets, which was first reduced to 11,818 after removing retweets, and then to 8,594 after a manual revision devoted to further deletion of duplicates and partial duplicates. A quantitative analysis of the collected data shows us that 4,244 users contributed to the debate on Twitter. Among them, only 1,238 (29,2%) posted at least 2 messages and produced 5,588 tweets, 65% of the total. If we consider the hashtags' occurrences, #labuonascuola appears 5,346 times, while its parodic reprise is very infrequent: 108 total occurrences for three hashtags #lacattivascuola -#thebadschool, #lascuolaingiusta -#theunfairschool, and #labuonasola -#thegoodswindle.
Furthermore, we coined two more general labels for tweets addressing a sub-topic not present in categories, and for tweets just indirectly targeted to school reform. In order to limit biases among annotators and to make well shared the meaning of all the labels to be annotated, we produced a document including guidelines for annotations, several examples of polarity-labeled tweets, three glossaries about the meaning of the topics-and some recurrent terms on the school reform. The final dataset, manually annotated by two independent human annotators and cleaned from duplicates, not related, and unintelligible tweets, consists of 7,049 posts. 4,813 out of the total amount of annotated tweets, were tagged with the same label by both annotators. This is the current result for TW-BS; the label distribution is shown in 1.
09% of the total) against 263 positive posts (10. 7%). The disparity is amplified when we take into account just ironic tweets. The use of irony for conveying a positive opinion is very rare (18 occurrences only). Table  1 : Labels distribution in TW-BS. A comparative analysis of polarity distribution in the TW-BS and the WEB-BS corpora has shown further important differences. The distribution of polarity is more balanced in the latter than in the former, where negative polarity prevails, while irony, frequently occurring in the Twitter corpus is almost absent in the other one. This confirms our theoretical hypothesis that the global contexts underlying these datasets are different, but also raises issues about the higher politeness and the cooperativeness applied by users in the consultation with respect to what is expressed in a social media context like Twitter.
Conclusions The paper describes a project for the analysis of a socio-political debate in a sentiment analysis perspective. A novel resource is presented by describing the collection and the annotation of the dataset organized in two subcorpora according to the source the texts have been extracted from: one from Twitter and one from the institutional online consultation platform. A first analysis of the resulting dataset is presented, which takes into account also a comparative perspective. Reference-free and Confidence-independent Binary Quality Estimation for Automatic Speech Recognition Hamed Zamani , José G. C. de Souza † ‡ , Matteo Negri ‡ , Marco Turchi ‡ , Daniele Falavigna ‡ School of ECE, College of Engineering, University of Tehran, Iran † University of Trento, Italy ‡ HLT research unit, Fondazione Bruno Kessler, Trento, Italy h.
In both cases, the resulting datasets will likely be rather imbalanced, which is a challenging condition from the learning perspective. Approaches. We experiment with two different strategies. The first one, classification via regression, represents the easiest way to adapt the method proposed in  (Negri et al. , 2014). It fits a regression model on the original training instances, applies it to the test data, and finally maps the predicted regression scores into good/bad labels according to τ. The second one is standard classification, which partitions the training data into good/bad instances according to τ , trains a binary classifier on such data, and finally applies the learned model on the test set. The two strategies have pros and cons that are worth to consider. On one side, classification via regression directly learns from the WER labels of the training points.
Experiments Dataset. We experiment with the ASR data released for the 2012 and 2013 editions of the IWSLT evaluation campaign  (Federico et al. , 2012; Cettolo et al. , 2013)  respectively consisting of 11 and 28 English TED talks. The 2012 test set, which has a total speech duration of around 1h45min, contains 1, 118 reference sentences and 18, 613 running words. The 2013 test set has a total duration of around 3h55min, it contains 2, 238 references and 41, 545 running words. In our experiments, we always use 1, 118 utterances for training and 1, 120 utterances for testing. To this aim, the (larger) IWSLT 2013 test set is randomly sampled three times in training and test sets of such dimensions. The use of two datasets is moti-vated by the objective of measuring variations in the classification performance of our quality estimators under different conditions: i) homogeneous training and test data from the same edition of the campaign, and ii) heterogeneous training and test data from different editions of the campaign.
Acknowledgements The authors thank all the persons who supported the work. We are grateful to our annotators, in particular to  Valentina Azer ,  Marta Benenti ,  Martina Brignolo ,  Enrico Grosso  and  Maurizio Stranisci. This research is supported in part by  Fondazione Giovanni Goria e Fordazione CRT  (grant Master dei  Talenti   2014 ;  Marco Stranisci ) and in part by the  National Council for Science and Technology, CONACyT Mexico  (Grant No. 218109 ,  CVU-369616 ;  D. I. Hernández Farías ).
La rete utilizza una finestra scorrevole di contesti delle parole per predire le loro etichette con associata probabilitá, la quale è us- ata come funzione di costo. Inoltre si uti- lizza un nuovo meccanismo di retroazione ricorrente per modellare le dipendenze tra le etichette di uscita. La valutazione della DNN sul dataset di Evalita 2009 indica che è alla pari con i migliori NER e migliora lo stato dell'arte quando si aggiungono delle features derivate dai dizionari. * Google Inc. ‡.
Word Sense Discrimination: A gangplank algorithm Flavio Massimiliano Cecchini, Elisabetta Fersini Università degli Studi di Milano-Bicocca Viale Sarca 336, Ed. U14, 20126 Milano {flavio. cecchini,fersiniel}@disco. unimib. it Philip J. Stone, Dexter C. Dunphy, Marshall S. Smith, and Daniel M. Ogilvie. 1966. The General In- quirer: A Computer Approach to Content Analysis. MIT Press. Partha Pratim Talukdar and Koby Crammer. 2009. New regularized algorithms for transductive learn- ing. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II, ECML PKDD '09, pages 442- 457, Berlin, Heidelberg. Springer-Verlag. Andrea Vanzo, Giuseppe Castellucci, Danilo Croce, and Roberto Basili. 2014a. A context based model for sentiment analysis in twitter for the italian lan- guage.
sezioni 2 e 4 ed ha realizzato e valutato i dataset; Alessandro Panunzi ha supervisionato il lavoro e la scrittura dell'articolo. La ricerca è stata condotta nellambito del progetto MODELACT, programma Futuro in Ricerca 2012. Project code: RBFR12C6O8 2012-2015. 2  In questo ambito è particolarmente rilevante l'iniziativa del Linguistic Linked Open Data Cloud (Chiarcos et al. , 2011) , che raccoglie e collega ontologie linguistiche in RDF e ad oggi contiene più di 500 risorse.
Multi-state models provide a relevant tool for studying the observations of a continuoustime process at arbitrary times. Markov models are often considered even if semi-Markov are better adapted in various situations. Such models are still not frequently applied mainly due to lack of available software. We have developed the R package SemiMarkov to fit homogeneous semi-Markov models to longitudinal data. The package performs maximum likelihood estimation in a parametric framework where the distributions of the sojourn times can be chosen between exponential, Weibull or exponentiated Weibull. The package computes and displays the hazard rates of sojourn times and the hazard rates of the semi-Markov process. The effects of covariates can be studied with a Cox proportional hazards model for the sojourn times distributions.
Introduction In multi-state models of longitudinal data usually a process is assumed to be Markovian that is that the conditional probability distribution of future states depends only on the present state, not on the whole sequence of past events. In a discrete-time framework, one can study models based on Markov chains. For instance, the R (R Core Team 2015) package VLMC  (Maechler 2015 ) can be used to fit a variable length Markov chain to a discrete time series. In a continuous-time framework, multi-state models based on Markov processes are often considered. In various applications, the intensities between states are supposed to be constant in time (homogeneity assumption) or piecewise constant  (Huszti, Abrahamowicz, Alioum, Binquet, and Quantin 2012; Saint-Pierre, Combescure, Daurès, and Godard 2003; Aguirre-Hernàndez and Farewell 2002).
A few R packages have been developed to simplify the usage of multi-state Markov models. The msm package  (Jackson 2011)  allows to fit homogeneous Markov or hidden Markov models in continuous-time and discrete-time. Non and semi parametric estimation of non homogeneous Markov models or competing risks models are possible using the mstate package (de  Wreede, Fiocco, and Putter 2011). The etm package  (Allignol, Schumacher, and Beyersmann 2011)  computes the Aalen-Johansen empirical transition matrix whereas p3state (Meira-Machado and Pardiñas 2011) focuses on the illness-death model. A non homogeneous Markov model is well adapted when the process evolution depends on calendar time, age or time since the beginning of the study. However, the memoryless property implies that the waiting times distributions in a Markov model is exponential.
Moreover, the parametric model allows to incorporate covariates in the distribution of sojourn times using a proportional-hazards regression model  (Cox 1972). Few R packages have been developed to handle semi-Markov or hidden semi-Markov models. The mhsmm package (O'Connell and Højsgaard 2011) performs estimation and prediction for multiple observation sequences in hidden semi-Markov models. The msSurv package  (Ferguson, Datta, and Brock 2012)  provides non parametric estimation in semi-Markov models but covariates are not considered. However, it seems that the parametric approach is not yet implemented in statistical software. We have developed an R package named SemiMarkov (Listwon-Krol and Saint-Pierre 2015) which performs parametric estimation in a homogeneous semi-Markov model. The waiting times distributions can be chosen to be the exponential, the Weibull, or the exponentiated Weibull distribution.