Sentences and Documents in Native Language Identification
Authors: Andrea Cimino, Felice Dell'Orletta, Dominique Brunato, Giulia Venturi (CNR-ILC, 2018)
Topic: Learner Corpora and Language Acquisition

Abstract
Starting from a wide set of linEnglish. Starting from a wide set of linguistic features, we present the ﬁrst in depth feature analysis in two different Native Language Identiﬁcation (NLI) scenarios. We compare the results obtained in a traditional NLI document classiﬁcation task and in a newly introduced sentence classiﬁcation task, investigating the different role played by the considered features. Finally, we study the impact of a set of selected features extracted from the sentence classiﬁer in document classiﬁcation.

Introduction
Native Language Identiﬁcation (NLI) is the research topic aimed at identifying the native language (L1) of a speaker or a writer based on his/her language production in a non-native language (L2). The leading assumption of NLI research is that speakers with the same L1 exhibit similar linguistic patterns in their L2 productions which can be viewed as traces of the L1 interference phenomena. Thanks to the availability of large-scale benchmark corpora, such as the TOEFL11 corpus (Blanchard et al., 2013), NLI has been recently gaining attention also in the NLP community where it is mainly addressed as a multi-class supervised classiﬁcation task. This is the approach followed by the more recent systems taking part to the last editions of the NLI Shared Tasks held in 2013 (Tetreault et al., 2013) and 2017 (Malmasi et al., 2017). Typically, these systems exploit a variety of features encoding the linguistic structure of L2 text in terms of e.g. ngrams of characters, words, POS tags, syntactic constructions. Such features are used as input for machine learning algorithms, mostly based on traditional Support Vector Machine (SVM) models. In addition, rather than using the output of a single classiﬁer, the most effective approach relies on ensemble methods based on multiple classiﬁers (Malmasi and Dras, 2017). In this paper we want to further contribute to NLI research by focusing the attention on the role played by different types of linguistic features in predicting the native language of L2 writers. Starting from the approach devised by (Cimino and Dell’Orletta, 2017), which obtained the ﬁrst position in the essay track of the 2017 NLI Shared Task, we carry out a systematic feature selection analysis to identify which features are more effective to capture traces of the native language in L2 writings at sentence and document level. Our Contributions (i) We introduce for the ﬁrst time a NLI sentence classiﬁcation scenario, reporting the classiﬁcation results; (ii) We study which features among a wide set of features contribute more to the sentence and to the document classiﬁcation task; (iii) We investigate the contribution of features extracted from the sentence classiﬁer in a stacked sentence-document system.

Conclusion
We introduced a new NLI scenario focused on sentence classiﬁcation. Compared to document classiﬁcation we obtained different results in terms of accuracy and distribution of errors across the L1s. We showed the different role played by a wide set of linguistic features in the two NLI scenarios. These differences may justify the performance boost we achieved with a stacked sentencedocument system. We also assessed which features extracted from the sentence classiﬁer maximizes NLI document classiﬁcation.