Determining the Ethno-nationality of Writers Using Written English Text

Ethno-nationality is where nations are defined by a shared heritage , for instance it can be a membership of a common language , nationality , religion or an ethnic ancestry . The main goal of this research is to determine a person ’ s country-oforigin using English text written in less controlled environments , employing Machine Learning ( ML ) and Natural Language Processing ( NLP ) techniques . The current literature mainly focuses on determining the native language of English writers and a minimal number of researches have been conducted in determining the country-of-origin of English writers . Further , most experiments in the literature are mainly based on the TOEFL , ICLE datasets which were collected in more controlled environments ( i.e. , standard exam answers ) . Hence , most of the writers try to follow some guidelines and patterns of writing . Subsequently , the creativity , freedom of writing and the insights of writers could be hidden . Thus , we believe it hides the real nativism of the writers . Further , those corpora are not freely available as it involves a high cost of licenses . Thus , the main data corpus used for this research was the International Corpus of English ( ICE corpus ) . Up to this point , none of the researchers have utilised the ICE corpus for the purpose of determining the writers ’ countryof-origin , even though there is a true potential . For this research , an overall accuracy of 0.7636 for the flat classification ( for all ten countries ) and accuracy of 0.6224∼1.000 for sub-categories were received . In addition , the best ML model obtained for the flat classification strategy is linear SVM with SGD optimizer trained with word ( 1,1 ) uni-gram model . 1 INTRODUCTION . Ethno-nationality can be determined as the identity in which jointly defined by ethnicity and nationality ( Sapp , 2012 ) . Identifying a writer ’ s country-of-origin is a part of identifying his ethnonationality ( Jerry , 2008 ) . Foremost focus of ethno-nationality in this research is to identify a writer ’ s country-of-origin based on their writings in a non-native or second language . In recent years , identification of ethno-nationality of a writer has gained a growing interest . In author profiling demographic features ( such as age , gender , education , native language , country of origin , etc. , ) of an author from a written text will be identified , which are commonly needed in forensic linguistics . For instance , Intelligence to build a profile of their suspect , to identify the author of an anonymous email threat . Therefore , this will enable to limit the search space as well ( Estival et al. , 2007 ) . Moreover , for business applications this can be useful ; for example , in marketing where the demographic features as stated above of customers is important to predict behaviors , upgrade the current products and to develop new products . People from different ethno-nationality make various language errors when learning a language . Identification of writer ’ s country of origin could have an impact on educational applications designed towards non-native speakers of a language . Besides , this can be used as a plug-in to online tutor systems to provide more tailored feedback to the students about their mistakes ( Tetreault et al. , 2013 ) . This can help researchers to identify specific teaching and learning issues in different ethnonationalities . This will enable them to develop pedagogical learning materials to address and solve those issues . In this research we focus on English as the second language . As English has become a ‘ universal communication ’ language due to the globalization . It is no longer restricted to the native countries such as England and United States ( Eric , 2013 ) . English language is now available in numerous emerging fields , and has become an essential requirement of labour market and further it is considered to have a cultural importance ( Marko , 2009 ) . We define , A controlled environment as a place or area where rules , regulations and norms are subject to strict enforcement . For instance , in an examination setting , candidates are required to produce answers limiting the scope of the question . 2 RELATED WORK . Distribution of English language around the world can be identified in two ways , mainly ; based on the geographic distribution and based on the Kachru ’ s theory ( Braj B , 2004 ) . In this study when segmenting according to the geographic distribution we have considered clusters namely ; ‘ Asia ’ , ‘ non-Asia ’ , ‘ South Asia ’ , ‘ non-South Asia ’ and ‘ North America ’ . According to the Kachru ’ s ‘ Concentric Circles ’ , English speakers were segmented in to three main categories , namely ; ‘ inner circle ’ , ‘ outer circle ’ and ‘ expanded circle ’ . As stated by his model , the inner circle comprises of the countries where the English is the native language ( i.e. , UK , USA , Canada , New Zealand , etc. ) . The outer circle includes countries where there are small communities of native English speakers however English is used as the second language in education and official purposes ( i.e. , Sri Lanka , India , Singapore , Nigeria , etc . ) . The expanding circle contains countries which considered English as foreign language ( i.e. , China , Indonesia , Japan , Saudi Arabia , etc. ) . The work of Koppel et al . ( 2005 ) is one of the first ethno-nationality identification work where it involves categorizing users based on their native language employing Support Vector Machines ( SVMs ) on various stylistic features focused on identification of common . Authors achieved 0.802 accuracy using five chosen languages ( i.e. , Czech , French , Bulgarian , Russian and Spanish ) from International Corpus of Learner English ( ICLE ) ( Granger , 2014 ) . The ICLE dataset contains argumentative essays writings of the university students and less nativism and creativity of the writer involved . Bykh & Meurers ( 2012 ) proposed use of recurring n-grams on three different classes ( word based , POS based , Open-Class-POS-based ) as features for training SVMs . Out of other SVM implementations LIBLINEAR produced best results . For this study , random data from seven native languages selected from the ICLE corpus . The highest performance was obtained for word-level n-grams with an accuracy of 0.8971 . Gebre et al . ( 2013 ) employs linear SVM , logistic regression and perceptron ( as baseline ) for the native language identification ( NLI ) and achieved accuracy of 0.814 for eleven languages of ‘ The Test of English as a Foreign Language ’ ( TOEFL ) data set ( Blanchard et al. , 2013 ) . Features used includes ; word n-grams , POS n-grams , character n-grams and spelling errors . TOEFL11 has become standard benchmark in NLI tasks since its introduction for the NLI Shared Task 2013 . The main limitation of the TOEFL11 dataset is that it is collected in a more controlled environment ( i.e. , exam for English ) . Cimino & Dell ’ Orletta ( 2017 ) utilizes a novel stacked classifier approach where linear logistic regression based sentence feature classifier is stacked with a SVM based document feature classifier with standard lexical , stylistic and syntactic features . However stacked classification approach has gained a minor gain compared to unstacked . Best results of the NLI shared task 2017 reported for this approach with F1-score of 0.8818 for the TOEFL11 dataset . Kulmizev et al . ( 2017 ) introduced ‘ Groningen ’ system for the NLI Shared Task 2017 which out performs employing linear SVM for character 1-9 n-grams with the F1-score of 0.8756 . Authors have reported that several experiments done with ensemble approach and other features such POS , word , lemma n-grams , skip-grams ; and those failed to match the performance of character 1-9 ngram system . Goutte & Léger ( 2017 ) explored use of voting ensemble SVM models with character , word and POS n-grams . Authors confirms that ensemble methods provide minor but systematic predictive perfor- mance gains . Highest F1-score performance was with best-vote approach consisting 10 models is 0.8740 for TOEFL11 dataset . Markov et al . ( 2018 ) has used punctuation-based features with POS n-grams for his experiments and accuracies of 0.4883 ( TOEFL11 ) and 0.6948 ( ICLE ) reported for best performing settings . SVM with one-vs-all ( OvA ) multi-class classification approach has been used to conduct these experiments . Malmasi & Dras ( 2018 ) employed a supervised multi-class classification approach and incorporated several corpora , including ; TOEFL11 , EF Cambridge Open Language Database Corpus ( EFCAMDAT ) , ASK Corpus ( Andresprȧkskorpus , Second Language Corpus ) , and Jinan Chinese Learner Corpus . The features extracted were ; word/lemma n-grams , character n-grams , function word ngrams , POS n-grams , dependencies , CFG rules , adaptor Grammars and TSG fragments . Highest accuracy of 0.871 is reported for the TOEFL11 dataset . According to the literature it is evident that most of the researchers in the computational linguistic community have employed ICLE in the early stages and TOEFL in the latest NLI tasks ( refer Table 1 ) which both were collected in more controlled environments . Finally , due to above controlled environments ; being unable to capture real nativism in written texts certainly inflate the performance of ethno-nationality identification . On the other hand , identifying the nation of the English writer opposed to his native language will be equally beneficial for author profiling as well . Thus , we introduce use of International Corpus of English ( ICE ) for the ethno-nationality identification . To the best of our knowledge , none of the researches have been conducted to determine the writer ’ s country of origin based on the International Corpus of English ( ICE ) corpus . Moreover , very limited number of researchers have identified the significant features which helps to distinguish Sri Lankan English writers using a large corpus like ICE . One limitation of these researches raised by the authors Tofıghı et al . ( 2012 ) is that , since for most of the web-based applications , automatic spell-checker has been applied , idiosyncratic features including misspelling and other anomalies are ignored . This may hide some of the features which will be useful in identifying the real categories . Nevertheless , in current context , usage of spell checkers and grammar checkers can be seen frequently . Thus , this will be a common limitation in similar types of researches . 3 RESEARCH METHODOLOGY . Supervised learning approach was chosen as it is suitable in classifying text documents into classes more accurately if the classes are known and the data set is labelled ( Slotte , 2018 ) . Hence , according to the literature most promising ML algorithms for the text classification such as ; Support Vector Machines with Stochastic Gradient Descent Optimizer ( linear SVM with SGD ) , Multinomial Naı̈ve Bayes ( MNB ) , Decision Tree ( DT ) and Random Forest ( an ensemble approach ) have been employed . For this research mostly Scikit-learn ( Pedregosa et al. , 2012 ) has been used for preprocessing , feature extraction and classification tasks . Scikit-learn is a Python module for machine learning which provides state-of-the-art implementations of many well-known machine learning algorithms , and maintains an easy-to-use interface ( Pedregosa et al. , 2012 ) . The workflow of the research is depicted in Figure 1 ( Left ) . The main assumptions of this research are authors from the same ethno-nationality share the same linguistic features in their writing and will often have an influence on the way they express themselves in writings ( Jain et al. , 2017 ) . The research questions addressed in this research are ;

Summary: This paper aims to determine the country of origin of English language writers. It uses documents from the International corpus of English (ICE) and employs various learning algorithms and text-based features. The findings are that linear SVM with unigram features works best, that some countries are better distinguished than others, and that some features are more informative than others.