P ars T wi NER : A Corpus for Named Entity Recognition at Informal P ersian (2021)

Abstract: As a result of unstructured sentences and some misspellings and errors, finding named entities in a noisy environment such as social media takes much more effort. ParsTwiNER contains about 250k tokens, based on standard instructions like MUC-6 or CoNLL 2003, gathered from Persian Twitter. Using Cohen’s Kappa coefficient, the consistency of annotators is 0.95, a high score. In this study, we demonstrate that some state-of-the-art models degrade on these corpora, and trained a new model using parallel transfer learning based on the BERT architecture. Experimental results show that the model works well in informal Persian as well as in formal Persian.

## introduction identifying named entities involves finding a sequence of words that refer to a specific concept, such as a person, a place, or an organization. this task is one of the essential components for other nlp tasks such as information extraction @xcite . when masking a named entity in a sentence, the context helps to identify the entity type. for example, in the sentence "doctor x says...", we find that the masked symbol "x" represents a person. on the other hand, sometimes a specific phrase refers to an entity, like "the united nations." hence, it is necessary to analyze word sequences to capture the context and the structure of a sentence. furthermore, recognizing named entities requires generating a sequence of tags. therefore, the problem type is a sequence-tosequence problem. often, informal languages contain a mess of words, misspellings, and grammar errors, which degrades the performance of models @xcite . accordingly, some pre-trained models based on formal corpora fail to perform well in this context. it also would be expensive to build a language model from scratch to resolve this issue. as a result, we can use some pretrained language models on formal language and train an end-to-end model using transfer learning. these models become more robust against noisy environments, such as twitter or other user-generated content. in this paper, we begin by preparing a corpus derived from persian twitter, called parstwiner. it has excellent diversity and coverage due to the variety of users and topics. then, we use transfer learning to use knowledge from the persian formal model, parsbert @xcite , and train a new model using parstwiner to overcome messy structure and misspellings. at the initial stages of training, we use a novel variation of data annealing for better learning. the rest of the paper is arranged as follows. we describe related work in section 2. next, we present the parstwiner corpus in section 3 and explain corpus statistics and annotation quality in 3.2 and 3.3, respectively. then, we describe the proposed approach in section 4 and the paper is concluded in section 5. ## related work this section describes ner corpora for informal languages like persian and english. we also discuss the ner methods for these languages. ## parstwiner corpus we describe the proposed method to create the corpus in this section. in addition to annotation instructions, we present statistics about the corpus and annotation quality based on the agreement of annotators. ## proposed model we use transfer learning to design a named entity recognizer in informal persian. current state-ofthe-art models, such as parsbert, were trained on the formal corpus, and their performance degrades on the collected corpus. this degradation is due to unstructured and noisy grammar in informal persian. as a first step, we fine-tune the parsbert language model on the corpus using sequential transfer learning. then, we use parallel multi-task learning to train a model in informal and formal persian simultaneously. experiments show the effectiveness of multi-task training. ## conclusions we demonstrated that the performance of existing ner models like parsbert degrades when applied to informal persian. a crawled persian twitter corpus, named parstwiner, was used for experiments. based on some corpus statistics, we showed that the corpus was built using various standard instructions, such as muc and conll 2003. using