# Named Entity Recognition Using Conditional Random Fields
###### Abstract  
Named entity recognition (NER) is an important task in natural language processing, as it is widely featured as a key information extraction sub-task with numerous application areas. A plethora of attempts was made for NER detection in Western and Asian languages. However, little effort has been made to develop techniques for the Urdu language, which is a prominent South Asian language with hundreds of millions of speakers across the globe. NER in Urdu is considered a hard problem owing to several reasons, including the paucity of large, annotated datasets; an inaccurate tokenizer; and the absence of capitalization in the Urdu language. To this end, this study proposed a conditional-random-field-based technique with both language-dependent and language-independent features, such as part-of-speech tags and context windows of words, respectively. As a second contribution, we developed an Urdu NER dataset (UNER-I) in which a large number of NE types were manually annotated. To evaluate the effectiveness of the proposed approach, as well as the usefulness of the dataset, experiments were performed using the dataset we developed and an existing dataset. The results of the experiments showed that our proposed technique outperformed the baseline technique for both datasets by improving the F1 scores by 1.5% to 3%. Furthermore, the results demonstrated that the enhanced dataset was useful for learning and prediction in a supervised learning approach. Keywords:natural language processing; information filtering; information extraction; machine learning; classification algorithms; named entity recognition +
Footnote †: journal:  
it is founded on constructed rules of grammar [7]. The second option is attempting to learn, which necessitates tagged samples and various characteristics in order to perform better [8]. The final system is a combination of learning-based and rule-based methods [9]. To get good outcomes, such methods allow language-specific expertise in rule-based methods and significant feature design in learning-based strategies. NER systems received the attention of researchers during the Message Understanding Conferences (MUCs) [4]. Subsequently, There have been numerous NER methodologies and systems developed [1; 10]. A large majority of these systems were developed for Western languages, especially English, and are fairly accurate [11]. Furthermore, several frameworks were developed for other languages, such as Arabic and Persian, as well as for the South Asian languages, such as Hindi and Bengali [12; 13; 14]. However, NER systems for the Urdu language are still in their infancy [15]. Typically, NER schemes rely on the manipulation of a corpus of extrinsic linguistic resources, e.g., annotated corpora, human-made dictionaries, and gazetteers, to enhance the accuracy of the system [16]. However, the Urdu language is deficient in linguistic resources. Furthermore, certain aspects of the Urdu language complicate NER tasks. For example, capitalization is a prominent feature that is used by NER systems for the Western languages, whereas there is no concept of capitalization in the Urdu language. Therefore, it is desired that novel features are developed that can be used for Urdu NER tasks. To this end, this study made the following key contributions:  
* The provision of a named entity annotated dataset consisting of 2161 news sentences with 5283 entities. The news articles were obtained from the BBC Urdu website ([[https://www.bbc.com/urdu](https://www.bbc.com/urdu)]([https://www.bbc.com/urdu](https://www.bbc.com/urdu)) (accessed on 15 February 2022)), which is a valuable resource of Urdu text in digital format. The newly developed Urdu NER dataset was named the UNER-I dataset. * We proposed a condition random field (CRF)-based approach for NER in Urdu. We also proposed the notation of a feature template and a novel set of features and feature functions. It includes dependent and independent features, such as the context-of-words and part-of-speech tags, to address the NER problem in Urdu.