STHAL : Location-mention Identification in Tweets of I ndian-context (2020)

Abstract: We investigate the problem of extracting Indian-locations from a given crowd-sourced textual dataset. The problem of extracting fine-grained Indian-locations has many challenges. One challenge in the task is to collect relevant dataset from the crowd-sourced platforms that contain locations. The second challenge lies in extracting the location entities from the collected data. We provide an in-depth review of the information collection process and our annotation guidelines such that a reliable dataset annotation is guaranteed. We evaluate many recent algorithms and models, including Conditional Random fields (CRF), Bi-LSTM-CNN and BERT (Bidirectional Encoder Representations from Transformers), on our developed dataset named . The study shows the best F1-score of 72.49% for BERT, followed by Bi-LSTM-CNN and CRF. As a result of our work, we prepare a publicly-available annotated dataset of Indian geolocations that can be used by the research community. Code and dataset are available at https://github.com/vkartik2k/STHAL .

## introduction location-based systems (gartner) are playing a vital role in multiple applications such as navigation services, tourist place recommendation, address standardization and safe routes recommendation. to implement such systems and provide locationbased services effectively, it requires to have upto-date information on location names and their associated events. one relevant source for such information is social media which is considered as a crowd-sourced dataset. social media has become the most potent medium for the real-time source of data (b. @xcite for analysis. the impact of social media is inevitable and massive. many case studies reflect on why people are such a lot active on social platforms and share information. this puts the means to connect anywhere, at any time. 1 journalists also use it as a medium of power to gain insights about the background of various events. however, social networks enable one to harvest recent location events; there lie various challenges due to the platforms being general in terms of sharing of information by individuals. one of the challenges is to select relevant posts out of streaming data that contain location information. the second challenge is to extract location entities from noisy text @xcite data. previous state-of-the-art techniques based on geolocation prediction in twitter @xcite , detecting location-centric communities @xcite and social media data location prediction @xcite do not perform well in terms of extracting fine-grained location entities in indian context from the crowd-source data. one of the specific reasons for inferior performance is the non-standardization in location naming conventions. for example, it is easy to find locations names having words such as gali, zila, village, vihar, nagar, gaon, etc. an example is given in figure 1 . it demands designing of specific methods to extract location names and associated events from crowd-source data for indian subcontinent. the application of this approach can help to demystify the route system, telematics vehicle tracking, and covid tracking using crowd sourced data. in this paper, we investigate various approaches for named entity recognition focusing mainly on location names. the approaches include conditional random field (crf) @xcite , neural network-based bi-directional long short term memory-convolution neural network (bi-lstm-cnn) @xcite , and bidirectional encoder representations from transformers-based @xcite models. we improve the finegrained location-based approaches by fine-tuning bert on our annotated dataset @xcite . we observe a performance improvement @xmath0. the remainder of the paper is organized as follows: in section 2, we describe the development of sthal dataset and the proposed approach. we present our experimental results and necessary analyses in section 3. finally, we conclude in section 4. ## methodology in this section, we describe our methodology in detail. first, we explain sthal's development process, and subsequently, present our system. a high-level diagram is depicted in figure 2 . ## experiments and evaluation results text check distance from gali no . 5 , dwarka to subzi mandi old , new delhi , delhi -110036 . gold o o o b-loc libraries, respectively. for the evaluation, we utilize conll-2002 evaluation script 5 for computing the precision, recall, and f1-score for the location mentions. in all the experiments, we randomly split our annotated dataset, sthal, into 75:25 ratio for the train and test sets. moreover, to establish our hypothesis that the exiting ner datasets do not adapt well to the location identifications for the indian context, we conduct our experiments in two setups. in it is evident that the best model in setup a (bert) does not have good f1-score, mainly due to lack of indian-styled location-mentions in the train set. in comparison, we observe significant improvements for all models in setup b. the best f1-score of 72.49% is obtained by bert, followed by bi-lstm-cnn (70.31%) and crf (69.99%). ## conclusion in this paper, we present our research on locationmention identification in indian-context. due to the lack of representation of indian-styled locationnames and addresses (e.g., gali no., chowk, etc.) in existing datasets, we develop a new twitter dataset, sthal, for location-mention identification in indian context. we benchmark sthalusing bertbased sequence classifier. evaluation shows that the underlying system leverages the presence of the indian-styled location-mentions in train set. in sthal, we include location-mentions primarily from delhi-ncr and northern part of india. thus, we hypothesis that it may not be adequately sufficient for discovering location-mentions across india, e.g., southern or north-eastern india. in future, we would like to explore the task of locationmentions suitable for the entire nation.