SUKHAN : Corpus of H indi Shayaris annotated with Sentiment Polarity Information (2020)

Abstract: Shayari is a form of poetry mainly popular in the Indian subcontinent, in which the poet expresses his emotions and feelings in a very poetic manner. It is one of the best ways to express our thoughts and opinions. Therefore, it is of prime importance to have an annotated corpus of Hindi shayaris for the task of sentiment analysis. In this paper, we introduce SUKHAN, a dataset consisting of Hindi shayaris along with sentiment polarity labels. To the best of our knowledge, this is the first corpus of Hindi shayaris annotated with sentiment polarity information. This corpus contains a total of 733 Hindi shayaris of various genres. Also, this dataset is of utmost value as all the annotation is done manually by five annotators and this makes it a very rich dataset for training purposes. This annotated corpus is also used to build baseline sentiment classification models using machine learning techniques.

## introduction sentiment analysis is simply 'the task of extraction and analysis of subjective information present in some natural language data with the use of natural language processing' 1 . but, the task of sentiment analysis becomes challenging for languages having annotated corpus only in some limited domains. one such language is hindi and shayari is one of its domains which has no annotated dataset for the task of sentiment analysis. shayari is a very rich tradition in south asia. it has generally 2 to 4 lines which have some kind of deep meaning in them. it is mainly written in languages like hindi, urdu, bangla, nepali and punjabi. whether you are sad, alone, happy or in love, you can use shayari to express your feelings and thoughts. that's why, it is very important to have an annotated corpus of shayaris for the task this paper is divided into 5 sections. section 2 discusses related work in this area. section 3 elaborates on the source and the creation of the corpus. section 4 elaborates on the annotation process and the annotation scheme used for getting sentiment labels. inter-annotator agreement has also been calculated and the details for interpretation of values for the fleiss kappa index have also been mentioned. section 5 describes the experimental setup for training a model using various classifiers which helps in establishing the baseline for sentiment classification of these hindi shayaris. all the results and conclusions using the annotated corpora have been briefly discussed in section 6. ## related work so far, no work has been done to run sentiment analysis on shayaris, neither in hindi nor in any other indian language, where similar constructs occur. however, work on sentiment analysis of hindi poems has been done previously @xcite but never on shayaris. sentiment analysis has also been done on odia poems by gaurav @xcite . music classification has been carried out using lyrics @xcite , audio @xcite and even multi-modal features @xcite for english. similar work has been carried out for mood classification of telugu @xcite and hindi songs @xcite nowadays, research mainly focuses on social media and very little attention is given to the traditional literature of which shayari is an important part. automatic analysis of poetry is done for poems written in various languages like english, chinese, arabic, malay, and spanish. @xcite tried to categorize poems based on their emotional content. in the case of traditional literary works such as poetry, a lexicon creation methodology has been discussed for analyzing classical chinese poetry @xcite . @xcite has also proposed a meter classification system for persian poems based on features extracted from uttered poem. @xcite tried to classify arabic poetry according to emotion associated with it. ## dataset due to the unavailability of annotated hindi shayari corpus with sentiment polarity information, the dataset was constructed manually. the advent of utf-8 encoding has led to text in indian scripts increasing day by day on the web. we collected shayaris from numerous online sources such as: https://poetrytadka.com , https://shayarifm.com/ https://shayarilovers.in/ https://bestnow.in/ these websites consist of many hindi shayaris from various categories. we have used hindi shayaris written using devanagari script only. a total of 845 shayaris were mined. we have not used the title of shayari because the title of shayari usually does not have a strong association with the theme of the shayari. that's why, title was not used for extracting emotions from shayaris in order to avoid wrong results. the name of the shayar 2 also do not carry any sentiment information and therefore does not serve the task at hand and therefore is not used in baseline experiments. ## conclusion shayaris and emotions share a very strong bonding. each shayari is penned by a shayar with a lot of emotions, feelings and values. different poetry elements such as diction, rhyme, rhythm, and imagery make shayari different from a normal piece of text. that's why in this research area, we have created a hindi shayari corpus which would help to create an automatic system for categorization of shayaris based on polarity identification questionnaire and emotional states present in it. sukhan is the first corpus of hindi shayaris of diverse themes, with shayaris, manually annotated as either having positive or negative sentiment values. the corpus was manually annotated with 733 hindi shayaris scripted in the devanagari script and based on the emotions they evoke. we also trained four different types of classifiers on our data. classification models have been built using tf-idf word-level features. linear-svm performed better as compared to other classifiers and the results of the experiments should serve as a good baseline. identifying the sentiment with the shayari is the first step towards identifying the emotions and thoughts which the shayari could evoke in the mind of the reader and this can further be used to build recommendation systems which are used by every major company in the e-commerce area.