Opinion Mining System for Processing H indi Text for Home Remedies Domain (2020)

Abstract: Opinion Mining (OM) is a field of study in Computer Science that deals with development of software applications related to text classifications and summarizations. Researchers working in this field contribute lexical resources, computing methodologies, text classification approaches, and summarization modules to perform OM tasks across various domains and different languages. Lexical and computational components developed for an Opinion Mining System that processes Hindi text taken from weblogs are presented in the paper for the demonstration. Text chosen for processing are the ones demonstrating cause and effect relationship between related entities ‘Food’ and ‘Health Issues’. The work is novel and lexical resources developed are useful in current research and may be of importance for future research in the field. The resources are developed for an algorithm ‘A’ such that for a sentence ‘Y’ which is a domain specific sentence from weblogs in Hindi, A(Y) returns a set F, HI, p, s such that F is a subset of set, FOOD=set of word or phrases in Hindi used for an edible item and HI is a subset of set, HEALTH_ISSUE= set of word or phrases in Hindi used for a part of body composition ‘BODY_COMPONENT’ UNION set of word or phrases in Hindi used for a health problem a human being face ‘HEALTH_PROBLEM’. Element ‘p’ takes numeric value ‘1’ or ‘-1’ where value ‘1’ means that from the text ‘Y’, algorithm ‘A’ computationally derived that the food entities mentioned in set ‘F’ have a positive effect in health issues mentioned in set ‘HI’ and the numeric value ‘-1’ means that the food entities in set ‘F’ have a negative effect in health issues in set ‘HI’. The element‘s’ may take value ‘1’ or ‘2’ indicating that the strength of polarity ‘p’ is medium or strong.

## introduction opinion mining (om) is a field of study in computer science that deals with development of software applications related to text classifications and summarizations. researchers working in this field contribute lexical resources, computing methodologies, text classification approaches, and summarization modules to perform om tasks across various domains and different languages. the common challenges encountered by the researchers in this field are; domain dependency, classification of sarcastic text, dealing with comments with thwarted expectations, language dependency, extensive domain knowledge, and categorising of text as a fact bearing text or an opinion bearing fact. there has been development in this field for processing hindi unstructured text @xcite @xcite @xcite @xcite @xcite . ## research problem an ongoing research in the field of om, which is dedicated towards the development of lexical and software components that facilitate opinion classification and summarization from hindi text appearing on web logs is presented here. hindi text showcasing cause and effect relationship between 'food' and 'health issues' entities are processed in the oms under study. the resources are developed for an algorithm 'a' such that for a sentence 'y' which is a domain specific sentence from weblogs in hindi, a(y) returns a set {f, hi, p, s} such that f is a subset of set, @xmath0, @xmath1. element 'p' takes numeric value '1' or '-1' where value '1' means that from the text 'y', algorithm 'a' computationally derived that the food entities mentioned in set 'f' have a positive effect in health issues mentioned in set 'hi' and the numeric value '-1' means that the food entities in set 'f' have a negative effect in health issues in set 'hi'. the element's' may take value '1' or '2' indicating that the strength of polarity 'p' is medium or strong. this research is undertaken with a sole objective to contribute towards bringing text appearing in hindi weblogs under the benefits that an om system offers. language dependent computational contributions based on english and chinese languages, in the previous studies motivated the proposal of the current work @xcite @xcite . this research is undertaken with a sole objective to contribute towards bringing text appearing in hindi ## key issues identified and addressed lexical resources required for named entity recognition (ner) and polarity classification were are not available for the current research. the same are developed in the research. a corpus in same domain and in same language as the text being processed in an om system helps in devising computational components for the system and evaluating them. an annotated corpus of hindi text relevant to the research did not pre-exist hence it is developed in the research. 4 major contributions a domain specific hindi corpus, with semantic and syntactic annotations is developed in the research. the semantic annotations of the corpus help in devising and evaluating the algorithms formulated in the research. a total of 3303 unique domain specific hindi sentences from weblogs are collected for the corpus. the corpus has approximately 2516 unique sentences with positive annotations for 'food' and 'health issue' associations and 787 unique sentences with negative annotations for associations between entities. the total number of syntactically annotated hindi words in the corpus is 60000 approximately. domain specific lexical datasets for entities; food, food_adjective, food_component, body_component and health_problem are also developed/identified in the research. the total number of hindi words/phrases identified for the lexical datasets are 8000 lexicons approximately. a lexical based polarity classification and polarity strength classification algorithm is developed in the research. there are two datasets developed to support the algorithm. the datasets are; (a) a set of approx. 15000 positive polarity bearing phrases and a set of approx. 10000 negative polarity bearing phrases that generate a vocabulary of approximately 35000 trigram words, (b) a set of approximately 14000 strength determining phrases that are useful in determining the strength of the polarity identified by the algorithm. ## methodology adopted firstly, a set of domain specific keywords were consolidated. then for approximately 2 years using some identified keywords domain specific hindi text from weblogs were collected. semi automated processes were adopted for data cleaning and refinement of the collected sentences for the corpus. the corpus is syntactically annotated with part of speech (pos) using a pos tool for hindi developed by cfilt, iit bombay. each word of the sentence is stemmed to give flexibility to the word usage for ner. the corpus is semantically annotated with the help of two annotators. a detailed guideline for semantic annotations was developed in the research. the annotators were supposed to annotate the text on their perception about the related entities and polarity of associations of the related entities. domain specific lexical datasets are developed using hindi wordnet of cfilt, iit bombay. the developed lexical datasets are used by the ner algorithm that extracts domain specific related entities from text under om processing. the polarity bearing and strength determining phrases are developed using a seed list from the corpus, later the dataset was extended using phrases from books that are candidate for appearing on weblogs and using synonyms of words in phrases with same pos using hindi wordnet. finally, a lexical based algorithm using naive bayes classifier that is trained on a vocabulary of n-gram words from polarity phrases and strength determining phrases is formalized for polarity classification of association and strength classification of polarity.