Parsing I ndian E nglish News Headlines (2020)

Abstract: Parsing news Headlines is one of the difficult tasks of Natural Language Processing. It is mostly because news Headlines (NHs) are not complete grammatical sentences. News editors use all sorts of tricks to grab readers’ attention, for instance, unusual capitalization as in the headline’ Ear SHOT ashok rajagopalan’; some are world knowledge demanding like ‘Church reformation celebrated’ where the ‘Church reformation’ refers to a historical event and not a piece of news about an ordinary church. The lack of transparency in NHs can be linguistic, cultural, social, or contextual. The lack of space provided for a news headline has led to creative liberty. Though many works like news value extraction, summary generation, emotion classification of NHs have been going on, parsing them had been a tough challenge. Linguists have also been interested in NHs for creativity in the language used by bending traditional grammar rules. Researchers have conducted studies on news reportage, discourse analysis of NHs, and many more. While the creativity seen in NHs is fascinating for language researchers, it poses a computational challenge for Natural Language Processing researchers. This paper presents an outline of the ongoing doctoral research on the parsing of Indian English NHs. The ultimate aim of this research is to provide a module that will generate correctly parsed NHs. The intention is to enhance the broad applicability of newspaper corpus for future Natural Language Processing applications.

## introduction nhs stands to be an excellent example of creative writing. headlines tend to be short, attention-grabbing, and giving out just enough information to attract readers' attention. to keep readers engage, news editors use all sorts of tricks possible. studies on indian english have been going on for some decades and being a part of south asian english, indian english has managed to grab attention for the past few years. like other stratification of the english language, indian english is unique as well. though it follows the basic underlying structure of british english, the syntax, semantics, and pronunciation of indian english differ from world english. translating from native language to english sometimes results in different sentence structures. due to linguistic diversity, code-mixing and code-switching are extremely common. these effects can be found in nhs as well. sometimes news editors tend to use local idiosyncrasies. nhs are different from standard text. for example, we can find unusual capitalization as in 'ear shot ashok rajagopalan', deliberate subject noun phrase drop as in 'arrested for theft', the deliberate dropping of main verb as in 'identity cards for all urban street vendors', dropping of auxiliaries as in '18th century stone inscription unearthed', so on. such structures are unique to nhs which make it difficult to parse them using existing parsers. this research attempts to solve such problem of nhs parsing. ## contributions this research contributes at following levels : ## methodology for the study, we deliberately chose indian english and collected data for two reasons: 1) apart from the linguistic analysis, a contrastive analysis between nhs of indian english and british english has been conducted. this study helped us to understand whether the structure of nhs in indian english is different from nhs in british english and other english varieties, 2) working on indian english, collecting indian english nhs data and building a parallel corpus will be fruitful for future endeavors as some computational works have been already conducted on indian english. on the other hand, indian english data collection is going on for quite a some time in many organizations. we collected data from three different newspapers to analyze the syntactic structures. the lack of space in hard copy newspapers leads to an opportunity for creative liberty for news editors, like excluding grammatical and lexical elements (dropping subject noun phrase, auxiliary drop), the-out-of-grammar use of punctuation, and so on. for a systematic analysis of the data, we adhered to an inductive research strategy. our very first objective was to analyze the data for a better understanding of the problem. this analysis helped us to come up with an idea about accuracy of the currently available parsers and areas where parsing output can be improved. after parsing (constituency) the data with existing open-source parsers like stanford and allennlp, we observed several errors like parsing of singular verbs as plural nouns, adjectives as verbs, and so on. there was a need to understand the structures of nhs and thus to analyze the nhs corpus linguistically. this study was exploratory and inter-pretive. for this we used both qualitative and quantitative methods for the data analysis. we found out that nhs can be either complete headlines (headlines following the necessary svo word order of english language) and fragments (comprises mostly of noun phrases, ex. 'a burning issue'). the complete nhs were further divided into four categories according to structures: declarative, imperative, interrogative, and exclamatory. the grammatical notions we used to describe the headlines are in a broader sense as headlines can not fulfill every linguistic need to belong in any particular category. instead, they fulfilled mere basic needs. the linguistic analysis of nhs helped us create a grammar and guideline to transform nhs to their canonical form to get the correct parse. based on the linguistic analysis, we have build a syntax and semantics-based feature model. in this model, we have tried to cover the various headlines' structures we have come across so far in our study. it has been created with the motive that it will help us in the annotation of the nhs. also, it will work as a guideline to create a transfer grammar analysis module. in the feature model, we covered all the aspects of nhs like headlines structure, which can be a complete nh or a fragment; there are nhs with or without subject noun phrase; nhs which is only a quote from a speaker but without the mention of the speaker, and so on. our next step is to automate the transformation. we decided to treat the problem as a machine translation problem, for which we are working on creating a parallel corpus of raw nhs and equivalent grammatically transformed sentences. we are also working on creating transfer grammar for this purpose. ## observations we have observed following specific vital issues during the research so far: ## results the results of the error analysis of the data after constituency parsing, shows that incorrect tags e.g. plural nouns as an adjective, singular verbs as plural nouns, common nouns as proper nouns and so on led to incorrect parsed output. following is an example of such output which shows multiple parsing errors: to cross-examine our observations, we provided the parsers with sample data of grammatically transformed raw nhs, where we added a few essential grammatical elements required. proving our theory (which we developed from error analysis) correct, the parsers provided the correct parse. this finding has important implications for developing the module we intend for the generation of correctly parsed nhs. in linguistic analysis, we observed specific structural constructions like dropping off the subject and out of the grammar style of using lin-guistic devices like punctuation, idioms, multiword expressions, and many more. ## research roadmap the research road map is as follows: we did data collection and sanitization during this research followed by constituency parsing of data, performed the error analysis, conducted the linguistic analysis of nhs, did manual transformations of nhs for the rule-based approach, and formulated a guideline for the transformations, and build a feature design model for the nhs. we are currently working on constructing a news headline grammar, automating the feature annotation process, and finally building a module to provide us with correctly parsed nhs as final output.