Learning Domain Terms - Empirical Methods to Enhance Enterprise Text Analytics Performance (2020)

Abstract: Performance of standard text analytics algorithms are known to be substantially degraded on consumer generated data, which are often very noisy. These algorithms also do not work well on enterprise data which has a very different nature from News repositories, storybooks or Wikipedia data. Text cleaning is a mandatory step which aims at noise removal and correction to improve performance. However, enterprise data need special cleaning methods since it contains many domain terms which appear to be noise against a standard dictionary, but in reality are not so. In this work we present detailed analysis of characteristics of enterprise data and suggest unsupervised methods for cleaning these repositories after domain terms have been automatically segregated from true noise terms. Noise terms are thereafter corrected in a contextual fashion. The effectiveness of the method is established through careful manual evaluation of error corrections over several standard data sets, including those available for hate speech detection, where there is deliberate distortion to avoid detection. We also share results to show enhancement in classification accuracy after noise correction.

## introduction a large part of enterprise data such as customer complaints, project management documents, client communications, risk reports, emails etc. can yield rich insights and actionable intelligence for improving enterprise processes. text analytics solutions built with machine learning methods are employed for the purpose. there is also an increasing emphasis on robotic process automation (rpa) where the focus is on automating enterprise tasks. tasks that do not involve active or deep cognitive abilities can be easily automated. however, if the task involves reading and interpretation of natural language content, there is a need to ensure that the machine interpretation of the content is correct before it is sent for automated downstream processing. given the volumes of such communication that is generated for any organization in the digital world, automated processing of these content is beneficial to ensure timely response to customers, timely resolution and also generate predictive insights. it is well known that the quality of analytical results are largely dependent on the quality of input text. thus, all text analytics solutions are preceded by pre-processing and cleaning steps to help yield better results. though there exists standard dictionaries to aid the cleaning process, these dictionaries are not enough to deal with enterprise text, due to their inherent nature. internal enterprise communication like emails, messages etc. contains words, abbreviations and terms that are very domain specific and not available in general purpose thesaurus. this includes terms representing names of services, products or groups etc. they also contain lots of acronyms that are not always standardized but get created along the way and are well understood within a community. consumer generated content like customer complaints, call logs etc. additionally contain spelling distortions of products and services. the real problem posed by enterprise text is that there is no demarcation between domain words that "appear to be noisy" since they are not part of a dictionary and words that are "true noise" because it is wrongly spelt or misused. for any cleaning to be done for enterprise text, it has to be first resolved whether the unrecognizable term is an enterprise term or a true noise term and then correct it appropriately. without this, even word or phrase frequencies cannot be generated properly. interestingly we found a resonance of the problem while dealing with social media text. there are not only spelling mistakes and typos -but deliberate distortion of traditional words and phrases while writing them down. some of these are conventional short-hands and some are done to avoid online detection. this is one of the problems faced by online hate speech detectors. hence detecting and replacing these words with the correct word is an important problem in that scenario also. let us illustrate the different types of errors and their significance through the following examples. the above examples illustrate that text cleaning, especially while dealing with enterprise text, involves segregation of non-dictionary domain terms from true noise words. since terminologies differ within groups in an enterprise, and also evolve over time, one time dictionary creation is not a solution. as products, processes and tasks evolve over time, the terminologies also change. hence, there is a need for designing methods that can ingest a given enterprise or social media repository and clean it correctly in an efficient fashion before embarking on a text analytics journey. in this work, we propose novel unsupervised method that can analyze any enterprise document repository and clean it effectively. initially, all words in the repository are segregated into two categoriesdictionary and non-dictionary words. any standard linguistic dictionary can be used for the purpose. the key idea is to segregate the domain words from true noise words and then correct the noise words accordingly. the domain words, when recognized, get included in the dictionary. the remaining noise words are corrected incrementally. the whole act is achieved via an iterative method using the following two steps of analyze and infer. analyze -this step analyzes the word distributions of the dictionary and non-dictionary words and classifies them into different frequency groups. infer -this step facilitates the movement of non-dictionary words to the dictionary and finally correct the noise words, based on word structures, their frequency groups and contextual similarities. similarities between words are established using multiple distance functions that combine traditional edit-distance based measures, phoneme based similarity as well as contextual similarities computed using word embeddings that are created for each repository. the word2vec skip-gram model @xcite is used for creating the word embeddings. the vector dimensions are adjusted to take care of small repositories. the above steps are applied in an iterative fashion till no movement of word is possible. the validity and effectiveness of the method are established through multiple experiments. we also provide performance analysis of the proposed algorithms to show that the proposed method is highly efficient. thus it can be used to clean any new repository before applying text analytics algorithms to achieve the best results. ## related work most of the noise detection and correction work for text center around spellchecking and spelling correction @xcite . a standard way to do so is to use edit distance based methods @xcite with respect to a standard dictionary. other methods include noisy channel model for detecting misspellings @xcite , linguistic and statistical approach @xcite and graph based approach to correct spelling errors in domain-centric search for emails @xcite . neural word embeddings for context sensitive spelling correction have also been considered in @xcite @xcite . the work by @xcite also uses orthographic and phonetic similarity and contextual information for automatic spelling correction. however, none of the above works consider segregation of non-dictionary domain words from noise. hence, most of these methods are not applicable for enterprise text which deserves special attention. in @xcite , methods have been specified for acronym disambiguation in enterprise text. however other errors are not addressed in this. in @xcite , methods are presented for correcting errors contextually, based on neighboring words and edit distance measures. however, these also do not take care of enterprise domain words and consider them as errors. although, @xcite has attempted to identify domain terms over true error terms based on the assumption that domain and correct terms are more uniformly distributed across documents, this assumption does not work always. it is often found that some spelling mistakes also occur uniformly and with high frequency. thus there is a need to work on unsupervised, robust and repeatable methods for detecting true noise terms in enterprise text and correct them. this work addresses that need. ## enterprise text cleaning -a prelude to analytics consumer generated text is inherently noisy. hence all text analytics tasks start with mandatory preprocessing and cleaning steps as follows. these texts are found to be fraught with random use of upper cases, spaces and punctuation marks. social media has also given rise to a set of new terms that represent phrases, and may be expressed by a non-traditional combination of characters. for example, the term 4u represents "for you". certain errors are also introduced due to the underlying digital platforms. incorrect conversion of characters to symbols or icons fall under this category. rule based pre-processors are implemented to remove the above noises. after the removal of unwanted elements, the text is tokenized using the modified twitter tokenizer @xcite . the modification is done in such a manner that it suits the semi-formal enterprise text tokenization. afterwards, stop words are removed and words are normalised to lower case letters. the next step is to perform noise correction, which involves recognition of noisy terms and restoring them to their correct forms. the next section explains the proposed steps to do this for enterprise text. ## correcting noisy terms in enterprise text in this section we present the details of the proposed noise correction algorithm. to begin with, all tokens in the text repository are segregated into two groups, dictionary and non-dictionary, depending on whether they belong to a standard language dictionary or not. each token is tagged with their category. further, the frequency of each token is computed and ranked. for a token w, its frequency is given by the number of occurrence of w in the corpus and is denoted by w f . the rank of w, denoted by w r , is determined by ordering the words in descending order of frequency. the word with highest frequency gets rank one, second highest will have rank two and so on. let the set of tokens that are detected as dictionary words be indicated by v, and the set of nondictionary tokens be denoted by n. the task now is to reduce the cardinality of n by identifying tokens that can be moved from it to v. this happens iteratively through the following actions. analyze -this step analyzes the relative frequencies and ranks of tokens and classifies them into different frequency groups. infer -this step checks the similarities among tokens in terms of their structures, frequency groups and contextual presence in text. the final movement is inferred based on a weighted similarity computed using multiple distance functions that combine traditional edit-distance based measures, phoneme based similarity as well as contextual similarities computed using word embeddings that are created for each repository. the next two subsections describe the above steps in detail. ## conclusion this work presents an efficient method for identifying domain terms over true noise terms from enterprise text and thereafter clean the text appropriately to enhance performance of text analytics programs. the methods, though empirical, have been derived after careful analysis of many large enterprise repositories. experimental evaluation shows that the method proposed is highly accurate in determining domain terms and also correct errors. the effectiveness of the approach is also established through improvement in classification accuracy after correction. though designed for enterprise text, it was found that the method can be very fruitfully engaged in detecting errors in hate speech data originating in social media. since these words are deliberately distorted, the errors in this case do not follow any specific pattern and are more difficult to determine. we have shown that our combined approach performs very well in such scenarios also. we are now working on formalizing the error detection and correction in enterprise text as an optimization problem. we are also working on an interactive system such that the method can receive rewards for a correct action and penalized for a wrong one. this can be effectively used for designing better systems which can utilize reinforcement learning for cleaning enterprise text. detecting and removing names and other low frequency errors is also on the agenda.