Mitigating Silence in Compliance Terminology during Parsing of Utterances (2020)

Abstract: This paper reports on an approach to increase multi-token-term recall in a parsing task. We use a compliance-domain parser to extract, during the process of parsing raw text, terms that are unlisted in the terminology. The parser uses a similarity measure (Generalized Dice Coefficient) between listed terms and unlisted term candidates to (i) determine term status, (ii) serve putative terms to the parser, (iii) decrease parsing complexity by glomming multi-tokens as lexical singletons, and (iv) automatically augment the terminology after parsing of an utterance completes. We illustrate a small experiment with examples from the tax-and-regulations domain. Bootstrapping the parsing process to detect out- of-vocabulary terms at runtime increases parsing accuracy in addition to producing other benefits to a natural-language-processing pipeline, which translates arithmetic calculations written in English into computer-executable operations.

## introduction the task of extracting multi-token terms 1 , i.e. terminological units which denote concepts and entities in a domain, is a core task of natural language processing (nlp). within the tax-and-regulations domain, some terms are compositional (nunberg et al., 1994; baldwin, 2006; krcmar et al., 2013; boguraev et al., 2015) @xcite in meaning and/or in form, such as unmarried college student or estimated tax payment; others are mixed instances of compositionality such as taxable sick leave pay or cannabis duty payable. in addition, terms can correspond either to the canonical form of the concept or to variant forms of concepts' names as in spouse or common-law partner credit versus spouse's or common-law partner's credit, or spouse amount versus spousal amount (park et al., 2002). from the perspective of parsing raw text, having multi-token terms not only simplifies the input by grouping multi-tokens as singletons but also removes syntactic complexity as the internal structure of these expressions remains opaque to parsing (korkontzelos et al., 2010; wehrli, 2014; boguraev et al., 2015; nerima et al., 2017). in addition, having multi-token terms allows parsers to output structurallysimilar parses for sentences that are constituent-wise similar even if the intra-phrasal complexity of multi-token terms vary. one major issue in term extraction, known as silence, is the failure to extract terms that appear infrequently in a domain-corpus but that domain-specialists would include in a term lexicon. an example of silence is when terms such as inventory valuation or combination money purchase do not make it into the terminology because they occur infrequently in our domain corpus @xcite . in the tax-and-regulations domain, the problem with terminology silence is particularly acute, as there are many instances of rules and arithmetic calculations concerning a specific entity which is mentioned once in the entire corpus. while the entity is not mentioned enough to be highly scored by collocation-based measures during the term extraction process, for the purposes of automatically interpreting and representing tax-and-regulations content, it is essential the entity be treated the same as other items in its class by the parser. this paper describes a simple method for extracting automatically multi-token terms that are not listed in the compliance terminology at the start of the parsing process of unlabelled utterances. the compliance terminology for taxes and regulations is the result of prior work on identifying, given the domain corpus, concepts and entities by means of co-occurrence/collocation-based surface statistical measures. in addition, linguistic-rule-based filters exclude a set of ill-formed terms from the final list. because out-of-vocabulary (oov) terms are tax-and-regulations-specific, we cannot rely on external lexical resources as these expressions are unlikely to be listed in general-purpose, financial or business lexicons. in our approach, the detection of oov terms takes place during parsing. we use generalized-dice-coefficient-based (gdc) metrics @xcite to estimate the degree of similarity between established terms and oov term candidates. when the gdc-based detection of multi-token terms is enabled during parsing of utterances, experiments show improvements of 93% in parsing accuracy for utterances with oov terms at the start of the parsing process. ## motivation the goal of the compliance-domain parser is to output, in a simplified logical form (slf) (wang et al., 2015, constant et al., 2016), a semantic representation of utterances taken from tax forms written in english foot_1 . of particular interest, are utterances that express entire or partial arithmetic calculations. downstream components of our nlp pipeline interpret the slfs and automatically transform them into executable operations. in our compliance domain, the language of arithmetic calculations written in english is distributed along a continuum of syntactic complexity. the slf of utterance 1 is an addition between the amount denoted by the multi-token term exclusion of income and the amount on a specific line of a specific form. utterance 2 is about a choice: the smallest amount of two amounts (min operator). the notion of smallest is conveyed by a discontinuous dependency as a relative clause at the end of the utterance, namely, whichever is less. the first amount for the choice is a dollar constant amount; the second is an addition of the amounts denoted by the term employment income, expressed by total ... on lines 101 and 104. in utterance 3, inputting the constant amount of $75,300 to a calculation is conditional on satisfying one of the disjuncts (or operator) for the filing status of the taxpayer, expressed by the multi-token terms married filing jointly and qualifying widow(er). when multi-token terms are missing from the terminology, the task of the parser is to determine the dependencies between the individual tokens that make up the utterances. with more available tokens to parse, the chance of an inaccurate parse increases. note that, with a large terminology (over 20,000 lexical entries), the prior acquisition of multi-token expressions that are in fact not multi-token domainconcepts is a possibility foot_3 . if multi-token terms contain pieces of calculations that should be discrete, the parser will not accurately break down the calculation into its constituents, since multi-token terms are monolithic literal strings to the parser-regardless of how many spaces there are between the tokens of a term. utterances 1 and 2 differ by the tokens eligible versus qualified where eligible is part of the term eligible dependant in utterance 1. while 3 contains the single four-token term single parent qualified dependant, utterance 4 counts two separate two-token terms, namely, single parent and eligible dependant. in the case of utterances 2 and 4 with oov terms, the parser parses qualified and single parent as left modifiers of the head of the noun phrase. in slfs, the modifiers are enclosed in parentheses as arguments to the predicates, respectively dependant in utterance 2 and eligible dependant in utterance 4. intuitively, each of the utterances in the slfs of utterances 2 and 4 of even with an increase in the size of the domain corpus, there is no guarantee that the oov terms qualified dependant or single parent eligible dependant of ## experiment in order to measure the impact of detecting gdc-based terms at runtime on the success of the nlp pipeline that extracts and interprets arithmetic calculations in the tax-and-regulations domain, we ran the following experiment. ## additional downstream benefits an additional advantage for this method occurs further downstream in the nlp pipeline, where elements of parsed phrases are matched to an internal data model. when executing the calculation for ifte(qualified dependant,85.00), we use a custom-built entity-recognition system to determine the value of qualified dependant. one of the features of this entity recognition system is consecutive token ## conclusion in this paper, we have described a method which increases term recall and improves parsing accuracy of utterances with oov terms at the start of the parsing process. in addition, multi-token terms detected at parsing runtime are automatically added to the existing terminology. we use a parser fitted with a term-generation preprocessor to identify similarity between oov multi-token term candidates and multi-token terms listed in the terminology. we observe improvements not only in the interpretation and representation of the utterances by our parser but also in the transformation of the slfs into executable operations in downstream components of our nlp pipeline. this method is best suited for domains where precision in a term lexicon, which has been automatically extracted, is important and where the problem with term silence can be severe @xcite in the future, we would like to experiment with beginning the parsing process by using a terminology manually curated by domain experts. because the method is domain-agnostic, we do not believe the discovery of oov terms and augmentation of a domain-specific terminology (as long as there is a preexisting terminology at outset of a parsing task) is constrained to our example domain. this work has shown that it is possible to overcome term silence by adding functionality to a parser with a preprocessor to discover oov terms on the fly.