WITH Context: Adding Rule-Grouping to VISL CG -3 (2023)

Abstract: This paper presents an extension to the VISL CG-3 compiler and processor which enables complex contexts to be shared between rules. This sharing substantially improves the readability and maintainability of sets of rules performing multi-step operations.

## introduction when writing constraint grammars for more complex tasks, such as parsing or translation, situations often arise in which a particular context triggers multiple operations. for example, when writing a dependency parser, the head of a word and its grammatical function label are often determined jointly. similarly, for tasks such as translation that involve modifying either the syntactic structure or the linear order of the words, a change in one word will typically necessitate changes to its dependents as well. one way to handle such cases in cg is to have each operation repeat the entire set of contextual tests, which is tedious to write, difficult to read, and error-prone to maintain. another way is to add an initial rule which checks the conditions and adds a label to the target word and then have each other rule simply check for the appropriate label. this, however, leads to a proliferation of single-use tags in the grammar (which may need to be documented), and does not solve the problem that rules which operate on relationships between words, such as setparent or addrelation still need to duplicate contextual tests in order to locate the second cohort. to address these difficulties, we extend the visl cg-3 processor @xcite with the operator with, which matches a context and then runs multiple rules, all with that same context. this new operator has been released as part of visl cg-3 version 1.4.0. section 2 describes the syntax of this operator, section 3 provides examples of its application in various domains, section 4 discusses its performance implications, and section 5 concludes. ## syntax an example of the with operator in use is given in (1). here the context being matched is a noun preceded at any distance by a determiner. the subsequent rules are then run with the noun as their target, so the target can be the any set (if a rule specifies a target set, then it will only be run if that set matches the target of the with). the rules can refer to the cohorts matched by the contextual tests of the with using either the position specifiers jc1, jc2, ... jc9 for the first through ninth tests, respectively, or using the magic sets c1 , c2 , ... c9 . thus the first setchild attaches the determiner (here matched with jc1 ( * )) to the noun and the second one finds any adjectives which are between the noun and the determiner (here matched with -1 * c1 ) and attaches them to the noun. by default, rules inside a with are run once when the with, but repeat has the usual effect of causing the rule to be repeated until it has no effect. as this example and those in the next section show, the with operator, while not strictly increasing the expressivity of cg, does allow many sets of rules to be written in a much more readable and maintainable manner. ## examples the following subsections give examples of applications of with to particular domains. ## performance the performance impact of adding a with group to a grammar is generally small, though measurable. in this section we present the effects on two grammars: the ancient hebrew dependency parser from @xcite and the apertium norwegian nynorsk morphological disambiguator @xcite . the results are listed in when with improves performance, it is generally due to a reduction in the number of contextual tests that need to be evaluated. however, the deduplication of tests is balanced by the fact that with must evaluate them sequentially in order to populate the cn magic sets whereas for most rules the visl cg-3 processor will internally update the order so as to start with the test that is most likely to fail. thus, when refactoring a complex grammar by hand where the total number of with groups added is likely to be small, the potential speedup is relatively small and is easily overwhelmed by the impact of new rules. in the hebrew parser, for example, the effect of rearranging the contextual tests of roughly 30 rules (9% of the grammar) into 9 with groups was negated by adding half a dozen new ones (overall a 2% slowdown), and both of these effects are minor compared to the effect of an unrelated change that removed one test from each of 135 rules (a 14% speedup). on the other hand, in the norwegian grammar, the relationships between rules are generally quite simple and we were thus able to write a script that automatically merged adjacent rules which shared the same target and had at least 5 contextual tests in common into a with group. the results of this conversion are not perfect (just over 5% of the 10k sentences in our test data have different output), but they are good enough for an approximate comparison. the script grouped 1636 rules (42% of the grammar) into 180 with groups, increasing the speed of the disambiguator by 44%. ## conclusion in this paper we have presented the with operator, an extension of visl cg-3 to allow collections of rules to be grouped into composite operations. as shown in the examples, this addition is likely to be useful to grammar authors approaching a wide variety of tasks and can even have a significant impact on grammar performance if deployed on a large scale.