trx : Regular-tree expressions , now in Scheme

Regular-tree expressions are to semi-structured data, such as XML and Lisp s-expressions, what standard regular expressions are to strings: a powerful “chainsaw” for describing, searching and transforming structure in large data sets. We have designed and implemented a little language, trx, for defining regular-tree patterns. We discuss the design of trx, its underlying mathematical formalisation with various kinds of tree automata, and its implementation technology. One of the attractions of trx is that, rather than being a complete, ad hoc language for computing with trees, it is instead embedded within Scheme by means of the Scheme macro system. The features of the design are demonstrated with multiple motivating examples. The resulting system is of general use to programmers who wish to operate on tree-structured data in Scheme. 1 LCD data representations Semi-structured and tree-structured data has become an important topic in the world of software engineering in the past few years, due to the widespread adoption of XML as a generic representation format for data. While this may be news to rest of the world, it is a very familiar picture to programmers in the Lisp family of languages. The Scheme and Lisp community has long been aware of the benefits of fixing on a general-purpose data structure for representing trees, and specifying a standard concrete character representation for these trees. Lisp s-expressions are essentially XML trees; the Lisp community has worked within the s-expression framework for representing data since the inception of the language in the 1960’s. Part of the power of the Lisp family of languages comes from this focus on s-expressions as the central data structure of the language. A Perlis aphorism [16] captures the benefit well: “It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures. ” S-expressions are the “least common denominator” (LCD) representation for data in the Lisp family of programming languages, in the same sense that strings are the LCD representation in the world of Unix tools: the multitude of functions that Permission to make digital or hard copies, to republish, to post on servers or to redistribute to lists all or part of this work is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To otherwise copy or redistribute requires prior specific permission. Fifth Workshop on Scheme and Functional Programming. September 22, 2004, Snowbird, Utah, USA. Copyright 2004 Ilya Bagrak and Olin Shivers. operate upon and produce results in this form can therefore easily be connected together to construct larger computations, providing for a large degree of code reuse. In the world of Scheme programming, s-expressions are the universal interchange format. The charm of s-expressions as an LCD representation is that, unlike strings, they come with some degree of existing structure. This eliminates much of the parsing/unparsing overhead that is typically required when computational agents interact using string intermediate representations (parsing that, in the Unix-tools setting, is frequently done by means of heuristic, incomplete, error-prone handwritten parsers). Another Perlis aphorism makes clear the downsides of using strings as an LCD form: “The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information. ” The problem with strings as a least-common denominator is that they are too “least,” that is, too low level. We operate upon strings a character at a time—a level where it is all too easy to break the invariants of the associated grammar that typically imposes structure and meaning on the strings. Even when s-expressions may not be the appropriate representation for the core data structures of an application, they still frequently find use around the application’s “fringe,” being converted to and from the internal, more highly-engineered core structures as they move across the application’s boundary—with the associated benefit that it is much simpler and more robust to parse a tree than a string. 2 Regular trees, little languages and Scheme Given that Lisp and Scheme programmers have been working with “semi-structured” tree data roughly three and half decades longer than XML has even existed, it is surprising that this community has never bothered to adopt one of the great, expressive tools for manipulating such data: regular trees and their associated patterns. Just as traditional regular expressions are an expressive tool for describing structure occurring within strings, regular trees can serve a similar role when dealing with recursively defined patterns occurring within trees—trees such as ones we frequently represent using Scheme s-expressions. We have long grumbled about the lack of such tools. Each time we write a low-level Scheme macro, for example, and we find ourselves writing an incomplete and awkward syntax-checker/parser for our new form directly in Scheme (Is the form exactly four elements long? Is the second element a list of identifier/expression pairs? Etc.), we pause to wish for a better way. When the XML world began wisely to exploit the extensive theoretical machinery