Requirements, Tools, and Architectures for Annotated Corpora

This paper provides an overview of the needs for corpus annotation and exploitation, and some suggested strategies for development of a widely usable and reusable corpus-handling environment. The central plank of our argument is that cross-disciplinary acceptability is no longer an optional extra. The overall goal is to provide a framework which can be adapted to meet the needs of a research community which is intellectually, geographically, and linguistically diverse. The ecology of corpora Annotated text and speech corpora are a staple of language processing research, as well as other applications such as lexicography and corpus linguistics. The cost of creating an annotated corpus can be very high, both in direct financial terms and in terms of the opportunity cost of allocating skilled labor. So funders, whether public or commercial, have come to expect that the cost of corpus creation will be amortized over multiple research and development efforts. The more costly the corpus, the wider the market which it must reach. For example, developments in representation and handling of multi-lingual, multi-media, and multi-modal data has opened up possibilities for linking text and speech data as well as audio, video, and image. Such corpora will be very expensive, to the extent that their cost will have to be justified not just within a single research community but across disciplinary boundaries. In extreme cases corpora may have to achieve worldwide distribution in order to justify the cost of their creation. It is no accident that multinational umbrella bodies have sprung up to coordinate the dissemination of these peculiarly valuable research products. Given these pressures, corpus designers have often been trailblazers for standardized data and annotation formats. Design ideas from corpus encoding and annotation have helped to shape the information architecture of the World Wide Web. As the flight from proprietary formats has spread from academia into the wider world, it is increasingly plausible to claim that we were right all along. But the original goals that motivated the involvement of corpus designers in the standardization process have not necessarily been achieved. XML will presumably be the encoding format of choice for the next few corpora; this tells us only that a measure of syntactic uniformity has been achieved. We still have to decide what information to encode, and how to ensure that it will meet future needs. This paper provides an overview of the needs for corpus annotation and exploitation, and some suggested strategies for development of a widely usable and reusable corpus-handling environment. The central plank of our argument is that cross-disciplinary acceptability is no longer an optional extra. The overall goal is to provide a framework which can be adapted to meet the needs of a research community which is intellectually, geographically, and linguistically diverse. Temporal diversity is a particular challenge. Corpora have a long active life, and as technology moves on, it becomes possible to exploit them in ways not envisaged by the original designers. This puts a high premium on flexibility. While we cannot predict future research needs, we can predict that there will be such needs. The corpus architecture must be such that it can be adapted to new situations as new research paradigms emerge. Also, in seeking to take account of the putative needs of potential future users we may (more or less by accident) build in the flexibility which is needed by current workers working in fields different from those envisaged by the designers of the corpus. These considerations will not give answers for every aspect of corpus design, but they will give some guidance, and help us to avoid egregious mistakes. In summary, good corpora will be both reusable, i.e., potentially usable in more than one research project and by more than one research team, and extensible, i.e., capable of further enhancement. Such corpora have a better chance of surviving in the changing environment of research and development. It is an open question how to achieve the necessary flexibility.