Architecture as a Problem of Information Fusion

In order to successfully integrate the contributions of different components in a complex system for spoken language processing, powerful and flexible techniques for information fusion are required. If applied at an appropriate level of granularity and supporting the functional autonomy of individual components based on a preferential combination of individual contributions, such an approach should allow to combine the available evidence from different sensory channels or independent knowledge sources and therefore might provide for higher accuracy and robustness. Experiences with the application of information fusion techniques within the Verbmobil system are reviewed and the prospects of an architecture based on weighted constraint dependency grammars are discussed. Spoken language processing requires to combine very different evidence from a wide variety of knowledge sources. This includes phone characteristics, prosodic cues, the lexical inventory, possibilities for combining lexical units into larger syntactic constructions, as well as their semantic coherence and pragmatic plausibility. Language processing in general can then be understood as the problem of bringing together all the relevant knowledge pieces for the task at hand and eventually deciding about the probably best target representation for an incoming utterance or a given communicative goal. From a decision theoretic point of view, the optimimum performance can only be expected if all the available evidence is combined into a single objective function. This suggests a monolithic system architecture where all the different aspects of language are comprised in a simple and uniform model structure. This view on the problem of system architecture has its roots in the pattern recognition tradition of spoken language processing, where the task of utterance recognition is decomposed into just two different components, namely the acoustic model, describing the probability of the observed signal given a particular word hypothesis, and the language model approximating word sequence probabilities. Nevertheless, the contributions of both components are fed into a single decision procedure, which tries to determine the optimum hypothesis given a speech signal observation. The only reason for separating the two components at all has to do with the very practical consideration that the acoustic and the language model can and should be trained on quite different kinds of data. More recently, the same principles of system decomposition have been applied to other language processing tasks like syntactic parsing or machine translation. In all those cases, a very small number of model components, which capture the input-output mapping, is identified and integrated by means of a single decision rule based on Bayes’ theorem. For machine translation, for instance, the relation between a source language sentence and its possible counterparts in the target language is described by the language model of the target language and a string translation model, which in turn is broken down into lexical correspondences and source-target-string alignments. This approach, although ideal from the decision theoretic point, comes with several disadvantages. Most obvious is the opacity of such a monolithic kind of model. Being trained to support relatively complex decision procedures, it usually comprises information which belongs to a wide variety of different phenomena, merged in a completely unseparable fashion. Acoustic phone models, for instance, are neither able to distinguish between prosodic nor dialectal variants. If they are designed to do, this results in considerably more phone classes which, of course, aremore difficult to train. Languagemodels, on the other hand, comprise syntactic, semantic and