Joint reranking of parsing and word recognition with automatic segmentation ?
Evaluation and optimization of automatic speech recognition (ASR) and parsing systems are often done separately. In the context of spoken language processing, however, these problems may be explored jointly via a reranking architecture. In this work, the effects of reranking for word error rate (WER) or reranking for the Sparseval parse-quality measure are examined in conversational speech recognition, while considering the impact of automatic segmentation. Under a WER criterion, the results indicate that the parse language model alone provides little benefit over a large n-gram model, but adding non-local syntactic features leads to improved performance. Under a Sparseval criterion, it is shown that including alternative word-sequence hypotheses has a much greater impact on parse accuracy than including alternate parse hypotheses. In both cases, the biggest performance improvements are obtained with high quality sentence segmentations. Qualitative analyses show that parse features help recover pronouns and improve recognition of main verbs.
parsing automatic speech recognition conversational speech sentence segmentation