text: In this paper , we present an alternative [[ approach ]] in which << multimodal parsing and understanding >> are achieved using a weighted finite-state device which takes speech and gesture streams as inputs and outputs their joint interpretation .