text: In this paper , we present an alternative approach in which multimodal parsing and understanding are achieved using a << weighted finite-state device >> which takes [[ speech and gesture streams ]] as inputs and outputs their joint interpretation .