text: In this paper , we present an alternative approach in which << multimodal parsing and understanding >> are achieved using a [[ weighted finite-state device ]] which takes speech and gesture streams as inputs and outputs their joint interpretation .