modality fusion for graphic design applications
Users must enter a complex mix of spatial and abstract information when operating a graphic design application. Speech / language provides a fluid and natural method for specifying abstract information while a spatial input device is often most intuitive for the entry of spatial information. Thus, the combined speech / gesture interface is ideally suited to this application domain. While some research has been conducted on multimodal graphic design applications, advanced research on modality fusion has typically focused on map related applications. This paper considers the particular demands of graphic design applications and what impact these demands will have on the general strategies employed when combining the speech and gesture channels. We also describe initial work on our own multimodal graphic design application (DPD) which uses these strategies.
modal modality fusion fusion graphic design applications complexity spatial abstraction informal operability speech language method input devices combinational gestural interfaces gesture research multimodality paper general strategies speech input pen interface multimodal interface