towards a computational model of sketching
Sketching is a powerful means of interpersonal communication. While many useful multimodal systems have been created, current systems are far from achieving human-like participation in sketching. A computational model of sketching would help characterize these differences and help us better understand how to overcome them. This paper is a first step towards such a model. We start with an example of a sketching system (nuSketch COA Creator) designed to aid military planners, to provide context and a source of examples. We then describe four dimensions of sketching, visual understanding, conceptual understanding, language understanding, and drawing, that can be used to characterize the competence of existing systems and identify open problems. The issues involved will be illustrated by examples from our experience with nuSketch. Three research challenges are posed, to serve as milestones towards a computational model of sketching that can explain and replicate human abilities in this area.
computer modeling model sketching interpersonal communication multimodality systems human participant help understandability paper examples context language competence experience research challenge replication multimodal interfaces