waashk/dblp huggingface dataset:
hierarchical parsing and recognition of hand-sketched diagrams. a long standing challenge in pen-based computer interaction is the ability to make sense of informal sketches. a main difficulty lies in reliably extracting and recognizing the intended set of visual objects from a continuous stream of pen strokes. existing pen-based systems either avoid these issues altogether, thus resulting in the equivalent of a drawing program, or rely on algorithms that place unnatural constraints on the way the user draws. as one step toward alleviating these difficulties, we present an integrated sketch parsing and recognition approach designed to enable natural, fluid, sketch-based computer interaction. the techniques presented in this paper are oriented toward the domain of network diagrams. in the first step of our approach, the stream of pen strokes is examined to identify the arrows in the sketch. the identified arrows then anchor a spatial analysis which groups the uninterpreted strokes into distinct clusters, each representing a single object. finally, a trainable shape recognizer, which is informed by the spatial analysis, is used to find the best interpretations of the clusters. based on these concepts, we have built simusketch, a sketch-based interface for matlab's simulink software package. an evaluation of simusketch has indicated that even novice users can effectively utilize our system to solve real engineering problems without having to know much about the underlying recognition techniques.
olfactory display. the last twenty years have seen enormous leaps forward in computers' abilities to generate sound and video. what happens when computers can produce scents on demand? in this talk, i present three approaches to this question. i first look at human olfactory processing: what is our olfactory bandwidth, and what are the limitations of our sense of smell? i then explore the use of scent to accompany other media, from historical examples like sense-o-rama and aromarama, to more recent work including firefighter training systems, augmented gaming, and food and beverage applications. finally, i look at the possibilities of olfactory output as an ambient display medium. i conclude with an overview of current computer-controlled olfactory output devices: off the shelf solutions for incorporating scent into user interface applications.
a remote control interface for large displays. we describe a new widget and interaction technique, known as a "frisbee," for interacting with areas of a large display that are difficult or impossible to access directly. a frisbee is simply a portal to another part of the display. it consists of a local "telescope" and a remote "target". the remote data surrounded by the target is drawn in the telescope and interactions performed within it are applied on the remote data. in this paper we define the behavior of frisbees, show unique affordances of the widget, and discuss design characteristics. we have implemented a test application and report on an experiment that shows the benefit of using the frisbee on a large display. our results suggest that the frisbee is preferred over walking back and forth to the local and remote spaces at a distance of 4.5 feet.
video-based document tracking: unifying your physical and electronic desktops. this paper presents an approach for tracking paper documents on the desk over time and automatically linking them to the corresponding electronic documents using an overhead video camera. we demonstrate our system in the context of two scenarios, <i>paper tracking</i> and <i>photo sorting</i>. in the paper tracking scenario, the system tracks changes in the stacks of printed documents and books on the desk and builds a complete representation of the spatial structure of the desktop. when users want to find a printed document buried in the stacks, they can query the system based on appearance, keywords, or access time. the system also provides a <i>remote desktop</i> interface for directly browsing the physical desktop from a remote location. in the photo sorting scenario, users sort printed photographs into physical stacks on the desk. the systemautomatically recognizes the photographs and organizes the corresponding digital photographs into separate folders according to the physical arrangement. our framework provides a way to unify the physical and electronic desktops without the need for a specialized physical infrastructure except for a video camera.
the designers' outpost: a tangible interface for collaborative web site. in our previous studies into web design, we found that pens, paper, walls, and tables were often used for explaining, developing, and communicating ideas during the early phases of design. these wall-scale paper-based design practices inspired the designers' outpost, a tangible user interface that combines the affordances of paper and large physical workspaces with the advantages of electronic media to support information design. with outpost, users collaboratively author web site information architectures on an electronic whiteboard using physical media (post-it notes and images), structuring and annotating that information with electronic pens. this interaction is enabled by a touch-sensitive smart board augmented with a robust computer vision system, employing a rear-mounted video camera for capturing movement and a front-mounted high-resolution camera for capturing ink. we conducted a participatory design study with fifteen professional web designers. the study validated that outpost supports information architecture work practice, and led to our adding support for fluid transitions to other tools.
considering the direction of cursor movement for efficient traversal of cascading menus. cascading menus are commonly seen in most gui systems. however, people sometimes choose the wrong items by mistake, or become frustrated when submenus pop up unnecessarily. this paper proposes two methods for improving the usability of cascading menus. the first uses the direction of cursor movement to change the menu behavior: horizontal motion opens/closes submenus, while vertical motion changes the highlight within the current menu. this feature can reduce cursor movement errors. the second causes a submenu to pop up at the position where horizontal motion occurs. this is expected to reduce the length of the movement path for menu traversal. a user study showed that our methods reduce menu selection times, shorten search path lengths, and prevent unexpected submenu appearance and disappearance.
sensing and visualizing spatial relations of mobile devices. location information can be used to enhance interaction with mobile devices. while many location systems require instrumentation of the environment, we present a system that allows devices to measure their spatial relations in a true peer-to-peer fashion. the system is based on custom sensor hardware implemented as usb dongle, and computes spatial relations in real-time. in extension of this system we propose a set of spatialized widgets for incorporation of spatial relations in the user interface. the use of these widgets is illustrated in a number of applications, showing how spatial relations can be employed to support and streamline interaction with mobile devices.
simplicial families of drawings. in this paper we present a method for helping artists make artwork more accessible to casual users. we focus on the specific case of drawings, showing how a small number of drawings can be transformed into a richer object containing an entire family of similar drawings. this object is represented as a simplicial complex approximating a set of valid interpolations in configuration space. the artist does not interact directly with the simplicial complex. instead, she guides its construction by answering a specially chosen set of yes/no questions. by combining the flexibility of a simplicial complex with direct human guidance, we are able to represent very general constraints on membership in a family. the constructed simplicial complex supports a variety of algorithms useful to an end user, including random sampling of the space of drawings, constrained interpolation between drawings, projection of another drawing into the family, and interactive exploration of the family.
shark: a large vocabulary shorthand writing system for pen-based computers. zhai and kristensson (2003) presented a method of speed-writing for pen-based computing which utilizes gesturing on a stylus keyboard for familiar words and tapping for others. in shark<sup>2</sup>:, we eliminated the necessity to alternate between the two modes of writing, allowing any word in a large vocabulary (e.g. 10,000-20,000 words) to be entered as a shorthand gesture. this new paradigm supports a gradual and seamless transition from visually guided tracing to recall-based gesturing. based on the use characteristics and human performance observations, we designed and implemented the architecture, algorithms and interfaces of a high-capacity multi-channel pen-gesture recognition system. the system's key components and performance are also reported.
tsi (teething ring sound instrument): a design of the sound instrument for the baby. in this paper, we will describe the tsi (teething ring sound instrument), a new sound instrument given to babies, which consists of a teething ring, a knob, an i-cubex digitizer [1] and a computer which processes midi messages. the tsi is designed to bring music experience to baby with the movement of the babies reflex sucking motion. we provided the tsi to a baby and observed her action to the tsi and her reaction to the generated sound. this experiment showed the high potential of the tsi.
talkback: a conversational answering machine. current asynchronous voice messaging interfaces, like voicemail, fail to take advantage of our conversational skills. talkback restores conversational turn-taking to voicemail retrieval by dividing voice messages into smaller sections based on the most significant silent and filled pauses and pausing after each to record a response. the responses are composed into a reply, alternating with snippets of the original message for context. talkback is built into a digital picture frame; the recipient touches a picture of the caller to hear each segment of the message in turn. the minimal interface models synchronous interaction and facilitates asynchronous voice messaging. talkback can also present a voice-annotated slide show which it receives over the internet.
bimanual and unimanual image alignment: an evaluation of mouse-based techniques. we present an evaluation of three mouse-based techniques for aligning digital images. we investigate the physical image alignment task and discuss the implications for interacting with virtual images. in a formal evaluation we show that a symmetric bimanual technique outperforms an asymmetric bimanual technique which in turn outperforms a unimanual technique. we show that even after mode switching times are removed, the symmetric technique outperforms the single mouse technique. subjects also exhibited more parallel interaction using the symmetric technique than when using the asymmetric technique.
a molecular architecture for creating advanced guis. this paper presents a new gui architecture for creating advanced interfaces. this model is based on a limited set of general principles that improve flexibility and provide capabilities for implementing information visualization techniques such as magic lenses, transparent tools or semantic zooming. this architecture also makes it possible to create multiple views and application-sharing systems (by sharing views on multiple computer screens) in a simple and uniform way and to handle bimanual interaction and multiple pointers. an experimental toolkit called ubit was implemented to test the feasibility of this approach. it is based on a pseudo-declarative c++ api that tries to simplify gui programming by providing a higher level of abstraction.
haptic pen: a tactile feedback stylus for touch screens. in this paper we present a system for providing tactile feedback for stylus-based touch-screen displays. the haptic pen is a simple low-cost device that provides individualized tactile feedback for multiple simultaneous users and can operate on large touch screens as well as ordinary surfaces. a pressure-sensitive stylus is combined with a small solenoid to generate a wide range of tactile sensations. the physical sensations generated by the haptic pen can be used to enhance our existing interaction with graphical user interfaces as well as to help make modern computing systems more accessible to those with visual or motor impairments.
automatic projector calibration with embedded light sensors. projection technology typically places several constraints on the geometric relationship between the projector and the projection surface to obtain an undistorted, properly sized image. in this paper we describe a simple, robust, fast, and low-cost method for automatic projector calibration that eliminates many of these constraints. we embed light sensors in the target surface, project gray-coded binary patterns to discover the sensor locations, and then prewarp the image to accurately fit the physical features of the projection surface. this technique can be expanded to automatically stitch multiple projectors, calibrate onto non-planar surfaces for object decoration, and provide a method for simple geometry acquisition.
the kinetic typography engine: an extensible system for animating expressive text. kinetic typography --- text that uses movement or other temporal change --- has recently emerged as a new form of communication. as we hope to illustrate in this paper, kinetic typography can be seen as bringing some of the expressive power of film --- such as its ability to convey emotion, portray compelling characters, and visually direct attention --- to the strong communicative properties of text. although kinetic typography offers substantial promise for expressive communications, it has not been widely exploited outside a few limited application areas (most notably in tv advertising). one of the reasons for this has been the lack of tools directly supporting it, and the accompanying difficulty in creating dynamic text. this paper presents a first step in remedying this situation --- an extensible and robust system for animating text in a wide variety of forms. by supporting an appropriate set of carefully factored abstractions, this engine provides a relatively small set of components that can be plugged together to create a wide range of different expressions. it provides new techniques for automating effects used in traditional cartoon animation, and provides specific support for typographic manipulations.
moveable interactive projected displays using projector based tracking. video projectors have typically been used to display images on surfaces whose geometric relationship to the projector remains constant, such as walls or pre-calibrated surfaces. in this paper, we present a technique for projecting content onto moveable surfaces that adapts to the motion and location of the surface to simulate an active display. this is accomplished using a projector based location tracking techinque. we use light sensors embedded into the moveable surface and project low-perceptibility gray-coded patterns to first discover the sensor locations, and then incrementally track them at interactive rates. we describe how to reduce the perceptibility of tracking patterns, achieve interactive tracking rates, use motion modeling to improve tracking performance, and respond to sensor occlusions. a group of tracked sensors can define quadrangles for simulating moveable displays while single sensors can be used as control inputs. by unifying the tracking and display technology into a single mechanism, we can substantially reduce the cost and complexity of implementing applications that combine motion tracking and projected imagery.
visual tracking of bare fingers for interactive surfaces. visual tracking of bare fingers allows more direct manipulation of digital objects, multiple simultaneous users interacting with their two hands, and permits the interaction on large surfaces, using only commodity hardware. after presenting related work, we detail our implementation. its design is based on our modeling of two classes of algorithms that are key to the tracker: image differencing segmentation (ids) and fast rejection filters (frf). we introduce a new chromatic distance for ids and a frf that is independent to finger rotation. the system runs at full frame rate (25 hz) with an average total system latency of 80 ms, independently of the number of tracked fingers. when used in a controlled environment such as a meeting room, its robustness is satisfying for everyday use.
topiary: a tool for prototyping location-enhanced applications. location-enhanced applications use the location of people, places, and things to augment or streamline interaction. location-enhanced applications are just starting to emerge in several different domains, and many people believe that this type of application will experience tremendous growth in the near future. however, it currently requires a high level of technical expertise to build location-enhanced applications, making it hard to iterate on designs. to address this problem we introduce topiary, a tool for rapidly prototyping location-enhanced applications. topiary lets designers create a map that models the location of people, places, and things; use this active map to demonstrate scenarios depicting location contexts; use these scenarios in creating storyboards that describe interaction sequences; and then run these storyboards on mobile devices, with a wizard updating the location of people and things on a separate device. we performed an informal evaluation with seven researchers and interface designers and found that they reacted positively to the concept.
papiercraft: a command system for interactive paper. knowledge workers use paper extensively for document reviewing and note-taking due to its versatility and simplicity of use. as users annotate printed documents and gather notes, they create a rich web of annotations and cross references. unfortunately, as paper is a static media, this web often gets trapped in the physical world. while several digital solutions such as xlibris [15] and digital desk [18] have been proposed, they suffer from a small display size or onerous hardware requirements.to address these limitations, we propose papiercraft, a gesture-based interface that allows users to manipulate digital documents directly using their printouts as proxies. using a digital pen, users can annotate a printout or draw command gestures to indicate operations such as copying a document area, pasting an area previously copied, or creating a link. upon pen synchronization, our infrastructure executes these commands and presents the result in a customized viewer. in this paper we describe the design and implementation of the papiercraft command system, and report on early user feedback.
automatic image retargeting with fisheye-view warping. image retargeting is the problem of adapting images for display on devices different than originally intended. this paper presents a method for adapting large images, such as those taken with a digital camera, for a small display, such as a cellular telephone. the method uses a non-linear fisheye-view warp that emphasizes parts of an image while shrinking others. like previous methods, fisheye-view warping uses image information, such as low-level salience and high-level object recognition to find important regions of the source image. however, unlike prior approaches, a non-linear image warping function emphasizes the important aspects of the image while retaining the surrounding context. the method has advantages in preserving information content, alerting the viewer to missing information and providing robustness.
augmenting conversations using dual-purpose speech. in this paper, we explore the concept of dual-purpose speech: speech that is socially appropriate in the context of a human-to-human conversation which also provides meaningful input to a computer. we motivate the use of dual-purpose speech and explore issues of privacy and technological challenges related to mobile speech recognition. we present three applications that utilize dual-purpose speech to assist a user in conversational tasks: the calendar navigator agent, dialogtabs, and speech courier. the calendar navigator agent navigates a user's calendar based on socially appropriate speech used while scheduling appointments. dialogtabs allows a user to postpone cognitive processing of conversational material by proving short-term capture of transient information. finally, speech courier allows asynchronous delivery of relevant conversational information to a third party.
dart: a toolkit for rapid design exploration of augmented reality experiences. in this paper [macintyre et al 2004]. we describe the designer's augmented reality toolkit (dart). dart is built on top of macromedia director, a widely used multimedia development environment. we summarize the most significant problems faced by designers working with ar in the real world, and discuss how dart addresses them. most of dart is implemented in an interpreted scripting language, and can be modified by designers to suit their needs. our work focuses on supporting early design activities, especially a rapid transition from storyboards to working experience, so that the experiential part of a design can be tested early and often. dart allows designers to specify complex relationships between the physical and virtual worlds, and supports 3d animatic actors (informal, sketch-based content) in addition to more polished content. designers can capture and replay synchronized video and sensor data, allowing them to work off-site and to test specific parts of their experience more effectively.
support for multitasking and background awareness using interactive peripheral displays. in this paper, we describe kimura, an augmented office environment to support common multitasking practices. previous systems, such as rooms, limit users by constraining the interaction to the desktop monitor. in kimura, we leverage interactive projected peripheral displays to support the perusal, manipulation and awareness of background activities. furthermore, each activity is represented by a montage comprised of images from current and past interaction on the desktop. these montages help remind the user of past actions, and serve as a springboard for ambient context-aware reminders and notifications.
letterwise: prefix-based disambiguation for mobile text input. a new technique to enter text using a mobile phone keypad is described. for text input, the traditional touchtone phone keypad is ambiguous because each key encodes three or four letters. instead of using a stored dictionary to guess the intended word, our technique uses probabilities of letter sequences --- "prefixes" --- to guess the intended letter. compared to dictionary-based methods, this technique, called letterwise, takes significantly less memory and allows entry of non-dictionary words without switching to a special input mode. we conducted a longitudinal study to compare letterwise to multitap, the conventional text entry method for mobile phones. the experiment included 20 participants (10 letterwise, 10 multitap), and each entered phrases of text for 20 sessions of about 30 minutes each. error rates were similar between the techniques; however, by the end of the experiment the mean entry speed was 36% faster with letterwise than with multitap.
the missing link: augmenting biology laboratory notebooks. using a participatory design process, we created three prototype augmented laboratory notebooks that provide the missing link between paper, physical artifacts and on-line data. the final a-book combines a graphics tablet and a pda. the tablet captures writing on the paper notebook and the pda acts as an "interaction lens" or window between physical and electronic documents. our approach is document-centered, with a software architecture based on layers of physical and electronic information.