waashk/dblp huggingface dataset:
viewpointer: lightweight calibration-free eye tracking for ubiquitous handsfree deixis. we introduce viewpointer, a wearable eye contact sensor that detects deixis towards ubiquitous computers embedded in real world objects. viewpointer consists of a small wearable camera no more obtrusive than a common bluetooth headset. viewpointer allows any real-world object to be augmented with eye contact sensing capabilities, simply by embedding a small infrared (ir) tag. the headset camera detects when a user is looking at an infrared tag by determining whether the reflection of the tag on the cornea of the user's eye appears sufficiently central to the pupil. viewpointer not only allows any object to become an eye contact sensing appliance, it also allows identification of users and transmission of data to the user through the object. we present a novel encoding scheme used to uniquely identify viewpointer tags, as well as a method for transmitting urls over tags. we present a number of scenarios of application as well as an analysis of design principles. we conclude eye contact sensing input is best utilized to provide context to action.
haptic techniques for media control. we introduce a set of techniques for haptically manipulating digital media such as video, audio, voicemail and computer graphics, utilizing virtual mediating dynamic models based on intuitive physical metaphors. for example, a video sequence can be modeled by linking its motion to a heavy spinning virtual wheel: the user browses by grasping a physical force-feedback knob and engaging the virtual wheel through a simulated clutch to spin or brake it, while feeling the passage of individual frames. these systems were implemented on a collection of single axis actuated displays (knobs and sliders), equipped with orthogonal force sensing to enhance their expressive potential. we demonstrate how continuous interaction through a haptically actuated device rather than discrete button and key presses can produce simple yet powerful tools that leverage physical intuition.
robust computer vision-based detection of pinching for one and two-handed gesture input. we present a computer vision technique to detect when the user brings their thumb and forefinger together (a pinch gesture) for close-range and relatively controlled viewing circumstances. the technique avoids complex and fragile hand tracking algorithms by detecting the hole formed when the thumb and forefinger are touching; this hole is found by simple analysis of the connected components of the background segmented against the hand. our thumb and fore-finger interface (taffi) demonstrates the technique for cursor control as well as map navigation using one and two-handed interactions.
circle & identify: interactivity-augmented object recognition for handheld devices. the first requirement of a "spatial mouse" is the ability to identify the object that it is aiming at. among many possible technologies that can be employed for this purpose, possibly the best solution would be object recognition by machine vision. the problem, however, is that object recognition algorithms are not yet reliable enough or light enough for hand-held devices. this paper demonstrates that a simple object recognition algorithm can become a practical solution when augmented by interactivity. the user draw a circle around a target using a spatial mouse, and the mouse captures a series of camera frames. the frames can be easily stitched together to give a target image separated from the background, with which we need only additional steps of feature extraction and object classification. we present here results from two experiments with a few household objects.
personalizing routes. navigation services (e.g., in-car navigation systems and online mapping sites) compute routes between two locations to help users navigate. however, these routes may direct users along an unfamiliar path when a familiar path exists, or, conversely, may include redundant information that the user already knows. these overly complicated directions increase the cognitive load of the user, which may lead to a dangerous driving environment. since the level of detail is user specific and depends on their familiarity with a region, routes need to be personalized. we have developed a system, called myroute, that reduces route complexity by creating user specific routes based on a priori knowledge of familiar routes and landmarks. myroute works by compressing well known steps into a single contextualized step and rerouting users along familiar routes.
projector-guided painting. this paper presents a novel interactive system for guiding artists to paint using traditional media and tools. the enabling technology is a multi-projector display capable of controlling the appearance of an artist's canvas. this display-on-canvas guides the artist to construct the painting as a series of layers. our process model for painting is based on classical techniques and was designed to address three main issues which are challenging to novices: (1) positioning and sizing elements on the canvas, (2) executing the brushstrokes to achieve a desired texture and (3) mixing pigments to make a target color. these challenges are addressed through a set of interaction modes. preview and color selection modes enable the artist to focus on the current target layer by highlighting the areas of the canvas to be painted. orientation mode displays brushstroke guidelines for the creation of desired brush texture. color mixing mode guides the artist through the color mixing process with a user interface similar to a color wheel. these interaction modes allow a novice artist to focus on a series of manageable subtasks in executing a complex painting. our system covers the gamut of the painting process from overall composition down to detailed brushwork. we present the results from a user study which quantify the benefit that our system can provide to a novice painter.
personal computing in the 21st century. ever since the dawn of the digital computer, invention, innovation, and creativity have been a hallmark of the industry. the mainframe computer seemed for a while to be the real player with experts or at least highly trained professionals operating these large and expensive machines. most users were allowed to see them through glass windows but "hands on" was a rare opportunity. in 1972, the xerox palo alto research center (parc), built a remarkable personal computer named the alto. except for the visionaries at parc and a few others, most people considered the personal computer a mere curiosity in this early period. today, the personal computer has become a tool that very few imagined. what might be yet to come.while prognosticating about the future is a risky endeavor at best, perhaps we can obtain a look ahead with a straightforward review of the current status of personal computing. we will look at operating systems, application software and peripherals, however, the real goal of this talk is to see what the user interface, tools and interactions with this future computing environment might be or perhaps even should be. will we still be using continuing variations of doug englebart's mouse in 2020 or might something new and much more advanced emerge? how might users seamlessly deal with terabytes of storage? how might multi-user environments be used and could multi-os machines be an economic and generally available personal computing environment? are there user experience issues that are critical in multi-os environments? how might the user's display be different from today? will tomorrow's displays be larger, have a significantly higher pixel density, be much more paper-like, etc.? might electronic printers and their requisite paper output still be with us by 2025, for example? will home and neighborhood network resources finally be a powerful ally of the computing environment? many exciting opportunities and questions beg for answers and industry insight.this talk will attempt to peer into the near future to see what we might expect of the personal computing environment based on what we can extrapolate from current experience and technology directions. while the exactitude of such projections may be limited, taken as a whole, there is perhaps much that can be learned from such an exercise. why do this? charles kettering, the great automotive inventor was asked why he spent so much time planning and thinking about the future. he wisely replied, "because i am going to spend the rest of my life there." thirty years ago, very few could have imagined all the wonderful things that personal computing has enabled. perhaps we have just begun our exciting journey. the acm portal is published by the association for computing machinery. copyright © 2010 acm, inc. terms of usage privacy policy code of ethics contact us useful downloads: adobe acrobat quicktime windows media player real player
cuetip: a mixed-initiative interface for correcting handwriting errors. with advances in pen-based computing devices, handwriting has become an increasingly popular input modality. researchers have put considerable effort into building intelligent recognition systems that can translate handwriting to text with increasing accuracy. however, handwritten input is inherently ambiguous, and these systems will always make errors. unfortunately, work on error recovery mechanisms has mainly focused on interface innovations that allow users to manually transform the erroneous recognition result into the intended one. in our work, we propose a mixed-initiative approach to error correction. we describe cuetip, a novel correction interface that takes advantage of the recognizer to continually evolve its results using the additional information from user corrections. this significantly reduces the number of actions required to reach the intended result. we present a user study showing that cuetip is more efficient and better preferred for correcting handwriting recognition errors. grounded in the discussion of cuetip, we also present design principles that may be applied to mixed-initiative correction interfaces in other domains.
rapid construction of functioning physical interfaces from cardboard, thumbtacks, tin foil and masking tape. rapid, early, but rough system prototypes are becoming a standard and valued part of the user interface design process. pen, paper, and tools like flash™ and director™ are well suited to creating such prototypes. however, in the case of physical forms with embedded technology, there is a lack of tools for developing rapid, early prototypes. instead, the process tends to be fragmented into prototypes exploring forms that look like the intended product or explorations of functioning interactions that work like the intended product - bringing these aspects together into full design concepts only later in the design process. to help alleviate this problem, we present a simple tool for very rapidly creating functioning, rough physical prototypes early in the design process - supporting what amounts to interactive physical sketching. our tool allows a designer to combine exploration of form and interactive function, using objects constructed from materials such as thumbtacks, foil, cardboard and masking tape, enhanced with a small electronic sensor board. by means of a simple and fluid tool for delivering events to "screen clippings," these physical sketches can then be easily connected to any existing (or new) program running on a pc to provide real or wizard of oz supported functionality.
under the table interaction. we explore the design space of a two-sided interactive touch table, designed to receive touch input from both the top and bottom surfaces of the table. by combining two registered touch surfaces, we are able to offer a new dimension of input for co-located collaborative groupware. this design accomplishes the goal of increasing the relative size of the input area of a touch table while maintaining its direct-touch input paradigm. we describe the interaction properties of this two-sided touch table, report the results of a controlled experiment examining the precision of user touches to the underside of the table, and a series of application scenarios we developed for use on inverted and two-sided tables. finally, we present a list of design recommendations based on our experiences and observations with inverted and two-sided tables.
citrine: providing intelligent copy-and-paste. we present citrine, a system that extends the widespread copy-and-paste interaction technique with intelligent transformations, making it useful in more situations. citrine uses text parsing to find the structure in copied text and allows users to paste the structured information, which might have many pieces, in a single paste operation. for example, using citrine, a user can copy the text of a meeting request and add it to the outlook calendar with a single paste. in applications such as excel, users can teach citrine by example how to copy and paste data by showing it which fields go into which columns, and can use this to copy or paste many items at a time in a user-defined manner. citrine can be used with a wide variety of applications and types of data and can be easily extended to work with more. it currently includes parsers that recognize contact information, calendar appointments and bibliographic citations. it works with internet explorer, outlook, excel, palm desktop, endnote and other applications. citrine is available to download on the internet.
content-aware scrolling. scrolling is used to navigate large information spaces on small screens, but is often too restrictive or cumbersome to use for particular types of content, such as multi-page, multi-column documents. to address this problem, we introduce content-aware scrolling (cas), an approach that takes into account various characteristics of document content to determine scrolling direction, speed, and zoom. we also present the cas widget, which supports scrolling through a content-aware path using traditional scrolling methods, demonstrating the advantages of making a traditional technique content-aware.
summarizing personal web browsing sessions. we describe a system, implemented as a browser extension, that enables users to quickly and easily collect, view, and share personal web content. our system employs a novel interaction model, which allows a user to specify webpage extraction patterns by interactively selecting webpage elements and applying these patterns to automatically collect similar content. further, we present a technique for creating visual summaries of the collected information by combining user labeling with predefined layout templates. these summaries are interactive in nature: depending on the behaviors encoded in their templates, they may respond to mouse events, in addition to providing a visual summary. finally, the summaries can be saved or sent to others to continue the research at another place or time. informal evaluation shows that our approach works well for popular websites, and that users can quickly learn this interaction model for collecting content from the web.
windowscape: a task oriented window manager. we propose windowscape, a window manager that uses a photograph metaphor for lightweight, post hoc task management. this is the first task management windowing model to provide intuitive accessibility while allowing windows to exist simultaneously in multiple tasks. windowscape exploits users' spatial and visual memories by providing a stable thumbnail layout in which to search for windows. a function is provided to let users search the window space while maintaining a largely consistent screen image to minimize distractions. a novel keyboard interaction technique is also presented.
multi-user, multi-display interaction with a single-user, single-display geospatial application. in this paper, we discuss our adaptation of a single-display, single-user commercial application for use in a multi-device, multi-user environment. we wrap google earth, a popular geospatial application, in a manner that allows for synchronized coordinated views among multiple instances running on different machines in the same co-located environment. the environment includes a touch-sensitive tabletop display, three vertical wall displays, and a tabletpc. a set of interaction techniques that allow a group to manage and exploit this collection of devices is presented.
automatic thumbnail cropping and its effectiveness. thumbnail images provide users of image retrieval and browsing systems with a method for quickly scanning large numbers of images. recognizing the objects in an image is important in many retrieval tasks, but thumbnails generated by shrinking the original image often render objects illegible. we study the ability of computer vision systems to detect key components of images so that automated cropping, prior to shrinking, can render objects more recognizable. we evaluate automatic cropping techniques 1) based on a general method that detects salient portions of images, and 2) based on automatic face detection. our user study shows that these methods result in small thumbnails that are substantially more recognizable and easier to find in the context of visual search.
in-stroke word completion. we present the design and implementation of a word-level stroking system called fisch, which is intended to improve the speed of character-level unistrokes. importantly, fisch does not alter the way in which character-level unistrokes are made, but allows users to gradually ramp up to word-level unistrokes by extending their letters in minimal ways. fisch relies on in-stroke word completion, a flexible design for fluidly turning unistroke letters into whole words. fisch can be memorized at the motor level since word completions always appear at the same positions relative to the strokes being made. our design for fisch is suitable for use with any unistroke alphabet. we have implemented fisch for multiple versions of edgewrite, and results show that fisch reduces the number of strokes during entry by 43.9% while increasing the rate of entry. an informal test of "record speed" with the stylus version resulted in 50-60 wpm with no uncorrected errors.
swingstates: adding state machines to the swing toolkit. this article describes swingstates, a library that adds state machines to the java swing user interface toolkit. unlike traditional approaches, which use callbacks or listeners to define interaction, state machines provide a powerful control structure and localize all of the interaction code in one place. swingstates takes advantage of java's inner classes, providing programmers with a natural syntax and making it easier to follow and debug the resulting code. swingstates tightly integrates state machines, the java language and the swing toolkit. it reduces the potential for an explosion of states by allowing multiple state machines to work together. we show how to use swingstates to add new interaction techniques to existing swing widgets, to program a powerful new canvas widget and to control high-level dialogues.
pen-top feedback for paper-based interfaces. current paper-based interfaces such as papiercraft, provide very little feedback and this limits the scope of possible interactions. so far, there has been little systematic exploration of the structure, constraints, and contingencies of feedback-mechanisms in paper-based interaction systems for paper-only environments. we identify three levels of feedback: discovery feedback (e.g., to aid with menu learning), status-indication feedback (e.g., for error detection), and task feedback (e.g., to aid in a search task). using three modalities (visual, tactile, and auditory) which can be easily implemented on a pen-sized computer, we introduce a conceptual matrix to guide systematic research on pen-top feedback for paper-based interfaces. using this matrix, we implemented a multimodal pen prototype demonstrating the potential of our approach. we conducted an experiment that confirmed the efficacy of our design in helping users discover a new interface and identify and correct their errors.
assieme: finding and leveraging implicit references in a web search interface for programmers. programmers regularly use search as part of the development process, attempting to identify an appropriate api for a problem, seeking more information about an api, and seeking samples that show how to use an api. however, neither general-purpose search engines nor existing code search engines currently fit their needs, in large part because the information programmers need is distributed across many pages. we present assieme, a web search interface that effectively supports common programming search tasks by combining information from web-accessible java archive (jar) files, api documentation, and pages that include explanatory text and sample code. assieme uses a novel approach to finding and resolving implicit references to java packages, types, and members within sample code on the web. in a study of programmers performing searches related to common programming tasks, we show that programmers obtain better solutions, using fewer queries, in the same amount of time spent using a general web search interface.
smart bookmarks: automatic retroactive macro recording on the web. we present a new web automation system that allows users to create a smart bookmark, consisting of a starting url plus a script of commands that returns to a particular web page or state of a web application. a smart bookmark can be requested for any page, and the necessary commands are automatically extracted from the user's interaction history. unlike other web macro recorders, which require the user to start recording before navigating to the desired page, smart bookmarks are generated retroactively, after the user has already reached a page, and the starting point of the macro is found automatically. smart bookmarks have a rich graphical visualization that combines textual commands, web page screenshots, and animations to explain what the bookmark does. a bookmark's script consists of keyword commands, interpreted without strict reliance on syntax, allowing bookmarks to be easily edited and shared.
gui --- phooey!: the case for text input. information cannot be found if it is not recorded. existing rich graphical application approaches interfere with user input in many ways, forcing complex interactions to enter simple information, requiring complex cognition to decide where the data should be stored, and limiting the kind of information that can be entered to what can fit into specific applications' data models. freeform text entry suffers from none of these limitations but produces data that is hard to retrieve or visualize. we describe the design and implementation of jourknow, a system that aims to bridge these two modalities, supporting lightweight text entry and weightless context capture that produces enough structure to support rich interactive presentation and retrieval of the arbitrary information entered.
gaze-enhanced scrolling techniques. scrolling is an essential part of our everyday computing experience. contemporary scrolling techniques rely on the explicit initiation of scrolling by the user. the act of scrolling is tightly coupled with the user?s ability to absorb information via the visual channel. the use of eye gaze information is therefore a natural choice for enhancing scrolling techniques. we present several gaze-enhanced scrolling techniques for manual and automatic scrolling which use gaze information as a primary input or as an augmented input. we also introduce the use off-screen gaze-actuated buttons for document navigation and control.