waashk/dblp huggingface dataset:
interacting with large displays from a distance with vision-tracked multi-finger gestural input. we explore the idea of using vision-based hand tracking over a constrained tabletop surface area to perform multi-finger and whole-hand gestural interactions with large displays from a distance. we develop bimanual techniques to support a variety of asymmetric and symmetric interactions, including fast targeting and navigation to all parts of a large display from the comfort of a desk and chair, as well as techniques that exploit the ability of the vision-based hand tracking system to provide multi-finger identification and full 2d hand segmentation. we also posit a design that allows for handling multiple concurrent users.
supporting interspecies social awareness: using peripheral displays for distributed pack awareness. in interspecies households, it is common for the non homo sapien members to be isolated and ignored for many hours each day when humans are out of the house or working. for pack animals, such as canines, information about a pack member's extended pack interactions (outside of the nuclear household) could help to mitigate this social isolation. we have developed a pack activity watch system: allowing broad interspecies love in telecommunication with internet-enabled sociability (pawsabilities) for helping to support remote awareness of social activities. our work focuses on canine companions, and includes, pawticipatory design, labradory tests, and canid camera monitoring.
interaction techniques for ambiguity resolution in recognition-based interfaces. because of its promise of natural interaction, recognition is coming into its own as a mainstream technology for use with computers. both commercial and research applications are beginning to use it extensively. however the errors made by recognizers can be quite costly, and this is increasingly becoming a focus for researchers. we present a survey of existing error correction techniques in the user interface. these mediation techniques most commonly fall into one of two strategies, repetition and choice. based on the needs uncovered by this survey, we have developed oops, a toolkit that supports resolution of input ambiguity through mediation. this paper describes four new interaction techniques built using oops, and the toolkit mechanisms required to build them. these interaction techniques each address problems not directly handled by standard approaches to mediation, and can all be re-used in a variety of settings.
physical embodiments for mobile communication agents. this paper describes a physically embodied and animated user interface to an interactive call handling agent, consisting of a small wireless animatronic device in the form of a squirrel, bunny, or parrot. a software tool creates movement primitives, composes these primitives into complex behaviors, and triggers these behaviors dynamically at state changes in the conversational agent's finite state machine. gaze and gestural cues from the animatronics alert both the user and co-located third parties of incoming phone calls, and data suggests that such alerting is less intrusive than conventional telephones.
a toolkit for managing user attention in peripheral displays. traditionally, computer interfaces have been confined to conventional displays and focused activities. however, as displays become embedded throughout our environment and daily lives, increasing numbers of them must operate on the periphery of our attention. <i>peripheral displays</i> can allow a person to be aware of information while she is attending to some other primary task or activity. we present the peripheral displays toolkit (ptk), a toolkit that provides structured support for managing user attention in the development of peripheral displays. our goal is to enable designers to explore different approaches to managing user attention. the ptk supports three issues specific to conveying information on the periphery of human attention. these issues are <i>abstraction</i> of raw input, rules for assigning <i>notification levels</i> to input, and <i>transitions</i> for updating a display when input arrives. our contribution is the investigation of issues specific to attention in peripheral display design and a toolkit that encapsulates support for these issues. we describe our toolkit architecture and present five sample peripheral displays demonstrating our toolkit's capabilities.
outlier finding: focusing user attention on possible errors. when users handle large amounts of data, errors are hard to notice. outlier finding is a new way to reduce errors by directing the user's attention to inconsistent data which may indicate errors. we have implemented an outlier finder for text, which can detect both unusual matches and unusual mismatches to a text pattern. when integrated into the user interface of a pbd text editor and tested in a user study, outlier finding substantially reduced errors.
c-blink: a hue-difference-based light signal marker for large screen interaction via any mobile terminal. to enable common mobile terminals to interact with contents shown on large screens, we propose "c-blink", a new light signal marker method that uses the color liquid-crystal display of a mobile terminal as a visible light source. we overcome the performance limitations of such displays by developing a hue-difference-blink technique. in combination with a screen-side sensor, we describe a system that detects and receives light signal markers sent by cell phone displays. evaluations of a prototype system confirm that c-blink performs well under common indoor lighting. the c-blink program can be installed in any mobile terminal that has a color display, and the installation costs are small. c-blink is a very useful way of enabling ubiquitous large screens to become interfaces for mobile terminals.
navigating documents with the virtual scroll ring. we present a technique for scrolling through documents that is simple to implement and requires no special hardware. this is accomplished by simulating a hardware scroll ring--a device that maps circular finger motion into vertical scrolling. the technique performs at least as well as a mouse wheel for medium and long distances, and is preferred by users. it can be particularly useful in portable devices where screen-space and space for peripherals is at a premium.
user interfaces when and where they are needed: an infrastructure for recombinant computing. users in ubiquitous computing environments need to be able to make serendipitous use of resources that they did not anticipate and of which they have no prior knowledge. the speakeasy recombinant computing framework is designed to support such ad hoc use of resources on a network. in addition to other facilities, the framework provides an infrastructure through which device and service user interfaces can be made available to users on multiple platforms. the framework enables uis to be provided for connections involving multiple entities, allows these uis to be delivered asynchronously, and allows them to be injected by any party participating in a connection.
generating remote control interfaces for complex appliances. the personal universal controller (puc) is an approach for improving the interfaces to complex appliances by introducing an intermediary graphical or speech interface. a puc engages in two-way communication with everyday appliances, first downloading a specification of the appliance's functions, and then automatically creating an interface for controlling that appliance. the specification of each appliance includes a high-level description of every function, a hierarchical grouping of those functions, and dependency information, which relates the availability of each function to the appliance's state. dependency information makes it easier for designers to create specifications and helps the automatic interface generators produce a higher quality result. we describe the architecture that supports the puc, and the interface generators that use our specification language to build high-quality graphical and speech interfaces.
join and capture: a model for nomadic interaction. the xweb architecture delivers interfaces to a wide variety of interactive platforms. xweb's subscribe mechanism allows multiple interactive clients to synchronize with each other. we define the concept of join as the mechanism for acquiring access to a service's interface. join also allows the formation of spontaneous collaborations with other people. we define the concept of capture as the means for users to assemble suites of interactive resources to apply to a particular problem. these mechanisms allow users to access devices that they encounter in their environment rather than carrying all their devices with them. we describe two prototype implementations of join and capture. one uses a java ring to carry a user's identification and to make connections. the other uses a set of cameras to watch where users are and what they touch. lastly we present algorithms for resolving conflicts generated when independent interactive clients manipulate the same information.
screencrayons: annotating anything. screencrayons is a system for collecting annotations on any type of document or visual information from any application. the basis for the system is a screen capture upon which the user can highlight the relevant portions of the image. the user can define any number of topics for organizing notes. each topic is associated with a highlighting "crayon." in addition the user can supply annotations in digital ink or text. algorithms are described that summarize captured images based on the highlight strokes so as to provide overviews of many annotations as well as being able to "zoom in" on particular information about a given note and the context of that note.
query-by-critique: spoken language access to large lists. spoken language interfaces provide highly mobile, small form-factor, hands-free, eyes-free interaction with information. uniform access to large lists of information using spoken interfaces is highly desirable, but problematic due to inherent limitations of speech. a speech widget for lists of attributed objects is described that provides for approximate queries to retrieve desired items. user tests demonstrate that this is an effective technique for accessing information using speech.
the actuated workbench: computer-controlled actuation in tabletop tangible interfaces. the actuated workbench is a device that uses magnetic forces to move objects on a table in two dimensions. it is intended for use with existing tabletop tangible interfaces, providing an additional feedback loop for computer output, and helping to resolve inconsistencies that otherwise arise from the computer's inability to move objects on the table. we describe the actuated workbench in detail as an enabling technology, and then propose several applications in which this technology could be useful.
tilttype: accelerometer-supported text entry for very small devices. tilttype is a novel text entry technique for mobile devices. to enter a character, the user tilts the device and presses one or more buttons. the character chosen depends on the button pressed, the direction of tilt, and the angle of tilt. tilttype consumes minimal power and requires little board space, making it appropriate for wristwatch-sized devices. but because controlled tilting of one's forearm is fatiguing, a wristwatch using this technique must be easily removable from its wriststrap. applications include two-way paging, text entry for watch computers, web browsing, numeric entry for calculator watches, and existing applications for pdas.
empirical measurements of intrabody communication performance under varied physical configurations. intrabody communication (ibc) is a wireless communications technology that uses a person's body as the transmission medium for imperceptible electrical signals. because communication is limited to the vicinity of a person's body, ambiguities arising from communication between personal devices and environmental devices when multiple people are present can, in theory, be solved simply. intrabody communication also potentially allows data to be transferred when a person touches an ibc-enabled device. we have designed and constructed an intrabody communication system, modeled after zimmerman's original design, and extended it to operate up to 38.4kbps and to calculate signal strength. in this paper, we present quantitative measurements of data error rates and signal strength while varying hand distance to transceiver plate, electrode location on the body, touch plate size and shape, and several other factors. we find that plate size and shape have only minor effects, but that the distance to plate and the coupling mechanism significantly effect signal strength. we also find that portable devices, with poor ground coupling, suffer more significant signal attenuation. our goal is to promote design guidelines for this technology and identify the best contexts for its effective deployment.
a gesture-based authentication scheme for untrusted public terminals. powerful mobile devices with minimal i/o capabilities increase the likelihood that we will want to annex these devices to i/o resources we encounter in the local environment. this opportunistic annexing will require authentication. we present a sensor-based authentication mechanism for mobile devices that relies on physical possession instead of knowledge to setup the initial connection to a public terminal. our solution provides a simple mechanism for shaking a device to authenticate with the public infrastructure, making few assumptions about the surrounding infrastructure while also maintaining a reasonable level of security.
that one there! pointing to establish device identity. computing devices within current work and play environments are relatively static. as the number of 'networked' devices grows, and as people and their devices become more dynamic, situations will commonly arise where users will wish to use 'that device there' instead of navigating through traditional user interface widgets such as lists. this paper describes a process for identifying devices through a pointing gesture using custom tags and a custom stylus called the gesturepen. implementation details for this system are provided along with qualitative and quantitative results from a formal user study. as ubiquitous computing environments become more pervasive, people will rapidly switch their focus between many computing devices. the results of our work demonstrate that our gesturepen method can improve the user experience in ubiquitous environments by facilitating significantly faster interactions between computing devices.
connectables: dynamic coupling of displays for the flexible creation of shared workspaces. we present the connectable, a new mobile, networked and context-aware information appliance that provides affordances for pen-based individual and cooperative work as well as for the seamless transition between the two. in order to dynamically enlarge an interaction area for the purpose of shared use, a flexible coupling of displays has been realized that overcomes the restrictions of display sizes and borders. two connectable displays dynamically form a homogeneous display area when moved close to each other. the appropriate triggering signal comes from built-in sensors allowing users to temporally combine their individual displays to a larger shared one by a simple physical movement in space. connected connectables allow their users to work in parallel on an ad-hoc created shared workspace as well as exchanging information by simply shuffling objects from one display to the other. we discuss the user interface and related issues as well as the software architecture. we also present the physical realization of the connectables.
tactile interfaces for small touch screens. we present the design, implementation, and informal evaluation of tactile interfaces for small touch screens used in mobile devices. we embedded a tactile apparatus in a sony pda touch screen and enhanced its basic gui elements with tactile feedback. instead of observing the response of interface controls, users can feel it with their fingers as they press the screen. in informal evaluations, tactile feedback was greeted with enthusiasm. we believe that tactile feedback will become the next step in touch screen interface design and a standard feature of future mobile devices.
ambient touch: designing tactile interfaces for handheld devices. this paper investigates the sense of touch as a channel for communicating with miniature handheld devices. we embedded a pda with a touchenginetm --- a thin, miniature lower-power tactile actuator that we have designed specifically to use in mobile interfaces (figure 1). unlike previous tactile actuators, the touchengine is a universal tactile display that can produce a wide variety of tactile feelings from simple clicks to complex vibrotactile patterns. using the touchengine, we began exploring the design space of interactive tactile feedback for handheld computers. here, we investigated only a subset of this space: using touch as the ambient, background channel of interaction. we proposed a general approach to design such tactile interfaces and described several implemented prototypes. finally, our user studies demonstrated 22% faster task completion when we enhanced handheld tilting interfaces with tactile feedback.
user interface continuations. dialog boxes that collect parameters for commands often create ephemeral, unnatural interruptions of a program's normal execution flow, encouraging the user to complete the dialog box as quickly as possible in order for the program to process that command. in this paper we examine the idea of turning the act of collecting parameters from a user into a first class object called a user interface continuation. programs can create user interface continuations by specifying what information is to be collected from the user and supplying a callback (i.e., a continuation) to be notified with the collected information. a partially completed user interface continuation can be saved as a new command, much as currying and partially evaluating a function with a set of parameters produces a new function. furthermore, user interface continuations, like other continuation-passing paradigms, can be used to allow program execution to continue uninterrupted while the user determines a command's parameters at his or her leisure.
fluid interaction techniques for the control and annotation of digital video. we explore a variety of interaction and visualization techniques for fluid navigation, segmentation, linking, and annotation of digital videos. these techniques are developed within a concept prototype called lean that is designed for use with pressure-sensitive digitizer tablets. these techniques include a transient position+velocity widget that allows users not only to move around a point of interest on a video, but also to rewind or fast forward at a controlled variable speed. we also present a new variation of fish-eye views called twist-lens, and incorporate this into a position control slider designed for the effective navigation and viewing of large sequences of video frames. we also explore a new style of widgets that exploit the use of the pen's pressure-sensing capability, increasing the input vocabulary available to the user. finally, we elaborate on how annotations referring to objects that are temporal in nature, such as video, may be thought of as links, and fluidly constructed, visualized and navigated.
side views: persistent, on-demand previews for open-ended tasks. we introduce side views, a user interface mechanism that provides on-demand, persistent, and dynamic previews of commands. side views are designed to explicitly support the practices and needs of expert users engaged in openended tasks. in this paper, we summarize results from field studies of expert users that motivated this work, then discuss the design of side views in detail. we show how side views' design affords their use as tools for clarifying, comparing, and contrasting commands; generating alternative visualizations; experimenting without modifying the original data (i.e., "what-if" tools); and as tools that support the serendipitous discovery of viable alternatives. we then convey lessons learned from implementing side views in two sample applications, a rich text editor and an image manipulation application. these contributions include a discussion of how to implement side views for commands with parameters, for commands that require direct user input (such as mouse strokes for a paint program), and for computationally-intensive commands.
zliding: fluid zooming and sliding for high precision parameter manipulation. high precision parameter manipulation tasks typically require adjustment of the scale of manipulation in addition to the parameter itself. this paper introduces the notion of zoom sliding, or zliding, for fluid integrated manipulation of scale (zooming) via pressure input while parameter manipulation within that scale is achieved via x-y cursor movement (sliding). we also present the zlider (figure 1), a widget that instantiates the zliding concept. we experimentally evaluate three different input techniques for use with the zlider in conjunction with a stylus for x-y cursor positioning, in a high accuracy zoom and select task. our results marginally favor the stylus with integrated isometric pressure sensing tip over bimanual techniques which separate zooming and sliding controls over the two hands. we discuss the implications of our results and present further designs that make use of zliding.
from desktop to phonetop: a ui for web interaction on very small devices. while it is generally accepted that new internet terminals should leverage the installed base of web content and services, the differences between desktop computers and very small devices makes this challenging. indeed, the browser interaction model has evolved on desktop computers having a unique combination of user interface (large display, keyboard, pointing device), hardware, and networking capabilities. in contrast, internet enabled cell phones, typically with 3-10 lines of text, sacrifice usability as web terminals in favor of portability and other functions. based on our earlier experiences building and using a web browser for small devices we propose a new ui that splits apart the integrated activities of link following and reading into separate modes: navigating to; and acting on web content. this interaction technique for very small devices is both simpler for navigating and allows users to do more than just read. the m-links system incorporates modal browsing interaction and addresses a number of associated problems. we have built our system with an emphasis on simplicity and user extensibility and describe the design, implementation and evolution of the user interface.
boom chameleon: simultaneous capture of 3d viewpoint, voice and gesture annotations on a spatially-aware display. we introduce the boom chameleon, a novel input/output device consisting of a flat-panel display mounted on a tracked mechanical boom. the display acts as a physical window into 3d virtual environments, through which a one-to-one mapping between real and virtual space is preserved. the boom chameleon is further augmented with a touch-screen and a microphone/speaker combination. we present a 3d annotation application that exploits this unique configuration in order to simultaneously capture viewpoint, voice and gesture information. design issues are discussed and results of an informal user study on the device and annotation software are presented. the results show that the boom chameleon annotation facilities have the potential to be an effective, easy to learn and operate 3d design review system.