waashk/dblp huggingface dataset:
two-finger input with a standard touch screen. most current implementations of multi-touch screens are still too expensive or too bulky for widespread adoption. to improve this situation, this work describes the electronics and software needed to collect more data than one pair of coordinates from a standard 4-wire touch screen. with this system, one can measure the pressure of a single touch and approximately sense the coordinates of two touches occurring simultaneously. naturally, the system cannot offer the accuracy and versatility of full multi-touch screens. nonetheless, several example applications ranging from painting to zooming demonstrate a broad spectrum of use.
searchtogether: an interface for collaborative web search. studies of search habits reveal that people engage in many search tasks involving collaboration with others, such as travel planning, organizing social events, or working on a homework assignment. however, current web search tools are designed for a single user, working alone. we introduce searchtogether, a prototype that enables groups of remote users to synchronously or asynchronously collaborate when searching the web. we describe an example usage scenario, and discuss the ways searchtogether facilitates collaboration by supporting awareness, division of labor, and persistence. we then discuss the findings of our evaluation of searchtogether, analyzing which aspects of its design enabled successful collaboration among study participants.
e-conic: a perspective-aware interface for multi-display environments. multi-display environments compose displays that can be at different locations from and different angles to the user; as a result, it can become very difficult to manage windows, read text, and manipulate objects. we investigate the idea of perspective as a way to solve these problems in multi-display environments. we first identify basic display and control factors that are affected by perspective, such as visibility, fracture, and sharing. we then present the design and implementation of e-conic, a multi-display multi-user environment that uses location data about displays and users to dynamically correct perspective. we carried out a controlled experiment to test the benefits of perspective correction in basic interaction tasks like targeting, steering, aligning, pattern-matching and reading. our results show that perspective correction significantly and substantially improves user performance in all these tasks.
evaluating user interface systems research. the development of user interface systems has languished with the stability of desktop computing. future systems, however, that are off-the-desktop, nomadic or physical in nature will involve new devices and new software systems for creating interactive applications. simple usability testing is not adequate for evaluating complex systems. the problems with evaluating systems work are explored and a set of criteria for evaluating new ui systems work is presented.
shadow reaching: a new perspective on interaction for large displays. we introduce shadow reaching, an interaction technique that makes use of a perspective projection applied to a shadow representation of a user. the technique was designed to facilitate manipulation over large distances and enhance understanding in collaborative settings. we describe three prototype implementations that illustrate the technique, examining the advantages of using shadows as an interaction metaphor to support single users and groups of collaborating users. using these prototypes as a design probe, we discuss how the three components of the technique (sensing, modeling, and rendering) can be accomplished with real (physical) or computed (virtual) shadows, and the benefits and drawbacks of each approach.
specifying label layout style by example. creating high-quality label layouts in a particular visual style is a time-consuming process. although automated labeling algorithms can aid the layout process, expert design knowledge is required to tune these algorithms so that they produce layouts which meet the designer's expectations. we propose a system which can learn a labellayout style from a single example layout and then apply this style to new labeling problems. because designers find it much easier to create example layouts than tune algorithmic parameters, our system provides a more natural workflow for graphic designers. we demonstrate that our system is capable of learning a variety of label layout styles from examples.
lucid touch: a see-through mobile device. touch is a compelling input modality for interactive devices; however, touch input on the small screen of a mobile device is problematic because a user's fingers occlude the graphical elements he wishes to work with. in this paper, we present lucidtouch, a mobile device that addresses this limitation by allowing the user to control the application by touching the back of the device. the key to making this usable is what we call pseudo-transparency: by overlaying an image of the user's hands onto the screen, we create the illusion of the mobile device itself being semi-transparent. this pseudo-transparency allows users to accurately acquire targets while not occluding the screen with their fingers and hand. lucid touch also supports multi-touch input, allowing users to operate the device simultaneously with all 10 fingers. we present initial study results that indicate that many users found touching on the back to be preferable to touching on the front, due to reduced occlusion, higher precision, and the ability to make multi-finger input.
capturing the user's attention: insights from the study of human vision. an effective user interface is a cooperative interaction between humans and their technology. for that interaction to work, it needs to recognize the limitations and exploit the strengths of both parties. in this talk, i will concentrate on the human side of the equation. what do we know about human visual perceptual abilities that might have an impact on the design of user interfaces? the world presents us with more information than we can process. just try to read this abstract and the next piece of prose at the same time. we cope with this problem by using attentional mechanisms to select a subset of the input for further processing. an inter-face might be designed to .capture. attention, in order to induce a human to interact with it. once the human is using an interface, that interface should .guide. the user.s atten-tion in an intelligent manner. in recent decades, many of the rules of attentional capture and guidance have been worked out in the laboratory. i will illustrate some of the basic principles. for example: do some colors grab attention better than others? are faces special? when and why do people fail to .see. things that are right in front of their eyes.
sidesight: multi-"touch" interaction around small devices. interacting with mobile devices using touch can lead to fingers occluding valuable screen real estate. for the smallest devices, the idea of using a touch-enabled display is almost wholly impractical. in this paper we investigate sensing user touch around small screens like these. we describe a prototype device with infra-red (ir) proximity sensors embedded along each side and capable of detecting the presence and position of fingers in the adjacent regions. when this device is rested on a flat surface, such as a table or desk, the user can carry out single and multi-touch gestures using the space around the device. this gives a larger input space than would otherwise be possible which may be used in conjunction with or instead of on-display touch input. following a detailed description of our prototype, we discuss some of the interactions it affords.
an application-independent system for visualizing user operation history. a history-of-user-operations function helps make applications easier to use. for example, users may have access to an operation history list in an application to undo or redo a past operation. to provide an overview of a long operation history and help users find target interactions or application states quickly, visual representations of operation history have been proposed. however, most previous systems are tightly integrated with target applications and difficult to apply to new applications. we propose an application-independent method that can visualize the operation history of arbitrary gui applications by monitoring the input and output gui events from outside of the target application. we implemented a prototype system that visualizes operation sequences of generic java awt/swing applications using an annotated comic strip metaphor. we tested the system with various applications and present results from a user study.
cinch: a cooperatively designed marking interface for 3d pathway selection. to disentangle and analyze neural pathways estimated from magnetic resonance imaging data, scientists need an interface to select 3d pathways. broad adoption of such an interface requires the use of commodity input devices such as mice and pens, but these devices offer only two degrees of freedom. cinch solves this problem by providing a marking interface for 3d pathway selection. cinch interprets pen strokes as pathway selections in 3d using a marking language designed together with scientists. its bimanual interface employs a pen and a trackball (see figure 1), allowing alternating selections and scene rotations without changes of mode. cinch was evaluated by observing four scientists using the tool over a period of three weeks as part of their normal work activity. event logs and interviews revealed dramatic improvements in both the speed and quality of scientists' everyday work, and a set of principles that should inform the design of future 3d marking interfaces. more broadly, cinch demonstrates the value of the iterative, participatory design process that catalyzed its evolution.
the prod framework for proactive displays. a proactive display is an application that selects content to display based on the set of users who have been detected nearby. for example, the ticket2talk [17] proactive display application presented content for users so that other people would know something about them. it is our view that promising patterns for proactive display applications have been discovered, and now we face the need for frameworks to support the range of applications that are possible in this design space. in this paper, we present the proactive display (prod) framework, which allows for the easy construction of proactive display applications. it allows a range of proactive display applications, including ones already in the literature. prod also enlarges the design space of proactive display systems by allowing a variety of new applications that incorporate different views of social life and community.
from information visualization to direct manipulation: extending a generic visualization framework for the interactive editing of large datasets. today's generic data management applications such as accounting, crm or logging and tracking software, rely on form and menu based interfaces. these applications take only marginal advantage of current graphical user interfaces. this is because the data they handle does not have intrinsic visual representations upon which direct manipulation principles can be used. this article presents how we have extended an information visualization framework with generic data manipulation functions. these new data editing capabilities are tuned to take advantage of the characteristics of each view. they enable us to generalize the direct manipulation mechanisms to address many abstract data manipulation needs. in this article we present five uses of the features we have implemented and deduce a general workflow applicable to a variety of contexts. the workflow comprises three steps and five editing actions. the steps are: adjust view, select, and edit. the editing actions are: edit a value or group of values, clone objects, remove objects, add attributes, and remove attributes. the workflow provides complete editing access to table and hierarchical data structures using particularly terse interaction methods. it defines a general data editing model that enables powerful data manipulation tasks without requiring end-user programming or scripting.
design as exploration: creating interface alternatives through parallel authoring and runtime tuning. creating multiple prototypes facilitates comparative reasoning, grounds team discussion, and enables situated exploration. however, current interface design tools focus on creating single artifacts. this paper introduces the juxtapose code editor and runtime environment for designing multiple alternatives of both application logic and interface parameters. for rapidly comparing code alternatives, juxtapose introduces selectively parallel source editing and execution. to explore parameter variations, juxtapose automatically creates control interfaces for "tuning" application variables at runtime. this paper describes techniques to support design exploration for desktop, mobile, and physical interfaces, and situates this work in a larger design space of tools for explorative programming. a summative study of juxtapose with 18 participants demonstrated that parallel editing and execution are accessible to interaction designers and that designers can leverage these techniques to survey more options, faster.
soap: a pointing device that works in mid-air. soap is a pointing device based on hardware found in a mouse, yet works in mid-air. soap consists of an optical sensor device moving freely inside a hull made of fabric. as the user applies pressure from the outside, the optical sensor moves independent from the hull. the optical sensor perceives this relative motion and reports it as position input. soap offers many of the benefits of optical mice, such as high-accuracy sensing. we describe the design of a soap prototype and report our experiences with four application scenarios, including a wall display, windows media center, slide presentation, and interactive video games.
browsing large html tables on small screens. we propose new interaction techniques that support better browsing of large html tables on small screen devices, such as mobile phones. we propose three modes for browsing tables: normal mode, record mode, and cell mode. normal mode renders tables in the ordinary way, but provides various useful functions for browsing large tables, such as hiding unnecessary rows and columns. record mode regards each row (or column) as the basic information unit and displays it in a record-like format with column (or row) headers, while cell mode regards each cell as the basic unit and displays each cell together with its corresponding row and column headers. for these table presentations, we need to identify row and column headers that explain the meaning of rows and columns. to provide users with both row and column headers even when the tables have attributes for only one of them, we introduce the concept of keys and develop a method of automatically discovering attributes and keys in tables. another issue in these presentations is how to handle composite cells spanning multiple rows or columns. we determine the semantics of such composite cells and render them in appropriate ways in accordance with their semantics.
phosphor: explaining transitions in the user interface using afterglow effects. sometimes users fail to notice a change that just took place on their display. for example, the user may have accidentally deleted an icon or a remote collaborator may have changed settings in a control panel. animated transitions can help, but they force users to wait for the animation to complete. this can be cumbersome, especially in situations where users did not need an explanation. we propose a different approach. phosphor objects show the outcome of their transition instantly; at the same time they explain their change in retrospect. manipulating a phosphor slider, for example, leaves an afterglow that illustrates how the knob moved. the parallelism of instant outcome and explanation supports both types of users. users who already understood the transition can continue interacting without delay, while those who are inexperienced or may have been distracted can take time to view the effects at their own pace. we present a framework of transition designs for widgets, icons, and objects in drawing programs. we evaluate phosphor objects in two user studies and report significant performance benefits for phosphor objects.
an exploration of pen rolling for pen-based interaction. current pen input mainly utilizes the position of the pen tip, and occasionally, a button press. other possible device parameters, such as rolling the pen around its longitudinal axis, are rarely used. we explore pen rolling as a supporting input modality for pen-based interaction. through two studies, we are able to determine 1) the parameters that separate intentional pen rolling for the purpose of interaction from incidental pen rolling caused by regular writing and drawing, and 2) the parameter range within which accurate and timely intentional pen rolling interactions can occur. building on our experimental results, we present an exploration of the design space of rolling-based interaction techniques, which showcase three scenarios where pen rolling interactions can be useful: enhanced stimulus-response compatibility in rotation tasks [7], multi-parameter input, and simplified mode selection.
mnemonic rendering: an image-based approach for exposing hidden changes in dynamic displays. managing large amounts of dynamic visual information involves understanding changes happening out of the user's sight. in this paper, we show how current software does not adequately support users in this task, and motivate the need for a more general approach. we propose an image-based storage, visualization, and implicit interaction paradigm called mnemonic rendering that provides better support for handling visual changes. once implemented on a system, mnemonic rendering techniques can benefit all applications. we explore its rich design space and discuss its expected benefits as well as limitations based on feedback from users of a small-screen and a wall-size prototype.
towards more paper-like input: flexible input devices for foldable interaction styles. this paper presents foldable user interfaces (fui), a combination of a 3d gui with windows imbued with the physics of paper, and foldable input devices (fids). fids are sheets of paper that allow realistic transformations of graphical sheets in the fui. foldable input devices are made out of construction paper augmented with ir reflectors, and tracked by computer vision. window sheets can be picked up and flexed with simple movements and deformations of the fid. fids allow a diverse lexicon of one-handed and two-handed interaction techniques, including folding, bending, flipping and stacking. we show how these can be used to ease the creation of simple 3d models, but also for tasks such as page navigation.
interacting with dynamically defined information spaces using a handheld projector and a pen. the recent trend towards miniaturization of projection technology indicates that handheld devices will soon have the ability to project information onto any surface, thus enabling interfaces that are not possible with current handhelds. we explore the design space of dynamically defining and interacting with multiple virtual information spaces embedded in a physical environment using a handheld projector and a passive pen tracked in 3d. we develop techniques for defining and interacting with these spaces, and explore usage scenarios.
interactive viscosity. when the macintosh first made graphical user interfaces popular the notion of each person having their own computer was novel. today's technology landscape is characterized by multiple computers per person many with far more capacity than that original mac. the world of input devices, display devices and interactive techniques is far richer than those macintosh days. despite all of this diversity in possible interactions very few of these integrate well with each other. the monolithic isolated user interface architecture that characterized the macintosh still dominates a great deal of today's personal computing. this talk will explore how possible ways to change that architecture so that information, interaction and communication flows more smoothly among our devices and those of our associates.
an infrastructure for extending applications' user experiences across multiple personal devices. users increasingly interact with a heterogeneous collection of computing devices. the applications that users employ on those devices, however, still largely provide user experiences that assume the use of a single computer. this failure is due in part to the difficulty of creating user experiences that span multiple devices, particularly the need to manage identifying, connecting to, and communicating with other devices. in this paper we present an infrastructure based on instant messaging that simplifies adding that additional functionality to applications. our infrastructure elevates device ownership to a first class property, allowing developers to provide functionality that spans personal devices without writing code to manage users' devices or establish connections among them. it also provides simple mechanisms for applications to send information, events, or commands between a user's devices. we demonstrate the effectiveness of our infrastructure by presenting a set of sample applications built with it and a user study demonstrating that developers new to the infrastructure can implement all of the cross-device functionality for three applications in, on average, less than two and a half hours.
videotater: an approach for pen-based digital video segmentation and tagging. the continuous growth of media databases necessitates development of novel visualization and interaction techniques to support management of these collections. we present videotater, an experimental tool for a tablet pc that supports the efficient and intuitive navigation, selection, segmentation, and tagging of video. our veridical representation immediately signals to the user where appropriate segment boundaries should be placed and allows for rapid review and refinement of manually or automatically generated segments. finally, we explore a distribution of modalities in the interface by using multiple timeline representations, pressure sensing, and a tag painting/erasing metaphor with the pen.
living better with robots. the emerging field of human-robot interaction is undergoing rapid growth, motivated by important societal challenges and new applications for personal robotic technologies for the general public. in this talk, i highlight several projects from my research group to illustrate recent research trends to develop socially interactive robots that work and learn with people as partners. an important goal of this work is to use interactive robots as a scientific tool to understand human behavior, to explore the role of physical embodiment in interactive technology, and to use these insights to design robotic technologies that can enhance human performance and quality of life. throughout the talk i will highlight synergies with hci and connect hri research goals to specific applications in healthcare, education, and communication.