waashk/dblp huggingface dataset:
mouse 2.0: multi-touch meets the mouse. in this paper we present novel input devices that combine the standard capabilities of a computer mouse with multi-touch sensing. our goal is to enrich traditional pointer-based desktop interactions with touch and gestures. to chart the design space, we present five different multi-touch mouse implementations. each explores a different touch sensing strategy, which leads to differing form-factors and hence interactive possibilities. in addition to the detailed description of hardware and software implementations of our prototypes, we discuss the relative strengths, limitations and affordances of these novel input devices as informed by the results of a preliminary user study.
enabling always-available input with muscle-computer interfaces. previous work has demonstrated the viability of applying offline analysis to interpret forearm electromyography (emg) and classify finger gestures on a physical surface. we extend those results to bring us closer to using muscle-computer interfaces for always-available input in real-world applications. we leverage existing taxonomies of natural human grips to develop a gesture set covering interaction in free space even when hands are busy with other objects. we present a system that classifies these gestures in real-time and we introduce a bi-manual paradigm that enables use in interactive systems. we report experimental results demonstrating four-finger classification accuracies averaging 79% for pinching, 85% while holding a travel mug, and 88% when carrying a weighted bag. we further show generalizability across different arm postures and explore the tradeoffs of providing real-time visual feedback.
using fnirs brain sensing in realistic hci settings: experiments and guidelines. because functional near-infrared spectroscopy (fnirs) eases many of the restrictions of other brain sensors, it has potential to open up new possibilities for hci research. from our experience using fnirs technology for hci, we identify several considerations and provide guidelines for using fnirs in realistic hci laboratory settings. we empirically examine whether typical human behavior (e.g. head and facial movement) or computer interaction (e.g. keyboard and mouse usage) interfere with brain measurement using fnirs. based on the results of our study, we establish which physical behaviors inherent in computer usage interfere with accurate fnirs sensing of cognitive state information, which can be corrected in data analysis, and which are acceptable. with these findings, we hope to facilitate further adoption of fnirs brain sensing technology in hci research.
activity analysis enabling real-time video communication on mobile phones for deaf users. we describe our system called mobileasl for real-time video communication on the current u.s. mobile phone network. the goal of mobileasl is to enable deaf people to communicate with sign language over mobile phones by compressing and transmitting sign language video in real-time on an off-the-shelf mobile phone, which has a weak processor, uses limited bandwidth, and has little battery capacity. we develop several h.264-compliant algorithms to save system resources while maintaining asl intelligibility by focusing on the important segments of the video. we employ a dynamic skin-based region-of-interest (roi) that encodes the skin at higher quality at the expense of the rest of the video. we also automatically recognize periods of signing versus not signing and raise and lower the frame rate accordingly, a technique we call variable frame rate (vfr). we show that our variable frame rate technique results in a 47% gain in battery life on the phone, corresponding to an extra 68 minutes of talk time. we also evaluate our system in a user study. participants fluent in asl engage in unconstrained conversations over mobile phones in a laboratory setting. we find that the roi increases intelligibility and decreases guessing. vfr increases the need for signs to be repeated and the number of conversational breakdowns, but does not affect the users' perception of adopting the technology. these results show that our sign language sensitive algorithms can save considerable resources without sacrificing intelligibility.
collabio: a game for annotating people within social networks. we present collabio, a social tagging game within an online social network that encourages friends to tag one another. collabio's approach of incentivizing members of the social network to generate information about each other produces personalizing information about its users. we report usage log analysis, survey data, and a rating exercise demonstrating that collabio tags are accurate and augment information that could have been scraped online.
interactive viscosity. your conscious experience is not a function of the world, it is a function of the neural networks of your brain. therefore, the biology of the brain and consciousness is as fundamental to understanding the universe, as we know it, as the high-energy physics of subatomic particles. this is especially true for the study of sensory and cognitive illusions, since they represent effects that clearly stand out as not representing the real world. that is, since illusions don't match reality we can know that by studying illusions we are studying exactly what the brain is actually doing, and not just what we think the brain should be doing. your brain does a staggering amount of pragmatic self-dealing guesswork and outright confabulation in order to construct the highly imperfect mental simulation of reality known as "consciousness." this is not to say that objective reality isn't "out there" in a very real sense--but no one lives there. no one's ever even been there for a visit. ironically, the fact that consciousness feels like a solid, robust, fact-rich transcript of reality is just one of the countless illusions your brain creates for itself. illusions are not errors of the brain. far from it. illusions arise from processes that are critical to our survival. our brains have developed illusory processes so that we may experience the world in a ready-to-consume manner. remove the machinery of illusion, and you unwind the entire tapestry of human awareness. illusions are those perceptual experiences that do not match the physical reality. they are therefore exquisite tools with which to analyze the neural correlates of human perception and consciousness. neuroscientists have long known that they can only be sure of where they stand, in terms of correlating neural responses to awareness, when they correlate the awareness of an illusion to the brain's response, specifically because of the illusions' mismatch with reality. the study of illusions is therefore of critical importance to the understanding of the basic mechanisms of sensory perception and conscious awareness. if you've ever seen a good magician perform, you know how thrilling it is to watch the impossible happening before your eyes. the laws of physics, probability, psychology and common sense--the four trusty compass points in your mental map of reality--are suddenly turned into liabilities. objects and people appear, vanish, levitate, transpose, transform, and with all your smarts you can't imagine how it's being done. magicians are the premier artists of attention and awareness, and they manipulate our cognition like clay on a potter's wheel. and the mechanisms underlying magic perception have implications for our daily lives. the magical arts work because humans have hardwired processes of attention and awareness that are hackable. by understanding how magicians hack our brains, we can better understand how we work.
detecting and leveraging finger orientation for interaction with direct-touch surfaces. current interactions on direct-touch interactive surfaces are often modeled based on properties of the input channel that are common in traditional graphical user interfaces (gui) such as x-y coordinate information. leveraging additional information available on the surfaces could potentially result in richer and novel interactions. in this paper we specifically explore the role of finger orientation. this property is typically ignored in touch-based interactions partly because of the ambiguity in determining it solely from the contact shape. we present a simple algorithm that unambiguously detects the directed finger orientation vector in real-time from contact information only, by considering the dynamics of the finger landing process. results of an experimental evaluation show that our algorithm is stable and accurate. we then demonstrate how finger orientation can be leveraged to enable novel interactions and to infer higher-level information such as hand occlusion or user position. we present a set of orientation-aware interaction techniques and widgets for direct-touch surfaces.
tapsongs: tapping rhythm-based passwords on a single binary sensor. tapsongs are presented, which enable user authentication on a single "binary" sensor (e.g., button) by matching the rhythm of tap down/up events to a jingle timing model created by the user. we describe our matching algorithm, which employs absolute match criteria and learns from successful logins. we also present a study of 10 subjects showing that after they created their own tapsong models from 12 examples (
everybodylovessketch: 3d sketching for a broader audience. we present everybodylovessketch, a gesture-based 3d curve sketching system for rapid ideation and visualization of 3d forms, aimed at a broad audience. we first analyze traditional perspective drawing in professional practice. we then design a system built upon the paradigm of ilovesketch, a 3d curve drawing system for design professionals. the new system incorporates many interaction aspects of perspective drawing with judicious automation to enable novices with no perspective training to proficiently create 3d curve sketches. everybodylovessketch supports a number of novel interactions: tick-based sketch plane selection, single view definition of arbitrary extrusion vectors, multiple extruded surface sketching, copy-and-project of 3d curves, freeform surface sketching, and an interactive perspective grid. finally, we present a study involving 49 high school students (with no formal artistic training) who each learned and used the system over 11 days, which provides detailed insights into the popularity, power and usability of the various techniques, and shows our system to be easily learnt and effectively used, with broad appeal.
contact area interaction with sliding widgets. we show how to design touchscreen widgets that respond to a finger's contact area. in standard touchscreen systems a finger often appears to touch several screen objects, but the system responds as though only a single pixel is touched. in contact area interaction all objects under the finger respond to the touch. users activate control widgets by sliding a movable element, as though flipping a switch. these sliding widgets resolve selection ambiguity and provide designers with a rich vocabulary of self-disclosing interaction mechanism. we showcase the design of several types of sliding widgets, and report study results showing that the simplest of these widgets, the sliding button, performs on-par with medium-sized pushbuttons and offers greater accuracy for small-sized buttons.
changing how people view changes on the web. the web is a dynamic information environment. web content changes regularly and people revisit web pages frequently. but the tools used to access the web, including browsers and search engines, do little to explicitly support these dynamics. in this paper we present diffie, a browser plug-in that makes content change explicit in a simple and lightweight manner. diffie caches the pages a person visits and highlights how those pages have changed when the person returns to them. we describe how we built a stable, reliable, and usable system, including how we created compact, privacy-preserving page representations to support fast difference detection. via a longitudinal user study, we explore how diffie changed the way people dealt with changing content. we find that much of its benefit came not from exposing expected change, but rather from drawing attention to unexpected change and helping people build a richer understanding of the web content they frequent.
augmenting interactive tables with mice & keyboards. this note examines the role traditional input devices can play in surface computing. mice and keyboards can enhance tabletop technologies since they support high fidelity input, facilitate interaction with distant objects, and serve as a proxy for user identity and position. interactive tabletops, in turn, can enhance the functionality of traditional input devices: they provide spatial sensing, augment devices with co-located visual content, and support connections among a plurality of devices. we introduce eight interaction techniques for a table with mice and keyboards, and we discuss the design space of such interactions.
the web page as a wysiwyg end-user customizable database-backed information management application. dido is an application (and application development environment) in a web page. it is a single web page containing rich structured data, an ajaxy interactive visualizer/editor for that data, and a "metaeditor" for wysiwyg editing of the visualizer/editor. historically, users have been limited to the data schemas, visualizations, and interactions offered by a small number of heavyweight applications. in contrast, dido encourages and enables the end user to edit (not code) in his or her web browser a distinct ephemeral interaction "wrapper" for each data collection that is specifically suited to its intended use. dido's active document metaphor has been explored before but we show how, given today's web infrastructure, it can be deployed in a small self-contained html document without touching a web client or server.
semfeel: a user interface with semantic tactile feedback for mobile touch-screen devices. one of the challenges with using mobile touch-screen devices is that they do not provide tactile feedback to the user. thus, the user is required to look at the screen to interact with these devices. in this paper, we present semfeel, a tactile feedback system which informs the user about the presence of an object where she touches on the screen and can offer additional semantic information about that item. through multiple vibration motors that we attached to the backside of a mobile touch-screen device, semfeel can generate different patterns of vibration, such as ones that flow from right to left or from top to bottom, to help the user interact with a mobile device. through two user studies, we show that users can distinguish ten different patterns, including linear patterns and a circular pattern, at approximately 90% accuracy, and that semfeel supports accurate eyes-free interactions.
photoelastictouch: transparent rubbery tangible interface using an lcd and photoelasticity. photoelastictouch is a novel tabletop system designed to intuitively facilitate touch-based interaction via real objects made from transparent elastic material. the system utilizes vision-based recognition techniques and the photoelastic properties of the transparent rubber to recognize deformed regions of the elastic material. our system works with elastic materials over a wide variety of shapes and does not require any explicit visual markers. compared to traditional interactive surfaces, our 2.5 dimensional interface system enables direct touch interaction and soft tactile feedback. in this paper we present our force sensing technique using photoelasticity and describe the implementation of our prototype system. we also present three practical applications of photoelastictouch, a force-sensitive touch panel, a tangible face application, and a paint application.
communitycommands: command recommendations for software applications. we explore the use of modern recommender system technology to address the problem of learning software applications. before describing our new command recommender system, we first define relevant design considerations. we then discuss a 3 month user study we conducted with professional users to evaluate our algorithms which generated customized recommendations for each user. analysis shows that our item-based collaborative filtering algorithm generates 2.1 times as many good suggestions as existing techniques. in addition we present a prototype user interface to ambiently present command recommendations to users, which has received promising initial user feedback.
a screen-space formulation for 2d and 3d direct manipulation. rotate-scale-translate (rst) interactions have become the de facto standard when interacting with two-dimensional (2d) contexts in single-touch and multi-touch environments. because the use of rst has thus far focused almost entirely on 2d, there are not yet standard techniques for extending these principles into three dimensions. in this paper we describe a screen-space method which fully captures the semantics of the traditional 2d rst multi-touch interaction, but also allows us to extend these same principles into three-dimensional (3d) interaction. just like rst allows users to directly manipulate 2d contexts with two or more points, our method allows the user to directly manipulate 3d objects with three or more points. we show some novel interactions, which take perspective into account and are thus not available in orthographic environments. furthermore, we identify key ambiguities and unexpected behaviors that arise when performing direct manipulation in 3d and offer solutions to mitigate the difficulties each presents. finally, we show how to extend our method to meet application-specific control objectives, as well as show our method working in some example environments.
bonfire: a nomadic system for hybrid laptop-tabletop interaction. we present bonfire, a self-contained mobile computing system that uses two laptop-mounted laser micro-projectors to project an interactive display space to either side of a laptop keyboard. coupled with each micro-projector is a camera to enable hand gesture tracking, object recognition, and information transfer within the projected space. thus, bonfire is neither a pure laptop system nor a pure tabletop system, but an integration of the two into one new nomadic computing platform. this integration (1) enables observing the periphery and responding appropriately, e.g., to the casual placement of objects within its field of view, (2) enables integration between physical and digital objects via computer vision, (3) provides a horizontal surface in tandem with the usual vertical laptop display, allowing direct pointing and gestures, and (4) enlarges the input/output space to enrich existing applications. we describe bonfire's architecture, and offer scenarios that highlight bonfire's advantages. we also include lessons learned and insights for further development and use.
mining web interactions to automatically create mash-ups. the deep web contains an order of magnitude more information than the surface web, but that information is hidden behind the web forms of a large number of web sites. metasearch engines can help users explore this information by aggregating results from multiple resources, but previously these could only be created and maintained by programmers. in this paper, we explore the automatic creation of metasearch mash-ups by mining the web interactions of multiple web users to find relations between query forms on different web sites. we also present an implemented system called tx2 that uses those connections to search multiple deep web resources simultaneously and integrate the results in context in a single results page. tx2 illustrates the promise of constructing mash-ups automatically and the potential of mining web interactions to explore deep web resources.
overview based example selection in end user interactive concept learning. interaction with large unstructured datasets is difficult because existing approaches, such as keyword search, are not always suited to describing concepts corresponding to the distinctions people want to make within datasets. one possible solution is to allow end users to train machine learning systems to identify desired concepts, a strategy known as interactive concept learning. a fundamental challenge is to design systems that preserve end user flexibility and control while also guiding them to provide examples that allow the machine learning system to effectively learn the desired concept. this paper presents our design and evaluation of four new overview based approaches to guiding example selection. we situate our explorations within cueflik, a system examining end user interactive concept learning in web image search. our evaluation shows our approaches not only guide end users to select better training examples than the best performing previous design for this application, but also reduce the impact of not knowing when to stop training the system. we discuss challenges for end user interactive concept learning systems and identify opportunities for future research on the effective design of such systems.
advanced narrow speech channeling algorithm for robot speech recognition. we propose a source extraction algorithm that provides robust speech recognition of intelligent service robots. this system consists of two stages. first, the target speech from a target direction is extracted from a mixture inputs using a binary time-frequency masking method. this stage can work alone. the next stage re-estimates the range of the target direction calculating the energy of the extracted speech by &delta; histogram. this algorithm was mounted on the speech recognition system of a service robot as a preprocessing step. in section 6, we show that our algorithm has excellent performances.
a study on enhanced algorithms for detecting defects of glasses. in this paper, we proposed image processing algorithms and developed the vision system for measuring blisters of flat glasses. the images captured by changing focus of camera because the depth of field of ccd camera is smaller than the depth of glasses and classified two types in blister as circle and ellipse types by the proposed criteria of measurement. we applied different methods to each type to improve the accuracy. in the pre-processing, it obtained binarized images to find blister. the information such as area, boundary, length, and position of blister has extracted from the binarized images by using chain code algorithm. for an illumination, the bright field illumination as back-light was applied. we tested the automatic measurement system, which we developed by using vision system, it successes 100% for detecting the blister at all glasses, and the error accuracy of information is about +/-10%.
context awareness inference engine for location based applications. in ubiquitous environments, location-based context awareness service provides additional services based on a user's location. in this paper, we design a context awareness inference engine model for working environment monitoring system of mobile object which is an issue in location-based application. we in the proposed a context awareness inference engine can use sensors, mobile communication network in order to achieve working environment information of mobile object. the proposed framework is consisted of data collector, context manager, knowledge base, inference engine, and context database. and it also has a structure that enables to serve users by embodying context awareness application interface for particular application.