Context-based Innovative Mobile User Interfaces

With the development of various sensors and devices, the computing is no longer limited to the desktop mode. However, the traditional user interface, used on the desktop computer, is no longer appropriate for ubiquitous computing. A sophisticated mobile environment requires dedicated design of interfaces involving the input and output techniques with the new emerging features beyond the ability of traditional ones. One of the possible solutions to enable the ubiquitous interaction and break the limitation of the desktop mode is the nomadism, another one is mobility. We propose three interfaces related to these two solutions: In-environment interface (IEI), Environment Dependent Interface (EDI) and Environment Independent Interface (EII). We exclude IEI and mainly focus on the wearable interaction. This thesis aims at an investigation of research issues involved in the design, implementation and evaluation of EDI and EII. First, we globally present our design approach of these three innovative interfaces (IEI, EDI and EII), their wearable configurations (camera-glasses device unit and camera-projector device unit), the real examples of use (including the Research Team Interaction Scenario) and the preliminary evaluation of input selection techniques to prove the feasibility of our prototypes. Second, in order to concretize the EDI and EII, we propose a continuum ranging from physical interface, physical-digital mixed interface, to digital interface. Based on the physical paper-based interface, the MobilePaperAccess is a wearable camera-glasses system allowing accessing to digital information from a paper interface. This system is devised to validate our concepts of EDI and EII. The two interfaces (EDI and EII) and three input techniques (finger input, mask input and page input) have been evaluated in both the quantitative and qualitative methods with this system. Based on the digital projected scalable interface, PlayAllAround system is a wearable camera-projector system with the scalable interface allowing mobile interaction and provides both the nearer small-size interface and the farther large-size interface supporting for the private and public use. In the design of this system we propose the design of reference-cell and the principle from decomposition of the application tasks to formation of scalable interface. The hover gesture and the scalable interface have been evaluated with this system. Results from our experiments have shown that PlayAllAround performs well with the hover gesture and the scalable interface. To go a step further, we explore the hand gestures including the hover gesture, the pinch gesture and the fist-palm gesture. We employ the pinch gesture and fist-palm gesture as the input for pointing, drag-drop action and painting. In order to satisfy the requirements of interaction in the sophisticated daily life, we investigate how the user might interact with these gestures in both the stationary and mobile settings. We compare the interactions of hover gesture and pinch gesture under three situations as standing, sitting and walking. Furthermore, the evaluation of the projected interface, as well as the satisfaction on the operation of the head worn configuration has been done and discussed. Finally, along with the continuum, we continue to investigate the physical-digital mixed interface by a wearable camera-projector device unit with the goal of concretizing the concepts of EDI and EII. The mixed interface contains the marker-based part with AR-toolkit tags and the projected digital part. Since the projection display can be an alternative method to provide the large presentation of image without any external support of device, the mixed interface has the capacity to contain more information as well as more dynamical choices compared with the physical interface. In a conclusion, in this thesis we have investigated the theoretical analysis, the innovative design, practical development, and both the quantitative and qualitative user studies. Our work is a many-sided investigation on innovative wearable interfaces, as well as input and output techniques, which will pave the foundation for the future research on the wearable interfaces.