. Two approaches allowing a subject-matter expert (S M E) to guide a local user to place the top of an aircraft engine combustion chamber relative to its bottom by interacting with a virtual replica of the top. (a) PO I N T 3D, as seen by the S M E in VR: The S M E places 3 D annotations (red spheres) on the transparent virtual replica, specifying contact points. The same annotations appear on the opaque virtual proxy corresponding to the virtual replica. (b) DE M O 3D, as seen by the S M E in AR, looking through the local user's cameras: The S M E has placed the transparent virtual replica on the chamber bottom. Pairs of metaobject annotations on the virtual replica and physical chamber top are linked with color-coded lines to show the correspondence between the physical top and its virtual replica, and are also seen by the local user. © 2015 Columbia University In many complex tasks, a remote subject-matter expert may need to assist a local user to guide actions on objects in the local user's environment. However, effective spatial refer encing and action demonstration in a remote physical envi ronment can be challenging. We introduce two approaches that use Virtual Reality (VR) or Augmented Reality (AR) for the remote expert, and AR for the local user, each wearing a stereo head-worn display. Both approaches allow the expert to create and manipulate virtual replicas of physical objects in the local environment to refer to parts of those physical ob jects and to indicate actions on them. This can be especially useful for parts that are occluded or difficult to access. In one approach, the expert points in 3 D to portions of virtual replicas to annotate them. In another approach, the expert demonstrates actions in 3 D by manipulating virtual replicas, supported by constraints and annotations. We performed a user study of a 6D O F alignment task, a key operation in many Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita tion on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re publish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. physical task domains, comparing both approaches to an ap proach in which the expert uses a 2D tablet-based drawing system similar to ones developed for prior work on remote assistance. The study showed the 3D demonstration approach to be faster than the others. In addition, the 3D pointing ap proach was faster than the 2D tablet in the case of a highly trained expert.
Oda et al. REF used an annotation-based solution to guide a user to match a 6DOF pose specified by a remote subject matter expert (SME).
Virtual Replicas for Remote Assistance in Virtual and Augmented Reality
UIST '15
Computer Science