Patent Publication Number: US-11042036-B2

Title: Device mount for mixed-reality capture

Description:
BACKGROUND 
     A conventional mixed-reality system is capable of presenting a mixed-reality environment to a user. For example, a mixed-reality headset may allow a user to see portions of her actual environment while displaying images of virtual objects and user interface elements such that the virtual objects and user interface elements appear to be present within the actual environment. The headset may also deliver sounds to the user which are not otherwise present within the actual environment. 
     In mixed-reality capture, the built-in camera of a mixed-reality headset captures footage of a user&#39;s view through the headset. This footage may include virtual objects, user interface elements, and portions of the user&#39;s actual environment as described above. However, such footage is typically shaky and unsuitable for accurately representing the mixed-reality experience to a viewer. The headset may be mounted to a stable platform during mixed-reality capture to address this issue, but such an arrangement prevents live monitoring of the actual footage being captured. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  depicts a mixed-reality headset according to some embodiments; 
         FIG. 2  depicts an apparatus and a camera mounted to the apparatus according to some embodiments; 
         FIG. 3  illustrates mounting of a headset on an apparatus according to some embodiments; 
         FIG. 4  is a bottom view of a headset according to some embodiments; 
         FIG. 5  illustrates mounting of a headset on an apparatus according to some embodiments; 
         FIG. 6  depicts a mixed-reality headset mounted to an apparatus according to some embodiments; 
         FIG. 7  depicts an apparatus supporting a mixed-reality headset and mounted to a camera stabilization device according to some embodiments; 
         FIG. 8  is a top right view of an apparatus according to some embodiments; and 
         FIG. 9  is a bottom right view of an apparatus according to some embodiments. 
     
    
    
     DETAILED DESCRIPTION 
     The following description is provided to enable any person in the art to make and use the described embodiments. Various modifications, however, will remain apparent to those in the art. 
     During traditional mixed-reality capture, for example, a mixed-reality headset presents graphical elements on a substantially transparent display screen. A user wearing the headset views the display screen so as to see the graphical elements as well as any actual objects located on an opposite side of the display screen. A camera system located in the headset captures images (e.g., video footage) of the display screen from the user&#39;s perspective. Since the camera system and the user&#39;s eyes are not co-located, the camera system is calibrated to allow registration of the captured images from a reference frame of the camera system to a reference frame of the user. 
     According to some embodiments, an apparatus is provided to efficiently monitor images captured by a device such as a mixed-reality headset. The apparatus may include elements to support a display device (e.g., a mixed-reality headset) and to support an image capture device, such as a camera, in a substantially fixed relationship to a display screen of the display device. The image capture device and the camera system of the display device may be calibrated based on the substantially fixed relationship so that footage captured by the image capture device is suitably similar to footage captured by the camera system during mixed-reality capture. Footage captured by the image capture device may be transmitted to an external display during mixed reality capture, thereby providing an operator/observer with a live preview of the actual footage being captured by the camera system of the display device. 
     By facilitating a substantially fixed relationship between the image capture device and the display screen of the display device, embodiments may reduce a need for recalibration of the system in response to bumping or other physical movement of the system. 
     Some embodiments also provide features to facilitate stabilization of the captured footage. For example, an apparatus according to some embodiments may include an interface for attaching the mixed-reality display device to a camera support system. Examples of a camera support system include but are not limited to a stabilizer, a tripod, a drone, a dolly, a slider, and a jib. 
     According to some embodiments, the apparatus includes features to facilitate replacement of a first display device with another display device (e.g., another mixed-reality headset) such that the camera remains in the same substantially fixed relationship with respect to a display screen of the new display device. Such features may reduce a need for recalibration in response to device replacement (e.g., if the first display device requires recharging). 
       FIG. 1  is a view of wearable (i.e., head-mounted) mixed-reality device  100  according to some embodiments. Embodiments are not limited to device  100 , and may utilize any past, current or future device capable of performing the functions attributed to device  100  herein. 
     Device  100  includes substantially transparent display screen  110  for presenting images to a wearer thereof. The images may completely occupy the wearer&#39;s field of view, or may be presented within the wearer&#39;s field of view such that the wearer may still view other objects in her vicinity. The images may be two-dimensional, three-dimensional, holographic, and/or any other type of presentable images that are or become known. 
     Device  100  also includes camera system  120  to capture images and video. The captured video may comprise mixed-reality capture footage as described above. Camera system  120  may also function as a sensor to assist, in conjunction with data from on-board accelerometers (not shown), in determining the position and motion of device  100  in three-dimensional space with six degrees of freedom. 
       FIG. 2  illustrates apparatus  200  to support a device such as device  100  according to some embodiments. Apparatus  200  also includes elements to support camera  300 . As described above, Apparatus  200  may support a device  100  and camera  300  to maintain a substantially fixed relationship therebetween. The substantially fixed relationship may position lens  320  of camera  300  with respect to a display screen/lens of device  100  so as to acquire suitable live footage of mixed-reality video captured by device  100 . Embodiments are not limited to use in conjunction with a camera having a form factor or of a type as illustrated in  FIG. 2 . The elements of apparatus  200  may comprise machined aluminum or any suitable one or more materials. 
     Elements operable to support device  100  are labeled in  FIG. 2  using numeric convention  210   x , while elements operable to support camera  300  are labeled using numeric convention  220   x . For convenience, elements  210   x  will be referred to as comprising a “first support” and elements  220   x  will be referred to as comprising a “second support”. 
     The first support includes a clamping mechanism consisting of engagement arm  210   a  and standoff  210   b , and a similar clamping mechanism consisting of engagement arm  210   c  and standoff  210   d . Each of standoffs  210   b  and  210   d  extend from surface  210   e  of the first support. Engagement arms  210   a  and  210   c  are rotatable around the couplings to their respective standoffs  210   b  and  210   d  to be disposed upward and substantially in parallel with their standoffs  210   b  and  210   d , and to be disposed substantially perpendicular to their standoffs  210   b  and  210   d . In the latter position, engagement arms  210   a  and  210   c  may engage portions of device  100  against surface  210   e , thus contributing to fixedly supporting device  100  against the first support. Element  210   f  may comprise a quick-release button for releasing arm  210   a , and therefore a corresponding engaged portion of device  100 . Projections  210   g  extend from surface  210   e  and mate with corresponding features of device  100 , as will be described below. 
     The second support includes rotatable element  220   a , which is rotatably inserted into an opening of base  210   h . Base  210   h  may therefore serve to couple the second support to the first support. Arm  220   c  is slidably inserted into element  220   a  and supports camera interface  220   d . Interface  220   d  includes features to engage with corresponding features of mounting element  320  of (or coupled to) camera  300 . As will be evident from the following figures and description, a position of camera lens  310  with respect to a display of device  100  may be adjusted via rotation of element  220   a , sliding of arm  220   c  and rotation of element  320  within interface  220   d , and then fixed using corresponding fixing mechanisms (e.g., screws). 
       FIG. 3  illustrates placement of device  100  upon apparatus  200  according to some embodiments. Engagement arms  210   a  and  210   c  have been rotated away from surface  210   e  to facilitate lowering of device  100  onto surface  210   e . As shown in  FIG. 4 , elements  130   a  and  130   b  of device  100  define respective channels  132   a  and  132   b . Channels  132   a  and  132   b  are aligned with and mate with protrusions  210   g  as device  100  settles on surface  210   e  as shown in  FIG. 3 . Protrusions  210   g  may be sized and disposed with respect to channels  132   a  and  132   b  such that movement of device  100  parallel to the plane of surface  210   e  is limited while protrusions  210   g  are mated with channels  132   a  and  132   b.    
     Also shown in  FIG. 4  are speaker elements  140   a  and  140   b  of device  100 . As shown in  FIG. 5 , speaker elements  140   a  and  140   b  may be disposed within corresponding cutouts of apparatus  200  after device  200  is lowered onto surface  210   e .  FIG. 6  depicts arms  210   a  and  210   c  after being manually rotated toward surface  210   e  to fixedly engage elements  130   a  and  130   b  against surface  210   e . Each of arms  210   a  and  210   c  may be coupled to standoffs  210   b  and  210   d  by a locking hinge which locks (or resists rotation in an opposite direction) in response to being rotated to a particular angle. The particular angle may be selected to result in secure fixing of device  100  to apparatus  200 . As mentioned above, the locking hinge may comprise a quick-release mechanism to selectively unlock the hinge (e.g., prior to removing device  100  from apparatus  200 ). 
     The foregoing features may thereby operate to fix device  100  in a known position with respect to apparatus  200  and to fix camera  300  in a known position with respect to apparatus  200 . Embodiments of the foregoing features may thereby create a substantially fixed relationship between an image capture device (e.g., camera  300 ) and the display screen of a display device (e.g., device  100 ). The fixed relationship may facilitate live monitoring of mixed-reality capture as described above. 
     Once device  100  is secured to apparatus  200  as shown in  FIG. 6 , lens  310  of camera  300  may be located at a position similar to the position of a left eye of a hypothetical wearer of device  100 . It is therefore assumed that a left side of display screen  110  of device  100  is configured to display virtual elements (and to allow light from actual objects to pass through) to a left eye of a wearer. Embodiments may position camera  300  at a position similar to the position of a right eye of a hypothetical wearer of device  100 , which would require corresponding changes to apparatus  200  as depicted herein. 
       FIG. 7  illustrates apparatus  200  attached to a camera support system according to some embodiments. Camera support system  700  may comprise a stabilization system, but embodiments are not limited thereto. As described below and illustrated in subsequent figures, apparatus  200  may comprise an interface compatible with standard camera support systems. Such an interface may include one or more screw holes compatible with ¼″×20 and/or ⅜″×16 screws. 
     According to some examples, an operator may operate device  100  to initiate mixed-reality capture using a camera system of device  100 . The operator may also operate camera  300  to begin capturing images and outputting the images (e.g., wirelessly) to an external display. Camera  300  may be positioned with respect to a display screen of device  100  such that the images captured by the camera system of device  100  are, after calibration, substantially identical to images output by camera  300 , although image settings, quality and/or format may differ therebetween. During mixed-reality capture, the operator may move system  700  to change the position and/or orientation of device  100  in three-dimensional space. Such movement should not negatively affect the suitability of the images output by camera  300  during mixed-reality capture, due to the secure fixing of device  100  and camera  300  to apparatus  200 . 
       FIG. 8  is a top right view which depict axes A and B around which arms  210   a  and  210   c  may respectively rotate.  FIG. 8  also shows separate elements  210   i  and  210   k , each of which supports a corresponding one of arms  210   a  and  210   c  and two protrusions  210   g . According to some embodiments, screws  210   j  and  210   l  may be loosened to slide their respective elements  210   i  and  210   k  toward and away from each other, and tightened to fix the spacing therebetween. Such adjustment changes the spacing between arms  210   a  and  210   c  and the two protrusions  210   g  of each of elements  210   i  and  210   k , thereby allowing apparatus to accommodate headsets of different widths. 
     Accessory area  210   g  includes screw holes to mount any suitable accessories, such as weights, battery packs, and lighting.  FIG. 8  also shows screw  220   e , which may be removed to allow insertion of element  320  into interface  220   d  and reinserted to fix element  320  therein. 
       FIG. 9  is a bottom right view of apparatus  200 . Shown are the underside of accessory area  210   g  and camera support interface  230 . As mentioned above, interface  230  includes screw hole  232  to accommodate a ⅜″×16 and screw hole  234  to accommodate a ¼″×20 screw. 
     Screw  220   f  may be loosened to allow rotational adjustment of element  220   a , and screw  220   g  may be loosened to allow for sliding of arm  220   c  in and out of element  220   a . Embodiments are not limited to the particular arrangement described herein for adjusting a position and an orientation of a camera with respect to other elements of apparatus  200 . 
     The figures and descriptions herein provide examples of some embodiments. Each element described herein may be composed of any number of suitable elements, and multiple elements may be implemented by a single element where suitable. Any element described herein may be composed of any one or more suitable materials, and elements are not necessarily composed of identical materials. Elements described as performing a function or providing a characteristic may be substituted with any other one or more elements to perform the function or provide the characteristic. 
     Those in the art will appreciate that various adaptations and modifications of the above-described embodiments can be configured without departing from the claims. Therefore, it is to be understood that the claims may be practiced other than as specifically described herein.