Patent Description:
Examples will now be described with reference to the accompanying drawings, in which:.

Head-mounted display (HMD) technology continues to evolve as the number of different applications and use scenarios for HMD devices (also referred to as "headsets") continue to increase. The user experience can be optimized for a given HMD application or use scenario when the HMD device being used is produced with specific characteristics that are better suited for that application or scenario. Producing a variety of HMD devices with different characteristics suitable for particular applications, however, presents manufacturers with an ongoing challenge of reducing costs and commoditizing HMD devices.

Most head-mounted display (HMD) devices are manufactured with fixed eye cup designs that can accommodate a particular type of lens and display. While lens types can vary between different HMD device manufacturers, the types of lenses often used in HMD device designs are those with focal lengths that can provide good performance and a quality user experience over a range of applications. The eye cup geometry of an HMD device (e.g., the size of the eye cup opening, and/or distances between the lens, the display, and the user's eye) can be determined and constructed based on the type of lens (i.e., the focal length) selected for use in the device. Therefore, most HMD devices come assembled with fixed lenses and displays in a "one size fits all" approach.

Furthermore, each lens, regardless of its type (i.e., focal length), comprises its own "lens profile" that defines optical characteristics such as chromatic aberrations and lens distortion (e.g., pincushion distortion). The optical characteristics can be associated with, and they can vary between, different manufacturers, makes, models, materials, and so on. Accordingly, for most HMD devices that come assembled with a fixed lens as noted above, an imaging system incorporates the lens profile of the fixed lens into an image calibration process that applies pre-distortions to images prior to rendering the images to the HMD display. The applied image pre-distortions are then neutralized as the images pass through the fixed lens so the images look correct to the user's eye.

In some HMD devices, lenses can be replaced with corrective lenses that are designed for users who wear glasses. The corrective lens replacements create image artifacts such as the chromatic aberrations and distortion noted above, which can be corrected through a proper re-calibration. However, in such HMD devices, the imaging system is unaware that a lens has been replaced, and it is unable to apply the underlying lens profile associated with the replacement corrective lens. Therefore, re-calibration to correct for these image artifacts involves a tedious, manual re-calibration using the lens profile associated with the corrective lens. The manual re-calibration is often further complicated due to there being a different corrective lens for each of the user's eyes. Alleviating the artifacts therefore involves a distinct manual calibration for each lens.

In general, as the number of applications and use cases for HMD devices continue to grow, there is an increasing desire to be able to optimize the optical characteristics of HMD lenses and displays to better accommodate the various applications and use cases. For example, a user seeking to engage in a fully immersive VR simulation may want to maximize the field of view provided by the lenses and accept lower detail and crispness of the image. Conversely, someone who wants to work with text or perform product design evaluations on small objects may want to optimize lenses to improve image crispness and detail while sacrificing some of the peripheral field of view. Optimizing experiences in these different use cases may call for using different lens types (e.g., different focal length lenses) and adjusting eye cup geometries, as well as re-calibrations of the HMD device according to specific lens profiles in order to provide images on the display that have appropriate corrective pre-distortions.

In addition to optimizing lens performance, users may also want to optimize certain display screen characteristics to enhance their experience with a given HMD application or use case. For example, in a particular application a user may want the display screen to provide brighter images, while in another application the user may want a higher contrast ratio from the display. In the first case, optimizing the user experience with brighter images may involve using an HMD device assembled with an LCD display. In the second case, optimizing the user experience with a higher contrast ratio may involve using an HMD device assembled with an OLED display.

However, as noted above most headsets or HMD devices are currently assembled with a fixed pair of lenses and a fixed display panel. There is presently limited ability for first and third-party manufacturers to provide custom lens and display solutions to HMD device consumers. Consumers are often faced with limited options that primarily include frequently refreshing their HMD devices with new generation devices that include lens and display upgrades. An unfortunate consequence of this process is that previous generation HMD devices are often entirely disposed of when next generation devices are acquired.

Accordingly, an example head-mounted display (HMD) device and methods described herein implement self-identifying, interchangeable lenses and displays to enable dynamic eye cup adjustment and device calibration. In some examples, interchangeable lens and/or displays can provide self-identification through the HMD device to enable the device's imaging system to apply the correct calibration for the lens and/or displays. In some examples, interchangeable lens and displays can be configured as interchangeable 'modules', where the modules can store and provide lens and display self-identification information. A module can comprise a structure such as a frame in which a lens and/or display is housed or integrated. A module can comprise a particular form factor that corresponds with the form factor of a module receiver or receptacle of an HMD device. Identification information stored on a module can include and/or enable the HMD device to access calibration profiles that are associated with an inserted lens and/or display so that an imaging system can correct for distortion, chromatic aberration, projection and lens centering, and other factors when displaying images on the display. The identification information also indicates a lens type (e.g., lens focal length) to enable a controller to make mechanical adjustments to the HMD eye cup shape based on the lens type.

The automatic calibration and mechanical adjustments enable users to implement different lenses having varying characteristics while maintaining a common HMD device platform. An example HMD device can include a variable lens and display receptacle that enables the insertion of lens and/or display modules containing different lenses and displays. A module can include stored identification information about an integrated lens or display, and the HMD device and/or receptacle can include a sensor to sense the identification information on the module, or the lens and display. A controller uses the identification information to determine a lens type and calibration profile, and controls mechanical adjustments to the HMD eye cup and makes distortion adjustments to images prior to displaying images on the HMD display. An actuator performs the mechanical adjustments based on geometric information from the controller, such as an applied distance between the lens at one end of the eye cup and a display panel at the opposite end of the eye cup.

In some examples, an HMD device includes a display, a variable lens receptacle to receive an interchangeable lens, a sensor to retrieve lens information from the lens, and a controller to obtain images that are pre-distorted according to the lens information, the controller to display the pre-distorted images on the display. The interchangeable lens can be housed in or integrated into a module having a form factor corresponding with the receptacle. In some examples, the lens information comprises a lens calibration profile, and obtaining the pre-distorted images comprises the controller applying image pre-distortions based on the lens calibration profile prior to displaying the pre-distorted images on the display.

A method of operating an HMD device includes receiving a lens module in the HMD device, retrieving lens information from the lens module, determining a lens type and a lens profile from the lens information, adjusting the shape of the HMD device eye cup based on the lens type, and calibrating the HMD device imaging system based on the lens profile.

In some examples, a head-mounted display (HMD) device includes a lens receptacle to receive a self-identifying, interchangeable lens module, a reading device to retrieve lens identification information stored on the lens module when the lens module is inserted into the lens receptacle, and a controller to calibrate the HMD device based on the lens identification information.

<FIG> shows a perspective view and block diagram view of an example head-mounted display (HMD) device <NUM> suitable for implementing self-identifying, interchangeable lenses and displays to enable dynamic eye cup adjustment and device calibration. The example HMD device <NUM> includes a strap <NUM> or other attachment mechanism such as a helmet that enables a user to mount the device <NUM> on the user's head. The HMD device <NUM> also includes an eye cup <NUM> and lens and display receptacles <NUM>, <NUM>, positioned near opposite ends of the eye cup <NUM>. The receptacles can comprise, for example, a frame structure with a particular form factor that enables the insertion of HMD device components having a corresponding form factor. Such components can include interchangeable lenses <NUM> and electronic displays <NUM>, or corresponding modules containing interchangeable lenses and/or electronic displays for insertion into a receptacle <NUM> or <NUM>. While lenses <NUM> and displays <NUM> are discussed and shown in the figures as being housed, packaged, or otherwise integrated within modules, other examples are possible, such as having the lenses and displays themselves manufactured with appropriate form factors for insertion into the HMD device receptacles <NUM> and <NUM>.

As shown in <FIG>, an example HMD device <NUM> can also include a controller <NUM> and a mechanical actuator <NUM>. An example controller <NUM> can include a processor (CPU) <NUM>, memory <NUM>, and other electronics (not shown) for communicating with and controlling components of the HMD device <NUM>. A memory <NUM> can include both volatile and nonvolatile memory components comprising non-transitory, machine-readable (e.g., computer/processor-readable) media that can provide for the storage of machine-readable coded program instructions, data structures, program instruction modules, and other data and/or instructions executable by a processor <NUM> of the HMD device <NUM>. Such instructions, data structures, and modules can include, for example, a lens profile <NUM>, a display profile <NUM>, an eye cup adjustment instruction module <NUM>, and an image pre-distortion instruction module <NUM>, which are discussed in more detail herein below.

The example HMD device <NUM> illustrated in <FIG> is configured as an "all-in-one" device. That is, the HMD device <NUM> shown in <FIG> not only operates as an image displaying device, but it also operates as an image processing device that processes images before they are displayed, for example, in accordance with a lens profile <NUM> and instructions from image pre-distortion module <NUM>. In some examples of an "all-in-one" HMD device <NUM>, images can be generated remotely and processed with pre-distortion on the HMD device <NUM>. In other examples of an "all-in-one" HMD device <NUM>, images can be both generated and processed with pre-distortion on the HMD device <NUM>.

In other examples however, an HMD device <NUM> may not be configured as an "all-in-one" device, but instead may be configured so that some or all of the image processing can be performed on a remote device prior to images being displayed on the HMD device <NUM>. <FIG> shows an example HMD device <NUM> that illustrates a number of different configurations in which image processing and other processing (eye cup adjustments implementing instructions from module <NUM>) can occur on a device that is remote from the HMD device <NUM>. As shown in <FIG>, in some examples an HMD device <NUM> can be coupled to a remote PC <NUM> or other processing device through a tethered or wired connection <NUM>, such as through an HDMI cable. In some examples, as shown in <FIG>, an HMD device <NUM> can be coupled to a remote PC <NUM> or other processing device through a wireless connection <NUM>. A wireless connection <NUM> can include any suitable wireless communication protocol such as Bluetooth™, ZigBee™, Z-Wave™, and the like. In some examples, as shown in <FIG>, an HMD device <NUM> can be coupled to a remote computing device such as a server <NUM> in a cloud network <NUM>. Cloud network <NUM> can represent any of a variety of network topologies and types (including optical, wired and/or wireless networks), employing any of a variety of network protocols (including public and/or proprietary protocols). Thus, a cloud network <NUM> may include, for example, a home network, a corporate network, and the Internet, as well as one or multiple local area networks (LANs) and/or wide area networks (WANs) and combinations thereof. In examples such as those shown in <FIG>, processing components including lens profile <NUM>, display profile <NUM>, eye cup adjustment instruction module <NUM>, and image pre-distortion instruction module <NUM> can be implemented on remote computing devices to provide image processing for an example HMD device <NUM> in the same or similar manner as described herein with respect to the "all-in-one" HMD device shown in <FIG>.

<FIG> shows an example of an interchangeable lens <NUM> integrated into a lens module <NUM>. <FIG> shows an example of an electronic display <NUM> integrated into a display module <NUM>. As noted above, the modules <NUM> and <NUM> comprise a form factor that facilitates their insertion and extraction with different lenses <NUM> and displays <NUM> into HMD device receptacles <NUM> and <NUM>. <FIG> show examples of HMD device receptacles <NUM> and <NUM>, respectively. Specifically, <FIG> shows an example of a lens receptacle <NUM> with a lens <NUM> and lens module <NUM> inserted into the receptacle <NUM>, and <FIG> shows an example of a display receptacle <NUM> with a display <NUM> and display module <NUM> inserted into the receptacle <NUM>.

Referring now generally to <FIG>, <FIG>, <FIG>, an interchangeable lens <NUM> can include any of a variety of different types of lenses that have associated lens calibration profiles. Therefore, an interchangeable lens <NUM> can include a lens with a different focal length and/or different optical center, a prescription lens to address problems such as astigmatism, myopia or hyperopia, a lens made from different materials including glass, plastic, and other materials, a Fresnel lens, a lens stack, and so on. In general, appropriate interchangeable lenses provided by different manufacturers can have varying levels of optical quality based on levels of refined materials, chemicals, and processes used to manufacture the lenses. A lens calibration profile <NUM> associated with such interchangeable lenses <NUM> enables the HMD device controller <NUM> (discussed below) to provide appropriate pre-distortions to images.

An interchangeable display <NUM> can include different types of display panels such as OLED display panels or LCD display panels. Displays can have different display screen characteristics such as the display resolution, the refresh rate, the contrast ratio, and the brightness level. Similar to the lens profile <NUM> discussed above, an interchangeable display <NUM> can include an associated display profile <NUM> that enables the HMD device controller <NUM> to provide appropriate image calibrations to help optimize images based on the display characteristics.

A lens module <NUM> can store lens information <NUM> that is readable by a reader <NUM> or sensor <NUM> on the HMD device <NUM> or on the lens receptacle <NUM> in the device <NUM>. Similarly, a display module <NUM> can store display information <NUM> that is readable by a reader/sensor <NUM>. Lens and display information <NUM>, <NUM> can be stored on modules <NUM>, <NUM> using various types of storage devices <NUM>, <NUM>, including, for example, an RFID tag or microchip, a QR code, an IR code, or an onboard memory accessible by electrical connections. Such electrical connections can include, for example, multiple electrical connectors to make contact with corresponding connections on a PC board or set of conductors on the lens module <NUM>. In some examples, such connections can be pre-wired, pre-routed, or otherwise configured to provide short or open electrical connections that can identify the lens without the involvement of active electronics or a power source. Thus, in different examples the readers <NUM> and <NUM> can comprise an RFID reader, a QR code reader, an IR reader, or the controller <NUM> that reads a memory <NUM>, <NUM> onboard the modules <NUM>, <NUM> when the modules are inserted into the receptacles <NUM>, <NUM>.

Lens information <NUM> can include lens identification information that identifies the lens type (e.g., lens focal length), as well as the manufacturer, model, materials, manufacturing processes, and so on. When the lens module <NUM> is inserted into the lens receptacle <NUM> of HMD device <NUM>, the controller <NUM> can access the lens information <NUM> from storage <NUM> through the reader <NUM>. Using lens information <NUM>, the controller <NUM> can access the lens focal length and other lens characteristics. Controller <NUM> can execute instructions from an eye cup adjustment instruction module <NUM> stored in memory <NUM> in order to determine an amount of mechanical adjustment to apply to the eye cup <NUM> to optimize the field of view for the inserted lens. <FIG> shows an example eye cup <NUM> formed from a flexible material <NUM> (e.g., accordion style material) to accommodate such mechanical adjustments that can expand and contract the size of the eye cup <NUM>, varying the distance <NUM> between the lens <NUM> and display <NUM>. Controller <NUM> can control mechanical actuator <NUM> to adjust the geometry of the eye cup <NUM>. In some examples, other geometries of the eye cup <NUM> can also be adjusted, such as the size of the eye cup opening.

In some examples, the controller <NUM> or other local or remote processing device (e.g., <FIG>; PC <NUM>, server <NUM>) can also match the lens information <NUM> with a lens profile <NUM>, and can use the lens profile <NUM> in an image pre-distortion calibration process (e.g., executing instructions from image pre-distortion instruction module <NUM>) to apply appropriate pre-distortions to images prior to their being displayed on the display. In some examples, a lens profile <NUM> can be stored in an onboard memory <NUM> of the lens module <NUM> and can be accessed by the controller <NUM> or remote processing device through the reader <NUM>.

Display information <NUM> can include display identification information that identifies the display type (e.g., OLED, LCD), as well as the manufacturer, model, display resolution, screen refresh rate, contrast ratio, brightness levels, and other characteristics. When a display module <NUM> is inserted into the display receptacle <NUM> of HMD device <NUM>, the controller <NUM> can access the display information <NUM> from storage <NUM> through the reader <NUM>. Using the display information <NUM>, the controller <NUM> can determine a display profile <NUM> that can be used in an image pre-distortion calibration process (e.g., executing instructions from image pre-distortion instruction module <NUM>) to apply appropriate pre-distortions to images prior to their being displayed on the display. In some examples, a display profile <NUM> can be stored in an onboard memory <NUM> of the display module <NUM> and can be accessed by the controller <NUM> through the reader <NUM>.

While example lens and display modules <NUM> and <NUM> have been discussed and shown in <FIG> as being separate modules that each comprise single interchangeable lens <NUM> and displays <NUM>, respectively, other modular configurations for interchangeable lenses and displays are possible and are contemplated herein. For example, <FIG> shows an example of a dual lens and display module <NUM> that accommodates two lenses and two display panels. In this example, the lens and displays can be in fixed relative positions. However, in other examples such a dual module <NUM> can comprise expandable and contractible components that enable the distance between the lenses and displays to be varied in a manner similar to that discussed above with regard to <FIG>.

<FIG> and <FIG> (i.e., <FIG> and <FIG>) are flow diagrams showing example methods <NUM> and <NUM> of operating a head-mounted display (HMD) device. Method <NUM> comprises extensions of method <NUM> and incorporates additional details of method <NUM>. Methods <NUM> and <NUM> are associated with examples discussed above with regard to <FIG>, <FIG>, and details of the operations shown in methods <NUM> and <NUM> can be found in the related discussion of such examples. The operations of methods <NUM> and <NUM> may be embodied as programming instructions stored on a non-transitory, machine-readable (e.g., computer/processor-readable) medium, such as memory/storage <NUM> shown in <FIG>. In some examples, implementing the operations of methods <NUM> and <NUM> can be achieved by a controller with a processor, such as a controller <NUM> with a processor <NUM> of <FIG>, reading and executing programming instructions stored in a memory <NUM>. In some examples, implementing the operations of methods <NUM> and <NUM> can be achieved using an ASIC and/or other hardware components alone or in combination with programming instructions executable by a processor <NUM>.

The methods <NUM> and <NUM> may include more than one implementation, and different implementations of methods <NUM> and <NUM> may not employ every operation presented in the respective flow diagrams of <FIG> and <FIG>. Therefore, while the operations of methods <NUM> and <NUM> are presented in a particular order within their respective flow diagrams, the order of their presentations is not intended to be a limitation as to the order in which the operations may actually be implemented, or as to whether all of the operations may be implemented. For example, one implementation of method <NUM> might be achieved through the performance of a number of initial operations, without performing other subsequent operations, while another implementation of method <NUM> might be achieved through the performance of all of the operations.

Referring now to the flow diagram of <FIG>, an example method <NUM> of operating a head-mounted display (HMD) device begins at block <NUM> with receiving a lens module in the HMD device, where the lens module comprises an interchangeable lens. The method continues with retrieving lens information from the lens module (block <NUM>), and determining a lens type and a lens profile from the lens information (block <NUM>). The method further includes adjusting an eye cup shape of the HMD device based on the lens type (block <NUM>), and calibrating images with pre-distortion based on the lens profile (block <NUM>).

Referring now to the flow diagram of <FIG> (i.e., <FIG>, <FIG>), another example method <NUM> of operating a head-mounted display (HMD) device is shown. Method <NUM> comprises extensions of method <NUM> and incorporates additional details of method <NUM>. Accordingly, method <NUM> begins at block <NUM> with receiving a lens module in the HMD device, where the lens module comprises an interchangeable lens. The method can continue with retrieving lens information from the lens module (block <NUM>), determining a lens type and a lens profile from the lens information (block <NUM>), adjusting an eye cup shape of the HMD device based on the lens type (block <NUM>), and calibrating images with pre-distortion based on the lens profile (block <NUM>).

Claim 1:
A head-mounted display (HMD) device (<NUM>) comprising:
a display (<NUM>);
a variable lens receptacle (<NUM>) to receive an interchangeable lens (<NUM>);
a sensor (<NUM>) to retrieve lens information from the lens;
a controller (<NUM>) to obtain images that are pre-distorted according to the lens information, the controller to display the pre-distorted images on the display;
characterized in that it further comprises:
an eye cup (<NUM>) comprising a flexible material (<NUM>) to enable adjustment to the eye cup shape, the eye cup comprising first and second opposing ends with the lens receptacle positioned at the first end and the display positioned at the second end;
an actuator (<NUM>) to vary the shape of the eye cup;
wherein the controller is to determine a lens type from the lens information, and to control the actuator to adjust the shape of the eye cup based on the lens type.