An information processing device according to an embodiment includes a detection unit, a determination unit, and a display control unit. The detection unit detects a wearing state of a wearer with respect to a display device that has optical transparency and projects a plurality of content images duplicated from a single content image toward the eyeballs of the wearer. The determination unit determines whether or not the mounting misalignment is occurring with the wearer on the basis of the detection result of the wearing state detected by the detection unit. In a case where the determination unit determines that the mounting misalignment is occurring, the display control unit displays a notification regarding the mounting misalignment on the display device.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International Patent Application No. PCT/JP2020/041886 filed on Nov. 10, 2020, which claims priority benefit of Japanese Patent Application No. JP 2019-209323 filed in the Japan Patent Office on Nov. 20, 2019. Each of the above-referenced applications is hereby incorporated herein by reference in its entirety.

FIELD

The present invention relates to an information processing device, an information processing method, and an information processing program.

BACKGROUND

In the related art, in head mounted displays, there are information processing devices that execute calibration of a display position in a head mounted display on the basis of a detection result in which a line of sight of a user is detected.

In such information processing devices, calibration is executed to match the motion of eyeballs of a wearer while a marker for the calibration is displayed on the head mounted display (see, for example, Patent Literature 1).

In addition, there are also display devices that cause light in a light-guiding plate to be emitted from the light-guiding plate while causing the light to be reflected and diffracted for a plurality of times by a diffraction grating member in order to widen an area (eyebox) in which a wearer can observe an image in the head mounted display (see, for example, Patent Literature 2).

CITATION LIST

Patent Literatures

SUMMARY

Technical Problem

In the prior art, a display configured to duplicate and project a content image in order to widen an eyebox allows, to some extent, a mounting misalignment in the wearer, that is, a misalignment of the display based on an optimum arrangement with respect to the eyeballs. As a result, such a head mounted display makes it difficult for the wearer to notice the mounting misalignment. For this reason, if calibration is performed in a state where there is a mounting misalignment of the head mounted display, there is a possibility that an error in the calibration increases.

The present invention has been made in view of the above, and an object of the present invention is to provide an information processing device, an information processing method, and an information processing program capable of eliminating a mounting misalignment of a display device that provides a so-called eyebox.

Solution to Problem

In order to solve the above-described disadvantage and to achieve the object, an information processing device according to an aspect of an embodiment includes a detection unit, a determination unit, and a display control unit. The detection unit detects a wearing state of the wearer with respect to a display device that has optical transparency and projects a plurality of content images duplicated from a single content image toward the eyeballs of the wearer. The determination unit determines whether or not a mounting misalignment with the wearer is occurring on the basis of a detection result of the wearing state detected by the detection unit. The display control unit displays a notification regarding the mounting misalignment on the display device in a case where the determination unit determines that a mounting misalignment is occurring.

Advantageous Effects of Invention

According to one aspect of the embodiment, even in a case where there is a mounting misalignment of a level that a wearer does not notice, the mounting misalignment of the display device can be easily eliminated.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present disclosure will be described in detail on the basis of the drawings. Note that in each of the following embodiments, the same parts are denoted by the same symbols, and redundant description will be omitted.

First, an overview of a display device according to an embodiment will be described by referring toFIGS.1to3.FIG.1is a diagram illustrating an example of the appearance of the display device according to the embodiment.FIG.2is a conceptual diagram of an eyebox.FIG.3is a schematic diagram of a mounting misalignment.

In the example illustrated inFIG.1, a display device10delivers augment reality (AR) in which a content image such as a virtual object is superimposed on the real world and thereby represented to a user. Note that, hereinafter, the display device10may also be referred to as a head mounted display (HMD). Furthermore, the content image may be a still image or a moving image.

As illustrated inFIG.1, the display device10includes display units11and is disposed at a position in front of both eyes of a user (hereinafter referred to as a wearer) so that the display units11are positioned in front of the wearer when worn by the wearer. Note that the display device10is not limited to a binocular type and may be a monocular type.

Furthermore, as illustrated inFIG.2, the display device10is a so-called optical see-through type display device that projects the duplicated content image toward the eyeballs of the wearer and transmits light of the real world via the display units11.

In such a display device10, three-dimensional eyeboxes I are defined between the eyes of the wearer and the display units11. Here, an eyebox I is an area in which an image (content image) is projected from a display unit11and in which the wearer can view the content image.

A display unit11that forms an eyebox I includes, for example, a light-guiding plate and a diffraction grating. The light-guiding plate is structured so as to propagate the content image output from a projector (light source) by total reflection. The diffraction gratings are provided on the light-guiding plates and are structured so as to project light propagated to the eyeballs of the user from a plurality of positions in the light-guiding plates in a substantially parallel manner. That is, a single content image output from the light source is duplicated as a plurality of content images and projected to the eyeballs of the user from a plurality of positions in the light-guiding plates. Note that the display device10of the present disclosure is not limited to this configuration and is only required to include a display unit that widens the area in which the content image can be observed by the wearer by projecting the plurality of duplicated content images.

For example, as illustrated inFIG.3, in a display device10, a virtual camera V is set at the center of an eyebox I, and the content image is projected from the display unit11using the virtual camera V as a reference, thereby providing the content image to the wearer. Note that the virtual camera V here indicates the optimum position of an eyeball of the wearer, and in the display device10, the content image is projected from a display unit11using the virtual camera V as the reference.

At this point, if the eyes of the wearer are within the eyeboxes I, the content image of the same angle of view can be viewed. Therefore, in the optical see-through type display device, even if a mounting misalignment of the HMD occurs, if the eyes of the wearer are in the eyeboxes I, although the display positions of the content image are shifted, the wearer is less likely to notice the mounting misalignment since the wearer can see the content image of the same angle of view.

Note that, in a case where a part of an eyeball of the wearer is deviates from the eyebox I, the content image is projected on the eyeball of the wearer with a missing part, but even in this case, it is not easy for the wearer to recognize the mounting misalignment.

For such a problem, the information processing device according to the embodiment detects the mounting misalignment and displays a notification regarding the mounting misalignment on the display units11in a case where the mounting misalignment is detected. That is, in the information processing device according to the embodiment, a mounting misalignment is detected, and in a case where the mounting misalignment is detected, the wearer is prompted to re-mount the display device10.

Hereinafter, the information processing device, an information processing method, and an information processing program according to an embodiment will be described in detail.

Next, a configuration example of the information processing device1according to the embodiment will be described by referring toFIG.4.FIG.4is a block diagram of the information processing device according to the embodiment. Note that the display device10is also illustrated inFIG.4. Furthermore, the information processing device1and the display device10can transmit and receive data bidirectionally in a wireless or wired manner.

First, the display device10will be described. As illustrated inFIG.4, the display device10includes display units11, a gyrosensor12, a camera13, a ranging sensor14, and a speaker15. A display unit11includes, for example, a half mirror or the like, has a light transmission type display surface (lens), and displays a content image input from the information processing device1. For example, the display unit11projects a content image displayed on a display (not illustrated) toward the eyeballs of the wearer by repeating total reflection in the lens.

The gyrosensor12detects angular velocities of three axes for detecting the motion of the display device10. As described above, since the display device10is an HMD, the gyrosensor12detects a change in the posture of the wearer of the display device10and outputs a posture signal corresponding to the detected change in the posture to the information processing device1.

The camera13includes an image sensor and captures an image ahead of the display device10. For example, the camera13has an angle of view that enables imaging the motion of the hands of the wearer and outputs the captured camera image to the information processing device1.

The ranging sensor14is an example of a sensor that performs sensing of the surrounding environment of the display unit11and is, for example, a time-of-flight (ToF) sensor. The ranging sensor14mainly measures the distance to the hands of the wearer. Note that, instead of the ranging sensor14, the image sensor of the camera13may be regarded as a sensor that performs sensing of the surrounding environment. That is, in a case where the distance to a gesture operation of the wearer can be measured by image analysis, the function of the ranging sensor14may be performed by the image sensor.

The speaker15is an example of an audio output unit and outputs audio corresponding to the content image. Furthermore, in a case where the information processing device1determines that there is a mounting misalignment of the HMD, the speaker15can also output a voice for notifying the mounting misalignment. Note that earphones connected in a wired or wireless manner may be used as the audio output unit.

Next, the information processing device1will be described. As illustrated inFIG.4, the information processing device1includes a storage unit2and a control unit3. Note that the information processing device1may include a communication unit (not illustrated) for performing wireless or wired communication with an external device and an operation unit (not illustrated) for receiving an operation by a user.

The storage unit2is implemented by, for example, a semiconductor memory element such as a RAM or a flash memory or a storage device such as a hard disk or an optical disk. In the example illustrated inFIG.4, the storage unit2stores default information20and threshold value information21.

The default information20includes, for example, information related to a correction coefficient based on a misalignment between content coordinates that are coordinates in the real space indicated by the display position of the content image displayed on the display units11and gesture coordinates indicated by a gesture operation of the wearer, information indicating a relationship between the content coordinates and the gesture coordinates, and other information. The default information20is registered by the control unit3at the time of setting defaults or every time a wearer wears the HMD.

The threshold value information21is information related to a threshold value at the time of detecting a mounting misalignment. Information regarding threshold values related to the distance or angle is stored in the storage unit2as the threshold value information21.

The control unit3is implemented by, for example, a central processing unit (CPU), a micro processing unit (MPU), or the like executing a program stored inside the information processing device1using a random access memory (RAM) or the like as a work area. The control unit3is also a controller and may be implemented by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

As illustrated inFIG.4, the control unit3includes an acquisition unit30, a detection unit31, a measurement unit32, a correction unit33, a determination unit34, and a display control unit35and implements or executes the functions or actions of information processing described below. Note that the internal configuration of the control unit3is not limited to the configuration illustrated inFIG.4and may be another configuration as long as information processing described below is performed. Note that the control unit3may be connected to a predetermined network in a wired or wireless manner using, for example, a network interface card (NIC) or the like and receive various types of information from an external server or the like via the network.

The acquisition unit30acquires various types of information input from the display device10. Specifically, the acquisition unit30acquires a posture signal from the gyrosensor12and acquires a camera image from the camera13.

The detection unit31detects a wearing state of the wearer with respect to the display device that has optical transparency and projects a plurality of content images duplicated from a single content image toward the eyeballs of the wearer. The detection unit31detects a positional relationship between the eyeballs of the wearer and the eyeboxes I as the wearing state.

Specifically, the detection unit31detects a gesture operation on the content image by the wearer on the basis of the camera image captured by the camera13and detects the wearing state on the basis of a gesture operation that has been detected.

For example, the detection unit31detects a hand of the wearer by performing predetermined image analysis on the camera image and detects a gesture operation by tracking the motion of the hand. Note that examples of the gesture operation include various operations such as an operation of touching the content image and an operation of grasping the content image. At this point, the detection unit31may set only the gesture operation associated with the content image being displayed on the display units11as a detection target of the gesture operation.

When detecting the gesture operation, the detection unit31requests the measurement unit32to measure gesture coordinates, which are three-dimensional coordinates of the gesture operation. As a result, the measurement unit32described later measures the gesture coordinates and notifies the detection unit31of the gesture coordinates that have been measured.

Then, the detection unit31detects the wearing state on the basis of the gesture coordinates. That is, the detection unit31estimates the three-dimensional position of the content image visible from the wearer on the basis of the gesture coordinates and also detects the positional relationship between the eyeballs of the wearer and the eyeboxes I as the wearing state on the basis of the three-dimensional position that has been estimated. The information regarding the wearing state detected by the detection unit31is notified to the determination unit34.

The measurement unit32measures gesture coordinates, which are three-dimensional coordinates of the gesture operation, based on sensor information obtained by measuring the surrounding environment of the display units11. Specifically, for example, in a case where the detection unit31detects the gesture operation on the content image, the measurement unit32measures the distance to the hand of the wearer performing the gesture operation on the basis of the detection result of the ranging sensor14.

Thereafter, the measurement unit32measures gesture coordinates that are three-dimensional coordinates of the gesture operation by using the installation position of the display device10(the camera13or the ranging sensor14) as reference coordinates. Such a three-dimensional coordinate system is a coordinate system that changes depending on the attitude or the position of the display device10and may also be referred to as a local coordinate system. Note that the measurement unit32may measure the gesture coordinates by performing predetermined image analysis on the camera image captured by the camera13.

At the time of default setting, the correction unit33corrects the display position of the content image displayed on the display units11with respect to the gesture coordinates. Note that the default setting herein refers to the first time of mounting the HMD by the wearer; however, it may be set to mean every time of mounting the HMD by the wearer.

Specifically, at the time of default setting, a content image related to the user interface (UI) for default setting is displayed on the display units11, and the correction unit33corrects the display position of the content image on the basis of gesture coordinates of a gesture operation for the UI.

That is, the correction unit33can appropriately display the content image for individual wearers by correcting misalignment of the display position or the like of the content image caused by individual differences of the wearers.

FIGS.5and6are conceptual diagrams of a correction process by the correction unit33according to the embodiment. Note that, inFIG.5, a content image actually viewed by the wearer via the display units11is illustrated as real content Cr, and a content image displayed on the display units11with reference to the virtual cameras V illustrated inFIG.2is illustrated as display content Cv.

In a state where the display content Cv is displayed, the information processing device1prompts the wearer to perform an operation such as touching the display content Cv. At this point, in a case where the display content Cv and the real content Cr substantially coincide with each other, that is, in a case where the display content Cv and the coordinates of the gesture operation substantially coincide with each other, the process by the correction unit33is unnecessary.

Furthermore, as illustrated inFIG.5, in a case where there is a misalignment between the display content Cv and the real content Cr, the wearer performs a gesture operation of touching the real content Cr which is away from the display content Cv.

At this point, the correction unit33estimates the three-dimensional coordinates of the real content Cr on the basis of the gesture coordinates of the gesture operation and calculates the misalignment between the real content Cr and the display content Cv on the basis of the three-dimensional coordinates that have been estimated and the content coordinates which are the three-dimensional coordinates of the display content Cv.

For example, the correction unit33calculates a correction coefficient for matching the display content Cv with the real content Cr on the basis of the misalignment that has been calculated and stores the correction coefficient in the storage unit2as the default information20. Thereafter, the display control unit35described later updates the display position of the display content Cv on the basis of the correction coefficient and corrects the display position of the display content Cv by performing the above process until the display content Cv and the real content Cr substantially coincide with each other.

Then, in subsequent processes, the information processing device1can display the content image at a display position suitable for the wearer by correcting the display position of the content image using the correction coefficient that has been finally determined.

Note that, although one piece of display content Cv and one corresponding piece of real content Cr are illustrated inFIG.5for convenience of description, the correction unit33may calculate the correction coefficient on the basis of a plurality of pieces of display content Cv.

In this case, it is preferable that the correction unit33calculate the correction coefficient on the basis of a plurality of pieces of display content Cv displayed at different display positions and corresponding real content Cr.

Furthermore, as illustrated inFIG.6, in a case where the misalignment between the display content Cv and the real content Cr includes a rotation component, the correction unit33can also calculate a correction coefficient corresponding to the rotation component.

Furthermore, the correction unit33sets a threshold value on the basis of the display content Cv and the real content Cr. The threshold value here is used for determination of the mounting misalignment and is information stored in the storage unit2as the threshold value information21.

For example, the correction unit33sets the threshold value on the basis of an error between the display content Cv and the real content Cr after the display position of the display content Cv has been corrected to coincide with the real content Cr.

For example, the correction unit33sets the threshold value higher as the error is larger and sets the threshold value lower as the error is smaller. In this case, by setting the threshold value higher as the error between the display content Cv and the real content Cr is larger, the allowable range for a mounting misalignment is widened.

Note that the correction unit33may set the threshold value higher as the misalignment between the display content Cv displayed first before correction and the real content Cr is larger and may set the threshold value lower when the misalignment between the display content Cv and the real content Cr is small. In this case, the threshold value is set lower as the eyeballs of the wearer is closer to the virtual cameras V, and the threshold value is set higher as the eyeballs of the wearer is farther from the virtual cameras V.

Note that the above method of setting the threshold value is an example, and the threshold value may be set by other methods. In this case, for example, a display unit11may be divided into a plurality of display areas, and a threshold value may be set for each display area.

Returning to the description ofFIG.4, the determination unit34will be described. The determination unit34determines whether or not a mounting misalignment with the wearer is occurring on the basis of the wearing state detected by the detection unit31. The determination unit34determines whether or not a mounting misalignment is occurring on the basis of the content coordinates based on the display content Cv and the gesture coordinates based on the real content Cr.

Here, a specific example of a process performed by the determination unit34will be described with reference toFIG.7.FIG.7is a diagram illustrating an example of a process performed by the determination unit34according to the embodiment. As illustrated inFIG.7, the determination unit34determines whether or not a mounting misalignment is occurring on the basis of an error distance d which is a distance between the content coordinates and the gesture coordinates.

More specifically, the determination unit34compares the error distance d with the threshold value stored in the storage unit2as the threshold value information21, determines that no mounting misalignment is occurring when the error distance d is less than or equal to the threshold value, and determines that a mounting misalignment is occurring when the error distance d exceeds the threshold value.

At this point, the determination unit34determines whether or not the mounting misalignment is occurring on the basis of a misalignment of the rotation component between the real content Cr and the display content Cv in addition to the error distance d. That is, even in a case where the error distance d is less than or equal to the threshold value, the determination unit34determines that a mounting misalignment is occurring when the misalignment of the rotation component is larger than a predetermined value.

That is, the determination unit34determines that a mounting misalignment is occurring when the gesture operation of the wearer with respect to the display content Cv deviates from the display content Cv by more than a predetermined value. Here, since the gesture operation by the wearer is performed on the basis of the real content Cr, a misalignment between the real content Cr estimated from the gesture operation and the display content Cv substantially indicates a misalignment of the viewpoint position of the wearer with respect to the virtual cameras V.

It is conceivable that the main cause of the misalignment of the viewpoint position is based on the mounting misalignment of the HMD. Therefore, the information processing device1can estimate the positional relationship between the eyeballs of the wearer and the eyeboxes I by estimating the three-dimensional coordinates of the real content Cr in the real space on the basis of the gesture coordinates and comparing the three-dimensional coordinates that have been estimated with the content coordinates.

Then, in a case where the misalignment between the real content Cr and the display content Cv exceeds the threshold value, it can be determined that the relative position between the eyeballs and the eyeboxes I is deviated from the initial position, and thus, it can be determined that a mounting misalignment is occurring. As described above, the information processing device1can determine a mounting misalignment of the HMD by estimating the real content Cr on the basis of the gesture operation.

Furthermore, in the information processing device1, the series of processes can be performed on the basis of a simple gesture operation such as the wearer touching the real content Cr, and thus a mounting misalignment can be easily determined every time an operation object is displayed.

In other words, the information processing device1can determine a mounting misalignment as needed at the timing when an operation object (UI) accompanied by a gesture operation is displayed as a content image. That is, the information processing device1can determine a mounting misalignment without interrupting the AR experience being provided to the wearer.

Moreover, in a case where it is determined that a mounting misalignment is occurring, the determination unit34notifies the display control unit35and causes the speaker15to output predetermined voice. At this point, the voice output from the speaker15includes guidance voice that prompts the user to remount the HMD, a warning sound indicating the mounting misalignment, or the like.

Returning to the description ofFIG.4, the display control unit35will be described. The display control unit35controls the content image to be displayed on the display devices10by drawing the content image and outputting an image signal related to the content image to the display devices10.

Here, the content image is not particularly limited and may be a two-dimensional image or a three-dimensional image. Furthermore, the content image may be a still image or a moving image. In addition, the content image includes those in which the display position does not change depending on the posture of the wearer and those in which the display position changes following the posture of the wearer.

At this point, in a case where a display position or a changing object is displayed as a content image following the posture of the wearer, the display control unit35draws the object so as to match the change in the current position or the posture of the wearer.

In addition, when the determination unit34determines that a mounting misalignment is occurring, the display control unit35displays a notification regarding the mounting misalignment on the display units11. Here, a specific example of the notification regarding the mounting misalignment will be described with reference toFIGS.8to10.

FIGS.8to10are diagrams illustrating an example of notification regarding the mounting misalignment. Note that the example illustrated inFIG.8is an example of a warning image. For example, in the example illustrated inFIG.8, a text such as “Please re-mount the HMD” and an image prompting to remount the HMD are displayed on the display units11.

The wearer can easily recognize that the HMD is in the state of a mounting misalignment on the basis of the notification. In addition, the example illustrated inFIG.9illustrates a case where an axis image indicating the current amount of misalignment from a reference axis is displayed on the display units11in addition to a text.

Note that the reference axis here is an axis connecting the virtual cameras V corresponding to both eyes and indicates a target inclination. The axis image is an axis connecting the current positions of the both eyes and indicates the current inclination. In this case, the wearer can know, at a glance, the current degree of a mounting misalignment and how to remount the HMD by visually recognizing the axis image, and thus, the wearer can easily remount the HMD.

In addition, the example illustrated inFIG.10illustrates a case where the content image is blurred and thereby displayed on the display units11in a case where it is determined that a mounting misalignment is occurring. In this case, it is assumed that the wearer recognizes that defocusing has occurred.

For this reason, an operation of remounting the HMD in order to correct the defocus by the wearer can be expected. That is, in this case, by blurring the content image, it is possible to indirectly prompt the wearer to remount the HMD.

Therefore, in this case, it is possible to prompt the wearer to remount the HMD in a natural manner without interrupting the display of the content image that is being displayed. Note that the display control unit35may display the blurred image by replacing the image with an image that has been blurred by an external device or the like in a case where it is determined that a mounting misalignment is occurring.

Furthermore, in this case, the display control unit35may change the intensity of the blurring process depending on the degree of the mounting misalignment. In other words, the display mode of the content image may be changed depending on the degree of the mounting misalignment.

Specifically, for example, as the degree of a mounting misalignment is larger, the intensity of the blurring process is increased, thereby strongly prompting the wearer to remount the HMD. In addition, at this point, in a case where it is determined that a mounting misalignment is occurring, the intensity of the blurring process may be gradually increased with the lapse of time. That is, the intensity of the blurring process may be gradually increased after it is determined that a mounting misalignment is occurring, thereby strongly prompting to remount the HMD with the lapse of time. Furthermore, the display mode in a case where there is a mounting misalignment of the HMD is not limited to the above-described example, and the display position may be corrected by the correction unit31.

A processing procedure executed by the information processing device1according to the embodiment will be described by referring toFIGS.11and12.FIGS.11and12are flowcharts illustrating a processing procedure executed by the information processing device1according to the embodiment. Note that the processing procedure illustrated below is executed by the control unit3.

First, a processing procedure performed by the information processing device1at the time of default setting will be described with reference toFIG.11. As illustrated inFIG.11, after displaying an object for default setting (step S101), the information processing device1acquires a sensing result of the surrounding environment (step S102).

Subsequently, the information processing device1detects a gesture operation of the wearer with respect to the object displayed in step S101on the basis of the sensing result acquired in step S102(step S103).

Subsequently, the information processing device1measures gesture coordinates on the basis of the gesture operation detected in step S103(step S104) and estimates three-dimensional coordinates of the real content on the basis of the gesture coordinates that have been measured (step S105).

Thereafter, the information processing device1determines whether or not a misalignment between the three-dimensional coordinates of the real content and the content coordinates indicated by the display content is less than or equal to a predetermined value (step S106), and if the misalignment is less than or equal to the predetermined value (step S106, Yes), the information processing device1registers the default information20indicating a relationship between the three-dimensional coordinates of the real content and the content coordinates of the display content (step S107) and ends the process.

Furthermore, if the misalignment exceeds the predetermined value in the determination of step S106(step S106, No), the information processing device1updates the display position of the display content (step S108) and proceeds to the process of step S102again.

Next, a series of processing procedures after the default setting by the information processing device1will be described with reference toFIG.12. As illustrated inFIG.12, the information processing device1determines whether or not it is the timing to display the operation object (operation UI) as the content image (step S201), and if it is the timing to display the operation object (step S202), the information processing device1displays the operation object (step S202).

Thereafter, the information processing device1detects a gesture operation with respect to the operation object (step S203) and calculates gesture coordinates corresponding to the gesture operation (step S204).

Subsequently, the information processing device1estimates three-dimensional coordinates of the real object on the basis of the gesture coordinates (step S205) and determines whether or not the distance between the three-dimensional position of the real object and the object coordinates indicated by the display object, which is the operation object, is larger than a threshold value (step S206).

If the distance exceeds the threshold value in the determination of step S208(step S206), the information processing device1determines that a mounting misalignment is occurring and displays a notification regarding the mounting misalignment (step S207). Note that the process of step S207is continued, for example, until the wearer remounts the HMD.

Alternatively, if it is determined in step S201that it is not the timing to display the operation object, or if where it is determined in step S206that the distance is less than the threshold value (step S201/step S206, No), the information processing device1ends the process. Note that to end the process here means to continuously display the content image that is being displayed.

In the above-described embodiment, the case where the positions of the eyeballs are indirectly detected on the basis of a gesture operation to determine a mounting misalignment has been described; however, the present invention is not limited thereto. For example, the information processing device1may determine a mounting misalignment directly on the basis of a detection result of a sensor that detects the positions of the eyeballs.

Furthermore, in the above-described embodiment, a case where a mounting misalignment is notified by an image or audio has been described; however, for example, the mounting misalignment may be notified by vibrating the HMD. Furthermore, in the above-described embodiment, the description has been made on the premise that the wearer performs the gesture operation using a hand; however for example, other parts such as a foot can be applied.

An information device such as the information processing devices according to the embodiments described above is implemented by, for example, a computer1000having a configuration as illustrated inFIG.13. Hereinafter, the information processing device1according to the embodiment will be described as an example.FIG.13is a hardware configuration diagram illustrating an example of the computer1000that implements the functions of the information processing device1. The computer1000includes a CPU1100, a RAM1200, a read only memory (ROM)1300, a hard disk drive (HDD)1400, a communication interface1500, and an input and output interface1600. The units of the computer1000are connected by a bus1050.

The CPU1100operates in accordance with a program stored in the ROM1300or the HDD1400and controls each of the units. For example, the CPU1100loads a program stored in the ROM1300or the HDD1400in the RAM1200and executes processes corresponding to various programs.

The ROM1300stores a boot program such as a basic input output system (BIOS) executed by the CPU1100when the computer1000is activated, a program dependent on the hardware of the computer1000, and the like.

The HDD1400is a computer-readable recording medium that non-transiently records a program to be executed by the CPU1100, data used by such a program, and the like. Specifically, the HDD1400is a recording medium that records a program according to the present disclosure, which is an example of program data1450.

The communication interface1500is an interface for the computer1000to be connected with an external network1550(for example, the Internet). For example, the CPU1100receives data from another device or transmits data generated by the CPU1100to another device via the communication interface1500.

The input and output interface1600is an interface for connecting an input and output device1650and the computer1000. For example, the CPU1100receives data from an input device such as a keyboard or a mouse via the input and output interface1600. In addition, the CPU1100transmits data to an output device such as a display, a speaker, or a printer via the input and output interface1600. Furthermore, the input and output interface1600may function as a media interface that reads a program or the like recorded in a predetermined recording medium (medium). A medium refers to, for example, an optical recording medium such as a digital versatile disc (DVD) or a phase change rewritable disk (PD), a magneto-optical recording medium such as a magneto-optical disk (MO), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, in a case where the computer1000functions as the information processing device1according to the embodiment, the CPU1100of the computer1000implements the function of the acquisition unit30by executing a program loaded on the RAM1200. The HDD1400also stores a program according to the present disclosure or data in the storage unit2. Note that although the CPU1100reads the program data1450from the HDD1400and executes the program data1450, as another example, these programs may be acquired from another device via the external network1550.

An information processing device comprising:a detection unit that detects a wearing state of a wearer with respect to a display device that has optical transparency and projects a plurality of content images duplicated from a single content image toward eyeballs of the wearer;a determination unit that determines whether or not a mounting misalignment with the wearer is occurring on a basis of a detection result of the wearing state detected by the detection unit; anda display control unit that displays a notification regarding the mounting misalignment on the display device in a case where the determination unit determines that the mounting misalignment is occurring.
(2)

The information processing device according to (1),wherein the determination unit determines whether or not the mounting misalignment is occurring on a basis of eyeboxes and three-dimensional positions of the eyeballs based on the wearing state, the eyeboxes being areas in which the plurality of content images that has been duplicated is projected and defined between the eyeballs of the wearer and the display device.
(3)

The information processing device according to (2),wherein the detection unit detects a positional relationship between the eyeballs of the wearer and the eyeboxes as the wearing state by detecting a gesture operation of the wearer with respect to the content images viewed by the wearer.
(4)

The information processing device according to (3), further comprising:a measurement unit that measures gesture coordinates indicating three-dimensional coordinates of the gesture operation on a basis of a sensing result of a sensor that performs sensing of a surrounding environment, the sensor provided at the display device,wherein the detection unit detects the wearing state on a basis of the gesture coordinates measured by the measurement unit.
(5)

The information processing device according to (4), wherein the determination unit determines that the mounting misalignment is occurring in a case where a distance between the gesture coordinates and content coordinates that are three-dimensional coordinates in the real world indicated by the content images displayed on the display device exceeds a threshold value.

The information processing device according to (5),wherein the determination unit determines whether or not the mounting misalignment is occurring on a basis of three-dimensional positions of the content images estimated from the gesture coordinates and the content coordinates.
(7)

The information processing device according to (4) or (5), further comprising:a correction unit that corrects display positions of the content images with respect to the gesture operation at a time of default setting,wherein the determination unit determines that the mounting misalignment is occurring in a case where a relationship between the gesture operation and the display positions of the content images changes by more than a predetermined value after correction.
(8)

The information processing device according to (7),wherein the correction unit sets the threshold value on a basis of the gesture operation at the time of the default setting or after the default setting and the display positions of the content image.
(9)

The information processing device according to (7) or (8),wherein the correction unit corrects the display positions of the content image in a case where it is determined that the mounting misalignment is occurring.
(10)

The information processing device according to any one of (6) to (9),wherein the display control unit displays an operation object for gesture operation on the display device as the content images, andthe detection unit detects the wearing state on a basis of the content coordinates of the operation object.
(11)

The information processing device according to (10),wherein the detection unit detects the wearing state every time the operation object is displayed, andthe determination unit determines whether or not the mounting misalignment is occurring every time the detection unit detects the wearing state.
(12)

The information processing device according to any one of (1) to (11),wherein the display control unit blurs and displays the content images as a notification regarding the mounting misalignment.
(13)

The information processing device according to (12),wherein the display control unit displays the content images having a stronger level of blurring as a degree of the mounting misalignment increases.
(14)

The information processing device according to any one of (1) to (13),wherein the display control unit displays an axis image indicating a misalignment from a reference axis caused by the mounting misalignment as a notification regarding the mounting misalignment.
(15)

The information processing device according to any one of (1) to (14),wherein the display control unit displays a warning image as a notification regarding the mounting misalignment.
(16)

The information processing device according to any one of (1) to (15),wherein the determination unit outputs predetermined sound in a case where it is determined that the mounting misalignment is occurring.
(17)

An information processing method comprising the steps of:by a computer,detecting a wearing state of a wearer with respect to a display device that has optical transparency and projects a plurality of content images duplicated from a single content image toward eyeballs of the wearer;determining whether or not a mounting misalignment with the wearer is occurring on a basis of a detection result of the wearing state; anddisplaying a notification regarding the mounting misalignment on the display device in a case where it is determined that the mounting misalignment is occurring.
(18)

An information processing program for causing a computer to function as:a detection unit that detects a wearing state of a wearer with respect to a display device that has optical transparency and projects a plurality of content images duplicated from a single content image toward eyeballs of the wearer;a determination unit that determines whether or not a mounting misalignment with the wearer is occurring on a basis of a detection result of the wearing state detected by the detection unit; anda display control unit that displays a notification regarding the mounting misalignment on the display device in a case where the determination unit determines that the mounting misalignment is occurring.

REFERENCE SIGNS LIST