Patent Description:
A display device which is worn on the head and is used to view video, i.e., a head-mounted display (HMD), is well known (see, for example, Patent Literature <NUM>, etc.). A head-mounted display has an optical unit for each of the left and right eyes, and is also used in combination with headphones, and is thereby configured to be capable of controlling visual sensation and auditory sensation. There have recently been advances in techniques for a so-called see-through head-mounted display, which allows the user wearing the display to see a real scene at which the user's line of sight is aimed.

From <CIT>, it is known that a see-through HMD reacts to sensor and other inputs to adjust how visual elements showing vide are displayed. From <CIT>, a head-mounted display provided with display units for displaying an image is known. It has a switching unit for switching between a display state for displaying an image on the display units and a non-display state for not displaying an image on the display units, brightness adjusting units for adjusting the brightness of the image displayed on the display units, and an external light detector for detecting the amount of external light. Moreover, it is known to adjust a display of information, based on, for example, a user's context (see, e.g., <CIT>) or based on a situation of a user (see, e.g., <CIT>). From <CIT> a head-mounted display and a control method for reducing motion sickness when using such a display in response to an external stimulus are known.

A see-through head-mounted display continues to be worn by the user during the use of the head-mounted display, so that the display is present in front of the user's eyes. Therefore, notification information which notifies the user of the arrival of a mail, call, or the like may be displayed on the display at a timing when the user is not expecting it. If notification information is suddenly displayed on the display, the user's field of vision is partially blocked by the notification information, which is dangerous.

Also, if notification information is caused to flicker or move during the use of a see-through head-mounted display in order to attract the user's attention, the user's attention is attracted by the flickering or movement, which is dangerous. In addition, if any information continues to be displayed in a portion of the field of view, the user's field of vision is partially blocked by the information, which is dangerous.

With the above in mind, the present disclosure provides a novel and improved display control device, display control method, and computer program which are capable of ensuring the user's field of vision while keeping the user safe when the user is using a see-through head-mounted display.

According to a first aspect of the present invention, there is provided a display control device in accordance with independent claim <NUM>. According to a second aspect, the invention provides a display control method in accordance with independent claim <NUM>. According to a third aspect, the invention provides a computer readable medium in accordance with independent claim <NUM>. Further aspects of the invention are set forth in the dependent claims, the drawings and the following description.

As described above, according to the present disclosure, a novel and improved display control device, display control method, and computer program which are capable of ensuring the user's field of vision while keeping the user safe when the user is using a see-through head-mounted display, can be provided.

Note that the effects described above are not necessarily limited, and along with or instead of the effects, any effect that is desired to be introduced in the present specification or other effects that can be expected from the present specification may be exhibited.

In this specification and the appended drawings, structural elements that have substantially the same function and structure are denoted with the same reference numerals, and repeated explanation of these structural elements is omitted.

Note that description will be provided in the following order.

Firstly, an external appearance example of a head-mounted display according to an embodiment of the present disclosure will be described. <FIG> is an illustrative diagram showing an external appearance example of a head-mounted display <NUM> according to an embodiment of the present disclosure.

As shown in <FIG>, the head-mounted display <NUM> is an eyeglasses display device provided with a display unit <NUM> including a right-eye display unit 110R and a left-eye display unit <NUM>. The head-mounted display <NUM> is a so-called see-through head-mounted display, which allows the user to view a screen superimposed on a real-world field of view.

The head-mounted display <NUM> is configured so that nose pads <NUM> serve as a sensor for determining whether the display is worn, by detecting the touch of the user's (viewer's) nose tip on the nose pads.

The display unit <NUM> may include, for example, an organic EL display. The display unit <NUM> is controlled so that the display of the display unit <NUM> is switched between a mode in which an image is not displayed (see-through mode), a mode in which an image is displayed and superimposed on the user's read-world field of view (image display mode), and a mode in which an image is displayed on only one of the two display units (single-eye display mode).

The head-mounted display <NUM> receives an image signal, by wire or wirelessly, from an external device (a personal computer, a smartphone, a tablet-type mobile terminal, etc.), and displays an image on the display unit <NUM> on the basis of the received image signal.

In the foregoing, an external appearance example of the head-mounted display <NUM> according to an embodiment of the present disclosure has been described with reference to <FIG>. Next, another external appearance example of the head-mounted display will be described. <FIG> is an illustrative diagram showing an external appearance example of a head-mounted display <NUM>' according to an embodiment of the present disclosure.

As shown in <FIG>, the head-mounted display <NUM>' is an eyeglasses display device which includes a mounting unit having a frame structure which extends across half the circumference of the head from one of both the sides of the head to the other through the back of the head. The head-mounted display <NUM>' is provided with a display unit <NUM>' including a right-eye the display unit 110R' and a left-eye the display unit <NUM>'.

The head-mounted display <NUM> shown in <FIG> and the head-mounted display <NUM>' shown in <FIG> can allow the user to view a screen superimposed on the read-world field of view. However, as described above, the see-through head-mounted display continues to be worn by the user during the use of the head-mounted display, so that the display is present in front of the user's eyes. Therefore, notification information which notifies the user of the arrival of a mail, call, or the like may be displayed on the display at a timing when the user is not expecting it. If notification information is suddenly displayed on the display, the user's field of vision is partially blocked by the notification information, which is dangerous.

Also, if notification information is caused to flicker or move during the use of the see-through head-mounted display in order to attract the user's attention, the user's attention is attracted by the flickering or movement, which is dangerous. In addition, if any information continues to be displayed in a portion of the field of view, the user's field of vision is partially blocked by the information, which is dangerous.

With the above in mind, in an embodiment of the present disclosure, a head-mounted display for ensuring the user's field of vision while keeping the user safe will be described.

Next, a functional configuration example of a head-mounted display according to an embodiment of the present disclosure will be described. <FIG> is an illustrative diagram showing a functional configuration example of a head-mounted display <NUM> according to an embodiment of the present disclosure. The functional configuration example of the head-mounted display <NUM> according to an embodiment of the present disclosure will now be described with reference to <FIG>.

As shown in <FIG>, the head-mounted display <NUM> according to an embodiment of the present disclosure includes a sensor unit <NUM>, a system controller <NUM>, a display unit <NUM>, a sensor signal processing unit <NUM>, a sensor control unit <NUM>, an image capture unit <NUM>, a captured image signal processing unit <NUM>, an image capture control unit <NUM>, an audio input unit <NUM>, an audio output unit <NUM>, an audio signal processing unit <NUM>, an audio generation unit <NUM>, an image generation unit <NUM>, a displayed image processing unit <NUM>, a display control unit <NUM>, a display drive unit <NUM>, a communication unit <NUM>, and a storage unit <NUM>.

The system controller <NUM>, which includes, for example, a microcomputer including a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a non-volatile memory unit, and an interface unit, may function as a control unit which controls the entirety of this embodiment. The system controller <NUM> controls each unit according to an internal operating program so that a required image is displayed on the display unit <NUM>. The system controller <NUM> may also function as an example of a situation acquisition unit of the present disclosure. Specifically, the system controller <NUM> acquires information about a situation where the head-mounted display <NUM> is being used, on the basis of details of signals sent from the sensor signal processing unit <NUM>, the captured image signal processing unit <NUM>, and the audio signal processing unit <NUM> (described below).

The sensor unit <NUM> is a sensor which is typically mounted in the vicinity of the display unit <NUM>. The sensor unit <NUM>, which includes, for example, an acceleration sensor, a gyroscopic sensor, a magnetic sensor, an atmospheric pressure sensor, or the like, can detect, for example, a motion of the head, a motion of the neck, a motion of the head-mounted display <NUM>, or the like, as a signal corresponding to a motion of the user. The sensor unit <NUM> sends the sensor signal to the sensor signal processing unit <NUM>.

The sensor unit <NUM> also processes signals from sensor systems which are used by the user to operate the head-mounted display <NUM>, such as a capacitive sensor, buttons, a global positioning system (GPS), and the like. Note that the positions where the input devices such as the capacitive sensor, buttons, GPS, and the like are provided are not limited to a region in the vicinity of the display unit <NUM>.

The image capture unit <NUM> is provided as a sensor for detecting the user's visual sensation, for example, in the vicinity of the display unit <NUM>, to capture an image of the user's eyes. The head-mounted display <NUM> can perform image analysis on the image of the user's eyes captured by the image capture unit <NUM>, to detect the user's line-of-sight direction, focal length, pupil size, or eye fundus pattern, whether the user's eyelid is open or closed, or the like. Note that the detection of the user's line-of-sight direction, focal length, pupil size, or eye fundus pattern, whether the user's eyelid is open or closed, or the like, may be performed by a light emission unit for emitting light to the user's eyes, which is provided in the vicinity of the display unit <NUM>, and a light reception unit for receiving light reflected by the eyes. For example, the thickness of the user's crystalline lens can be detected on the basis of a light signal received by the light reception unit.

The sensor control unit <NUM> performs sensor control according to an instruction from the system controller <NUM> to determine which sensor is driven at what timing, or what drive method is used, or the like. The sensor signal processing unit <NUM> performs various numerical processes such as average, variance, and the like, as processes which are performed before sending a sensor signal from the sensor unit <NUM> to the system controller <NUM>.

In this embodiment, the user's operation may be required in order to turn on/off power supply, start or end displaying various information images, change details of an image, adjust display such as a luminance level, a color, or the like, change a display area on a display screen, or the like. For example, in order to allow these user's operations (detection of trigger for processing operations), an operation unit provided as an operation key or operation dial which is operated by the user, and an operation detection mechanism for detecting an operation on the operation unit, are provided so that the system controller <NUM> can detect the user's operation, although not shown in <FIG>. Alternatively, instead of providing such an operation unit, the system controller <NUM> may determine the user's intention to operate or an appropriate operation process, on the basis of the user's situation detected by the sensor unit <NUM> (e.g., a motion of the eye, a behavior or state of the body, etc.), and perform a process corresponding to the result of the determination.

Moreover, the head-mounted display <NUM> may be configured to be capable of detecting external information (detection information, such as a situation around the display device, the place, the date and time, conditions of a subject, etc.) using the sensor unit <NUM>. The system controller <NUM> may determine an appropriate operation process on the basis of the external information, and perform the process.

The audio generation unit <NUM> performs an audio synthesis process to generate an audio signal such as a message voice or the like, or generates an audio signal such as an electronic sound or the like which is to be presented to the user, according to an instruction from the system controller <NUM>. The audio output unit <NUM> has, for example, a speaker or earphone speaker (not shown) provided in the head-mounted display <NUM>, and an amplifier circuit for the speaker. The audio signal generated by the audio generation unit <NUM> is supplied to the audio output unit <NUM>, so that the user can hear a message voice, an electronic sound, or the like. Note that the audio output unit <NUM> may be configured as a so-called bone-conduction speaker.

The audio input unit <NUM>, which includes a microphone amplifier unit for amplifying an audio signal obtained by a microphone and an A/D converter, outputs audio data to the audio signal processing unit <NUM>. The audio signal processing unit <NUM> includes, for example, a digital signal processor, a D/A converter, and the like. Audio data obtained by the audio input unit <NUM> or audio data generated by the audio generation unit <NUM> is supplied to the audio signal processing unit <NUM>.

The audio signal processing unit <NUM> performs processes such as sound volume adjustment, sound quality adjustment, sound effect, and the like on the supplied audio data under the control of the system controller <NUM>. Thereafter, the audio signal processing unit <NUM> converts the processed audio data into an analog signal, and supplies the analog signal to the audio output unit <NUM>. Note that the audio signal processing unit <NUM> is not limited to a configuration for performing digital signal processing, and may perform signal processing using an analog amplifier or an analog filter. The audio signal output from the audio signal processing unit <NUM> is output as a sound from the earphone speaker of the audio output unit <NUM>. Such a configuration allows the user to hear external sounds collected by the audio input unit <NUM> or sounds generated by the audio generation unit <NUM>.

The image generation unit <NUM> generates an image signal under the control of the system controller <NUM>. The system controller <NUM> causes the image generation unit <NUM> to generate an image signal from which an image to be presented to the user is generated, on the basis of information or numerical values from the components, and thereby generates an image signal as a picture image, a graph image, a character display image, an image for warning the user, or other images. The image generation unit <NUM> supplies the generated image signal to the displayed image processing unit <NUM>.

The displayed image processing unit <NUM>, which includes, for example, a video processor, performs various display processes on the image signal supplied from the image generation unit <NUM>. The displayed image processing unit <NUM> can perform luminance level adjustment, color correction, contrast adjustment, sharpness (contour emphasis) adjustment, and the like on the image signal. The displayed image processing unit <NUM> also adjusts settings for a display position on a display screen in the display unit <NUM>. In addition, the displayed image processing unit <NUM> may be configured to be capable of performing processes, including: image effect processes, such as generation of an enlarged image by enlarging a portion of an image signal, generation of a reduced image by reducing a portion of an image signal, soft focus, mosaic, luminance reversal, highlight display (emphasized display) of a portion of an image, changing of the entire chromatic atmosphere, and the like; separation and combination of an image signal for image division display; the process of generating a character image or a picture image, and combining the generated image with an image signal supplied from the image generation unit <NUM>; and the like. The displayed image processing unit <NUM> receives an image signal generated by the image generation unit <NUM>, performs a signal process required for displaying on the image signal, and supplies the resultant signal to the display drive unit <NUM>.

The display drive unit <NUM> includes a pixel drive circuit for displaying, on the display unit <NUM>, the image signal supplied from the displayed image processing unit <NUM>. Specifically, the display drive unit <NUM> applies a drive signal to pixels arranged in a matrix in the display unit <NUM>, at respective predetermined horizontal/vertical drive timings, according to the image signal, and thereby causes the display unit <NUM> to display. The display drive unit <NUM> can also control the luminances of the pixels of the display unit <NUM> so that the entirety or a portion of the screen is in the see-through mode.

The display control unit <NUM> controls a process operation of the displayed image processing unit <NUM>, an operation of the display drive unit <NUM>, and an image displayed on the display unit <NUM> (including a process for each of the right and left display units 110R and <NUM>) according to an instruction from the system controller <NUM>, and instructs the displayed image processing unit <NUM> to perform signal processing. The display control unit <NUM> also controls the display drive unit <NUM> so that the display drive unit <NUM> is switched between the see-through mode, the image display mode, and the single-eye display mode.

The image capture unit <NUM> includes: a lens system including an image capture lens, a diaphragm, a zoom lens, a focusing lens, and the like; a drive system for causing the lens system to perform a focusing operation or a zooming operation; a solid-state imaging element array for detecting captured image light obtained by the lens system, and performing photoelectric conversion to generate a captured image signal; and the like. Examples of the solid-state imaging element array include a charge coupled device (CCD) sensor array, and a complementary metal oxide semiconductor (CMOS) sensor array. In this embodiment, the image capture unit <NUM> captures an image of a scene in front of the user wearing the head-mounted display <NUM>. Of course, the image capture unit <NUM> may capture an image of a scene in other directions, such as a scene behind the user or the like.

The captured image signal processing unit <NUM>, which includes a sample hold/automatic gain control (AGC) circuit which performs gain adjustment or wave shaping on a signal obtained by the solid-state imaging element of the image capture unit <NUM>, and a video A/D converter, obtains a captured image signal in the form of digital data. The captured image signal processing unit <NUM> may also perform a white balance process, a luminance process, a color signal process, a shake correction process, or the like on a captured image signal.

The image capture control unit <NUM> controls operations of the image capture unit <NUM> and the captured image signal processing unit <NUM> according to an instruction from the system controller <NUM>. The image capture control unit <NUM> performs control to turn on/off operations of the image capture unit <NUM> and the captured image signal processing unit <NUM>, for example. The image capture control unit <NUM> may also perform control (motor control) to cause the image capture unit <NUM> to perform operations such as autofocusing, automatic exposure adjustment, diaphragm adjustment, zooming, changing of the focal point, and the like. Note that when the image capture unit <NUM> is provided with a movable mechanism capable of changing the direction of the image capture lens toward a subject, the image capture control unit <NUM> may change the direction of the image capture lens in the image capture unit <NUM> by controlling the operation of the movable mechanism according to an instruction from the system controller <NUM>.

The image capture control unit <NUM> may also include a timing generator. The image capture control unit <NUM> may control signal processing operations of the solid-state imaging element, and the sample hold/AGC circuit and video A/D converter of the captured image signal processing unit <NUM>, according to a timing signal generated by the timing generator. In addition, the timing control may be used to perform variable control on the frame rate of image capture.

Moreover, the image capture control unit <NUM> may control the image capture sensitivity or signal processing of the solid-state imaging element and the captured image signal processing unit <NUM>. For example, in order to control the image capture sensitivity, the image capture control unit <NUM> may control the gain of a signal read from the solid-state imaging element, black level settings, various coefficients for captured image signal processing in the digital data phase, a correction amount for a shake correction process, or the like. In order to control the image capture sensitivity, the image capture control unit <NUM> can perform a general sensitivity adjustment in which a wavelength band is not particularly taken into account, a sensitivity adjustment in which the sensitivity of image capture in a particular wavelength band, such as an infrared region, an ultraviolet region, or the like, is adjusted (e.g., image capture where the particular wavelength band is cut off), or the like. The sensitivity adjustment according to wavelength can be carried out by insertion of a wavelength filter in the image capture lens system, or a wavelength filter calculation process performed on a captured image signal. In these cases, the image capture control unit <NUM> can perform sensitivity control by controlling the insertion of a wavelength filter, specifying a filter calculation coefficient, or the like.

A captured image signal obtained by the image capture unit <NUM> and the captured image signal processing unit <NUM> is supplied together with an information image signal generated by the image generation unit <NUM> to the displayed image processing unit <NUM>. The displayed image processing unit <NUM> performs the above signal processes on the captured image signal and the information image signal, and also performs a signal process for screen division (image synthesis process) on the captured image signal and the information image signal in order to simultaneously display a captured image and an information image on the display unit <NUM>.

An image signal obtained by the synthesis process in the displayed image processing unit <NUM> is supplied to the display drive unit <NUM>, and displayed on the display unit <NUM>, so that a captured image and other images are simultaneously displayed on the display unit <NUM>. In other words, the user can view various images while viewing an image captured by the image capture unit <NUM>.

The user may need to perform operations for starting and ending of an image capture operation, a zooming operation, a focusing operation, a captured image adjustment, and the like. Of course, the user may also need to perform operations for turning on/off of power supply, starting and ending of displaying of various information images, changing of details of an image, display adjustment of a luminance level, a color, etc., changing of a display region on a display screen, and the like. For these operations (triggers for the operations), the head-mounted display <NUM> may be provided with an operation unit such as an operation key or the like. Alternatively, the system controller <NUM> may determine the user's intention to operate or an appropriate operation process on the basis of the user's conditions (e.g., a motion of the eye, a behavior or state of the body, etc.) detected by the sensor unit <NUM>, and perform a process corresponding to the determination result. Moreover, the head-mounted display <NUM> may be configured to be capable of detecting external information (detection information, such as a situation around the display device, the place, the date and time, conditions of a subject, etc.), and the system controller <NUM> may determine an appropriate operation process on the basis of the external information, and perform the process.

The storage unit <NUM>, which is a part that records and retrieves data with respect to a predetermined recording medium, is implemented as, for example, a hard disk drive (HDD) or a solid-state drive (SSD). Of course, the recording medium may be various media, such as a solid-state memory (a flash memory etc.), a memory card including a fixed memory, an optical disc, a magneto-optic disc, a hologram memory, and the like. The storage unit <NUM> may be adapted, according to the employed recording medium, to be capable of recording and retrieving. For example, image data as a captured image signal which has been captured by the image capture unit <NUM> and processed by the captured image signal processing unit <NUM>, image data received by the communication unit <NUM>, and various information image signals generated by the image generation unit <NUM> may be supplied to the storage unit <NUM>. In addition, audio data obtained by the audio input unit <NUM>, audio data received by the communication unit <NUM>, and audio data generated by the audio generation unit <NUM> may also be supplied to the storage unit <NUM>.

The storage unit <NUM> performs an encoding process on the supplied image data or audio data in order to record the data to a predetermined recording medium, and records the data to the recording medium, under the control of the system controller <NUM>. The storage unit <NUM> also retrieves image data or audio data recorded in the recording medium, and outputs the data, under the control of the system controller <NUM>. The data retrieved by the storage unit <NUM> may be any data that is to be displayed, such as moving image contents (movies, video clips, etc.), still image contents which are captured by a digital still camera or the like and recorded in a recording medium, data of electronic books and the like, data used on a computer (image data, text data, spreadsheet data and the like which are created by the user using a personal computer or the like), game images, and the like.

The communication unit <NUM> exchanges data with external devices. Examples of the external devices include an image capture device, such as a video camera, a digital still camera, and the like, which has a communication function, a computer device, a smartphone, a smartwatch, a video storage device, AV devices, such as a television set and the like, a network server device, and the like. The communication unit <NUM> may be configured to perform network communication, for example, through short-distance radio communication with respect to a network access point, according to a scheme such as wireless LAN, Bluetooth (registered trademark), or the like, or may be configured to perform direct radio communication with external devices having a corresponding communication function. When the external device is an image capture device, data transmitted from the external device to the head-mounted display <NUM> may be image data captured by the image capture device. Alternatively, when the external device is a content source device, data transmitted from the external device to the head-mounted display <NUM> may be any data that is to be displayed, such as moving image contents (movies, video clips, etc.), still image contents which are captured by a digital still camera or the like and recorded in a recording medium, data of electronic books and the like, data used on a computer (image data, text data, spreadsheet data and the like which are created by the user using a personal computer or the like), game images, and the like.

Audio data obtained by the audio input unit <NUM>, audio data retrieved by the storage unit <NUM>, and audio data received by the communication unit <NUM> are supplied to the audio signal processing unit <NUM> according to an instruction from the system controller <NUM>. Therefore, the user can hear external sounds when an image is captured, view and hear an image and sounds retrieved by the storage unit <NUM>, and view and hear an image and sounds received by the communication unit <NUM> while wearing the device. In particular, an image obtained by the image generation unit <NUM> is supplied together with the captured image, or retrieved image, or received image to the displayed image processing unit <NUM>, so that various information images are displayed together with the captured image, or retrieved image, or received image on the display unit <NUM>. Audio data generated by the audio generation unit <NUM> is supplied, at a timing when the audio data is generated, to the audio signal processing unit <NUM>, so that the user can hear a message voice, a warning sound, or the like generated by the audio generation unit <NUM> while hearing, for example, external sounds, retrieved sounds, or received sounds.

In addition to operations of the display system and operations involved in the image capture function, the system controller <NUM> may determine whether to trigger operation controls for retrieval, cuing up, fast-forwarding/fast-backwarding, pausing, recording, and the like in the storage unit, and operation controls involved in transmission and reception by the communication unit. Also, in this case, an operation unit such as an operation key or the like which is operated by the user may be provided, and a process corresponding to the operation may be performed. Alternatively, the system controller <NUM> may determine the user's intention to operate or an appropriate operation process on the basis of the user's conditions (e.g., a motion of the eye, a behavior or state of the body, etc.) detected by the sensor unit <NUM>, and perform a process corresponding to the determination result. Moreover, the head-mounted display <NUM> may be configured to be capable of detecting external information (detection information, such as a situation around the display device, the place, the date and time, conditions of a subject, etc.), and the system controller <NUM> may determine an appropriate operation process on the basis of the external information, and perform the process.

When the head-mounted display <NUM> configured as shown in <FIG> is operating in the image display mode or the single-eye display mode, then if there is an incoming mail, call, or the like to a device external to the head-mounted display <NUM> or the head-mounted display <NUM> itself, the system controller <NUM> performs a process of displaying, on the display unit <NUM>, an image (e.g., a predetermined icon) indicating the arrival of a mail, call, or the like. In this case, the system controller <NUM> displays the image indicating the arrival on the display unit <NUM> while changing the transparency of the icon (or the luminance of the display unit <NUM>) instead of suddenly displaying the image on the display unit <NUM>. By thus displaying an image indicating the arrival of a mail, call, or the like on the display unit <NUM> while changing the transparency of the icon (or the luminance of the display unit <NUM>), the head-mounted display <NUM> according to this embodiment can ensure the field of vision of the user wearing the head-mounted display <NUM> while keeping the user safe.

Examples of an image displayed on the display unit <NUM> of the head-mounted display <NUM> according to an embodiment of the present disclosure will be described. <FIG> are illustrative diagrams showing examples of an image displayed on the display unit <NUM> of the head-mounted display <NUM> according to an embodiment of the present disclosure, where an icon is displayed when an interrupt, such as the arrival of a mail or the like, occurs. Note that <FIG> show a scene which is viewed through the display unit <NUM> by the user of the head-mounted display <NUM>, in addition to an image which is displayed on the display unit <NUM> by the head-mounted display <NUM>.

<FIG> shows an example of an image which is displayed on the display unit <NUM> immediately after the occurrence of an interrupt, such as the arrival of a mail or the like. The system controller <NUM> displays an icon indicating the occurrence of an interrupt on the display unit <NUM>. In this case, the system controller <NUM> instructs the display control unit <NUM> to increase the transparency of an icon <NUM> displayed on the display unit <NUM> or decrease the luminance of the display unit <NUM> immediately after the occurrence of an interrupt, such as the arrival of a mail or the like. In other words, the system controller <NUM> performs control so that the displayed icon <NUM> is less easily recognized immediately after the occurrence of an interrupt, such as the arrival of a mail or the like.

<FIG> shows an example of an image which is displayed on the display unit <NUM> a predetermined period of time (e.g., <NUM> to <NUM> msec) after the state shown in <FIG>. <FIG> shows an example of an image which is displayed on the display unit <NUM> a predetermined period of time (e.g., <NUM> to <NUM> msec) after the state shown in <FIG>. As shown in <FIG> and <FIG>, the system controller <NUM> instructs the display control unit <NUM> to gradually decrease the transparency of the icon <NUM> or increase the luminance of the display unit <NUM> so that the icon <NUM> can be recognized as a display.

<FIG> shows an example of an image which is displayed on the display unit <NUM> a predetermined period of time (e.g., <NUM> to <NUM> msec) after the state shown in <FIG>. As shown in <FIG>, the system controller <NUM> instructs the display control unit <NUM> to minimize the transparency of the icon <NUM> or maximize the luminance of the display unit <NUM> so that the icon <NUM> can be perfectly recognized as a display. The display control unit <NUM> performs control according to the instruction from the system controller <NUM> to minimize the transparency of the icon <NUM> or maximize the luminance of the display unit <NUM>.

<FIG> is an illustrative diagram showing a graph indicating an example of changes in the transparency of the icon <NUM> displayed on the display unit <NUM> (or the luminance of the display unit <NUM>) of the head-mounted display <NUM> according to an embodiment of the present disclosure. In the graph shown in <FIG>, the horizontal axis represents time, and the vertical axis represents the transparency of the icon <NUM> (or the luminance of the display unit <NUM>). It is assumed that time t1 is a time point corresponding to the state shown in <FIG>, time t2 is a time point corresponding to the state shown in <FIG>, time t3 is a time point corresponding to the state shown in <FIG>, and time t4 is a time point corresponding to the state shown in <FIG>. In the head-mounted display <NUM> according to an embodiment of the present disclosure, the transparency of the icon <NUM> (or the luminance of the display unit <NUM>) indicating the arrival of a mail, call, or the like is thus gradually changed so that the user is prevented from being involuntarily attracted by the animation.

After the icon <NUM> is clearly displayed at time t4, the system controller <NUM> may adjust the transparency of the icon <NUM> or the luminance of the display unit <NUM> so that the icon <NUM> is again not displayed, as indicated by a solid-state line in the graph shown in <FIG>. Alternatively, as indicated by a dashed line in the graph shown in <FIG>, the transparency of the icon <NUM> or the luminance of the display unit <NUM> may not be changed.

After displaying the icon <NUM> on the display unit <NUM> as shown in <FIG>, the system controller <NUM>, when receiving a second incoming mail, call, or the like, may newly display an icon <NUM> corresponding to the second incoming mail, call, or the like on the display unit <NUM> as shown in <FIG>, or alternatively, may erase the previously displayed icon <NUM> from the display unit <NUM>, and then display an icon <NUM> corresponding to the second incoming mail, call, or the like on the display unit <NUM> as shown in <FIG>.

Thus, the head-mounted display <NUM> according to an embodiment of the present disclosure gradually display the icon <NUM> on the display unit <NUM> by changing the transparency of the icon <NUM> (or the luminance of the display unit <NUM>) indicating the arrival of a mail, call, or the like over time. When only one icon <NUM> is displayed on the display unit <NUM>, the system controller <NUM> may change either the transparency of the icon <NUM> or the luminance of the display unit <NUM>. However, if two or more icons <NUM> are displayed on the display unit <NUM>, then when all the icons <NUM> are uniformly changed, the system controller <NUM> may change the luminance of the display unit <NUM> over time, or then when only a particular icon(s) <NUM> is changed, the system controller <NUM> may change the transparency of the icon(s) <NUM> over time.

Note that the head-mounted display <NUM> according to an embodiment of the present disclosure may be capable of switching between a display process of displaying the icon <NUM> on the display unit <NUM> so that the icon <NUM> gradually becomes clearly visible as shown in <FIG>, and a display process of clearly displaying the icon <NUM> on the display unit <NUM> from the beginning. Such switching may be set by the user or may be automatically set according to a situation where the head-mounted display <NUM> is used. For example, in a situation where a problem will arise if the user's field of vision is suddenly blocked, such as when the user is outside, the head-mounted display <NUM> may perform a display process of displaying the icon <NUM> on the display unit <NUM> so that the icon <NUM> gradually becomes clearly visible as shown in <FIG>, according to settings provided by the user themselves or settings automatically provided by the head-mounted display <NUM>.

The head-mounted display <NUM> according to an embodiment of the present disclosure may change the transparency of an image indicating the arrival of a mail, call, or the like (or the luminance of the display unit <NUM>), according to the intensity of ambient light. Specifically, when ambient light is weak, then even if the amount of a change in the transparency of an image (or the luminance of the display unit <NUM>) is small, the user can satisfactorily recognize the change. When ambient light is strong, it is difficult for the user to recognize such a change. Therefore, the head-mounted display <NUM> according to an embodiment of the present disclosure may measure the intensity of ambient light using a sensor capable of measuring the intensity of ambient light, such as a luminance sensor, a camera, or the like, in the sensor unit <NUM>, and adjust the amount of a change in transparency or luminance according to the intensity of ambient light.

<FIG> is an illustrative diagram showing a graph indicating an example of changes in the transparency of the icon <NUM> displayed on the display unit <NUM> (or the luminance of the display unit <NUM>) of the head-mounted display <NUM> according to an embodiment of the present disclosure. In <FIG>, reference sign <NUM> indicates the transition of a display of the transparency (or the luminance of the display unit <NUM>) when the intensity of ambient light is weak, reference sign <NUM> indicates the transition of a display of the transparency (or the luminance of the display unit <NUM>) when the intensity of ambient light is stronger than in the case of reference sign <NUM>, and reference sign <NUM> the transition of a display of the transparency (or the luminance of the display unit <NUM>) when the intensity of ambient light is stronger than in the case of reference sign <NUM>. The head-mounted display <NUM> according to an embodiment of the present disclosure thus gradually changes the transparency of the icon <NUM> (or the luminance of the display unit <NUM>) indicating the arrival of a mail, call, or the like according to the intensity of ambient light, and thereby prevents the user from being involuntarily attracted by the animation.

Note that, as in the example shown in <FIG>, after the icon <NUM> is clearly displayed, the system controller <NUM> may adjust the transparency of the icon <NUM> or the luminance of the display unit <NUM> so that the icon <NUM> is again not displayed, or alternatively, may not change the transparency of the icon <NUM> or the luminance of the display unit <NUM>.

Although, in the example shown in <FIG>, the time it takes to fully recognize the icon <NUM> as a display is the same irrespective of the intensity of ambient light, the present disclosure is not limited to such an example. The head-mounted display <NUM> according to an embodiment of the present disclosure may perform display control so that the time it takes to fully recognize the icon <NUM> as a display is increased with an increase in the intensity of ambient light.

<FIG> is an illustrative diagram showing a graph indicating an example of changes in the transparency of the icon <NUM> displayed on the display unit <NUM> (or the luminance of the display unit <NUM>) of the head-mounted display <NUM> according to an embodiment of the present disclosure. <FIG> shows an example of changes in the transparency of the icon <NUM> (or the luminance of the display unit <NUM>) in a case where the time it takes to fully recognize the icon <NUM> as a display is increased with an increase in the intensity of ambient light.

The above technique of changing the transparency of an icon or display information (or the luminance of the display unit <NUM>) can be combined with a sensor value which is obtained by detecting a motion of the user's line of sight, head, or the like using the sensor unit <NUM>. For example, if an operable icon or a message continues to be displayed in front of the eyes, a portion of the field of view is blocked, and therefore, the user is likely to feel uncomfortable. With this in mind, a display control example will be described in which display control is performed in combination with the user's line of sight so that the user is prevented from feeling uncomfortable.

<FIG> are illustrative diagrams showing examples of an image displayed on the display unit <NUM> of the head-mounted display <NUM> according to an embodiment of the present disclosure, where an icon is displayed in an illustrative manner when an interrupt, such as the arrival of a mail or the like, occurs.

In <FIG>, a point <NUM> indicates the position of the line of sight of the user wearing the head-mounted display <NUM>. As shown in <FIG>, when the line of sight is located away from a region where the icon <NUM> is displayed, the system controller <NUM> instructs the display control unit <NUM> to cause the display unit <NUM> to increase the transparency of the icon <NUM> or decrease the luminance of the display unit <NUM>.

As shown in <FIG> and <FIG>, when the user's line of sight is close to the region where the icon <NUM> is displayed, the system controller <NUM> instructs the display control unit <NUM> to cause the display unit <NUM> to change the transparency of the icon <NUM> (or the luminance of the display unit <NUM>) so that the icon <NUM> is clearly displayed instead of being transparent.

The head-mounted display <NUM> according to an embodiment of the present disclosure controls the display of an icon according to the place where the user's line of sight is located, so that the icon <NUM>, which is usually not displayed in front of the eyes, is displayed on the display unit <NUM> only when the user wearing the head-mounted display <NUM> performs an operation of changing the line of sight with a thought of viewing information.

<FIG> is an illustrative diagram showing a specific example of a calculation method which is used during execution of display control on the icon <NUM> according to the user's line of sight. A coordinate point of the icon <NUM> in a display region is represented by (X1, Y1), and a coordinate point of the line of sight in the display region is represented by (X2, Y2). A distance A between the two points can be calculated by A = {(X2 - X1)<NUM> + (Y2 - Y1)<NUM>}<NUM>. The system controller <NUM> may instruct the display control unit <NUM> to increase the transparency (or decreases the luminance) so that the icon <NUM> becomes transparent when A is great, and decrease the transparency (or increase the luminance) so that the icon <NUM> becomes clearly visible when A is small.

The head-mounted display <NUM> according to an embodiment of the present disclosure may detect an angle of the user's head using the sensor unit <NUM> instead of the user's line of sight, and similarly perform display control on the icon using the angle.

As shown in <FIG>, the system controller <NUM>, when detecting, on the basis of data acquired by the sensor unit <NUM>, that the user faces front without inclining their neck, instructs the display control unit <NUM> to cause the display unit <NUM> to increase the transparency of the icon <NUM> or decrease the luminance of the display unit <NUM>.

As shown in <FIG> and <FIG>, when the inclination of the user's neck gradually increases, so that the sensor unit <NUM> detects that the user faces their feet, on the basis of acquired data, the system controller <NUM> instructs the display control unit <NUM> to cause the display unit <NUM> to change the transparency of the icon <NUM> (or the luminance of the display unit <NUM>) so that the icon <NUM> is clearly displayed instead of being transparent.

Although the head-mounted display <NUM> according to an embodiment of the present disclosure thus performs display control to clearly display the icon <NUM> according to the angle of the user's head, the present disclosure is not limited to this example. For example, the system controller <NUM> may perform display control to clearly display the icon <NUM> if the amount of a motion of the user's head exceeds a predetermined threshold.

The head-mounted display <NUM> according to an embodiment of the present disclosure may detect the rate of movement of the user using the sensor unit <NUM>, and similarly perform display control on an icon using the movement rate. Specifically, when the user is walking, then if an icon is displayed on the display unit <NUM> to attract the user's attention, the user's attention is distracted from the scene in front of the user, which is dangerous. Therefore, the head-mounted display <NUM> according to an embodiment of the present disclosure may perform display control to display a thin icon when the user is walking, and a clearly visible icon when the user is standing.

When the head-mounted display <NUM> according to an embodiment of the present disclosure detects the rate of movement of the user, and performs display control on an icon using the movement rate, the head-mounted display <NUM> may perform display control to clearly display the icon if the icon is intended for emergency notification, even when the user is walking. Examples of emergency notification may include the reception of a call or the arrival of a mail from a particular person, a reminder having high priority, and the like. The user of the head-mounted display <NUM> may decide what is emergency notification, and emergency notification is not limited to particular things.

The head-mounted display <NUM> according to an embodiment of the present disclosure may detect the state of the user's activity using the image capture unit <NUM> or the audio input unit <NUM> in addition to the sensor unit <NUM>, and perform display control on an icon to be displayed on the display unit <NUM>, using the result of the detection of the user's activity state.

For example, when the user is driving a car while wearing the head-mounted display <NUM>, then if it is found that the steering wheel of the car is included in an image captured by the image capture unit <NUM>, or the sound of the engine is included in sounds collected by the audio input unit <NUM>, the system controller <NUM> determines that the user is driving a car. When the user is driving a car, then if an icon is displayed on the display unit <NUM> to attract the user's attention, the user's attention is distracted from the scene in front of the user, which is very dangerous, as in the above case where the user is walking. Therefore, when it is found that the user is driving a car, the system controller <NUM> may perform display control to display a thin icon or no icon when the user is driving a car, and a clearly visible icon as described above when the car is standing.

In the head-mounted display <NUM> according to an embodiment of the present disclosure, if it is found that the sound of human conversation is included in sounds collected by the audio input unit <NUM>, the system controller <NUM> determines that the user using the head-mounted display <NUM> is talking with another person. When the user using the head-mounted display <NUM> is talking with another person, then if an icon is displayed on the display unit <NUM> to attract the user's attention, the user's attention is distracted from the conversation with that person, which hinders smooth communication. Therefore, if it is found that the user is talking with another person, the system controller <NUM> may perform display control to display a thin icon or no icon when the user is talking, and a clearly visible icon as described above when the conversation is interrupted.

Thus, the head-mounted display <NUM> according to an embodiment of the present disclosure can recognize a situation where the user is using the head-mounted display <NUM>, on the basis of an image captured by the image capture unit <NUM> or sounds collected by the audio input unit <NUM>, and control the display of an icon on the display unit <NUM> according to the user's situation.

In the foregoing, examples have been described in which an icon is gradually displayed on the display unit <NUM> by changing the transparency of the icon or the luminance of the display unit <NUM>. For example, the head-mounted display <NUM> according to an embodiment of the present disclosure may gradually clearly display an icon on the display unit <NUM> by gradually increasing the size of the icon.

In the foregoing, functional configuration examples of the head-mounted display <NUM> according to an embodiment of the present disclosure have been described. Next, operation examples of the head-mounted display <NUM> according to an embodiment of the present disclosure will be described.

<FIG> is a flowchart showing an operation example of the head-mounted display <NUM> according to an embodiment of the present disclosure. <FIG> shows an operation example of the head-mounted display <NUM> which is performed when the head-mounted display <NUM> performs a process of displaying, on the display unit <NUM>, an image (e.g., a predetermined icon) indicating the presence of information of which the user is to be notified, such as the arrival of a call or mail, or the like. The operation example of the head-mounted display <NUM> according to an embodiment of the present disclosure will now be described with reference to <FIG>.

The head-mounted display <NUM> according to an embodiment of the present disclosure initially waits until the occurrence of a process of displaying, on the display unit <NUM>, an icon indicating the presence of information of which the user is to be notified, such as the arrival of a call or mail, or the like (step S101). The head-mounted display <NUM>, when detecting the occurrence of the process of displaying an icon on the display unit <NUM> (step S101: Yes), displays an icon on the display unit <NUM> while adjusting the transparency of the icon (or the luminance of the display unit <NUM>) (step S102). The adjustment and display process in step S102 is, for example, performed by the display control unit <NUM> according to an instruction from the system controller <NUM>.

The transparency of an icon (or the luminance of the display unit <NUM>) may be adjusted so that, as described above, the icon may be more clearly displayed over time, the icon may be more clearly displayed according to the position of the line of sight, the icon may be more clearly displayed according to the inclination of the head, or the icon may be more clearly displayed according to the rate of movement of the user.

By thus displaying an image indicating the arrival of a mail, call, or the like on the display unit <NUM> while changing the transparency of an icon (or the luminance of the display unit <NUM>), the field of vision of the user wearing the head-mounted display <NUM> can be ensured while the user is kept safe.

The head-mounted display <NUM> according to an embodiment of the present disclosure may perform display control to gradually clearly display an icon using information about color in the user's field of vision. For example, when the user using the head-mounted display <NUM> is in a room having white walls, the system controller <NUM> initially performs display control to display a white icon on the display unit <NUM>. By initially displaying a white icon on the display unit <NUM> when the user using the head-mounted display <NUM> is in a room having white walls, the head-mounted display <NUM> according to an embodiment of the present disclosure can cause the user not to notice that the icon is displayed on the display unit <NUM>.

After displaying the white icon on the display unit <NUM>, the system controller <NUM> performs display control to gradually change the color of the icon to the original color, or a different color if the original color is white. By such display control, the head-mounted display <NUM> according to an embodiment of the present disclosure can avoid displaying information which is involuntarily noticed by the user or blocks the field of view.

Alternatively, for example, the system controller <NUM> initially performs display control to display, on the display unit <NUM>, an icon having colors from which a green component is removed. Green is a color which is most strongly sensed by humans. Therefore, by displaying an icon having colors from which a green component is removed, the head-mounted display <NUM> according to an embodiment of the present disclosure can avoid displaying information which is involuntarily noticed by the user or blocks the field of view.

As described above, according to an embodiment of the present disclosure, provided is the see-through head-mounted display <NUM> which, when displaying information which blocks a portion of the user's field of view, displays the information gradually clearly while adjusting the transparency of an icon or the luminance of the display unit <NUM>. The head-mounted display <NUM> according to an embodiment of the present disclosure, when displaying information of which the user is to be notified on the display unit <NUM>, displays the information gradually clearly while adjusting the transparency of an icon or the luminance of the display unit <NUM>, for example, over time, according to the position of the line of sight, according to the inclination of the head, or according to the rate of movement of the user.

By thus displaying information gradually clearly while adjusting the transparency thereof or the luminance of the display unit <NUM>, the head-mounted display <NUM> according to an embodiment of the present disclosure can avoid displaying information which is involuntarily noticed by the user or blocks the field of view, and thereby reduce a danger to the user when the user is using the head-mounted display <NUM>.

In the above embodiments, examples have been described in which the display control on the display unit <NUM> is performed by the display control unit <NUM> according to an instruction from the system controller <NUM> included in the head-mounted display <NUM>. The present disclosure is not limited to these examples. For example, the display control on the display unit <NUM> may be performed by a device separated from the head-mounted display <NUM>, which is connected to the head-mounted display <NUM> by wire or wirelessly.

Steps in processes executed by devices in this specification are not necessarily executed chronologically in the order described in a sequence chart or a flow chart. For example, step S in processes executed by devices may be executed in a different order from the order described in a flow chart or may be executed in parallel.

Further, a computer program can be created which causes hardware such as a CPU, ROM, or RAM, incorporated in each of the devices, to function in a manner similar to that of structures in the above-described devices. Furthermore, it is possible to provide a recording medium having the computer program recorded thereon. Moreover, by configuring respective functional blocks shown in a functional block diagram as hardware, the hardware can achieve a series of processes.

Claim 1:
A display control device comprising:
a system controller (<NUM>) configured to:
acquire a state of a line of sight of a user viewing a see-through display which is worn on the user in front of the user's eye,
send a first instruction to increase a visibility of a display of notification information when the system controller (<NUM>) receives an interrupt related to the notification information while the see-through display is being used when the system controller (<NUM>) determines that the line of sight of the user is close to the display of the notification information,
send a second instruction to decrease the visibility of the display of the notification information when the system controller (<NUM>) determines that the line of sight of the user is located away from the display of the notification information; and
a display control unit (<NUM>) configured to perform, according to the first instruction from the system controller (<NUM>), display control on the see-through display (<NUM>) so that the visibility of the display of the notification information gradually increases immediately after the system controller (<NUM>) receives the interrupt, and configured to perform the display control according to the second instruction from the system controller (<NUM>),
wherein the system controller (<NUM>) is configured to acquire an intensity of ambient light,
wherein the display control unit (<NUM>) is configured perform the display control so that the visibility of the display of the notification information gradually increases according to the intensity of the ambient light.