Source: https://patents.google.com/patent/WO2014041871A1/en
Timestamp: 2019-04-23 06:21:18
Document Index: 467326645

Matched Legal Cases: ['art 2101', 'art 2101', 'art 2101', 'art 2201', 'art 2201', 'art 2201', 'art 2201', 'art 2201']

WO2014041871A1 - Image display device, image display method, and recording medium - Google Patents
WO2014041871A1
WO2014041871A1 PCT/JP2013/068002 JP2013068002W WO2014041871A1 WO 2014041871 A1 WO2014041871 A1 WO 2014041871A1 JP 2013068002 W JP2013068002 W JP 2013068002W WO 2014041871 A1 WO2014041871 A1 WO 2014041871A1
PCT/JP2013/068002
宮島　靖
楢原　立也
高井　基行
正資 武田
浩明 小川
和田　成司
いちご 早川
一英 劔持
2012-09-12 Priority to JP2012200902 priority Critical
2012-09-12 Priority to JP2012-200902 priority
2013-07-01 Application filed by ソニー株式会社 filed Critical ソニー株式会社
2014-03-20 Publication of WO2014041871A1 publication Critical patent/WO2014041871A1/en
Provided is an image display device which is worn on the head of a user to allow the user to view an image while providing various information for even non-wearers of the device. The image display device not only displays an internal image to be seen by the user wearing the device but also displays an external image which can be seen from the outside of the device. Non-wearers of the device can be aware of, via the external image, the condition of the user wearing the device, what the user is doing, what the user is viewing, and how much the user is focused on or immersed in viewing. Furthermore, the user may convey the user's own condition and information relating to the content the user is viewing via the external image, even when the user is wearing the device.
An image display device and an image display method, and a recording medium
Technology disclosed in the present specification includes an image display device and an image display method and mounted on the head is used for viewing the image, and relates to a recording medium, particularly, an observer as an enlarged virtual image of the display image by a virtual image optical system the image display apparatus and image display method for displaying such that observed, and a recording medium.
And mounted on the head image display apparatus of the head-mounted utilized for viewing of the image, i.e., there is known a head-mounted display. Generally, head-mounted image display apparatus has an image display portion of the right and left each eyes, also when used in combination with headphones is configured to be able to control visual and audible. Further, head-mounted image display apparatus, it is also possible to project a different image to the left and right eyes, it is possible to present a three-dimensional image by displaying an image with parallax to the right and left eyes.
Head-mounted image display apparatus, as the display portions of the left and right eyes, for example, be equipped with high-resolution display panel made of liquid crystal or organic EL (Electro-Luminescence) element. Further, the head-mounted image display device can be classified into types of permeable and light-shielding type. Light-shielding of the head-mounted image display apparatus, when worn on the head, is configured to cover the user's eyes directly, immersive increases during image viewing. And enlarging and projecting the display screen by using a virtual image optical system, causes observed user as an enlarged virtual image comprising a suitable angle, if reproduced multi-channel headphones, to reproduce the realistic sensation as view in a theater can (for example, see Patent Document 1). On the other hand, if the permeability of the head-mounted image display apparatus, even while the user is viewing an image mounted on the head, look at the outside scene to an image over (i.e., see-through) since it is possible (for example, see Patent Document 2), in use at the time or walking outdoor use, the user can avoid the danger of collision with the obstacle.
Incidentally, in the thin display device, proposals have been made on the display can display an image in both front and back (for example, refer to Patent Documents 3 and 4). Double-side display device is also possible to display the same content on both sides, it is also possible to display different content on both sides. User since it is possible to observe an image in both front and back sides of the apparatus main body, the double-side display device can be a good information providing tools.
On the other hand, a head-mounted image display apparatus described above, although it is possible to face the eye of the front side i.e. the wearer to view the images, it is not able to display an image to the outside of the rear side or image display device . For this reason, the periphery of the person, you can not be sure at all whether the wearer is looking at what. Moreover, since the wearer's eye is covered, it is difficult around the person to grasp the status of the wearer (e.g., whether or speaking, should be later).
Head-mounted image display apparatus, said one of the individual ultimate display. However, the non-user person wearing the device, its or his or her state, or the person is doing, what you're watching, it is not possible to know the degree of concentration or immersion in viewing. In addition, and if the user himself wants to notify their state around, or if you want to inform the content being viewed in the outside, but in some cases you want to convey useful information related to the content, from once disconnect the equipment it is not possible to tell not equal, viewing is interrupted.
The purpose of the technology disclosed herein is mounted on the head is used to view the image, relative to the surrounding persons, or the wearer is looking at what, for example, the current state of a wearer, various You can present information, excellent image display apparatus and image display method, as well as a recording medium.
This application has been made in consideration of the above problems, the technique described in claim 1 is an image display device used by mounting on the head or face of the user,
A first display unit for displaying an inner image viewed from the user side,
A second display section for displaying the outside image viewed from the outside of the image display device,
And a control unit which controls the display of the outer picture and the inner image,
An image display device location having a.
According to the technique described in claim 2 of the present application, the display unit of the image display apparatus according to claim 1, a single display device that displays the inner image and the outer image, displayed on said display device guides the inner images in a location visible from the user side, and a guide portion for guiding the outer image displayed on the display device to a location visible from the outside of the image display device.
According to the technique described in claim 3 of the present application, an image display apparatus according to claim 1 further comprising a projection unit for projecting the outer image toward the outside of the image display device.
According to the present claims 4 to described technique, an image display apparatus according to claim 1 further comprising an input operation unit which the user performs input operation. Then, the control unit, in accordance with an input operation by the user, is configured to control the display of the inner image and the outer image.
According to the technique described in claim 5 of the present application, the control unit of the image display apparatus according to claim 4, in accordance with an input operation on the input operation unit of the user, of the inner image or the outer image color adjustment display on / off, the inner image or the outer image, the luminance adjustment of the inner image or the outer image, switching the display size of the outer image, or is configured to perform display area movement of the outer image ing.
According to the technique described in claim 6 of the present application, the control unit of the image display apparatus according to claim 4, in accordance with an input operation on the input operation unit of the user, as the outer image, the inner image the same image, the image different from the inner image, the ambient environment information, wherein to display at least one of the current state of the user, or is configured so as to display the outer image as the inner image and .
According to the present claims 7 techniques described, the controller of the image display apparatus according to claim 4, in accordance with an input operation based on the motion of the blinking operation or ocular of the user, of the outer image It is configured to switch the display.
According to the technique described in claim 8 of the present application, the control unit of the image display apparatus according to claim 4, in accordance with an input operation on the input operation unit of the user, continuous display of the outer image, intermittent display, or is configured to perform any given time each of the display.
According to the present claims 9 techniques described, the image display apparatus according to claim 1, further comprising an environmental information acquisition unit for acquiring information on the surrounding environment of the image display device. Then, the control unit, on the basis of the ambient environment, and is configured to control the display of the inner image and the outer image.
According to the technique described in claim 10 of the present application, the control unit of the image display apparatus according to claim 9, in response to detecting the brightness change of the ambient light to control brightness of the outer image It is configured to.
According to the technique described in claim 11 of the present application, the control unit of the image display apparatus according to claim 9, in response to the lapse of the current time, so as to control the intermittent display or timer display of the outer image It is configured.
According to the technique described in claim 12, wherein the control unit of the image display apparatus according to claim 9, depending on the number of people who are surrounding the display on / off of the outer image, luminance level, a display area , and is configured to control the display size.
According to the present claim 13 techniques described, the image display apparatus according to claim 1 further comprising a status information acquisition unit that acquires information on the state of the user. Then, the control unit, on the basis of the user's state, is configured to control the display of the inner image and the outer image.
According to the technique described in claim 14 of the present application, the control unit of the image display apparatus according to claim 13, the current working status of the user, wherein the state information acquiring unit acquires, behavioral state, mental state, or physiological state, and is configured to display as the outer image.
According to the technique described in claim 15, wherein the control unit of the image display apparatus according to claim 13, identifies the mental state of the user based on blinking operation information of the user, the spirit It is configured to control the display of the outer image depending on the state.
According to the technique described in claim 16, wherein the control unit of the image display apparatus according to claim 13, wherein in response to lateral tilt of the user's head, the horizontal character information in said outer image and it is configured to control so as to keep the.
According to the technique described in claim 17, wherein the control unit of the image display apparatus according to claim 13, wherein in response to rotation of the user's head, keeping the position of the inner image or the outer image It is configured to control so.
According to the technique described in claim 18, the image display apparatus according to claim 1 further comprising a content information obtaining section that obtains content information on the image content to be displayed on the inner display. Then, the control unit, in response to the content information, is configured to control the display of the outer image.
The technique according to claim 19 of the present application is an image display method used by mounting on the head or face of the user,
A first display step of displaying the interior image viewed from the user side,
A second display step of displaying the outer image viewed from the outside of the image display device,
A control step for controlling the display of the outer image and the inner image,
An image display method having.
The technique described herein of claim 20 is a recording medium readable to a computer which records a computer program for controlling an image display device used by mounting on the head or face portion of the user, the computer program the computer,
A first display unit for displaying the inside image viewed from the user side,
A second display unit for displaying the outer image viewed from the outside of the image display device,
Control unit for controlling display of the outer image and the inner image,
To function as a recording medium.
According to the technology disclosed herein, and mounted on the head is used to view the image, relative to the surrounding persons, or the wearer is looking at what, for example, the current state of a wearer, various it is possible to present information, it is possible to provide an excellent image display device and an image display method, and recording medium.
Head-mounted image display apparatus according to the technology disclosed herein, in addition to the inner display unit viewed from the user side mounted, provided with an outer display unit visible from the outside, that the wearer is looking the same image and the current state of the wearer and is displayed on the outer display unit, it is possible to provide various information to the outside.
1, the user wearing light-shielding types of head-mounted image display apparatus 1 is a diagram showing a state as viewed from the front. Figure 2 is a diagram showing a state as viewed head-mounted image display apparatus 1 of the light-shielding type from above. 3, the user where the permeability of types of head-mounted image display device 3 is attached is a diagram showing a state as viewed from the front. Figure 4 is a diagram showing a state as viewed permeability types of head-mounted image display device 3 from above. Figure 5 is a diagram schematically illustrating a functional configuration of a light-shielding type head-mounted image display apparatus 1 shown in FIGS. Figure 6 is a diagram of a functional configuration schematically showing the permeability of the types of head-mounted image display apparatus 3 shown in FIGS. Figure 7 is a diagram illustrating a display interval of the inner image and outer image. Figure 8 is a diagram showing a configuration example of a virtual image-real image optical unit 612. Figure 9 is the virtual-real image optical unit 612 shown in FIG. 8 is a diagram showing the polarization characteristics of the display light immediately after emitted from the display panel 611. 10, in the virtual-real image optical unit 612 shown in FIG. 8, the optical axis azimuth is a diagram showing the polarization characteristics of the display light passing through the wave plate 802 for 2 minutes set at 0 degrees. Figure 11 is the virtual-real image optical unit 612 shown in FIG. 8 is a diagram showing the polarization characteristics of the display light passing through the wave plate 802 for 2 minutes by tilting the optical axis azimuth 45 degrees. 12, in the virtual-real image optical unit 612 shown in FIG. 8 is a diagram showing the polarization characteristics of the display light transmitted through the polarizing beam splitter 804. 13, the virtual-real image optical unit 612 shown in FIG. 8, and is reflected and condensed by the concave mirror 806 is a diagram showing the polarization characteristics of the display light transmitted through the polarizing beam splitter 804. Figure 14 is a diagram showing an example of the operation of the optical axis orientation of the wave plate 802 for 2 minutes. Figure 15 is a diagram showing a display operation of the display panel 611 in synchronization with the operation of the optical axis orientation of the half-wave plate 802. Figure 16 is an optical axis direction of wave plate 802 half a diagram showing a state set in the middle of the angle θ of 0 degrees and 45 degrees. Figure 17 is a diagram showing the S-polarized light component which is emitted when an optical axis direction of wave plate 802 half was set at an angle theta. Figure 18 is a diagram illustrating a P-polarized light component which is emitted when an optical axis direction of wave plate 802 half was set at an angle theta. Figure 19 is a diagram P-polarized light component showing a state that has been converted through the wave plate 805 quarter into circularly polarized light shown in FIG. 18. Figure 20 is a diagram circularly polarized light showing a state that has been converted through the wave plate 805 of the quarter to S-polarized light shown in FIG. 19. Figure 21 is a diagram showing another configuration example of a virtual image-real image optical unit 612. Figure 22 is a diagram showing another configuration example of the virtual-real image optical unit 612. Figure 23 is a state transition diagram showing an operation example of the head-mounted image display apparatus 1 or 3 according to an instruction from the user via the input operation unit 502. Figure 24 is a flowchart showing an operation example of the head-mounted image display apparatus 1 or 3 according to the environment information. Figure 25 is a flowchart showing an operation example of the head-mounted image display apparatus 1 or 3 according to the user status information. Figure 26 is a diagram showing a display example of the icon representing the state (eye opening degree) closing the eyelids. Figure 27 is a diagram showing a display example of icons representing the direction (direction of the pupil) of the line of sight. Figure 28 is a diagram illustrating a method of determining mental state of the user in accordance with the blink frequency and blink time per hour were detected (wake / sleepiness / concentration) by the eye electrostatic system. Figure 29 is an illustrated diagram of method of determining the mental state of the user in accordance with the blink interval and blink time detected (wake / sleepiness / concentration) by the eye electrostatic system. Figure 30 is a diagram illustrating a method of determining mental state of the user in accordance with the blink interval and blink time were detected by imaging method (wake / sleepiness / concentration). Figure 31 is a diagram showing a state of controlling to keep the character information 3101 displayed in the outer image horizontally. Figure 32 is a diagram showing a state of controlling so as to keep the position of the mapped inside image or outside the image in the real space. Figure 33 is a flowchart showing an operation example of the head-mounted image display apparatus 1 or 3 according to the content information. Figure 34 is a diagram showing a state in which projects the display image of the head-mounted image display apparatus 1 or 3 to the wall.
It will be described in detail embodiments of the technology disclosed herein with reference to the drawings.
As already mentioned, the head-mounted image display apparatus of the type of the light-shielding (e.g., Patent Document 1 see) classified into transmissive type (e.g., see Patent Document 2) be able to. Technology disclosed herein may be applied to any of these types.
FIG. 1, a user wearing light-shielding types of head-mounted image display apparatus 1 shows a state as viewed from the front.
As shown in FIG. 1, the user is uncertain of wearing the head-mounted image display apparatus 1, the right and left eyes are covered directly. The side facing the user's face of the head-mounted image display apparatus 1 main body "inner", defines the side surface of the opposite side is called "outside". When a user wears the head-mounted image display apparatus 1, the inner surface can be only the user observes the other outer surface can be observed from the outside of a person exposed to the outside. A position facing to the left and right eyes of the inner apparatus 1 main body, the inner display unit the user to observe (in FIG. 1 not shown) is disposed. Further, on the outside of the apparatus main body 1, the outer display unit outsiders can be observed is disposed. Inner display unit and the outer display unit, for example, a micro-display such as an organic EL element or a liquid crystal display.
FIG. 2 shows a state as viewed head-mounted image display apparatus 1 of the light-shielding type from above. As shown, the head-mounted image display apparatus 1, the side surface of inner or facing the user's face, with the inner display unit of the left- and right-eye. Inner display unit, for example, a micro-display such as an organic EL element or a liquid crystal display. Display image of the inner display unit is observed to the user as enlarged virtual image by passing through the virtual image optical unit. Further, since the height and eye width of an eye of individual differences in each user, it is necessary to align the user's eye wearing a left and right of the display system. Thus, head-mounted image display apparatus 1, between the display unit for the right eye and the display unit for the left eye is equipped with a pupil distance adjusting mechanism for adjusting the interpupillary distance.
In the example shown in FIGS. 1 and 2, the head-mounted image display device 1 is outside the display portion is disposed at a position on the front and back relationship with the inner display unit, outside display elsewhere part may be arranged. Although head-mounted image display apparatus 1 is provided with a pair of left and right outer display unit may comprise a single or three or more outer display unit.
Although not shown in FIGS. 1 and 2, the head-mounted image display apparatus 1 includes a sound output unit that outputs audio accompanying the picture. Further, head-mounted image display apparatus 1 is provided with mounting portions for mounting the device 1 on the head or face of the user (not shown). For example by a member such as a belt wound around the back of the user's head wearing, it is secured to the head (e.g., already see 2011-48424 Japanese Patent Application No., which is assigned to the present applicant). Although FIGS. 1 and head-mounted image display apparatus 1 shown in FIG. 2 is a binocular type having a display portion on both left and right eyes, as monocular type with a display unit only one left or right it is also possible to configure.
On the other hand, in FIG. 3, the permeability i.e. user wearing the head-mounted image display apparatus 3 types of see-through shows a state as viewed from the front.
In the example shown in FIG. 3, head-mounted image display apparatus 3 is composed of a structure similar to glasses for vision correction. The side facing the user's face of the head-mounted image display apparatus 3 by the "inner", defines the side surface of the opposite side is called "outside". When a user wears the head-mounted image display apparatus 3, the inner surface can be only the user observes the other outer surface can be observed from the outside of a person exposed to the outside. A position opposed to the left and right eyes of a user of the apparatus 3 body is a virtual image optical unit is arranged made of transparent light guiding portion, on the inner side of the virtual image optical unit, an image that the user observes is displayed. Further, on the outside of the virtual image optical unit, an outer image outsiders can be observed is displayed. The virtual image optical unit, for example, is supported by the eyeglass frame-like support.
Figure 4 shows a state in which viewing the head-mounted image display device 3 from above. As shown, the left and right ends of the head-mounted image display device 3, a display panel for each display output image left-eye and right-eye is provided. Each display panel is comprised of micro-displays, such as liquid crystal display or an organic EL device, respectively, and displays and outputs the multiplexed by time division frame inside the image and outside the image of the left- and right-eye, for example. Left and right display images, while being guided to the vicinity of the right and left eyes, respectively, by the virtual image optical unit, are separated inside the image and outside the image. Then, enlarged virtual image of the inner image while being focused on the user's pupil, enlarged virtual image of the outer image is imaged on a screen which is disposed on the outside of the virtual image optical unit. Note that guided the output image of the virtual image optical unit is a display panel, for details of the configuration for forming an enlarged virtual image with separating inside the image and outside the image, it will be described later.
In the example shown in FIGS. 3 and 4, head-mounted image display apparatus 3, the display area outside the image at a position on the front and back relationship are arranged in the display region of the inner image, other location may be displayed outside the image. Moreover, although the head-mounted image display apparatus 3 to display the exterior image pair of left and right location, it may be displayed outside the image in a single or three or more locations.
Although not shown in FIGS. 3 and 4, head-mounted image display apparatus 3 is provided with a sound output unit that outputs audio accompanying the picture. Further, head-mounted image display apparatus 3 is provided with a mounting portion for mounting the device 3 to the head or face of the user. For example, the temple portion which is rotatably supported at each end left and right of the apparatus 3 main body which forms a shape similar to a spectacle frame (not shown) by being caught in the auricle of the left and right user, are mounted on the head that. Further, although FIG. 3 and head-mounted image display apparatus 3 shown in FIG. 4 is a binocular type having a display portion on both left and right eyes, as monocular type with a display unit only one left or right it is also possible to configure.
FIG. 5 shows schematically the functional structure of FIG. 1 and FIG. 2 light-shielding type head-mounted image display apparatus shown in 1. As mentioned above, head-mounted image display apparatus 1 comprises an inner image viewed from the user side who wears the device 1, a function of displaying respectively the outer image viewed from the outside of the device. The following describes each part.
Control unit 501 includes a ROM (Read Only Memory) 501A and RAM (Random Access Memory) 501B. Within ROM501A, stores program codes and various data to be executed by the control unit 501. Control unit 501 by executing the program loaded into RAM501B, starting display control of the outer image viewed from inside the image and outside the user-visible side wearing the device 1, overall the device 1 overall operation to control. As data to be stored in ROM501A, mention may be made of apparatus identification information and other device-specific information for identifying the device 1.
The input operation unit 502, keys and buttons, switches, etc., the user includes one or more operator to perform input operation, accepts the user's instruction via the operation device, and outputs to the control unit 501. The input operation unit 502, a user instruction consisting of a remote control command received by the remote control receiving portion 503 accepts the same manner, and outputs to the control unit 501. Further, the input operation unit 502, myoelectric sensor, through eye movements and blinking operation resulting from condition sensors such as eyes electrostatic sensor (described later), may be accepted for the user's instruction.
In the present embodiment, the control unit 501, in response to an instruction from the user via the input operation unit 502, and an inner image viewed from the user side who wears the device 1, the display operation of the outer image viewed from the outside of the device to control. For example, the control unit 501, in response to an instruction from the user, or outside the image of on / off, display method (the size of the display image, brightness, contrast, hue, display position, such as continuous display or intermittent display), displays to control the information. The control unit 501, an outer image in conjunction or cooperation with the content to be displayed as an inner image, controlled in response to an instruction from the user.
Environmental information acquisition unit 504 acquires information about the environment external of the head-mounted image display apparatus 1, and outputs to the control unit 501. Environmental information acquisition unit 504, environmental information, for example, ambient light intensity, sound intensity, position or location, acquires temperature, weather, time, ambient image, and people are outside. The environmental information acquisition unit 504, in order to obtain these environmental information, and the light amount sensor, a microphone, GPS (Global Positioning System) sensor, a temperature sensor, humidity sensor, clock, image sensor (camera), radiation sensor various environmental sensors, such as may be provided (both not shown in FIG. 5). Environment information acquired from these environmental sensors, for example, is temporarily held in the RAM501B.
In the present embodiment, the control unit 501, according to the environment information environmental information acquisition unit 504 has acquired an inner image viewed from the user side who wears the device 1, the display operation of the outer image viewed from the outside of the device to control. For example, the control unit 501, according to the environment information, and the outside image on / off, display method (the size of the display image, brightness, contrast, hue, display position, such as continuous display or intermittent display), the information to be displayed to control. The control unit 501, an outer image in conjunction or cooperation with the content to be displayed as an inner image, controlled according to the environment information.
The state information acquiring unit 505 acquires information about the viewer's state of mounting the head-mounted image display apparatus 1, and outputs to the control unit 501. State information acquisition unit 505, as the state information, for example, (the presence or absence of the user's installed) user of the work state and the posture of the user's head of user behavior state (during wear, movement such as walking, opening and closing state of the eyelids ), mental state (either immersed or concentrated in the observation of the inner image (or watching, or have a look flow) excitement factor such as alertness, such as feelings and emotions), further obtains the physiological condition to. The state information acquiring unit 505, in order to obtain these state information from the user, and attachment sensor made of a mechanical switch, a gyro sensor, an acceleration sensor, speed sensors, pressure sensors, temperature sensors, perspiration sensors, muscle electrostatic sensor, an electrooculogram sensor may comprise various condition sensors such as EEG sensors (both not shown in FIG. 5). State information obtained from these condition sensors, for example, is temporarily held in the RAM501B.
In the present embodiment, the control unit 501, depending on the current state information of the user status information acquisition unit 505 has acquired an inner image viewed from the user side who wears the device 1, the outer image viewed from the outside of the device to control the display operation. For example, the control unit 501, depending on the user's status information, and the outside image on / off, display method (the size of the display image, brightness, contrast, hue, display position, such as continuous display or intermittent display), displays to control the information (display of icons and character information that represents the direction of the condition and the line-of-sight opening of the eye). The control unit 501, an outer image in conjunction or cooperation with the content to be displayed as an inner image, controlled depending on the user's status information.
The communication unit 506 includes a communication processing with other devices, and performs demodulation and coding and decoding process of the communication signals. For example, the communication unit 506 receives an image signal for displaying the output as an inner image, the image source from an external device (not shown). Of course, the communication unit 506 also differs outer image and inner image may be received from an external device.
Received by the communication unit 506 is demodulated and decoded processed, inside the image or outside the image, or other received data is supplied to the control unit 501. The control unit 501 sends the transmission data to an external device from the communication unit 506.
Configuration of the communication unit 506 is optional. For example, according to the communication standard used in the transmitting and receiving operations of an external device as a communication partner, it is possible to construct a communication unit 506. Communication standard, wired, may be any form of radio. As a communication standard referred to here, mention may be made of MHL (Mobile High-definition Link) and USB (Universal Serial Bus), HDMI (registered trademark) (High Definition Multimedia Interface), Bluetooth (registered trademark) communication, an infrared communication .
Content information acquiring unit 507 acquires information of the image content to be input via the communication unit 506. In the illustrated example, the content information obtaining unit 507 is realized by a program executed in the control unit 501, it is also possible to configure the content information acquisition section 507 as dedicated hardware outside the controller 501. Content information acquiring unit 507, for example, metadata associated with the image content (content title, genre, detailed information, relevant site URL (Uniform Resouorce Locator), etc.) and the total playing time of the image content, the current playback position or remaining playback time, to acquire the content information, such as the presence or absence of parental control and other use restrictions. The content information obtaining unit 507 includes a decoder for decoding the data, such as encoded strings embedded in the image content within the body.
In the present embodiment, the control unit 501, depending on the content information the content information acquisition section 507 has acquired an inner image viewed from the user side who wears the device 1, the display operation of the outer image viewed from the outside of the device to control. For example, the control unit 501, depending on the content information, and the outside image on / off, display method (the size of the display image, brightness, contrast, hue, display position, such as continuous display or intermittent display), information to be displayed ( string such as a URL that provides detailed information and related information of the content, or to control the encoding string was QR code (registered trademark)). The control unit 501, an outer image in conjunction or cooperation with the content to be displayed as an inner image, controlled depending on the content information.
The image processing unit 508 generates an inner image generating unit 508-1 which generate the inner image based on the image signal outputted from the control unit 501, an outer image based on the image signal outputted from the control unit 501 outside includes an image generation unit 508-2, further performs a signal processing such as image quality correction for the generated inside the image and outside the image display panel 511 for the inner image respectively, adapted to the screen of the outer image display panel 512 to convert to the resolution. However, when displaying the same image as the inner image outside, the outer image generation unit 508-2 is omitted. Then, each of the display driving unit 509 and 510, a display panel 511 for the inner image, sequentially selects a pixel outside the image display panel 512 for each row, in line sequential scanning the pixel signals based on signal processed image signals and supplies.
Inner image display panel 511, an outer image display panel 512 is, for example, an organic EL element and composed of a micro-displays, such as liquid crystal displays (display inside the image panel 511 and the outer image display panel 512 is the same material need not). Inner image display panel 511, the inside of the head-mounted image display apparatus 1 (i.e., the side surface of the apparatus 1 main body facing the user's face) is disposed.
The virtual image optical unit 513 is disposed in front of the display surface of the inner image display panel 511. The virtual image optical unit 513 magnifying and projecting the displayed image of the inner image display panel 511, the user is observed as an enlarged virtual image. On the other hand, the display panel 512 for the outer image, outside of the head-mounted image display apparatus 1 (i.e., the opposite side of the apparatus 1 main body and the inner side) is disposed.
Further, as an option, in front of the display surface of the outer image display panel 512 may be disposed a projection optical unit 514. The projection optical unit 514, a real image of the displayed outer image outside the image display panel 512 is enlarged and projected in a head-mounted image display apparatus 1 close to the wall (not shown) and the like. That is, the head-mounted image display apparatus 1 can be utilized as a compact projector. Figure 34 is a display image of the head-mounted image display apparatus 1 shows a state in which projected onto the wall surface (when the inner image and outer image of the same).
Incidentally, in FIG. 5, for simplification of the drawings, but the inner image display panel 511 and the virtual image optical unit 513 not drawn only one, when the head-mounted image display apparatus 1 is binocular is , the display panel 511 and the virtual image optical unit 513 for the inner image is provided for each the left and right eyes.
Further, in FIG. 6, the functional configuration of FIG. 3 and permeability of types of head-mounted image display apparatus 3 shown in FIG. 4 is schematically shown. As mentioned above, head-mounted image display device 3 comprises an inner image viewed from the user side wearing the device 3, a function of displaying respectively the outer image viewed from the outside of the device. Are denoted by the same reference numerals refer to like functional elements and head-mounted image display apparatus 1 shown in FIG. Hereinafter, mainly described different functional elements as head-mounted image display device 1.
Image multiplexing unit 609, a frame of the inner image and outer images generated respectively inside image generation unit 508-1 and the outer image generation unit 508-2, to display output in a single display panel 611, multiplexing processing. A method of multiplexing is optional, is described as time division multiple operation in the following. For example, as shown in FIG. 7, the image multiplexing unit 609, per inner image frame N Like continuous on the time axis, inserting one outer picture frame. Luminance ratio of the case, the inner image and outer image (or, the sharpness of the image), simply the N: 1 and becomes (however, it is assumed that the same inner image and outer image brightness of a respective time of output) . Here, N is a positive integer of 1 or more (in the illustrated example, N = 4). When the value of N increases, the outer image finally displayed output is more bright and clear images. However, when displaying the same image as the inner image outside the multiplexing process in the image multiplexing unit 609 is omitted.
Permeable That see-through type head-mounted image display apparatus 3, the incident as a display system of the image, a display panel 611 for displaying by multiplexing inner image and outer image, the display light displayed on the display panel 611 to separates the inner image and outer image, guides the enlarged virtual image of the inner image the pupil of the observer, and a virtual-real image optical unit 612 for guiding the real image of the outer image to the outside.
Display panel 611, for example, an organic EL (Electro Luminescence) display, inorganic EL display, a liquid crystal display: consisting (LCD Liquid Crystal Display) microdisplay such. The following description assumes that applied to the display panel 611 of the liquid crystal display for outputting image light comprising linearly polarized light. The display driving unit 610 sequentially selects the pixels of the display panel 611 for each row, and line-sequential scanning, supplies a pixel signal based on the signal processed image signals.
Virtual-real image optical unit 612 separates the inner image and outer image multiplexed example optical effect, provided the enlarged virtual inner image while imaging the user's pupil, a real image of the outside image outside been screen (see FIG. 4) forms an image on.
As options, in front of the screen real image of the outside image is projected may be further provided a projection optical unit 514. The projection optical unit 514, a real image of the displayed outer image outside the image display panel 512, a head-mounted close to the wall surface to the image display device 1 (not shown) for enlarging and projecting the like (supra). That is, the head-mounted image display device 3 can be utilized as a small-sized projector (see FIG. 34).
B. The structure of the optical system in the permeability of the head-mounted image display device
Figure 8 shows a configuration example of a virtual image-real image optical unit 612. Virtual-real image optical unit 612 shown includes a collimation optical unit 801, a half-wave plate: and (Half Wave Plate HWP) 802, a light guide unit (wave guide) 803, distribution inside the light guide portion 803 setting has been polarized beam splitter (Poparization beam splitter: PBS) 804 and a quarter wave plate with (quarter wave plate) 805, a concave mirror (concave mirror) 806 formed on the end face of the light guide portion 803, the outer image It is equipped with a screen 807 for projecting.
Polarizing beam splitter 804 is arranged to be inclined by 45 degrees with respect to the optical axis of the incident light from the display panel 611. Further, quarter-wave plate 805 is arranged so as to be perpendicular to the optical axis of the incident light from the display panel 611. Further, the concave mirror 806 is formed on the end face of the light guide portion 803 on the side opposite to the incident side of the display light from the display panel 611.
Collimating optical unit 801, for example, a convex lens is an optical system to a parallel beam group incident to a light flux emitted from each pixel of the display panel 611, a display panel 611 at a predetermined position of the focal length of the convex lens It is placed. Parallel light pencil groups emitted from the collimating optical unit 801 is incident on the light guide portion 803 through the wave plate 802 for 2 min each. Display light after emitted from the display panel 611 consists of only the S-polarized light component (see Fig. 9).
Between the display panel 611 and the polarization beam splitter 804, wave plate 802 half is arranged. In the example shown in FIG. 8, the display light emitted from the display panel 611, after a parallel light pencil groups by a collimating optics unit 801 and enters the wave plate 802 for 2 minutes. Wave plate 802 half as used herein may be polarizing the optical axis direction dynamically. As an example of the wave plate 802 for 2 minutes possible polarization direction of the optical axis dynamically, mention may be made of light doubler composed of a ferroelectric liquid crystal (e.g., see Patent Document 5).
Here, when the optical axis orientation of the wave plate 802 for 2 minutes to set to 0 degrees, from half-wave plate 802, display light consisting of S-polarized light as it passes through (see FIG. 10). Furthermore, when tilting the optical axis orientation of the wave plate 802 for 2 minutes by 45 degrees, from wave plate 802 of 2 minutes through the display light converted into P-polarized light (see FIG. 11). Therefore, by changing the angle of the optical axis of the wave plate 802 for 2 minutes, the polarization of display light passing through the wave plate 802 for 2 minutes can be switched alternately into S polarized light and P-polarized light. Further, by switching the cycle of the angle setting of the optical axis of wave plate 802 of 2 minutes, it is possible to arbitrarily adjust the duty of the S polarized light and P-polarized light passing through the wave plate 802 for 2 minutes.
The light guide portion 803 is formed of a substantially transparent material. Parallel light pencil groups consisting of S-polarized light or P-polarized light after passing through the wave plate 802 for 2 minutes is incident from one end surface of the light guide portion 803, is totally reflected inside the light guide portion 803, the straight direction propagating.
Polarizing beam splitter 804 on the optical path of the parallel light flux group, it is arranged inclined by 45 degrees with respect to the optical axis of the incident light from the display panel 611. Polarizing beam splitter 804 transmits P-polarized light, having a property of reflecting S-polarized light (known). The parallel light flux group by changing the angle of the optical axis of the wave plate 802 for 2 minutes alternately switched into S-polarized light and P-polarized light, as previously mentioned.
The optical axis azimuth of the wave plate 802 for 2 minutes and set to 0 °, when its passing light becomes S polarized light, it is reflected by the front side of the polarization beam splitter 804 (reflected light A in Fig. 8). The reflected light A, after exiting from the inner side surface of the light guide portion 803, is observed by the observer's pupil as an enlarged virtual image of the display image of the display panel 611. Therefore, at the timing of displaying the inside image on the display panel 611, the optical axis orientation of the wave plate 802 for 2 minutes may be set to 0 degrees.
On the other hand, tilting the optical axis orientation of the wave plate 802 for 2 minutes by 45 degrees, the transmitted light is at the P-polarized light is transmitted through the polarizing beam splitter 804. The transmitted light is once converted into circularly polarized light yet wave plate 805 of the fourth of the rear (see FIG. 12). This circularly polarized light is reflected and condensed by the concave mirror 806 formed on the other end face of the light guide portion 803, by passing through the wave plate 805 for 4 minutes again, and is converted into S-polarized light (Fig. 13 see). The S polarized light when it is reflected by the back side of the polarization beam splitter 804 (reflected light B in FIG. 8), the observer's pupil traveling in opposite or outer.
As shown in FIG. 8, on the opposite side, i.e. outside the observer's pupil, the screen 807 is arranged consisting of scatterers. S-polarized light B reflected by the back side of the polarizing beam splitter 804, after exiting from the outer side surface of the light guide portion 803, is projected onto the screen 807, to project and display a real image of the display image of the display panel 611 Become. Therefore, at the timing of displaying the outer image display panel 611 may be set to the optical axis orientation of the wave plate 802 for 2 minutes to be inclined by 45 degrees.
To ensure a high luminance of the outside image projected on the screen 807, increase the reflectivity of the polarizing beam splitter 804, preferably a reflectance of 90% or more. In addition, without much increasing the size of the screen 807, preferably not more than 2 inches. Furthermore, by increasing the gain of the screen 807, preferably a gain 2 or more (the gain at full spread is 1, and diffusion properties, such as front luminance than when a gain of 1 is equal to or greater than 2-fold).
As the screen 807, a polymer dispersed liquid crystal (Polymer Dispersed Liquid Crystal: PDLC) may be used. PDLC, depending on the applied voltage, it is possible to scatter light by inducing irregular state array of liquid crystal molecules by the action of the polymer network. Thus, displays an outer image display panel 611, and converts the display light by the wave plate 802 for 2 minutes to P-polarized light, only when projecting displaying the real image, which cause scattering effect of the screen 807 Bayoi. On the other hand, displays an inner image in the display panel 611, the display light in half-wave plate 802 to pass through remains S-polarized light, when projecting the pupil of the observer as an enlarged virtual image of the display panel 611, the screen 807 by suppressing the scattering effect, it is possible to take advantage of the permeability i.e. properties as see-through head-mounted image display device 3.
Alternatively, the screen 807, can be used holographic screen. Holographic screen consists interference fringes on the resin film from the laminated structure, diffracted light of a particular wavelength, for light having other wavelengths having the property of the transparent. Thus, it displays an outer image display panel 611, and converts the display light by the wave plate 802 for 2 minutes to P-polarized light, only when projecting displaying the real image, the specific wavelength a display image of the display panel 611 in may be output. On the other hand, displays an inner image in the display panel 611, the display light in half-wave plate 802 to pass through remains S-polarized light, when projecting the pupil of the observer as an enlarged virtual image of the display panel 611, display panel 611 by outputting the display image at other than the specific wavelength, maintaining the transparency of the screen 807, permeability i.e. properties as see-through head-mounted image display apparatus 3 may take advantage of.
As options, the front of the screen 807 may be further provided a projection optical unit 514. The projection optical unit 514, a real image of the displayed outer image on the display panel 611 is enlarged and projected in a head-mounted image display apparatus 3 such as a nearby wall surface (see FIG. 34). That is, the head-mounted image display device 3 can be utilized as a small-sized projector (above).
According to the configuration example shown in FIG. 8, the optical axis orientation of the wave plate 802 for 2 minutes when set to 0 degrees, enlarged virtual image of the display image of the display panel 611 is viewed by the viewer. On the other hand, when tilting the optical axis orientation of the wave plate 802 for 2 minutes by 45 degrees, the real image of the display image of the display panel 611 is projected onto the screen 807. Therefore, in synchronization with the angular change of the optical axis of the wave plate 802 for 2 minutes, by alternately switching the frame of the inner image and outer image display panel 611, as an enlarged virtual image of the observer's pupil to the inner image outputs can be output outside the image as a real image on the screen 807. That is, the optical axis orientation of the wave plate 802 for 2 minutes in synchronism with the period set to 0 degrees, and displays an inner image in the display panel 611, 45 degrees the optical axis azimuth of the wave plate 802 for 2 minutes in synchronism with the period for tilting only, we are sufficient to display an outer image display panel 611.
Figure 14 shows an example of the operation of the optical axis orientation of the half-wave plate 802. In the figure, the section where the optical axis orientation of the wave plate 802 for 2 minutes to set to 0 degrees, the section to be set to 45 degrees alternately switched.
In the section where the optical axis orientation of the wave plate 802 for 2 minutes to set to 0 °, S-polarized light emitted from the display panel 611 is incident on the polarization beam splitter 802, enlarged virtual image of the display image of the display panel 611 by the observer a virtual image display section is observed. Therefore, in synchronization with the virtual image display section may be displayed inside the image display panel 611.
In the section that sets the optical axis orientation of the wave plate 802 for 2 minutes at 45 degrees, P-polarized light emitted from the display panel 611 is incident on the polarization beam splitter 802, a real image of the display image of the display panel 611 screenshot 807 the projected the real image display interval. Therefore, in synchronization with the real image display section may be displayed outside the image display panel 611.
The display panel 611 is the optical axis direction of wave plate 802 of 2 minutes should be displayed inside the image in a virtual image display section to be set to 0 degrees, it is not necessary to light the image throughout interval. FIG. 15 shows a display example of the operation of the display panel 611 in synchronization with the operation of the first wavelength plate 802 optical axis direction of 2 minutes. In the illustrated example, the display panel 611, thereby lighting the virtual image for or inner image only a part of the virtual image display section. The ratio of lighting time of the image in the virtual image display section, it is possible to adjust the brightness of the enlarged virtual image observed by the observation. On the other hand, the outer image is a projection image projected on the screen 807, due to the relatively low brightness, the display unit 611 over the entire real image display section is Shiryo and it is preferable to light the image.
Up to this point, as shown in FIGS. 10 and 11, the optical axis orientation of the wave plate 802 for 2 minutes by switching in two stages 0 and 45 degrees, alternately in a time-division inner image or outside the image It has been described for an example to be displayed in. This contrast, the optical axis orientation of the wave plate 802 of the half by setting the middle of the angle θ of 0 degrees and 45 degrees (see FIG. 16), S-polarized light component and P-polarized component of the emitted light it is possible to adjust the ratio. In changing the optical axis direction of wave plate 802 of 2 minutes may be rotated a half-wave plate 802 itself physically.
S-polarized light component having passed through the wave plate 802 for 2 minutes (see FIG. 17), since it is reflected by the front side of the polarization beam splitter 804 (reflected light A in FIG. 8), the display panel 611 It is viewed by the viewer as an enlarged virtual image of the displayed image. Also, P polarized light component having passed through the wave plate 802 for 2 minutes (see FIG. 18), is transmitted through the polarization beam splitter 804, once the circular polarization behind the quarter wave plate 805 after being converted (see Figure 19), by passing through the wave plate 805 quarter again reflected by the concave mirror 806 and is converted into S-polarized light (see FIG. 20), a polarization beam the splitter 804 from being reflected by the back side (reflected light B in FIG. 8), is projected on a screen 807 as an outer image.
For example, when the optical axis orientation of the wave plate 802 for 2 minutes was set to theta is, S-polarized light component and P ratio of polarized light components, in other words, the luminance ratio of the inner image and outer image cos 2 theta: the sin 2 theta Become. When displaying the same outer image the inner image, by changing the optical axis azimuth θ-wave plate 802 of the 2 minutes, it is possible to adjust the brightness of the outside image.
In the configuration example of a virtual-real image optical unit 612 shown in FIG. 8, since the separation of multiplexed inside the image and outside the image, it is possible to present different outer image and inner image. In contrast, if the presenting common outer image and inner image, it is possible to simplify the configuration of the virtual-real image optical unit 612.
Figure 21 shows another configuration example of the virtual-real image optical unit 612. Forming virtual-real image optical unit 612 shown, a light guide unit 2101, a half mirror 2102 disposed in the light guide portion, a concave mirror 2103 formed on the end face of the light guide part 2101, an inner image a collimation optical unit 2104 of the image, and a screen 2105 for projecting the outer image.
Display light emitted from the display panel 611 is incident from one end surface of the light guide part 2101. Half mirror 2102 is arranged to be inclined by 45 degrees with respect to the optical axis of the incident light from the display panel 611, thereby reflecting part of the incident display light, it transmits part. Intensity of the reflected light and transmitted light are substantially equal, may be adjusted, for example, as the intensity towards the reflected light increases.
Display light reflected by the front side of the half mirror 2102 (reflected light A in FIG. 21), after exiting from the inner side surface of the light guide portion 2101 is condensed by the collimating optical portion 2104, a display panel 611 displaying images It is of observed in the observer's pupil as an enlarged virtual image.
The light passing through the other hand half mirror 2102 is reflected and condensed by the concave mirror 2103 formed on the other end face of the light guide part 2101, it is reflected by the back side of the half mirror 2102 (in FIG. 21 reflected light B). The reflected light B, after exiting from the outer side surface of the light guide portion 2101, is projected onto the screen 2105, the projecting view a real image of the display image of the display panel 611.
To ensure a high luminance of the outside image projected on the screen 2105, to increase the reflectivity of the half mirror 2102, preferably reflectivity of 90% or more. In addition, without much increasing the size of the screen 2105, preferably not more than 2 inches. Furthermore, by increasing the gain of the screen 2105, preferably gain 2 or more (the gain at full spread is 1, and diffusion properties, such as front luminance than when a gain of 1 is equal to or greater than 2-fold).
Further, FIG. 22 shows still another example of the configuration of a virtual-real image optical unit 612. Virtual-real image optical unit 612 shown, a light guide unit 2201, a half mirror 2202 disposed inside of the light guide part 2201, a plane mirror 2203 formed on the end face of the light guide part 2201, an inner image a collimation optical unit 2204 for imaging, and a projection optical unit 2205 and a screen 2206 for projecting the outer image.
Display light emitted from the display panel 611 is incident from one end surface of the light guide part 2201. Half mirror 2202 is arranged to be inclined by 45 degrees with respect to the optical axis of the incident light from the display panel 611, thereby reflecting part of the incident display light and transmits a part (ibid). Intensity of the reflected light and transmitted light are substantially equal, may be adjusted, for example, as the intensity towards the reflected light increases.
Display light reflected by the front side of the half mirror 2202 (reflected light A in Fig. 22), after exiting from the inner side surface of the light guide portion 2201 is condensed by the collimating optical portion 2204, a display panel 611 displaying images It is of observed in the observer's pupil as an enlarged virtual image.
The light passing through the other hand half mirror 2202, which has been totally reflected by the other end plane mirrors formed on surface 2203 of the light guide part 2201, and further reflected by the back side of the half mirror 2202, the light guide part 2201 emitted from the outer side surface (reflected light B in FIG. 22). The reflected light B is enlarged and projected onto a screen 2206 by the projection optical unit 2205 will display a projected image of the display image.
To ensure a high luminance of the outside image projected on the screen 2206, to increase the reflectivity of the half mirror 2202, together, without much increasing the size of the screen 2206, and, to increase the gain of the screen 2206 that It is preferred (ibid.).
C. Image display to the outside in the head-mounted image display device
Head-mounted image display device can be said to be one of the individual ultimate display. However, the conventional head-mounted image display apparatus is not provided with a means for transmitting information to the outside, around the person, you can not receive any information from the user wearing. In addition, the user himself, once disconnect the equipment, and does not interrupt the viewing of the content, it is impossible to convey the information to the outside.
In contrast, a head-mounted image display apparatus 1 and 3 according to this embodiment, in addition to the inner image viewed from the user side wearing the device, also provided with a function of displaying the outer image viewed from the outside of the device ing. Thus, around the person, through the outer image, and the user himself state, whether the person is doing, what you're watching, it is possible to know the degree of concentration or immersion in view. Also, the user himself, even while wearing device (i.e., while continuing to view the content), through the outer image, and its status, can transmit information about the content being watched.
C-1. Image control in accordance with the user operation
As described above, the control unit 501, in response to an instruction from the user via the input operation unit 502, and an outer image of on / off, the size of the display method (display image, brightness, contrast, hue, display position, such continuous display or intermittent display), controls the information to be displayed. The control unit 501, an outer image in conjunction or cooperation with the content to be displayed as an inner image, controlled in response to an instruction from the user. An example of a display control method of the outer image and inner image according to an instruction from the user, are summarized in Table 1 below.
Figure 23 illustrates in the form of a state transition diagram of an operation example of the head-mounted image display apparatus 1 or 3 according to an instruction from the user via the input operation unit 502.
In the initial state, the head-mounted image display apparatus 1 or 3, the inner image on the outer image off. Here, the display of the outer image is instructed through the input operation unit 502, turns on the outer image, the process proceeds to either side image on state. Further, the display-off outside the image on both sides image on state is instructed, by turning off the outer image returns to the initial state.
Although not shown in FIG. 23, the display-off of the inner image on either side image on state is instructed, turn off the inner image, it becomes only the outer image on state.
In both sides image on state, further in response to an instruction from the user via the input operation unit 502, the switching control of an image to be displayed as an outer image and inner image, the color adjustment of the outer image and inner image, outside the image and inner image brightness control of the display size switching and outside image, performs like movement of the display area of ​​the outer image (entire screen display, one of the left and right display, such as display using a part of the screen).
Further, in the initial state, the display-off of the inner image is instructed through the input operation section 502, turns off the inner image, the process proceeds to either side image off state. Further, the display on the inner image on either side image off state is indicated, turn on interior image returns to the initial state.
The on-state only both side image on state and outside the image outside the image is turned on, as an outer image, the same image as the inner image, or to display the images of different outer dedicated inner image (e.g., watching users inner image when you want to hide or camouflage against surrounding who are displays image content swap as an outer image). Further, it can be displayed as an outside image, environmental sensors (described above) is acquired information of the surrounding environment, such as condition sensors (described above) the user's acquired state. Further, in response to an instruction from the user via the input operation unit 502 (including the movement of the blinking operation and eye detected by the myoelectric sensor and eye electrostatic sensor), the information to be displayed as an outer image, (e.g., slideshow may be so) I switched as. Further, in response to an instruction from the user via the input operation unit 502, and also displays the outer image inside the user during wear may be allowed to check the outer image visually.
The on-state only both side image on state and outside the image outside the image is turned on, the need to constantly i.e. continuous display outside the image not always, intermittent display or may be performed a timer display for displaying the a predetermined time . Control unit 501, in response to an instruction from the user via the input operation unit 502, continuous display, intermittent display, may be made to perform any of the timer display.
In the initial state or both images on state inside the image is on, in response to an input operation for instructing prohibition of input operations, shifts to prohibition state for prohibiting the user input operation on the inner image, further instructs the inhibition release in response to an input operation to, it cancels the prohibition of the user input operation on the inner image. Further, in the binocular image on state outside image is on, in accordance with an input operation for instructing prohibition of input operations, shifts to prohibition state for prohibiting the user input operation with respect to the outer image, further instructs the inhibition release in response to an input operation to, cancels the prohibition of the user input operation to the outside the image.
C-2. Image control according to the ambient environment
Controller 501 according to the environment information environmental information acquisition unit 504 has acquired, to control the inner image viewed from the user side who wears the device 1 or 3, the display operation of the outer image viewed from the outside of the device. An example of a display control method of the outer image and inner image corresponding to environmental information is summarized in Table 2 below.
Figure 24 illustrates an operation example of the head-mounted image display apparatus 1 or 3 according to the environmental information in the form of a flowchart. Illustration of the processing operation starts for example periodically.
First, environmental information acquisition unit 504 acquires the output information of various environmental sensors (described above) as the environment information (step S2401). Then, the control unit 501 analyzes the acquired environment information (step S2402), identifies the ambient environment, to check whether a change in the environment in which to switch the inner image or outside image is generated (step S2403 ).
Then, when a change in the environment to perform the switching of the inner picture or outside the image occurs (Yes in step S2403), the control unit 501 controls the display of the outer image corresponding to the current environment (step S2404).
For example, the environmental information acquisition unit 504, and when the brightness changes in the ambient light is detected based on the position information and an image captured by a camera to the indoor outdoor (or outdoors from indoors) when it has moved is detected or to determine that the clock is day or after sunset according to the current time measured, the control unit 501, to match the current environment, to adjust the brightness level of the outer image.
The control unit 501 according to the current time measured by the clock, may be performed intermittently display or timer display outside the image.
The control unit 501, based on the recognition result of the image captured by the surrounding camera, to determine whether there is a person on the periphery of the head-mounted image display apparatus 1 or 3, in response to a person appears, to start the display of the outside image. Further, the control unit 501, depending on the location (orientation) and distance there are emerging person, to control the display area and the display size of the outer image. The control unit 501 controls the display area and the display size of the outer image in accordance with the number of people who are around. When people is large, since the secret level of the image is assumed to be reduced, the control unit 501 may control the brightness level of the external image according to the number of people who are around.
C-3. Image control corresponding to the user state
Control unit 501, depending on the current state information of the user status information acquisition unit 505 has acquired an inner image viewed from the user side who wears the device 1 or 3, the outer image viewed from the outside of the apparatus 1 or 3 to control the display operation. An example of a display control method of the outer image and inner image in accordance with the state information is summarized in Table 3 below.
FIG 25 illustrates an operation example of the head-mounted image display apparatus 1 or 3 according to the user state information in the form of a flowchart. Illustration of the processing operation starts for example periodically.
First, the state information acquiring unit 505 acquires the output information of various condition sensors (described above) as the status information (step S2501). Then, the control unit 501 analyzes the acquired state information (step S2502), the user's current working state, behavioral state, mental state, and to identify the physiological state, the user status to be notified to the surrounding persons but to check whether or not occurred (step S2503).
Then, when the user status to be notified to the surrounding persons occurs (Yes in step S2503), the control unit 501 controls the display of the outer image corresponding to the user state (step S2504).
For example, the control unit 501, myoelectric sensor and eye electrostatic sensor, based on the output information of the condition sensors, such as cameras and identifies the direction of the eyelid condition and sight, the closed state of the eyelids (eye opening degree) and, an icon representing the direction (direction of the pupil) of the line of sight (e.g., FIG. 26, see FIG. 27) is displayed on the outside image. Also, the icon shown in FIG. 26 and FIG. 27 may be displayed superimposed on the outside image the same as the inner image.
The control unit 501, based on the output information of the state sensor, mental state of the user (excitement level such as if immersed or concentrated in the observation of the inner image, alertness, feelings and emotions, and so forth) specifying the, displaying icons representing the mental state (not shown) outside the image. Furthermore, based on the mental state, or may be speaking to the user, or may be interrupted viewing inside the image, determining a situation or may be disturbed, icon of the situation representation is determined (not shown), or "right now immersive", to display the message text such as "Please do not talk!" on the outside image.
For example, based on the output information of the condition sensors, such as myoelectric sensor and eye electrostatic sensors, it is known to be able to detect blinking behavior. Control unit 501, in accordance with the blink frequency and blink time per hour, it is possible to determine the mental state of the user. Figure 28 illustrates a method of determining mental state of the user in accordance with the blink frequency and blink hourly time detected by the eye electrostatic system (wake / sleepiness / concentration).
The control unit 501, in accordance with the blink interval and blink time, it is possible to determine the mental state of the user. Figure 29 illustrates a method of determining mental state of the user in accordance with the blink interval and blink time detected (wake / sleepiness / concentration) by the eye electrostatic system.
Here, tb blink time detected by the eye electrostatic method, the blink interval is ti. Further, Thb the blink time threshold, Thi the blink interval threshold, as Mi blink interval the maximum threshold, at the sampling number n, the average blink time A (tb), the average blink interval A the (ti), respectively the following formula (1) is calculated as (2).
Then, the above equation (1), (2) average blink time was calculated according to A (tb), on the basis of the average blink interval A (ti), the detection priority order indicated below, to estimate the state.
The imaging method of detecting a blink operation based on an image captured by a camera are also known. FIG 30 illustrates a method of determining mental state of the user in accordance with the blink interval and blink time detected (wake / sleepiness / concentration) by photography method. In shooting mode, the above equation (1), the average blink time A (tb) calculated according to (2), based on the average blink interval A (ti), the detection priority order indicated below, to estimate the state .
In the blink operation, there is a case where it is difficult to distinguish closed eyes and sleep. Closed eyes and sleep is a state that is not both looking at the screen, there is whether or not the user is aware of the display content differences. EEG during the closed-eye can be obtained from the eye electrode. Eyes closed resting alpha wave components from EEG during (frequency 10Hz or so, the amplitude several tens .mu.V) can be observed, alpha wave disappears from the electroencephalogram during sleep.
Based on the above estimation result, whether or talk to user state can be determined as follows.
The control unit 501, based on the output information of the state sensor, when a specific user behavior state, to be compatible with its action state, and displays the outer image and inner image.
Gyro sensor or an acceleration sensor, based on the output information of the state sensor such as a camera, can detect the orientation of the user's head (or head-mounted image display apparatus 1 or 3 user is wearing) .
Head horizontal when it is detected that the inclined (the roll direction), the control unit 501 controls to maintain the character information 3101 displayed in the outer image horizontally (see Figure 31).
When the head is detected and is rotated in the (yaw direction), the control unit 501 to maintain the position of the mapped inside image or outside the image in the real space. For example, an image shown in reference numeral 3201 in FIG. 32, are provided for the display of the inner picture or outside the image, the rotational position of the current head to an area indicated by reference numeral 3202 is displayed. In this case, when the head is rotated in the yaw direction (left), moves the display area indicated by reference numeral 3203. Further, when the head is rotated in the yaw direction (rightward), moves the display area indicated by reference numeral 3204.
Also, a gyro sensor or an acceleration sensor, based on the output information of the state sensor such as a camera, the user still (supine, sitting, including standing), walking, running, the action state such as movement of the mobile it is possible to detect. Control unit 501, the display of the outer image is controlled so as to adapt to such action state.
C-4. Image control in accordance with the display content
Control unit 501, according to the information of the content displayed as an inner image, controls an inner image viewed from the user side who wears the device 1 or 3, the display operation of the outer image viewed from the outside of the device. An example of a display control method of the outer image and inner image corresponding to the content information is summarized in Table 5 below.
Figure 33 illustrates an operation example of the head-mounted image display apparatus 1 or 3 according to the content information in the form of a flowchart. Illustration of the processing operation starts for example periodically.
First, when the content information acquisition section 507 acquires the content information (step S3301), the control unit 501 analyzes the acquired state information (step S3302).
Content information acquiring unit 507, for example, metadata associated with the image content (content title, genre, detailed information, relevant site URL (Uniform Resouorce Locator), etc.) and the total playing time of the image content, the current playback position or remaining playback time, to acquire the content information, such as the presence or absence of parental control and other use restrictions.
Then, the control unit 501 checks whether the acquired content information to be notified about the person (step S3103).
Here, the user when to retrieve the content information to be notified to the surrounding persons (Yes in step S3303), the control unit 501, in response to the content information by the content information acquisition section 507 has acquired, fitted with the device 1 controlling an inner image viewed from the side, the display operation of the outer image viewed from the outside of the device (step S3304).
For example, the control unit 501, depending on the content information, and the outside image on / off, display method (the size of the display image, brightness, contrast, hue, display position, such as continuous display or intermittent display), information to be displayed ( string such as a URL that provides detailed information and related information of the content, or, these strings to control the encoded QR code (registered trademark)). The control unit 501, an outer image in conjunction or cooperation with the content to be displayed as an inner image, controlled depending on the content information.
D. Configuration example of the technology disclosed herein
The technique disclosed herein, it is also possible to adopt the following configurations.
(1) An image display device used by mounting on the head or face of the user, a first display unit for displaying an inner image viewed from the user side, the outer image viewed from the outside of the image display device a second display unit for displaying a control unit for controlling display of the inner image and the outer image,
(2) the display unit, the first display device is arranged in a location visible from the user side to view the inner image, is disposed in a location visible from the outside of the image display device displays the outer image a second display device, image display device according to (1).
(3) The display unit is displayed on the display device and guides the single display device for displaying the inner image and the outer image, the inner image displayed on the display device to a location visible from the user side has been outside the image comprises a light guide portion for guiding the location visible from the outside of the image display apparatus, an image display device according to (1).
(4) further comprising a projection unit that projects toward the outside of the outer image the image display apparatus, an image display device according to (1).
(5) further comprising an input operation unit which the user performs input operation, the control unit, in accordance with an input operation by the user, controls the display of the inner image and the outer image, according to the above (1) image display device.
(6) wherein, in response to the input operation on the input operation unit of the user, the inner image or the outer image display ON / OFF, color adjustment of the inner image or the outer image, the inner image or the brightness adjustment of the outer image, switching the display size of the outer image, or performs display area movement of the outer image, the image display device according to (5).
(7) wherein, in response to the input operation on the input operation unit of the user, as the outer image, the same image as the inner image, the image different from the inner image, information of the surrounding environment, the user displaying at least one of the current state, the image display device according to (5).
(8) wherein the control unit is configured in accordance with the blink operation or an input operation based on the motion of the eyeball user switches the display of the outer image, the image display device according to (5).
(9) wherein, in response to the input operation on the input operation unit of the user, and displays the outer image as the inner image, the image display device according to (5).
(10) wherein, in response to the input operation on the input operation unit of the user, to prohibit the operation with respect to the outer image or the inner image, or to release the prohibition, image according to the above (5) display device.
(11) wherein, in response to the input operation on the input operation unit of the user, the continuous display of the outer image, intermittent display, or to perform any every display of a predetermined time, the above (5) the image display apparatus according.
(12) further comprising an environmental information acquisition unit for acquiring information on the surrounding environment of the image display device described above, on the basis of the surrounding environment, to control the display of the inner image and the outer image, the ( the image display apparatus according to 1).
(13) the control unit, in response to detecting the brightness change of the ambient light to control brightness of the outer image, the image display device according to (12).
(14) the control unit, in response to the lapse of the current time, controls the intermittent display or timer display of the outer image, the image display device according to (12).
(15) the control unit, depending on the number of people who are surrounding the display on / off, brightness level of the outer image display area, controls the display size, the image display device according to (12).
(16) further comprising a status information acquisition unit for acquiring the information about the user's state, the control unit, on the basis of the user's status and controls the display of the inner image and the outer image, the (1) the image display apparatus according to.
(17) the control unit, the current working status of the user, wherein the state information acquiring unit acquires, behavioral state, mental state or physiological state, displayed as the outer image, images according to (16) display device.
(18) the control unit, on the basis of the blinking operation of the information of the user to identify the mental state of the user, controls the display of the outer image in accordance with the mental state, according to the above (16) image display device.
(19) wherein the control unit controls the display of the outer image to fit the current activity state of the user, wherein the state information acquiring unit acquires, the image display device according to (16).
(20) the control unit, in response to said lateral inclination of the user's head, said controls to keep the character information in the outer image horizontally, the image display device according to (16).
(21) the control unit, in response to said rotation of the user's head, said controls to keep the position of the inner image or the outer image, the image display device according to (16).
(22) further comprises the content information obtaining section that obtains content information on the image content to be displayed inside the display unit, the control unit, in response to the content information, controls the display of the outer image, the (1 the image display apparatus according to).
(23) A user of the image display method used by mounting on the head or face portion, a first display step of displaying the interior image viewed from the user side, the outer image viewed from the outside of the image display device image displaying method having a second display step, and a control step for controlling the display of the inner image and the outer picture to see.
(24) A recording medium readable to a computer which records a computer program for controlling an image display device used by mounting on the head or face portion of the user, the computer program the computer, the user a first display unit for displaying the inside image viewed from the side, a second display unit for displaying the outer image viewed from the outside of the image display apparatus, functions as a control unit, which controls the display of the inner image and the outer image make, recording medium.
JP 2012-141461 JP JP 2012-42654 JP JP 2000-75815 JP JP 2007-251331 JP U.S. Patent No. 6198523 Publication
Head-mounted image display device, can be classified as Type permeability and the light-shielding type, the technology disclosed herein can be applied to any of these types. Further, head-mounted image display apparatus includes a binocular type having a display portion on both left and right eyes, can be classified into monocular type with a display unit only one left or right, hereby technology disclosed in writing may be applied to any of these types. Further, head-mounted image display apparatus, it may be a glasses-type or ear-hook type is a matter of course.
1 ... head-mounted image display apparatus (light-shielding type)
3 ... head-mounted image display device (transmissive type)
501 ... control unit, 501A ... ROM, 501B ... RAM
502 ... input operating unit, 503 ... remote control receiving portion 504 ... environment information acquisition unit, 505 ... status information acquisition unit 506 ... communication unit, 507 ... content information obtaining unit 508 ... image processing unit 508-1 ... inner image generation unit, 508 -2 ... outer image generating unit 509, 510 ... display driving unit 511, 512 ... display panel 513 ... virtual image optical unit, 514 ... projection optical unit 609 ... image multiplexing unit, 610 ... display driving unit 611 ... display panel, 612 ... virtual-real image optical unit 801 ... collimating optical unit, wave plate 803 ... light guide portion 802 ... 2 minutes, 804 ... polarizing beam splitter 805 ... quarter-wave plate, 806 ... concave mirror 807 ... screen 2101 ... light Department, 2102 ... half-mirror 2103 ... concave mirror, 2104 ... collimating optical unit 2105 ... screen 2201 ... the light guide portion, 202 ... half-mirror 2203 ... concave mirror, 2204 ... collimated optical section 2205 ... the projection optical unit, 2206 ... screen
An image display device used by mounting on the head or face of the user,
A control unit for controlling display of the outer image and the inner image,
The display unit includes a single display device that displays the inner image and the outer image, guides the inner image displayed on the display device to a location visible from the user side, which is displayed on the display device comprising a light guide portion for guiding the outer image in a location visible from the outside of the image display device,
Further comprising a projection unit for projecting the outer image toward the outside of the image display device,
Further comprising an input operation unit which the user performs input operation,
Wherein, in response to an input operation by the user, it controls the display of the inner image and the outer image,
Wherein, in response to the input operation on the input operation unit of the user, the inner image or the outer image display ON / OFF, color adjustment of the inner image or the outer image, the inner image or the outer image brightness control, switching the display size of the outer image, or performs display area movement of the outer image,
The image display apparatus according to claim 4.
Wherein, in response to the input operation on the input operation unit of the user, as the outer image, the same image as the inner image, the image different from the inside image of the surrounding environment information, the user current state to display at least one of, or to display the outer image as the inner image,
Wherein, in response to said blinking operation or an input operation based on the motion of the eyeball user switches the display of the outer image,
Wherein, in response to the input operation on the input operation unit of the user, the continuous display of the outer image, intermittent display, or to perform any given time each of the display,
Further comprising an environmental information acquisition unit for acquiring information on the surrounding environment of the image display device,
The control unit, on the basis of the surrounding environment, to control the display of the inner image and the outer image,
Wherein, in response to detecting the brightness change of the ambient light to control brightness of the outer image,
The image display apparatus according to claim 9.
Wherein, in response to the lapse of the current time, it controls the intermittent display or timer display of the outer image,
Wherein, in response to the number of people are around, the display on / off, brightness level of the outer image display area, controls the display size,
Further comprising a status information acquisition unit that acquires information on the state of the user,
The control unit, on the basis of the user's status and controls the display of the inner image and the outer image,
Wherein, the current working status of the user, wherein the state information acquiring unit acquires, behavioral state, mental state or physiological state, displayed as the outer image,
The image display apparatus according to claim 13.
Wherein the control unit is configured to identify the mental state of the user based on blinking operation information of the user, controls the display of the outer image in accordance with the mental state,
Wherein, in response to said lateral inclination of the user's head, is controlled to keep horizontally the character information in said outer image,
Wherein the control unit is configured in accordance with the rotation of the user's head is controlled so as to keep the position of the inner image or the outer image,
Further comprising a content information obtaining section that obtains content information on the image content to be displayed on the inner display unit,
Wherein, in response to the content information, controls the display of the outer image,
An image display method used by mounting on the head or face of the user,
Image display method having the.
A recording medium readable to a computer which records a computer program for controlling an image display device used by mounting on the head or face portion of the user, the computer program the computer,
Function as the cause, the recording medium.
PCT/JP2013/068002 2012-09-12 2013-07-01 Image display device, image display method, and recording medium WO2014041871A1 (en)
US14/426,203 US9798144B2 (en) 2012-09-12 2013-07-01 Wearable image display device to control display of image
CN201380046423.0A CN104603675B (en) 2012-09-12 2013-07-01 Image display, image display method and recording medium
JP2014535407A JP6350283B2 (en) 2012-09-12 2013-07-01 An image display device and an image display method, and a recording medium
KR1020157001384A KR20150056521A (en) 2012-09-12 2013-07-01 Image display device, image display method, and recording medium
BR112015004958A BR112015004958A2 (en) 2012-09-12 2013-07-01 device and image display method, and recording medium readable by computer.
EP13836975.6A EP2896986A4 (en) 2012-09-12 2013-07-01 Image display device, image display method, and recording medium
US15/708,782 US20180003983A1 (en) 2012-09-12 2017-09-19 Image display device, image display method, and recording medium
US201514426203A A-371-Of-International 2015-03-05 2015-03-05
US15/708,782 Continuation US20180003983A1 (en) 2012-09-12 2017-09-19 Image display device, image display method, and recording medium
WO2014041871A1 true WO2014041871A1 (en) 2014-03-20
CN (1) CN104603675B (en)
BR (1) BR112015004958A2 (en)
CN104951259A (en) * 2014-03-28 2015-09-30 索尼公司 Display device and display control method thereof
JP2016109760A (en) * 2014-12-03 2016-06-20 日本電信電話株式会社 Display device and display method
US20180004478A1 (en) * 2016-06-30 2018-01-04 Sony Interactive Entertainment Inc. Display Screen Front Panel of HMD for Viewing by Users Viewing the HMD Player
JPH05328257A (en) * 1992-05-26 1993-12-10 Olympus Optical Co Ltd Head mount type display device
JP2008065169A (en) * 2006-09-08 2008-03-21 Sony Corp Display device and display method
JP2011048424A (en) 2009-08-25 2011-03-10 Square Enix Co Ltd Server, method and program for community support
JP2012093767A (en) * 2011-11-28 2012-05-17 Olympus Corp Information display apparatus
CN107179607A (en) * 2010-12-24 2017-09-19 奇跃公司 Ergonomic head mounted display device and optical system
2013-07-01 WO PCT/JP2013/068002 patent/WO2014041871A1/en active Application Filing
2013-07-01 US US14/426,203 patent/US9798144B2/en active Active
2013-07-01 BR BR112015004958A patent/BR112015004958A2/en active Search and Examination
2013-07-01 JP JP2014535407A patent/JP6350283B2/en active Active
2013-07-01 CN CN201380046423.0A patent/CN104603675B/en active IP Right Grant
2013-07-01 EP EP13836975.6A patent/EP2896986A4/en active Pending
2013-07-01 KR KR1020157001384A patent/KR20150056521A/en not_active Application Discontinuation
2017-09-13 JP JP2017176226A patent/JP2018026832A/en active Pending
2017-09-19 US US15/708,782 patent/US20180003983A1/en not_active Abandoned
See also references of EP2896986A4 *
EP2896986A1 (en) 2015-07-22
JP2018026832A (en) 2018-02-15
US20150253573A1 (en) 2015-09-10
JP6350283B2 (en) 2018-07-04
EP2896986A4 (en) 2016-05-11
US20180003983A1 (en) 2018-01-04
KR20150056521A (en) 2015-05-26
CN104603675B (en) 2019-01-18
JPWO2014041871A1 (en) 2016-08-18
CN104603675A (en) 2015-05-06
BR112015004958A2 (en) 2017-07-04
US9798144B2 (en) 2017-10-24
Ref document number: 13836975
Ref document number: 2014535407
Ref document number: 20157001384
Ref document number: 14426203
Ref document number: 112015004958