Patent Description:
Virtual reality (VR) is a technology that enables a person to experience life in a computer-generated virtual world. Augmented reality (AR) is a technology that allows virtual images to be mixed with physical environments or spaces in the real world. Near-eye displays in which VR displays or AR displays are implemented focus a virtual image using a combination of optical and stereoscopic images. In such displays, display resolution and processing are important.

Images displayed to a user through a near-eye display apparatus may include a virtual image including highly detailed graphics or a real image. When the near-eye display apparatus processes an image using a software method, image processing may be slow due to a large amount of image processing calculation. To reduce the calculation amount, a foveated display has been developed to provide a high resolution image to a fovea area in the center of a person's view and a low resolution image to the remaining peripheral area.

<CIT> discloses a peripheral display for use with a near-eye display device. The peripheral display is positioned by a near-eye support structure of the near-eye display device for directing a visual representation of an object towards a side of an eye area associated with the near-eye display device. The peripheral display has a lower resolution than a resolution of a front display of the near-eye display device. The peripheral display may include a Fresnel structure. The peripheral display may be used for augmented reality, virtual reality and enhanced vision applications.

<CIT> discloses a system with multiple displays, including a first, three-dimensional display, a second display, and a computer for coordinating displaying a scene using the first display to display a first portion of the scene in three dimensions and using the second display to display a second portion of the scene. Related apparatus and methods are also described.

<CIT> discloses a display for displaying a wide Field of View (FoV) scene including a holographic image within the scene, including a first Spatial Light Modulator (SLM) and an optical system for producing a first holographic image at a center of a displayed scene, and a second image display for producing at least a first additional image adjacent to the first holographic image. In some embodiments an augmented reality display is used for the displaying of the first holographic image at the center of a field of view and the second image adjacent to the first holographic image. In some embodiments a virtual reality display is used for the displaying of the first holographic image near the center of a field of view and the second image adjacent to the first holographic image. Related apparatus and methods are also described.

Provided are foveated display apparatuses.

In accordance with an aspect of the disclosure, a foveated display apparatus includes a first display panel configured to form a two-dimensional (2D) image; a second display panel configured to form a hologram image comprising a three-dimensional (3D) image; a light guide plate configured to transmit the 2D image at a first angle of view to an eye of a user; and a holographic optical element configured to transmit the hologram image at a second angle of view to the eye of the user, the second angle of view being smaller than the first angle of view.

The 2D image may include a peripheral image having a first resolution, the hologram image may include a fovea image having a second resolution that is higher than the first resolution.

The 2D image may include a single depth image, and the hologram image may include a multi depth image.

Preferably, the second angle of view is <NUM> degrees or less.

The light guide plate may include a first surface from which light is emitted; a second surface facing the first surface; and an input coupler on one side of the first surface or the second surface.

The light guide plate may include a first surface from which light is emitted; a second surface facing the first surface; and an output coupler on one side of the first surface or the second surface.

The light guide plate may include a first surface through which light is emitted and a second surface facing the first surface, and the holographic optical element may be provided on the first surface or the second surface.

The second display panel may include a light-emitting diode (LED) display, an organic LED (OLED) display, a liquid crystal display (LCD) display, or a liquid crystal on silicon (LCoS) display.

The foveated display apparatus may further include a light source configured to radiate light to the second display panel; a beam splitter configured to reflect the light radiated from the light source to be incident on the second display panel and transmit the light reflected from the second display panel; and a filter configured to filter the light passing through the beam splitter.

The foveated display apparatus may further include a light path converter configured to direct the light that has passed through the filter to the holographic optical element.

The foveated display apparatus may further include a light source configured to radiate light to the second display panel; and a filter configured to filter the light emitted from the light source that has passed through the second display panel.

The light guide plate may include a first surface; a second surface facing the first surface; and a sub light guide plate on the first surface, wherein the first display panel faces the second surface, and wherein the hologram image formed on the second display panel is transmitted through the sub light guide plate.

The sub light guide plate may include an inclined incidence surface on one side thereof, and the hologram image formed on the second display panel may be incident on the inclined incidence surface.

The holographic optical element may be provided between the light guide plate and the sub light guide plate or may be provided on an exit surface of the sub light guide plate.

The first display panel and the second display panel may be integrally formed, and the foveated display apparatus may further include a light source configured to radiate light to the first display panel and the second display panel; a first beam splitter between the first display panel and the light source; and a second beam splitter between the second display panel and the light source.

A 2D image reflected from the first beam splitter may be incident on the light guide plate, and the hologram image reflected from the second beam splitter may be incident on the holographic optical element.

The 2D image may be transmitted through the light guide plate, and the hologram image may be transmitted to the holographic optical element without passing through the light guide plate.

The 2D image and the hologram image may be combined to display a single image.

The foveated display apparatus may further include at least one of a micro electro-mechanical system (MEMS) mirror, a galvano mirror, a liquid crystal lens, a liquid crystal beam deflector, a geometric phase lens, and a meta lens for controlling an area on which the hologram image is displayed.

The foveated display apparatus may further include a pupil tracker configured to track a user's pupil.

In accordance with an aspect of the disclosure, a foveated display apparatus includes a first optical system that provides a two-dimensional (2D) image to a user's eye via a first optical path; and a second optical system that provides a three-dimensional hologram image to the user's eye via a second optical path different from the first optical path.

The first optical system may provide the 2D image to a peripheral region of a field of view of the user's eye, and the second optical system may provide the hologram image to a fovea region of the field of view of the user's eye.

The first optical system may include a first display panel configured to form the 2D image; and a light guide plate configured to transmit the 2D image to the user's eye.

The second optical system may include a second display panel configured to form the hologram image; and a holographic optical element configured to transmit the hologram image to the user's eye.

The holographic optical element may be positioned on a surface of the light guide plate.

The above and other aspects, features, and advantages of certain example embodiments of the disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:.

In this regard, embodiments may have different forms and should not be construed as being limited to the descriptions set forth herein. Accordingly, example embodiments are merely described below, by referring to the figures, to explain aspects.

A foveated display apparatus according to various example embodiments are described in detail with reference to the accompanying drawings. In the accompanying drawings, wherein like reference numerals refer to like elements throughout. Also, the size of each layer illustrated in the drawings may be exaggerated for convenience of explanation and clarity. Terms such as "first" and "second" are used herein merely to describe a variety of constituent elements, but the constituent elements are not limited by the terms. Such terms are used only for the purpose of distinguishing one constituent element from another constituent element.

An expression used in a singular form in the present specification also includes the expression in its plural form unless clearly specified otherwise in context. Throughout the specification, when a portion "includes" an element, another element may be further included, rather than excluding the existence of the other element, unless otherwise described. Also, the size or the thickness of each layer illustrated in the drawings may be exaggerated for clarity of explanation. Also, in the following description, when a material layer is described to exist on another layer, the material layer may exist directly on the other layer or a third layer may be interposed therebetween. Because a material forming each layer in the following embodiments is an example, other materials may be used therefor.

Terms such as a "portion", a "unit", a "module", and a "block" stated in the specification may signify a unit to process at least one function or operation and the unit may be embodied by hardware, software, or a combination of hardware and software.

The particular implementations shown and described herein are illustrative examples of the disclosure and are not intended to otherwise limit the scope of the disclosure in any way. For the sake of brevity, conventional electronics, control systems, software development and other functional aspects of the systems (and components of the individual operating components of the systems) may not be described in detail. Furthermore, the connecting lines, or connectors shown in the various figures presented are intended to represent functional relationships and/or physical or logical couplings between the various elements. It should be noted that many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device.

The use of terms "a" and "an" and "the" and similar referents in the context of describing the disclosure (especially in the context of the following claims) are to be construed to cover both the singular and the plural.

The steps of all methods described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or language (e.g., "such as") provided herein, is intended merely to better illuminate the disclosure and does not pose a limitation on the scope of the disclosure unless otherwise claimed.

<FIG> illustrates a foveated display apparatus <NUM> according to an example embodiment.

The foveated display apparatus <NUM> may include a first display panel <NUM> forming a two-dimensional (2D) image, a second display panel <NUM> forming a hologram image including a three-dimensional (3D) image, a light guide plate <NUM> transmitting the 2D image, and a holographic optical element <NUM> transmitting the hologram image. For example, the first display panel <NUM> may be included in a first optical system providing the 2D image via a first optical path and the second display panel <NUM> may be included in a second optical system providing the hologram image via second optical path different from the first optical path.

The 2D image may include, for example, an image having a single depth, and the hologram image may include an image having multiple depths. Accordingly, the hologram image may display the 3D image. The first display panel <NUM> and the second display panel <NUM> may be provided to be spaced apart. The first display panel <NUM> may include, for example, an liquid crystal display (LCD), a liquid crystal on silicon (LCoS), an organic light-emitting diode (OLED) display, or an light-emitting diode (LED) display. The first display panel <NUM> may process an image signal to form the 2D image. The second display panel <NUM> may include, for example, an LCD, an LCoS, an OLED display, or an LED display. The second display panel <NUM> may process an image signal to form the 3D image. The second display panel <NUM> may process a computer generated holography (CGH) image signal to form the hologram image, and may process a relatively large amount of image signals and calculations compared to the 2D image.

The 2D image output from the first display panel <NUM> may be incident on the light guide plate <NUM> and may be transmitted to a user's eye E through the light guide plate <NUM>. In the disclosure, an image is assumed to include the concept of an image light that displays a corresponding image. The 2D image may be transmitted with a first angle of view <NUM> to the user's eye E. That is, the 2D image may be transmitted to have a first field of view corresponding to the first angle of view <NUM>. The holographic optical element <NUM> may transmit the hologram image with a second angle of view Θ2 to the user's eye E. That is, the hologram image may be transmitted to have a second field of view corresponding to the second angle of view Θ2. The second angle of view Θ2 may be smaller than the first angle of view θ1, such that the second field of view is smaller than the first field of view. The second angle of view Θ2 may be, for example, greater than <NUM> and equal to or less than <NUM> degrees. When the second angle of view Θ2 is relatively small, and the second display panel <NUM> has the same or similar number of pixels as the first display panel <NUM>, the hologram image displayed with the second angle of view Θ2 may have a relatively high resolution image compared to that of the 2D image. The hologram image may be displayed on a fovea area (i.e., a fovea region of the field of view of the user's eye), and the 2D image may be displayed on a peripheral area of the fovea area (i.e., a peripheral region of the field of view of the user's eye). Therefore, because the hologram image displayed on the fovea area is provided in high resolution, a user may see an image stereoscopically.

The holographic optical element <NUM> may react only to incident light having a specific incidence angle and a specific wavelength. In addition, the holographic optical element <NUM> may output incident light with a narrow angle of view. In addition, the holographic optical element <NUM> may be configured to record hologram information on, for example, a thin film and is very thin, and thus the holographic optical element <NUM> may be easily disposed in an optical device and may reduce the size of the entire system. The holographic optical element <NUM> may be combined in contact with, for example, a first surface <NUM> of the light guide plate <NUM> as shown in <FIG>. That is, the holographic optical element <NUM> may be integrally attached to the light guide plate <NUM>. However, a position of the holographic optical element <NUM> is not limited thereto.

An input coupler <NUM> may be further provided on one side of the light guide plate <NUM>. The input coupler <NUM> may be provided, for example, on a part of one surface of the light guide plate <NUM>. The light guide plate <NUM> may include, for example, the first surface <NUM> through which an image is emitted to the user's eye E, and a second surface <NUM> facing the first surface <NUM>. The input coupler <NUM> may be provided on, for example, a part of the first surface <NUM>. Alternatively, the input coupler <NUM> may be provided on the second surface <NUM>. The input coupler <NUM> may allow light to be coupled into the light guide plate <NUM>. The input coupler <NUM> may guide the light to travel through the inside of the light guide plate <NUM> while the light is totally reflected.

In addition, an output coupler <NUM> may be provided on a different part of the light guide plate <NUM>. The output coupler <NUM> may allow light that has traveled through the light guide plate <NUM> to be output to the outside of the light guide plate <NUM>. The output coupler <NUM> may be provided on, for example, the first surface 131and spaced apart from the input coupler <NUM>. However, the position of the output coupler <NUM> is not limited thereto, and the output coupler <NUM> may be provided on the second surface <NUM>. The input coupler <NUM> and the output coupler <NUM> may include, for example, a diffractive element, and may be provided on the light guide plate <NUM> with an embossed structure or an engraved structure.

A first lens <NUM> may be further provided between the first display panel <NUM> and the input coupler <NUM>. The first lens <NUM> may magnify the 2D image output from the first display panel <NUM>. A second lens <NUM> may be further provided between the second display panel <NUM> and the holographic optical element <NUM>. However, the number or positions of lenses is not limited thereto.

The foveated display apparatus <NUM> may further include a beam steering element <NUM> controlling an area on which the hologram image is displayed. The beam steering element <NUM> may include, for example, at least one of a micro electro-mechanical system (MEMS) mirror, a galvano mirror, a liquid crystal lens, a liquid crystal beam deflector, a geometric phase lens, and a meta lens. The beam steering element <NUM> may be disposed between the second display panel <NUM> and the holographic optical element <NUM>.

The foveated display apparatus <NUM> according to an example embodiment may further include a pupil tracker <NUM> that tracks a user's pupil. The pupil tracker <NUM> may be connected to the first display panel <NUM> and the second display panel <NUM>. Positions of the 2D image formed by the first display panel <NUM> and the hologram image formed by the second display panel <NUM> may change according to a change in a position of the pupil tracked by the pupil tracker <NUM>.

The person's eye has the best resolution in the vicinity of the fovea, where the optic cells are most densely distributed, and the resolution is relatively low in a peripheral area of the fovea (i.e., an area around the fovea). <FIG> illustrates a number of receptors according to an angle of view viewed through a person's eye. Referring to <FIG>, it may be seen that a person uses a fovea area when distinguishing detailed objects through eyes.

Therefore, it is inefficient to increase the resolution with respect to all areas of a person's view. Accordingly, a relatively high resolution image is provided in a foveated region that the person focuses and views sensitively, and a relatively low resolution image is provided in a peripheral region, and thus both a high resolution and a reduction in an image processing computation may be satisfied.

<FIG> illustrates a conceptual operation of the foveated display apparatus <NUM>. The foveated display apparatus <NUM> may measure a person's gaze using the pupil tracker <NUM> and provide a high resolution 3D image to a part of a person's pupil.

Referring to <FIG>, a person's view may include a fovea area corresponding to a human pupil and a peripheral area in the periphery of the fovea area. For example, when time t=t1 and when time t=t2, the fovea area and the peripheral area may be different. In the foveated display apparatus, a relatively high resolution image having a narrow angle of view may be provided to the fovea area and a relatively low resolution image having a wide angle of view may be provided to the peripheral area. The foveated display apparatus <NUM> may provide a high resolution image as an image of the fovea area and a low resolution image as an image of the peripheral area, thereby reducing a computational amount used for image processing. In addition, in an example embodiment, the foveated display apparatus <NUM> may provide a hologram image including a high resolution 3D image in the fovea area, thereby increasing a stereoscopic effect.

In most augmented reality (AR) devices using only a holographic optical element, a parallel light or a light emitted from a display hits the holographic optical element and is diffracted, and then converges to a single point. Accordingly, an eyebox, which is an area in which a user of the AR device may observe a virtual image, is limited to one point. In this case, the user may observe the image only when the eye is positioned exactly at a dot where light is collected, and the image is not visible when the eye rotates or an eyeglass device shakes even a little on the face. In addition, when the eyebox is limited to the dot, because a calibration process is performed that adjusts the dot where the image is visible after wearing glasses to fit a gap of a user's eyes, it is difficult for users with various eye gaps to use one eyeglass device. However, the foveated display apparatus <NUM> according to an example embodiment may solve this problem, and this will be described.

The light guide plate <NUM> may allow a 2D image formed by the first display panel <NUM> to be displayed with the relatively wide second angle of view Θ2. For example, the second angle of view Θ2 may be <NUM> degrees or more. Because the light guide plate <NUM> provides a wide eyebox, the light guide plate <NUM> may be applied to, for example, a glasses-type AR device. However, in the case of an AR device having only a light guide plate, the light guide plate has a depth of an image generally fixed to infinity, and is difficult to independently control the depth of the image. Accordingly, in order to adjust the depth of the image in the AR device having only the light guide plate, for example, an additional optical element is required to bring a virtual image to a specific depth in front and rear of the light guide plate. When the additional optical element is provided as described above, the overall size may increase due to the additional optical element even though a thickness of the light guide plate is thin. In addition, in the case of the light guide plate method, because the depth is fixed, the user may become dizzy in an AR environment due to a discrepancy between the depth of the image generated by the AR device and the depth of the real-world image.

In consideration of this point, the foveated display apparatus <NUM> according to an example embodiment may provide a hologram image so that a user may view a 3D image having multiple depths without dizziness. In the case of the hologram image, the depth of the image may be adjusted by software through calculation of a CGH image, and optical distortion may be solved. However, it is difficult to increase the angle of view of the hologram image, and when the angle of view increases, a viewing area may be limited and a depth expression power may decrease as the size of the eyebox decreases. That is, the hologram image has a trade-off relationship between the angle of view and the size of the eyebox.

Therefore, in the foveated display apparatus <NUM> according to an example embodiment, by using this characteristic of the hologram image, the hologram image is provided to only the fovea area, not the entire field of view, and a 2D image is provided to the remaining peripheral area, and thus the hologram image with a relatively narrow angle of view and the 2D image with a relatively wide angle of view may be combined into a foveated display. The 2D image may be displayed on the peripheral area having the relatively wide angle of view, and the hologram image employing a holographic method may be displayed on the fovea area having the relatively narrow angle of view through the light guide plate <NUM>. For example, the second angle of view Θ2 at which the hologram image is displayed may be <NUM> degrees or less, and the first angle of view <NUM> at which the 2D image is displayed may be <NUM> degrees or more. Even if the second angle of view Θ2 is <NUM> degrees or less, it is possible to display the hologram image having the second angle of view Θ2 on the fovea area, and because the second angle of view Θ2 is relatively small, a high resolution image may be provided. Therefore, the user may view a high resolution image with a sense of depth without an eye strain in the fovea area that a user mainly perceives with the eyes. In addition, because the peripheral area around the fovea area is relatively less sensitive than the fovea area, even if the 2D image with a relatively wide angle of view is displayed on the peripheral area, the user may view the image without recognizing a difference in the resolution. As described above, an image with the difference in the resolution between the fovea area and the peripheral area is displayed, thereby providing a 3D image with a sense of depth while reducing the computational amount of the hologram image. In addition, instead of narrowing the angle of view of the hologram image, the eyebox may increase. Therefore, it is possible to cover different distances between the user's two eyes using a wide eyebox.

In addition, because the light guide plate <NUM> provides a fixed focus while the holographic optical element <NUM> provides various focus information through a CGH computation, it is possible to correct the image in advance in an appropriate shape according to an eye condition of a person with eye disease.

The foveated display apparatus <NUM> may include a first display panel <NUM> forming a 2D image, a second display panel <NUM> forming a hologram image having a 3D image, a light guide plate <NUM> transmitting a 2D image, and a holographic optical element <NUM> transmitting a hologram image.

The 2D image may be displayed on a peripheral area of a fovea area, and transmitted through the light guide plate <NUM> so that the 2D image may enter the user's eyes with a relatively wide first angle of view. The hologram image may be displayed on the fovea area and may enter the user's eyes with a relatively narrow second angle of view through the holographic optical element <NUM>.

A first lens <NUM> may be further provided between the first display panel <NUM> and the light guide plate <NUM>. The first lens <NUM> may include, for example, a Fourier transformation lens. The Fourier transformation lens may transform spatial information of the 2D image formed on the first display panel <NUM> into an angular component, thereby causing image information to be incident into the light guide plate <NUM>. An input coupler <NUM> may be further provided between the first display panel <NUM> and the light guide plate <NUM>. The input coupler <NUM> may couple the 2D image to the light guide plate <NUM>. The light guide plate <NUM> may include a first surface <NUM> through which an image is emitted to the user's eye E, and a second surface <NUM> facing the first surface <NUM>. The input coupler <NUM> may be provided, for example, on the first surface <NUM>.

An output coupler <NUM> may be further provided on another part of the light guide plate <NUM>. The output coupler <NUM> may be provided on the first surface <NUM> to be spaced apart from the input coupler <NUM>. The output coupler <NUM> may guide 2D image information propagated through the light guide plate <NUM> to be emitted toward the user's eye E.

The holographic optical element <NUM> may be provided on one side of the light guide plate <NUM>. The hologram image formed on the second display panel <NUM> may be incident on the holographic optical element <NUM> and reproduced with a narrow angle of view by the holographic optical element <NUM> to enter the fovea area of the user's eye E.

A hologram image reproduction optical system may include a light source <NUM> and a beam splitter <NUM> reflecting light emitted from the light source <NUM> to the second display panel <NUM>. In addition, the hologram image output from the second display panel <NUM> may be transmitted through the beam splitter <NUM>. Here, the second display panel <NUM> may be a reflective panel. For example, the second display panel <NUM> may include an LED display, an OLED display, or an LCoS.

A lens and a filter <NUM> may be further provided on a light path between the beam splitter <NUM> and the holographic optical element <NUM>. The filter <NUM> may be a spatial filter for reproducing a CGH.

For example, a second lens <NUM> may be provided between the beam splitter <NUM> and the filter <NUM>, and a third lens <NUM> and a fourth lens <NUM> may be provided between the filter <NUM> and the holographic optical element <NUM>. However, the number and positions of lenses are not limited thereto.

A light path converter <NUM> may be further provided between the filter <NUM> and the holographic optical element <NUM>. The light path converter <NUM> may be a mirror for converting a path of light. The light path converter <NUM> may be employed for spatial utilization between the second display panel <NUM> and the holographic optical element <NUM>. The holographic optical element <NUM> may be directly coupled to one surface of the light guide plate <NUM>. The holographic optical element <NUM> may be provided to overlap with, for example, the output coupler <NUM>. The holographic optical element <NUM> may be provided in the center of the output coupler <NUM>. The holographic optical element <NUM> may be provided, for example, on a part of a light incident surface on which light is incident on the light guide plate <NUM>.

In <FIG>, the first display panel <NUM> and the second display panel <NUM> may be provided in the same direction with respect to the light guide plate <NUM>. In other words, the first display panel <NUM> and the second display panel <NUM> may both be positioned on the same side of the light guide plate <NUM>. However, the positions of the first display panel <NUM> and the second display panel <NUM> are not limited thereto, and the first display panel <NUM> and the second display panel <NUM> may be provided in opposite directions with respect to the light guide plate <NUM>.

<FIG> illustrates a change in a position of the light source <NUM> in the foveated display apparatus <NUM> shown in <FIG>. In <FIG>, elements using the same reference numerals as in <FIG> have substantially the same configurations and functions, and thus detailed descriptions thereof are omitted.

In a foveated display apparatus 200A, an input coupler 235a may be provided on the second surface <NUM> of the light guide plate <NUM>, and an output coupler 238a may be spaced apart from the input coupler 235a and provided on the second surface <NUM>. In addition, the light source <NUM> radiates light to a second display panel 220a, and the light passes through the second display panel 220a so that a hologram image may be formed. The second display panel 220a may be, for example, a transmissive panel. The second display panel 220a may include an LCD. In an example embodiment, the light source <NUM> and the second display panel 220a may be arranged in a line. A fourth lens <NUM> may be further provided between the light source <NUM> and the beam splitter <NUM>.

In the hologram image includes the spatial filter <NUM> so that a CGH may be reproduced well. The reproduced hologram image may propagate to enter a pupil of a user's eye E with a narrow angle of view by the holographic optical element <NUM> coupled to the light guide plate <NUM>. In an example embodiment, because the first display panel <NUM> and the second display panel <NUM> are independently spaced apart from each other, an optical system displaying a 2D image and an optical system displaying a hologram image may be separately designed.

<FIG> illustrates a foveated display apparatus 200B according to an example embodiment. <FIG> illustrates a change in a configuration of the foveated display apparatus <NUM> illustrated in <FIG>. In <FIG>, elements using the same reference numerals as in <FIG> have substantially the same configurations and functions, and thus detailed descriptions thereof are omitted.

In the foveated display apparatus 200B, the first display panel <NUM> may be provided to face the second surface <NUM> of the light guide plate <NUM>. The input coupler 235a and the output coupler 238a may be provided to be spaced apart from each other on the second surface <NUM> of the light guide plate <NUM>. A sub light guide plate <NUM> may be provided on the first surface <NUM> of the light guide plate <NUM>. The holographic optical element <NUM> may be provided between, for example, the light guide plate <NUM> and the sub light guide plate <NUM>. The holographic optical element <NUM> may be provided in a recess <NUM> provided in the sub light guide plate <NUM>. However, the recess <NUM> may be provided in the light guide plate <NUM>. Alternatively, the holographic optical element <NUM> may be provided on an exit surface <NUM> of the sub light guide plate <NUM>.

The second display panel <NUM> may be provided in a direction inclined with respect to the light guide plate <NUM>. For example, the second display panel <NUM> may be positioned so that a direction A perpendicular to the second display panel <NUM> is not parallel to a direction B perpendicular to the second surface <NUM> of the light guide plate <NUM>. The second display panel <NUM> may be disposed not to overlap with a position of the light guide plate <NUM> and may be disposed in a diagonal direction with respect to the light guide plate <NUM>.

The sub light guide plate <NUM> may be combined in contact with, for example, the light guide plate <NUM>. Further, the sub light guide plate <NUM> may include an inclined surface <NUM> on one side thereof. The inclined surface <NUM> may be provided to correspond to the second display panel <NUM>. For example, the inclined surface <NUM> may be provided parallel to the second display panel <NUM>, but is not limited thereto. The hologram image from the second display panel <NUM> may be incident on the inclined surface <NUM>. In an example embodiment, because the hologram image propagates through the sub light guide plate <NUM>, for example, when the foveated display apparatus 200B is applied to a glasses-type display, the image may be displayed regardless of a shape of a wearer's face. Although the inclined surface <NUM> is provided on one side of the sub light guide plate <NUM>, the configuration of the inclined surface <NUM> is not limited thereto, and may be implemented as a separate prism having an inclined surface.

<FIG> is a diagram illustrating a foveated display apparatus <NUM> according to an example embodiment.

The foveated display apparatus <NUM> may include a display panel <NUM> forming a 2D image and a hologram image including a 3D image, a light guide plate <NUM> transmitting the 2D image, and a holographic optical element <NUM> transmitting the hologram image.

The display panel <NUM> may include a first display panel <NUM> and a second display panel <NUM>. The first display panel <NUM> may form the 2D image, and the second display panel <NUM> may form the hologram image. In an example embodiment, the first display panel <NUM> and the second display panel <NUM> may be integrally provided. Although the first display panel <NUM> and the second display panel <NUM> are integrally provided, the first display panel <NUM> and the second display panel <NUM> may independently process an image. The first display panel <NUM> and the second display panel <NUM> are arranged in parallel. Although <FIG> illustrates an example in which the first display panel <NUM> and the second display panel <NUM> are integrally formed, the first display panel <NUM> and the second display panel <NUM> may be independently provided and may be in parallel arranged in contact with or adjacent to each other.

A light source <NUM> that radiates light to the display panel <NUM> may be provided. A first beam splitter <NUM> and a second beam splitter <NUM> may be provided to face the first display panel <NUM> and the second display panel <NUM>, respectively. The first beam splitter <NUM> may be provided between the first display panel <NUM> and the light source <NUM>, and the second beam splitter <NUM> may be provided between the second display panel <NUM> and the light source <NUM>. The first beam splitter <NUM> and the second beam splitter <NUM> may be arranged in parallel. In addition, the first beam splitter <NUM> and the second beam splitter <NUM> may be combined in contact with each other or disposed adjacent to each other.

The first beam splitter <NUM> and the second beam splitter <NUM> may reflect a partial light and pass the remaining light therethrough. Accordingly, light emitted from the light source <NUM> may pass through the first beam splitter <NUM> and the second beam splitter <NUM> to be respectively incident on the first display panel <NUM> and the second display panel <NUM>. Using the incident light, the 2D image may be formed on the first display panel <NUM> and the hologram image may be formed on the second display panel <NUM>.

The first display panel <NUM> and the second display panel <NUM> may be reflective. Accordingly, the 2D image formed by the first display panel <NUM> may be reflected by the first beam splitter <NUM>, and the hologram image formed by the second display panel <NUM> may be reflected by the second beam splitter <NUM>. The 2D image may be reflected by the first beam splitter <NUM> and incident on the light guide plate <NUM>. A 3D image may be reflected by the second beam splitter <NUM> to travel in a direction opposite to a direction in which the 2D image is reflected as shown in <FIG>.

The light guide plate <NUM> may include a first surface <NUM> through which light is output toward the user's eye E, and a second surface <NUM> facing the first surface <NUM>. An input coupler <NUM> may be provided on one side of the first surface <NUM>, and an output coupler <NUM> may be provided on a different part of the first surface <NUM> to be spaced apart from the input coupler <NUM>. The input coupler <NUM> and the output coupler <NUM> may be provided on the second surface <NUM>. In an example embodiment, the input coupler <NUM> and the output coupler <NUM> may be provided on the light guide plate <NUM> in an engraved structure.

A first lens <NUM> may be further provided between the light source <NUM> and the first beam splitter <NUM> and the second beam splitter <NUM>. For example, the first lens <NUM> may collimate the light emitted from the light source <NUM>.

A second lens <NUM> may be provided between the first beam splitter <NUM> and the input coupler <NUM>. The second lens <NUM> may, for example, magnify the 2D image output from the first beam splitter <NUM> to be incident on the input coupler <NUM>. The 2D image incident on the input coupler <NUM> may be transmitted to the output coupler <NUM> through the light guide plate <NUM> and displayed on a peripheral area of the user's eye E with a wide angle of view.

A light path converter <NUM> converting a light path so that the hologram image output from the second beam splitter <NUM> is directed to the holographic optical element <NUM> may be further provided. The light path converter <NUM> may include, for example, a mirror. In <FIG>, one light path converter <NUM> is provided, but a plurality of light path converters may be provided.

A filter <NUM> may be further provided on the light path between the second beam splitter <NUM> and the holographic optical element <NUM>. The filter <NUM> may include, for example, a Fourier transformation filter. In addition, a third lens <NUM> may be further provided on the optical path between the second light path beam splitter <NUM> and the holographic optical element <NUM>. The third lens <NUM> may focus light so that the light emitted from the second beam splitter <NUM> may pass through the filter <NUM>. As described above, the hologram image output from the second beam splitter <NUM> may be incident on the holographic optical element <NUM> through the filter <NUM> and the light path converter <NUM>. The holographic optical element <NUM> may narrow an angle of view of the hologram image to enter a fovea area of the user's eye E. Accordingly, the hologram image may have a high resolution and be provided in the fovea area of the user's eye E.

In an example embodiment, the first display panel <NUM> and the second display panel <NUM> may be arranged in parallel, and an optical system for the 2D image and an optical system for the hologram image may be compactly arranged.

The foveated display apparatus <NUM> according to an example embodiment may provide a low resolution 2D image having a wide angle of view and a high resolution hologram image having a narrow angle of view sequentially or simultaneously, and a user may view a natural stereoscopic image obtained by combining the 2D image and the hologram image.

A foveated rendering technology may be applied to the foveated display apparatus. Foveated rendering is a virtual reality (VR) or AR technology that follows the movement of a user's pupil and shows an image like a real field of view. When a person is looking at a certain direction or object, a part entering the center of the field of view may be seen at a high resolution and a peripheral part may be seen at a low resolution, thereby reducing the amount of image processing calculation and increasing the image processing speed.

The foveated display apparatuses according to various example embodiments described above may be applied to various devices such as augmented reality glasses, virtual reality (VR) glasses, or head-up displays (HUD). In addition, the foveated display apparatuses according to various example embodiments may be applied to various fields such as entertainment, education, and medicine.

<FIG> illustrates an example of a foveated display apparatus applied to AR glasses <NUM> according to an example embodiment. In the disclosure, AR may mean overlaying and displaying a virtual image on an image of a physical environment space in the real world or a real world object together. In the disclosure, an AR device is a device capable of expressing AR, and may include not only AR glasses in the shape of glasses that a user wears on a face part, but also a head mounted display (HMD) and an AR helmet that the user wears on a head part.

In the disclosure, a real scene is a scene of the real world viewed by an observer or a user through an AR device, and may include a real world object(s). A virtual image may represent an image generated through an optical system. The virtual image may include both static and dynamic images. Such a virtual image is observed together with the real scene, and may be an image showing information about the real world object in the real scene, information about an operation of the AR device, or a control menu. A virtual object may be expressed as a partial area of the virtual image. The virtual object may represent information related to the real world object. The virtual object may include, for example, at least one of letters, numbers, symbols, icons, images, and animations.

<FIG> illustrates an example of a foveated display apparatus applied to a VR glasses <NUM> according to an example embodiment. <FIG> illustrates an example of a foveated display apparatus <NUM> applied to a vehicle according to an example embodiment. The foveated display apparatus may be applied to a head-up display apparatus <NUM> for the vehicle. The head-up display apparatus <NUM> may include the foveated display apparatus <NUM> provided in an area of the vehicle, and at least one light path conversion member <NUM> converting a path of light so that a driver may view an image generated by the foveated display apparatus <NUM>.

<FIG> illustrates an example of a foveated display apparatus <NUM> applied to a mobile device <NUM> according to an example embodiment. The display apparatus described with reference to <FIG> may be applied to the foveated display apparatus <NUM>.

The foveated display apparatus according to various example embodiments may be applied to a tablet or a smart tablet <NUM> as shown in <FIG>. In addition, the foveated display apparatus according to various example embodiments may be applied to a notebook computer <NUM> as illustrated in <FIG>, and may also be applied to a computer monitor <NUM> or a TV as illustrated in <FIG>. In addition, as illustrated in <FIG>, a foveated display apparatus <NUM> according to various example embodiments may be applied to a multi folder display <NUM>. As illustrated in <FIG>, the foveated display apparatus according to various example embodiments may be applied to a large display used in a signage <NUM>, a large billboard, a theater screen, and the like. In addition, as illustrated in <FIG>, a foveated display apparatus according to various example embodiments may be applied to a smart watch <NUM>.

<FIG> is a block diagram of an electronic device <NUM> including a foveated display apparatus according to an example embodiment.

Referring to <FIG>, the electronic device <NUM> may be provided in a network environment <NUM>. In the network environment <NUM>, the electronic device <NUM> may communicate with another electronic device <NUM> through a first network <NUM> (a short-range wireless communication network, etc.), or may communicate with another electronic device <NUM> and/or a server <NUM> through a second network <NUM> (a remote-range wireless communication network, etc.) The electronic device <NUM> may communicate with the electronic device <NUM> through the server <NUM>. The electronic device <NUM> may include a processor <NUM>, a memory <NUM>, an input device <NUM>, a sound output device <NUM>, a display apparatus <NUM>, an audio module <NUM>, a sensor module <NUM>, an interface <NUM>, a haptic module <NUM>, a camera module <NUM>, a power management module <NUM>, a battery <NUM>, a communication module <NUM>, a subscriber identification module <NUM>, and/or an antenna module <NUM>. Some of these elements may be omitted or other elements may be added to the electronic device <NUM>. Some of these elements may be implemented as one integrated circuit. For example, the sensor module <NUM> (a fingerprint sensor, an iris sensor, an illumination sensor, etc.) may be embedded in the display apparatus <NUM> (a display, etc.).

The processor <NUM> may execute software (a program <NUM>, etc.) to control one element or a plurality of other elements (hardware or software elements, etc.) of the electronic device <NUM> connected to the processor <NUM>, and perform various data processing or operations. As part of data processing or operations, the processor <NUM> may load commands and/or data received from another element (the sensor module <NUM>, the communication module <NUM>, etc.) into a volatile memory <NUM>, process the commands and/or data stored in the volatile memory <NUM> and store resultant data in a nonvolatile memory <NUM>. The processor <NUM> may include a main processor <NUM> (a central processing unit, an application processor, etc.) and an auxiliary processor <NUM> (a graphic processing unit, an image signal processor, a sensor hub processor, a communication processor, etc.) operable independently or together therewith. The auxiliary processor <NUM> may use less power than a main processor <NUM> and may perform specialized functions.

On behalf of the main processor <NUM> while the main processor <NUM> is in an inactive state (a sleep state), or together with the main processor <NUM> while the main processor <NUM> is in an active state (an application execution state), the auxiliary processor <NUM> may control functions and/or states related to some of the elements (the display apparatus <NUM>, the sensor module <NUM>, the communication module <NUM>, etc.) of the electronic device <NUM>. The auxiliary processor <NUM> (an image signal processor, a communication processor, etc.) may be implemented as part of other functionally related elements (the camera module <NUM>, the communication module <NUM>, etc.).

The memory <NUM> may store various data required by the elements (the processor <NUM>, the sensor module <NUM>, etc.) of the electronic device <NUM>. The data may include, for example, software (the program <NUM>, etc.) and input data and/or output data for commands related thereto. The memory <NUM> may include the volatile memory <NUM> and/or the nonvolatile memory <NUM>.

The program <NUM> may be stored as software in the memory <NUM> and may include an operating system <NUM>, middleware <NUM>, and/or an application <NUM>.

The input device <NUM> may receive commands and/or data to be used for the elements (the processor <NUM>, etc.) of the electronic device <NUM> from the outside (a user, etc.) of the electronic device <NUM>. The input device <NUM> may include a remote controller, a microphone, a mouse, a keyboard, and/or a digital pen (such as a stylus pen).

The sound output device <NUM> may output a sound signal to the outside of the electronic device <NUM>. The sound output device <NUM> may include a speaker and/or a receiver. The speaker may be used for general purposes such as multimedia playback or recording playback, and the receiver may be used to receive incoming calls. The receiver may be combined as a part of the speaker or may be implemented as an independent separate device.

The display apparatus <NUM> may visually provide information to the outside of the electronic device <NUM>. The display apparatus <NUM> may include a display, a hologram device, or a projector and a control circuit for controlling the device. The display apparatus <NUM> may include any of the foveated display apparatuses described with reference to <FIG>. The display apparatus <NUM> may include a touch circuitry set to sense a touch, and/or a sensor circuit (such as a pressure sensor) set to measure the strength of a force generated by the touch.

The audio module <NUM> may convert sound into an electric signal, or vice versa. The audio module <NUM> may obtain sound through the input device <NUM>, or may output sound through the sound output device <NUM>, and/or a speaker and/or a headphone of another electronic device (the electronic device <NUM>, <NUM>, etc.) directly or wirelessly connected to the electronic device <NUM>.

The sensor module <NUM> may sense an operating state (power, temperature, etc.) of the electronic device <NUM> or an external environmental state (a user state, etc.), and generate an electrical signal and/or a data value corresponding to the sensed state. The sensor module <NUM> may include a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, and/or an illuminance sensor.

The interface <NUM> may support one or more designated protocols that may be used for the electronic device <NUM> to be directly or wirelessly connected to another electronic device (the electronic device <NUM>, <NUM>, etc.) The interface <NUM> may include a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, an SD card interface, and/or an audio interface.

A connection terminal <NUM> may include a connector through which the electronic device <NUM> may be physically connected to another electronic device (the electronic device <NUM>, etc.) The connection terminal <NUM> may include an HDMI connector, a USB connector, an SD card connector, and/or an audio connector (such as a headphone connector).

The haptic module <NUM> may convert an electrical signal into a mechanical stimulus (vibration, movement, etc.) or an electrical stimulus that a user may perceive through a tactile or motor sense. The haptic module <NUM> may include a motor, a piezoelectric element, and/or an electrical stimulation device.

The camera module <NUM> may capture a still image and a video. The camera module <NUM> may include a lens assembly including one or more lenses, image sensors, image signal processors, and/or flashes. The lens assembly included in the camera module <NUM> may collect light emitted from a subject that is a target of image capture.

The power management module <NUM> may be implemented as a part of a power management integrated circuit (PMIC).

The battery <NUM> may supply power to the elements of the electronic device <NUM>. The battery <NUM> may include a non-rechargeable primary cell, a rechargeable secondary cell, and/or a fuel cell.

The communication module <NUM> may establish a direct (wired) communication channel and/or a wireless communication channel between the electronic device <NUM> and other electronic devices (the electronic device <NUM>, the electronic device <NUM>, the server <NUM>, etc.), and support communication through the established communication channel. The communication module <NUM> may include one or more communication processors operating independently of the processor <NUM> (an application processor, etc.) and supporting direct communication and/or wireless communication. The communication module <NUM> may include a wireless communication module <NUM> (a cellular communication module, a short-range wireless communication module, a Global Navigation Satellite System (GNSS) communication module, etc.) and/or a wired communication module <NUM> (a Local Area Network (LAN) communication module, a power line communication module, etc.) Among these communication modules, the corresponding communication module may communicate with other electronic devices through a first network <NUM> (a short-range communication network such as Bluetooth, WiFi Direct, or Infrared Data Association (IrDA)) or through a second network <NUM> (a remote range communication network such as a cellular network, the Internet, or a computer network (LAN), WAN, etc.) These various types of communication modules may be integrated into one element (a single chip, etc.), or may be implemented as a plurality of elements (multiple chips) separate from each other. The wireless communication module <NUM> may use subscriber information (International Mobile Subscriber Identifier (IMSI), etc.) stored in the subscriber identification module <NUM> to confirm and authenticate the electronic device <NUM> in a communication network such as the first network <NUM> and/or the second network <NUM>.

The antenna module <NUM> may transmit signals and/or power to the outside (such as other electronic devices) or receive signals and/or power from the outside. The antenna module <NUM> may include a radiator formed in a conductive pattern on a substrate (a PCB, etc.). The antenna module <NUM> may include one or a plurality of antennas. When the antenna module <NUM> include the plurality of antennas, an antenna suitable for a communication method used in a communication network such as the first network <NUM> and/or the second network <NUM> may be selected from among the plurality of antennas by the communication module <NUM>. Signals and/or power may be transmitted or received between the communication module <NUM> and other electronic devices through the selected antenna. In addition to the antenna, other elements (such as an RFIC) may be included as a part of the antenna module <NUM>.

Some of the elements may be connected to each other through a communication method (a bus, General Purpose Input and Output (GPIO), Serial Peripheral Interface (SPI), Mobile Industry Processor Interface (MIPI), etc.) between peripheral devices and may exchange signals (commands, data, etc.).

The commands or data may be transmitted or received between the electronic device <NUM> and the external electronic device <NUM> through the server <NUM> connected to the second network <NUM>. The other electronic devices <NUM> and <NUM> may be the same type or different types of devices as the electronic device <NUM>. All or some of operations executed by the electronic device <NUM> may be executed by one or more devices of the other electronic devices <NUM>, <NUM>, and <NUM>. For example, when the electronic device <NUM> needs to perform a function or service, instead of executing the function or service by itself, the electronic device <NUM> may request one or more other electronic devices to partly or wholly perform the function or service. One or more other electronic devices that have received a request may execute an additional function or service related to the request, and transmit a result of the execution to the electronic device <NUM>. To this end, cloud computing, distributed computing, and/or client-server computing technology may be used.

The above-described embodiments are merely examples, and various modifications and equivalent other embodiments are possible from those of ordinary skill in the art. Therefore, the true technical protection scope according to embodiments should be determined by the technical idea of the disclosure described in the following claims.

The foveated display apparatuses according to one or more example embodiments may provide a high resolution image having a narrow angle of view and a low resolution image having a wide angle of view using a display panel providing a 2D image and a display panel providing a 3D hologram image.

The foveated display apparatuses according to one or more example embodiments provide a high resolution image only in a fovea area, and thus reducing the amount of calculation for image processing, thereby increasing an image display speed and efficiently providing a 3D image.

Claim 1:
A foveated display apparatus (<NUM>; <NUM>; 200A; 200B; <NUM>; <NUM>; <NUM>; <NUM>) comprising:
a first display panel (<NUM>; <NUM>; <NUM>) configured to form a two-dimensional, 2D, image;
a second display panel (<NUM>; <NUM>; 220a; <NUM>) configured to form a hologram image comprising a three-dimensional, 3D, image;
a light guide plate (<NUM>; <NUM>; <NUM>) configured to transmit the 2D image at a first angle of view (<NUM>) to an eye (E) of a user; and
characterized by comprising a holographic optical element (<NUM>; <NUM>; <NUM>) configured to transmit the hologram image at a second angle of view (θ2) to the eye (E) of the user, the second angle of view (θ2) being smaller than the first angle of view (θ1).