Patent Publication Number: US-2022229297-A1

Title: Display device and method of controlling the same

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2021-0006772, filed on Jan. 18, 2021 in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety. 
     TECHNICAL FIELD 
     Embodiments of the present disclosure relate to a display device and a method of controlling the same. 
     DISCUSSION OF RELATED ART 
     Some display devices may have a form configured to be worn on a user&#39;s body. These electronic devices are commonly called wearable devices. Since wearable devices are directly worn on the body, portability and user accessibility are factors to be taken into consideration when designing and manufacturing these devices. 
     An example of a wearable electronic devices is a head-mounted display (head-mounted electronic device) that can be mounted on a user&#39;s head. Head-mounted displays may be broadly classified as a see-through type that provides augmented reality (AR) and a see-closed type that provides virtual reality (VR). 
     SUMMARY 
     Aspects of the present disclosure provide a display device which can accurately calculate the position of a user&#39;s pupils and prevent interference with the user when worn by the user, and a method of controlling the display device. 
     According to an embodiment, a display device includes a display unit including a first surface that emits first light, a second surface disposed opposite to the first surface, a first transmission area that transmits light incident on the first surface, and a second transmission area that transmits the light incident on the first surface. The second transmission area is spaced apart from the first transmission area. The display device further includes a first lens disposed on the first surface of the display unit and including a first hole overlapping the first transmission area in a thickness direction, a second lens disposed on the first surface of the display unit and spaced apart from the first lens, and including a second hole overlapping the second transmission area in the thickness direction, and a plurality of light sources disposed on the first lens and the second lens. The light sources emit a second light. The display device includes a first camera sensor disposed on the second surface of the display unit and overlapping the first transmission area, and a second camera sensor disposed on the second surface of the display unit and overlapping the second transmission area. 
     According to an embodiment, a method of controlling a display device includes emitting light from a light source disposed on a multi-channel lens toward a user&#39;s pupil, focusing the light, which is reflected from the user&#39;s pupil, passed through a hole formed at a center of the multi-channel lens and then transmitted through a transmission area formed in a display unit, on a camera sensor, and calculating a position of the user&#39;s pupil based on the light received by the camera sensor. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The above and other features of the present invention will become more apparent by describing in detail embodiments thereof with reference to the accompanying drawings, in which: 
         FIG. 1  is a perspective view of a display device according to an embodiment; 
         FIG. 2  is a plan view of a plurality of lenses and a display unit according to an embodiment; 
         FIG. 3  is a perspective view of a first lens according to an embodiment; 
         FIG. 4  is a plan view of a display unit according to an embodiment; 
         FIG. 5  is a cross-sectional view taken along line A-A′ of  FIG. 2  according to an embodiment; 
         FIG. 6  is a plan view of part P 1  of  FIG. 4  according to an embodiment; 
         FIG. 7  is a cross-sectional view taken along line B-B′ of  FIG. 6  according to an embodiment; 
         FIG. 8  is a plan view of part P 1  of  FIG. 4  according to an embodiment; 
         FIG. 9  is a cross-sectional view of a display device according to an embodiment; 
         FIG. 10  is a cross-sectional view of a display device according to an embodiment; 
         FIG. 11  is a cross-sectional view of part P 2  of  FIG. 10  according to an embodiment; and 
         FIG. 12  is a flowchart illustrating a method of controlling a display device according to an embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     Embodiments of the present invention will be described more fully hereinafter with reference to the accompanying drawings. Like reference numerals may refer to like elements throughout the accompanying drawings. 
     It will be understood that when a component such as a film, a region, a layer, or an element, is referred to as being “on”, “connected to”, “coupled to”, or “adjacent to” another component, it can be directly on, connected, coupled, or adjacent to the other component, or intervening components may be present. It will also be understood that when a component is referred to as being “between” two components, it can be the only component between the two components, or one or more intervening components may also be present. It will also be understood that when a component is referred to as “covering” another component, it can be the only component covering the other component, or one or more intervening components may also be covering the other component. Other words used to describe the relationships between components should be interpreted in a like fashion. 
     It will be understood that the terms “first,” “second,” “third,” etc. are used herein to distinguish one element from another, and the elements are not limited by these terms. Thus, a “first” element in an embodiment may be described as a “second” element in another embodiment. 
     It should be understood that descriptions of features or aspects within each embodiment should typically be considered as available for other similar features or aspects in other embodiments, unless the context clearly indicates otherwise. 
     As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. 
     Herein, when two or more elements or values are described as being substantially the same as or about equal to each other, it is to be understood that the elements or values are identical to each other, the elements or values are equal to each other within a measurement error, or if measurably unequal, are close enough in value to be functionally equal to each other as would be understood by a person having ordinary skill in the art. For example, the term “about” as used herein is inclusive of the stated value and means within an acceptable range of deviation for the particular value as determined by one of ordinary skill in the art, considering the measurement in question and the error associated with measurement of the particular quantity (e.g., the limitations of the measurement system). For example, “about” may mean within one or more standard deviations as understood by one of the ordinary skill in the art. Other uses of these terms and similar terms to describe the relationships between components should be interpreted in a like fashion. 
       FIG. 1  is a perspective view of a display device  1  according to an embodiment. 
     The display device  1  to be described below may include a head-mounted display that is mounted on a user&#39;s head to provide a screen on which an image or a video is displayed to the user. 
     Head-mounted displays may include a see-through type head-mounted display that provides augmented reality (AR) based on actual external objects, and a see-closed type head-mounted display that provides virtual reality (VR) to a user with a screen independent from external objects. A see-closed type head-mounted display will be described below as an example. However, embodiments of the present disclosure are not limited thereto. 
     Referring to  FIG. 1 , the display device  1  may include a main frame MF, a display unit DP, multi-channel lenses LS including a first lens LS 1  and a second lens LS 2 , and a cover frame CF. 
     The main frame MF may be worn on a user&#39;s head, and may have a shape corresponding to the shape of the user&#39;s head. 
     The first lens LS 1 , the second lens LS 2 , the display unit DP, and the cover frame CF may be mounted on the main frame MF. The main frame MF may include a space or structure in which the display unit DP, the first lens LS 1 , and the second lens LS 2  can be accommodated. In an embodiment, the main frame MF may further include a structure such as a strap or a band to affix the main frame MF to the user&#39;s head. In an embodiment, a control unit, an image processing unit and a lens housing unit may be further mounted on the main frame MF. 
     The display unit DP displays an image and/or a video. The display unit DP may include a front surface DP_FS on which an image and/or a video is displayed and a rear surface DP_RS which is opposite the front surface DP_FS. Light for providing an image and/or a video may be emitted from the front surface DP_FS of the display unit DP. Thus, the front surface DP_FS may be described as a surface that emits light. As will be described below, the first lens LS 1  and the second lens LS 2  may be disposed on the front surface DP_FS of the display unit DP, and a plurality of camera sensors IFR_C 1  and IFR_C 2  (see  FIGS. 4 and 5 ) may be disposed on the rear surface DP_RS of the display unit DP. 
     The display unit DP may be fixed to the main frame MF or may be detachable from the main frame MF. The display unit DP may be opaque, transparent, or translucent according to the design of the display device  1 , for example, the type of the display device  1 . The display unit DP may include an electronic component such as a display module including a display panel. which will be described below, or may include a display device such as a mobile terminal including a display panel. 
     The display unit DP may include a display panel for displaying an image or a video. 
     The display panel may be a light emitting display panel including light emitting elements. For example, the display panel may be an organic light emitting display panel using organic light emitting diodes that include organic light emitting layers, a micro light emitting diode display panel using micro light emitting diodes, a quantum dot light emitting display panel using quantum dot light emitting diodes that include quantum dot light emitting layers, or an inorganic light emitting display panel using inorganic light emitting elements that include inorganic semiconductors. An organic light emitting display panel will be described below as an example of the display panel. However, embodiments of the present disclosure are not limited thereto. 
     The multi-channel lenses LS may provide light emitted from the display unit DP to a user by allowing the light to pass therethrough. The multi-channel lenses LS may provide a plurality of channels through which the light emitted from the display unit DP passes. The channels may provide the light emitted from the display unit DP to the user by allowing the light to pass through different paths. The light emitted from the display unit DP may be incident on each of the channels. Thus, an enlarged image may be focused on the user&#39;s eyes. 
     In an embodiment, each of the multi-channel lenses LS is exemplified as a focusing optical system interposed between a user&#39;s eye and the front surface DP_FS of the display unit DP. However, embodiments of the present disclosure are not limited thereto. The focusing optical system may also include various types of lenses such as, for example, a convex lens, a concave lens, a spherical lens, an aspherical lens, a single lens, a compound lens, a standard lens, a narrow angle lens, a wide angle lens, a fixed focus lens, and a variable focus lens. 
     The first lens LS 1  and the second lens LS 2  of the multi-channel lenses LS may be disposed on the front surface DP_FS of the display unit DP. The first lens LS 1  and the second lens LS 2  may be arranged on the front surface DP_FS of the display unit DP to correspond to positions of left and right eyes of a user. The first lens LS 1  and the second lens LS 2  may be accommodated in the main frame MF. 
     The first lens LS 1  and the second lens LS 2  may reflect and/or refract light for providing an image and/or a video displayed on the display unit DP and provide the reflected and/or refracted light to the user. As will be described below, a plurality of light sources through IFR_L 8  (see  FIG. 2 ) may be disposed on a side of the first lens LS 1  and a side of the second lens LS 2  which face the main frame MF (the user&#39;s eyes). 
     The cover frame CF may be disposed on the rear surface DP_RS of the display unit DP and may protect the display unit DP. A plurality of infrared cameras IFR_C 1  and IFR_C 2  (see  FIGS. 4 and 5 ) to be described below may be interposed between the cover frame CF and the display unit DP. However, embodiments of the present disclosure are not limited thereto. 
     In an embodiment, the display device  1  may further include a control unit which controls the overall operation of the display device  1 . 
     The control unit may control the operation of at least one of the display unit DP, the infrared light sources IFR_L 1  through IFR_L 8  (see  FIG. 2 ), and the infrared cameras IFR_C 1  and IFR_C 2  (see  FIGS. 4 and 5 ). The control unit may perform operations to analyze images obtained by the infrared cameras IFR_C 1  and IFR_C 2  (see  FIGS. 4 and 5 ), calculate the position of a user&#39;s pupils PP (see  FIG. 5 ), calculate the user&#39;s gaze direction, perform image processing (image mapping) based on the position of the user&#39;s pupils PP (gaze direction), and display an image processed by the display unit DP. 
     The control unit may be implemented as, but is not limited to, a dedicated processor including an embedded processor and/or a general-use processor including a central processing unit or an application processor. 
       FIG. 2  is a plan view of a plurality of lenses and a display unit DP according to an embodiment.  FIG. 3  is a perspective view of a first lens LS 1  according to an embodiment.  FIG. 4  is a plan view of the display unit DP according to an embodiment.  FIG. 5  is a cross-sectional view taken along line A-A′ of  FIG. 2  according to an embodiment.  FIG. 6  is a plan view of part P 1  of  FIG. 4  according to an embodiment.  FIG. 7  is a cross-sectional view taken along line B-B′ of  FIG. 6  according to an embodiment. 
     Referring to  FIGS. 1 and 2 , as illustrated in  FIG. 2 , the first lens LS 1  and the second lens LS 2  may be disposed on the front surface DP_FS of the display unit DP and may be located at positions corresponding to each of a user&#39;s eyes, respectively. 
     For example, the display unit DP may have a substantially rectangular shape that is long (extends lengthwise) in a left-right direction (a horizontal direction in  FIG. 2 ) in a plan view, the first lens LS 1  may be located on one side (a left side in  FIG. 2 ) on the front surface DP_FS of the display unit DP, and the second lens LS 2  may be located on the other side (a right side in  FIG. 2 ) on the front surface DP_FS of the display unit DP. In an embodiment, the first lens LS 1  and the second lens LS 2  may be disposed symmetrically with respect to the center of the display unit DP and may have substantially the same or a similar structure. However, embodiments of the present disclosure are not limited thereto. 
     Referring to  FIGS. 2, 3 and 5 , first lens LS 1  may include a plurality of sub-lenses LS 11  through LS 14  and the second lens LS 2  may include a plurality of sub-lenses LS 21  through LS 24 . Each of the sub-lenses LS 11  through LS 14  may include a lenslet that forms a part of the first lens LS 1 , and each of the sub-lenses LS 21  through LS 24  may include a lenslet that forms a part of the second lens LS 2 . 
     In an embodiment, the first lens LS 1  may include a first sub-lens LS 11 , a second sub-lens LS 12 , a third sub-lens LS 13 , and a fourth sub-lens LS 14 . The second lens LS 2  may include a fifth sub-lens LS 21 , a sixth sub-lens LS 22 , a seventh sub-lens LS 23 , and an eighth sub-lens LS 24 . However, the number of sub-lenses LS 11  through LS 14  and LS 21  through LS 24  is not limited thereto. 
     In an embodiment, the second lens LS 2  is substantially the same as or similar to the first lens LS 1 . Therefore, for convenience of explanation, the first lens L 1  will be mainly described below, and a redundant description of the second lens L 2  will be omitted. 
     Referring to  FIG. 2 , the first lens LS 1  may have a substantially circular shape in a plan view. The first sub-lens LS 11 , the second sub-lens LS 12 , the third sub-lens LS 13 , and the fourth sub-lens LS 14  may be arranged, for example, in a clover shape to surround the center of the circular shape in a plan view. 
     For example, as illustrated in  FIG. 2 , the first sub-lens LS 11 , the second sub-lens LS 12 , the third sub-lens LS 13 , and the fourth sub-lens LS 14  may be disposed on upper right, upper left, lower left, and lower right sides of the center of the first lens LS 1 , respectively. The first sub-lens LS 11 , the second sub-lens LS 12 , the third sub-lens LS 13 , and the fourth sub-lens LS 14  may be integrally connected to each other or may be separated from each other. 
     Part (a) of  FIG. 3  is a perspective view of a side of the first lens LS 1  which faces a user&#39;s eye. part (b) of  FIG. 3  is a perspective view of the other side of the first lens LS 1  which faces the display unit DP. 
     Referring to  FIG. 3 , the first lens LS 1  may have a substantially hemispherical shape. A side of the first lens LS 1  which faces the main frame MF (or a user&#39;s eye) may have a convex shape, and the other side of the first lens LS 1  which faces the display unit DP may have a concave shape. 
     Referring to  FIGS. 2 and 3 , a first hole H 1  penetrating the first lens LS 1  may be disposed at the center of the first lens LS 1 . 
     The first hole H 1  may penetrate the first lens LS 1  in a thickness direction. The thickness direction may be a thickness direction of the display unit DP, a direction in which a user gazes at the display unit DP, and/or a direction substantially perpendicular to the front surface DP_FS of the display unit DP (a rear surface of the display unit DP). In a plan view, the first hole H 1  may be located at the center of the first lens LS 1  and may be surrounded by the first sub-lens LS 11 , the second sub-lens LS 12 , the third sub-lens LS 13 , and the fourth sub-lens LS 14 . As will be described below, light reflected from a user&#39;s pupil PP (see  FIG. 5 ) may pass through the first hole HE For example, the first hole H 1  may have a circular shape in a plan view, and a diameter of the first hole H 1  may be about 0.5 mm. However, the shape and size of the first hole H 1  are not limited thereto. 
     A first center lens CLS 1  may be disposed in the first hole H 1 . The first center lens CLS 1  may be mounted in the first hole HE The first center lens CLS 1  may include various types of lenses such as, for example, a convex lens and a concave lens. The first center lens CLS 1  may cause light IFR (see  FIG. 5 ) reflected from a user&#39;s pupil PP (see  FIG. 5 ) to be focused on a first camera sensor IFR_C 1  by appropriately refracting and/or reflecting the light IFR, thereby reducing the thickness of the display device  1 . However, embodiments of the present disclosure are not limited thereto. In an embodiment, the first hole H 1  may remain empty (e.g., may be filled with air). 
     Still referring to  FIGS. 2 and 3 , like the first lens LS 1  and the second lens LS 2  may have a substantially hemispherical shape. The fifth sub-lens LS 21 , the sixth sub-lens LS 22 , the seventh sub-lens LS 23 , and the eight sub-lens LS 24  of the second lens LS 2  may be arranged, for example, in a clover shape to surround the center of the second lens LS 2  in a plan view. Like the first hole H 1  of the first lens LS 1 , a second hole H 2  penetrating the second lens LS 2  and surrounded by the fifth sub-lens LS 21 , the sixth sub-lens LS 22 , the seventh sub-lens LS 23  and the eighth sub-lens LS 24  may be disposed at the center of the second lens LS 2 . A second center lens CLS 2 , which is substantially the same as or similar to the first center lens CLS 1 , may be disposed in the second hole H 2 , or the second hole H 2  may remain empty (e.g., may be filled with air). 
     Still referring to  FIGS. 2, 3 and 5 , the sub-lenses LS 11  through LS 14  and LS 21  through LS 24  may respectively provide a plurality of channels through which light emitted from the front surface DP_FS of the display unit DP passes. Light emitted from different areas of the front surface DP_FS of the display unit DP may pass through the channels through different paths. The sub-lenses LS 11  through LS 14  and LS 21  through LS 24  may magnify the light emitted from different areas of the front surface DP_FS of the display unit DP to have the same magnification ratio or to have different magnification ratios. The light passing through each of the sub-lenses LS 11  through LS 14  and LS 21  through LS 24  may include a partial video and/or a partial image for forming a complete VR image (or a complete VR video) and may be focused on a user&#39;s pupil PP to provide one complete VR image (or a complete VR video) to the user. 
     For example, as illustrated in  FIG. 5 , the first sub-lens LS 11  may provide a channel through which light IMG 1  emitted from an area of the display unit DP (e.g., an upper part of the display unit DP of  FIG. 5 ) passes, and the fourth sub-lens LS 14  may provide a channel through which light IMG 2  emitted from another area of the display unit DP (e.g., a lower part of the display unit DP of  FIG. 5 ) passes. The above-described areas of the display unit DP may respectively include an area overlapping the first sub-lens LS 11  and an area overlapping the fourth sub-lens LS 14  in the thickness direction. 
     Similarly, in an embodiment, the second sub-lens LS 12  and the third sub-lens LS 13 , like the first sub-lens LS 11  and the fourth sub-lens LS 14 , may respectively provide channels through which light emitted from different areas of the display unit DP pass. 
     In an embodiment, light passing through each of the sub-lenses LS 11  through LS 14  and LS 21  through LS 24  may be provided to a user through two refractions and two reflections. However, embodiments of the present disclosure are not limited thereto. 
     Referring to  FIGS. 2 and 5 , the display device  1  may further include a plurality of light sources IFR_L 1  through IFR_L 8 . 
     The light sources IFR_L 1  through IFR_L 8  may be disposed on the first lens LS 1  and the second lens LS 2 . The light sources IFR_L 1  through IFR_L 8  may be disposed on a convex side of the first lens LS 1  and a convex side of the second lens LS 2  which face the main frame MF (or a user&#39;s eyes). 
     The light sources IFR_L 1  through IFR_L 8  may emit light IFR for detecting the position of a user&#39;s pupils PP (see  FIG. 5 ). The light sources IFR_L 1  through IFR_L 8  may emit, for example, infrared light or visible light. However, embodiments of the present disclosure are not limited thereto. 
     The light sources IFR_L 1  through IFR_L 8  may include a first light source IFR_L 1 , a second light source IFR_L 2 , a third light source IFR_L 3 , a fourth light source IFR_L 4 , a fifth light source IFR_L 5 , a sixth light source IFR_L 6 , a seventh light source IFR_L 7 , and an eighth light source IFR_L 8 . 
     The first light source IFR_L 1 , the second light source IFR_L 2 , the third light source the fourth light source the fifth light source IFR_L 5 , the sixth light source IFR_L 6 , the seventh light source IFR_L 7 , and the eighth light source IFR_L 8  may be disposed on the first sub-lens LS 11 , the second sub-lens LS 12 , the third sub-lens LS 13 , the fourth sub-lens LS 14 , the fifth sub-lens LS 21 , the sixth sub-lens LS 22 , the seventh sub-lens LS 23 , and the eighth sub-lens LS 24 , respectively. 
     As illustrated, in an embodiment, eight light sources IFR_L 1  through IFR_L 8  are disposed on the sub-lenses LS 11  through LS 14  and LS 21  through LS 24 , respectively. However, embodiments of the present disclosure are not limited thereto. For example, according to embodiments, the number of light sources IFR_L 1  through IFR_L 8  may be greater than or less than eight, and the light sources IFR_L 1  through IFR_L 8  may be disposed only on some of the sub-lenses LS 11  through LS 14  and LS 21  through LS 24 . 
     As illustrated in  FIG. 2 , the first light source IFR_L 1 , the second light source IFR_L 2 , the third light source IFR_L 3 , and the fourth light source IFR_L 4  may be located inside a boundary of the first lens LS 1  and may surround the center of the first lens LS 1  and/or the first hole H 1  in a plan view. The fifth light source IFR_L 5 , the sixth light source IFR_L 6 , the seventh light source IFR_L 7 , and the eighth light source IFR_L 8  may be located inside a boundary of the second lens LS 2  and may surround the center of the second lens LS 2  and/or the second hole H 2  in a plan view. 
     As illustrated in  FIG. 5 , in a cross-sectional view, the first light source IFR_L 1  and the fourth light source IFR_L 4  may be disposed on an inclined surface of the first sub-lens LS 11  and an inclined surface of the fourth sub-lens LS 14 , respectively. The inclined surface of the first sub-lens LS 11  and the inclined surface of the fourth sub-lens LS 14  may extend from the display unit DP toward a user&#39;s eyeball EB such that a width of the first lens LS 1  is reduced. The width of the first lens LS 1  may be a width measured in the thickness direction and/or a direction intersecting the user&#39;s gaze direction, for example, a vertical direction in  FIG. 5 . 
     Similarly, in an embodiment, the second light source IFR_L 2  and the third light source IFR_L 3  may be disposed on the second sub-lens LS 12  and the third sub-lens LS 13  in substantially the same or a similar manner as the first light source and the fourth light source IFR_L 4 , respectively. In addition, the fifth light source IFR_L 5 , the sixth light source IFR_L 6 , the seventh light source IFR_L 7 , and the eighth light source IFR_L 8  may be disposed on the fifth sub-lens LS 21 , the sixth sub-lens LS 22 , the seventh sub-lens LS 23 , and the eighth sub-lens LS 24  in substantially the same or a similar manner as the first light source the second light source the third light source IFR_L 3 , and the fourth light source IFR_L 4 , respectively. 
     Referring to  FIGS. 4 and 5 , the display device  1  may further include the camera sensors IFR_C 1  and IFR_C 2 . Although two camera sensors are exemplified as the camera sensors IFR_C 1  and IFR_C 2  according to an embodiment, embodiments of the present disclosure are not limited thereto. 
     The camera sensors IFR_C 1  and IFR_C 2  may be disposed on the rear surface of the display unit DP. The camera sensors IFR_C 1  and IFR_C 2  may include various types of cameras or image sensors capable of detecting light IFR reflected from a user&#39;s pupils PP. The camera sensors IFR_C 1  and IFR_C 2  may capture an image ahead. The position of the user&#39;s pupils PP and the user&#39;s gaze direction may be calculated based on the captured image. In an embodiment, each of the camera sensors IFR_C 1  and IFR_C 2  may be, but is not limited to, an infrared camera. 
     The first camera sensor IFR_C 1  may overlap the first lens LS 1  and the second camera sensor IFR_C 2  may overlap the second lens LS 2 . 
     As illustrated in  FIG. 4 , the first camera sensor and the second camera sensor IFR_C 2  may respectively overlap the center of the first lens LS 1  and the center of the second lens LS 2  in the thickness direction in a plan view. The first camera sensor IFR_C 1  and the second camera sensor IFR_C 2  may be disposed on the rear surface DP_RS of the display unit DP to overlap a first transmission area TR 1  and a second transmission area TR 2  to be described below, respectively. 
     As illustrated in  FIG. 5 , the first camera sensor may overlap the first hole H 1  in the thickness direction in a cross-sectional view. The first camera sensor may overlap a virtual reference line RL passing through the first center lens CLS 1 , the first hole H 1 , and the center of a user&#39;s eyeball EB. Similarly, in an embodiment, the second camera sensor IFR_C 2  may overlap the second hole H 2  in the thickness direction in a cross-sectional view. 
     Still referring to  FIG. 5 , the display device  1  may further include a first sensor lens SL interposed between a camera sensor and the display unit DP. 
     As illustrated in  FIG. 5 , in a cross-sectional view, the first sensor lens SL may be interposed between the first camera sensor IFR_C 1  and the first transmission area TR 1 , which will be described further below. The first sensor lens SL may refract and/or reflect light IFR passing through the first transmission area TR 1  to enter the first camera sensor IFR_C 1 . The first sensor lens SL may include various types of lenses such as, for example, a convex lens and a concave lens. 
     The first sensor lens SL may include a focusing optical system which may focus infrared light reflected from a user&#39;s pupil PP onto the first camera sensor IFR_C 1 . The first sensor lens SL may also form the focusing optical system together with the first center lens CLS 1 . Similarly, in an embodiment, the display device  1  may further include a second sensor lens disposed between the second camera sensor IFR_C 2  and the second transmission area TR 2  and focusing light IFR passing through the second transmission area TR 2 , which will be described further below, onto the second camera sensor IFR_C 2 . 
     Referring to  FIGS. 4, 5, 6 and 7 , the display device  1  may include a plurality of transmission areas including the first transmission area TR 1  and the second transmission area TR 2 . Although two transmission areas TR 1  and TR 2  are exemplified as the transmission areas TR 1  and TR 2 , embodiments of the present disclosure are not limited thereto. 
     The transmission areas TR 1  and TR 2  may transmit most of the light incident on the front surface DP_FS of the display unit DP. The transmission areas TR 1  and TR 2  may selectively transmit light in a specific wavelength region, for example, infrared light. However, embodiments of the present disclosure are not limited thereto. Light IFR transmitted through the transmission areas TR 1  and TR 2  may be sent to the camera sensors and IFR_C 2  disposed on the rear surface DP_RS of the display unit DP. 
     The first transmission area TR 1  may overlap the first lens LS 1  and the second transmission area TR 2  may overlap the second lens LS 2 . 
     As illustrated in  FIG. 4 , in a plan view, the first transmission area TR 1  and the second transmission area TR 2  may overlap the center of the first lens LS 1  and the center of the second lens LS 2  in the thickness direction, respectively. 
     As illustrated in  FIG. 5 , the first transmission area TR 1  may overlap the first camera sensor IFR_C 1  and the first hole H 1  (the first center lens CLS 1 ) in the thickness direction in a cross-sectional view. The first transmission area TR 1  may be interposed between the first hole H 1  (the first center lens CLS 1 ) and the first camera sensor IFR_C 1 . The first transmission area TR 1  may overlap the virtual reference line RL passing through the first camera sensor IFR_C 1 , the first hole H 1  (the first center lens CLS 1 ), and the center of a user&#39;s eyeball EB. Similarly, in an embodiment, the second transmission area TR 2  may overlap the second camera sensor IFR_C 2  and the second hole H 2  (the second center lens CLS 2 ) in the thickness direction in a cross-sectional view. 
     In an embodiment, the transmission areas TR 1  and TR 2  have substantially the same or a similar structure. Therefore, for convenience of explanation, the first transmission area TR 1  will be mainly described below, and a redundant description of the second transmission area TR 2  will be omitted. 
     Referring to  FIG. 6 , the display unit DP may include a plurality of pixels PX. In an embodiment, at least some of the pixels PX are not disposed in the first transmission area TR 1 . That is, in an embodiment, at least some of the pixels PX may be removed from the first transmission area TR 1 . 
     In an embodiment, the pixels PX are not disposed in the first transmission area TR 1 , and a transmission hole TH penetrating at least some of a plurality of layers constituting the display unit DP may be formed. The transmission hole TH may denote a window through which light in a specific wavelength region, for example, infrared light, passes. In  FIG. 6 , the transmission hole TH may have a circular shape in a plan view. However, embodiments of the present disclosure are not limited thereto. 
     Referring to  FIG. 7 , the display unit DP may include the display panel as described above. 
     The display panel may include a substrate SUB, a plurality of conductive layers disposed on the substrate SUB, a plurality of insulating layers  111 ,  112 ,  113 , VIA 1  and VIA 2  insulating the conductive layers from each other, and a light emitting layer EL. 
     The substrate SUB may be disposed over a display area and a non-display area. The substrate SUB may be a flexible substrate SUB including a flexible material such as flexible glass or polyimide. In  FIG. 7 , the substrate SUB is illustrated as a single layer. However, embodiments of the present disclosure are not limited thereto. In an embodiment, the substrate SUB may include a plurality of layers, for example, a plurality of polyimide substrate layers and at least one barrier layer interposed therebetween. 
     A buffer layer BF may be disposed on the substrate SUB. The buffer layer BF may prevent penetration of moisture and oxygen from outside of the display unit through the substrate SUB. The buffer layer BF may include any one of, for example, a silicon nitride (SiNx) layer, a silicon oxide (SiO 2 ) layer, and a silicon oxynitride (SiOxNy) layer. 
     A semiconductor layer  105  may be disposed on the buffer layer BFF. The semiconductor layer  105  forms a channel of a thin-film transistor (TFT). The semiconductor layer  105  may be disposed in each pixel PX of the display area and, in some cases, may also be disposed in the non-display area. The semiconductor layer  105  may include polycrystalline silicon. 
     A first insulating layer  111  may be disposed on the semiconductor layer  105 . The first insulating layer  111  may be disposed over the entire substrate SUB. The first insulating layer  111  may be a gate insulating film having a gate insulating function. 
     The first insulating layer  111  may include, for example, a silicon compound, a metal oxide, or the like. For example, the first insulating layer  111  may include silicon oxide, silicon nitride, silicon oxynitride, aluminum oxide, tantalum oxide, hafnium oxide, zirconium oxide, titanium oxide, and the like. These materials may be used alone or in combination with each other. 
     A first conductive layer among the plurality of conductive layers may be disposed on the first insulating layer  111 . The first conductive layer ay include a gate electrode GE of the thin-film transistor TFT and a first electrode CE 1  of a storage capacitor. In an embodiment, a plurality of gate electrodes GE may be disposed with at least one insulating layer interposed between them. 
     The first conductive layer may include one or more metals selected from, for example, molybdenum (Mo), aluminum (Al), platinum (Pt), palladium (Pd), silver (Ag), magnesium (Mg), gold (Au), nickel (Ni), neodymium (Nd), iridium (Ir), chromium (Cr), calcium (Ca), titanium (Ti), tantalum (Ta), tungsten (W), and copper (Cu). The first conductive layer  120  may be a single layer or a stacked layer made of any one or more of the above-described materials. 
     A second insulating layer  112  may be disposed on the first conductive layer. The second insulating layer  112  may insulate the first conductive layer from a second conductive layer among the plurality of conductive layers. The second insulating layer  112  may be selected from the example materials described above with reference to the first insulating layer  111 . 
     The second conductive layer may be disposed on the second insulating layer  112 . The second conductive layer may include a second electrode CE 2  of the storage capacitor. The material of the second conductive layer may be selected from the example materials of the first conductive layer described above. The first electrode CE 1  of the storage capacitor and the second electrode CE 2  of the storage capacitor may form a capacitor through the second insulating layer  112 . 
     A third insulating layer  113  may be disposed on the second conductive layer. The third insulating layer  113  may include at least one of the example materials of the first insulating layer  111  described above. In some embodiments, the third insulating layer  113  may include an organic insulating material. The organic insulating material may be selected from example materials of a first via layer VIA 1  to be described below. 
     A third conductive layer may be disposed on the third insulating layer  113 . The third conductive layer may include a source electrode SE, a drain electrode DE, a high potential voltage electrode ELVDDE, and a signal line pad. 
     The third conductive layer may include at least any one of, for example, molybdenum (Mo), aluminum (Al), platinum (Pt), palladium (Pd), silver (Ag), magnesium (Mg), gold (Au), nickel (Ni), neodymium (Nd), iridium (Ir), chromium (Cr), calcium (Ca), titanium (Ti), tantalum (Ta), tungsten (W), and copper (Cu). The third conductive layer may be a single layer made of any one of the above example materials. However, embodiments of the present disclosure are not limited thereto. For example, in an embodiment, the third conductive layer may be a stacked layer. For example, the third conductive layer may have a stacked structure of Ti/Al/Ti, Mo/Al/Mo, Mo/AlGe/Mo, or Ti/Cu. In an embodiment, the third conductive layer may include Ti/Al/Ti. 
     The first via layer VIA 1  may be disposed on the third conductive layer. The first via layer VIA 1  may include an organic insulating material. The organic insulating material may include at least any one of, for example, polyacrylates resin, epoxy resin, phenolic resin, polyamides resin, polyimides resin, unsaturated polyesters resin, polyphenylenethers resin, polyphenylenesulfides resin, and benzocyclobutene (BCB). 
     A fourth conductive layer may be disposed on the first via layer VIAL The forth conductive layer may include a data line DL, a connection electrode CNE, and a high potential voltage line ELVDDL. The data line DL may be electrically connected to the source electrode SE of the thin-film transistor TFT through a contact hole penetrating the first via layer VIAL The connection electrode CNE may be electrically connected to the drain electrode DE of the thin-film transistor TFT through a contact hole penetrating the first via layer VIAL The high potential voltage line ELVDDL may be electrically connected to the high potential voltage electrode ELVDDE through a contact hole penetrating the first via layer VIAL The fourth conductive layer may include a material selected from the example materials of the third conductive layer. 
     A second via layer VIA 2  may be disposed on the fourth conductive layer. The second via layer VIA 2  may include at least one of the example materials of the first via layer VIA 1  described above. 
     An anode ANO may be disposed on the second via layer VIA 2 . The anode ANO may be electrically connected to the connection electrode CNE through a contact hole penetrating the second via layer VIA 2 . 
     A bank layer BANK may be disposed on the anode ANO. The bank layer BANK may include a contact hole exposing the anode ANO. The bank layer BANK may be made of an organic insulating material or an inorganic insulating material. For example, the bank layer BANK may include at least one of a photoresist, polyimide resin, acrylic resin, a silicone compound, and polyacrylic resin. As illustrated in  FIG. 7 , a pixel PX may be defined by an opening of the bank layer BANK which exposes the light emitting layer EL. 
     The light emitting layer EL may be disposed on an upper surface of the anode ANO and in the opening of the bank layer BANK. A cathode CAT is disposed on the light emitting layer EL and the bank layer BANK. The cathode CAT may be a common electrode disposed over the pixels PX. 
     A thin-film encapsulation layer  170  may be disposed on the cathode CAT. The thin-film encapsulation layer  170  may cover an organic light emitting diode (OLED). The thin-film encapsulation layer  170  may be a stacked layer in which an inorganic layer and an organic layer are alternately stacked. For example, the thin-film encapsulation layer  170  may include a first encapsulating inorganic layer  171 , an encapsulating organic layer  172 , and a second encapsulating inorganic layer  173  stacked sequentially. 
     In an embodiment, the display panel may further include an under-panel sheet disposed on a lower surface of the substrate SUB. The under-panel sheet may include at least one functional layer such as, for example, a digitizer, a heat dissipation member, a shield member, and a buffer member. 
     Referring to  FIGS. 4 through 7 , the first transmission area TR 1  may include the transmission hole TH as described above. The transmission hole TH may penetrate at least one of a plurality of layers constituting the display unit DP (the display panel) in the thickness direction. 
     In an embodiment, the transmission hole TH may penetrate the first via layer VIA 1 , the second via layer VIA 2  and the bank BANK, and may expose the third insulating layer  113 . In this case, the first encapsulating inorganic layer  171  may directly contact the third insulating layer  113 , and the transmission hole TH may be filled with the encapsulating organic layer  172 . 
     However, embodiments of the present disclosure are not limited thereto. For example, in an embodiment, the transmission hole TH may be formed to further penetrate at least one layer, for example, the third insulating layer  113  disposed under the first via layer VIA 1 , or may be formed to penetrate only up to the bank layer BANK and/or the first via layer VIA 1 . Alternatively, the transmission hole TH may be formed to completely penetrate the display panel. 
     The second transmission area TR 2  has substantially the same or a similar structure as the first transmission area TR 1 , and thus, for convenience of explanation, a redundant description thereof is omitted. 
     In the display device  1  according to an embodiment, the light sources IFR_L 1  through IFR_L 8  are disposed on a side of the first lens LS 1  and a side of the second lens LS 2  adjacent to a user&#39;s eyeballs EB, and the camera sensors and IFR_C 2  are disposed in front of the eyeballs EB to overlap the transmission areas TR 1  and TR 2  of the display unit DP. Therefore, according to an embodiment, performing error correction for images obtained by the camera sensors IFR_C 1  and IFR_C 2  may be omitted. In addition, when the user wears the display device  1 , interference between the light sources IFR_L 1  through IFR_L 8  and the user and/or interference between the camera sensors IFR_C 1  and IFR_C 2  and the user may be prevented or reduced. 
       FIG. 8  is a plan view of part P 1  of  FIG. 4  according to an embodiment. 
     Referring to  FIG. 8 , at least one pixel PX may be disposed in the first transmission area TR 1 , unlike in the embodiment of  FIG. 6 . In other words, a plurality of pixels PX may be arranged at regular intervals, for example, in a lattice shape and/or a matrix shape in a plan view, but some of the pixels PX located in the first transmission area TR 1  may be removed. That is, the number of pixels PX per unit area in the first transmission area TR 1  may be smaller than that in an area excluding the first transmission area TR 1  from an area in which the pixels PX of a display unit DP are disposed, for example, at least a part of a front surface DP_FS of the display unit DP (display panel). 
     The pixels PX located in the first transmission area TR 1  may be removed in units of pixel groups. A pixel group may include pixels emitting light of different colors. For example, the pixel group may include one pixel emitting first light, one pixel emitting second light, and two pixels emitting third light. However, embodiments of the present disclosure are not limited thereto. 
     A plurality of transmission holes TH may be disposed in the first transmission area TR 1 . The pixels PX may be disposed between the transmission holes TH. The transmission holes TH may have substantially the same or a similar structure as the transmission hole TH of  FIG. 7 . Each of the transmission holes TH may have a substantially ‘L’ shape in a plan view. However, embodiments of the present disclosure are not limited thereto. The transmission holes TH may have various shapes such as, for example, a quadrangle, a triangle, a polygon, a circle, and an oval. 
     Similarly, in an embodiment, a plurality of pixels PX and a plurality of transmission holes TH disposed between pixels PX may be disposed in the second transmission area TR 2 . 
     An embodiment according to  FIG. 8  is substantially the same as or similar to an embodiment according to  FIGS. 1 through 7  except that the pixels PX are disposed in the first transmission area TR 1 . Thus, for convenience of explanation, a redundant description thereof is omitted. 
       FIG. 9  is a cross-sectional view of a display device according to an embodiment. For convenience of explanation, a further description of elements and technical aspects previously described with reference to  FIG. 5  may be omitted. 
     Referring to  FIG. 9 , the plurality of light sources may further include a light source IFR_LC disposed in the first hole H 1  of the first lens LS 1 . In this case, the first through fourth light sources IFR_L 1  through LFR_L 4  may be omitted. However, embodiments of the present disclosure are not limited thereto. 
     The light source IFR_LC may overlap the first hole H 1  in the thickness direction in a cross-sectional view. The light source IFR_LC may overlap the virtual reference line RL passing through the first camera sensor the first transmission area TR 1 , the first hole H 1 , and the center of a user&#39;s eyeball EB in a cross-sectional view. 
     The light source IFR_LC may be mounted in the first hole H 1  and may be disposed not to protrude from a side of the first lens LS 1  adjacent to the user&#39;s eyeball EB toward the user&#39;s eyeball EB. The light source IFR_LC may be disposed in a space between the other side of the first lens LS 1  facing the display unit DP and having a concave shape and a front surface of the display unit DP. 
     As described above, the light source IFR_LC may emit light for detecting the position of a pupil PP, for example, infrared light and/or ultraviolet light toward the user&#39;s eyeball EB. 
     In an embodiment, a light source IFR_LC which is substantially the same as or similar to the light source IFR_LC disposed in the first hole H 1  may also be disposed in the second hole H 2  of the second lens LS 2 . 
     As illustrated in  FIG. 9 , since the light source IFR_LC is disposed in the first hole H 1 , the angle of view of the first camera sensor IFR_C 1  limited by the size of the first hole H 1  and the size of the first sensor lens SL may be increased. In addition, since a smaller number of light sources IFR_LC are utilized in an embodiment according to  FIG. 9  than in an embodiment according to  FIGS. 1 through 7 , effective arrangement of the light sources IFR_LC is possible. 
     An embodiment according to  FIG. 9  is substantially the same as or similar to an embodiment according to  FIGS. 1 through 7  except for the arrangement of the light sources IFR_LC, and thus, for convenience of explanation, a redundant description thereof is omitted. 
       FIG. 10  is a cross-sectional view of a display device  1  according to an embodiment.  FIG. 11  is a cross-sectional view of part P 2  of  FIG. 10  according to an embodiment. For convenience of explanation, a further description of elements and technical aspects previously described with reference to  FIGS. 5 and 7  may be omitted. 
     Referring to  FIG. 10 , the display device  1  may further include a plurality of microlenses SL_M disposed in the first transmission area TR 1 . 
     The microlenses SL_M may focus light IFR incident on the first transmission area TR 1  on the first camera sensor IFR_C 1  by refracting and/or reflecting the light IFR. In an embodiment, when the microlenses SL_M are disposed, the first sensor lens SL is omitted. Thus, the overall thickness of the display device  1  may be reduced. 
     The microlenses SL_M may be manufactured using, for example, a mold, or may be formed using a method of dropping and curing droplets containing a highly viscous photoreactive polymer in the first transmission area TR 1 . However, embodiments of the present disclosure are not limited thereto. 
     Referring to  FIG. 11 , the microlenses SL_M may be disposed in the transmission hole TH. Although one microlens SL_M is illustrated in  FIG. 11  for ease of description, embodiments of the present disclosure are not limited thereto. For example, in an embodiment, two or more microlenses SL_M may be disposed in the transmission hole TH. 
     The microlenses SL_M may be disposed on the substrate SUB of the display panel. The first camera sensor IFR_C 1  may be overlapped by the microlenses SL_M in the thickness direction with the substrate SUB interposed therebetween. The curvature of each microlens SL_M may be determined by the thickness of the substrate SUB and the thickness of at least one layer stacked between the substrate SUB and the microlens SL_M. Since the first sensor lens SL is omitted, and the first camera sensor IFR_C 1  is in close contact with a rear surface of the display unit DP (the substrate SUB), and the thickness of the display device  1  may be reduced. 
     In  FIG. 11 , the microlens SL_M are directly disposed on the first encapsulating inorganic layer  171 . However, embodiments of the present disclosure are not limited thereto. A layer on which the microlens SL_M is disposed may vary according to a depth and/or a layer to/through which the transmission hole TH penetrates. For example, the microlens SL_M may be directly disposed on the substrate SUB, the buffer layer BF, the first insulating layer  111 , the second insulating layer  112 , the third insulating layer  113 , or the second encapsulating inorganic layer  173 . However, embodiments of the present disclosure are not limited thereto. 
     An embodiment according to  FIG. 11  is substantially the same as or similar to an embodiment according to  FIGS. 1 through 7  except for the arrangement of the microlenses SL_M, and thus, for convenience of explanation, a redundant description thereof is omitted. 
       FIG. 12  is a flowchart illustrating a method of controlling a display device according to an embodiment. 
     The following method of controlling the display device may be performed by a control unit. 
     Referring to  FIGS. 1 through 12 , a method of controlling the display device according to an embodiment may include emitting light IFR from the light sources IFR_L 1  through IFR_L 4  or IFR_L 5  through IFR_L 8  disposed on the multi-channel lens LS toward the user&#39;s pupil PP (S 1201 ), focusing the light IFR, which is reflected from the user&#39;s pupil PP, passed through the hole H 1  or H 2  formed at the center of the multi-channel lens LS and then transmitted through the transmission area TR 1  or TR 2  formed in the display unit DP, on the camera sensor IFR_C 1  or IFR_C 2  (S 1202 ), and calculating the position of the user&#39;s pupil PP based on the light IFR received by the camera sensor IFR_C 1  or IFR_C 2  (S 1203 ). 
     The method of controlling the display device according to an embodiment is not limited to the above example. For example, according to embodiments, at least some of the above operations may be omitted, or at least one other operation may be further included with reference to other descriptions in the present disclosure. 
     Since the display device  1  has a symmetrical structure corresponding to both eyes of a user, the first lens LS 1 , the first transmission area TR 1 , the first camera sensor IFR_C 1 , and elements adjacent to them, all of which correspond to one eye of the user, will be mainly described below. 
     Referring to  FIGS. 1, 2 and 5 , in an embodiment, it is determined whether a user is wearing the display device  1 . When the user is wearing the display device  1 , light IFR, for example, infrared light, may be emitted from the light sources IFR_L 1  through IFR_L 8  toward the user&#39;s eyeballs EB. 
     As illustrated in  FIG. 5 , the light IFR emitted from the light sources IFR_L 1  through IFR_L 4  may be reflected from the user&#39;s eyeball EB to pass through the first hole HE The light IFR passing through the first hole H 1  may be refracted and/or reflected by the first center lens CLS 1  to enter the first transmission area TR 1 . 
     Referring to  FIG. 9 , when the first center lens CLS 1  is not mounted in the first hole H 1 , the light IFR reflected from the user&#39;s eyeball EB may pass through the first hole H 1  as it is about to enter the first transmission area TR 1 . 
     Still referring to  FIG. 9 , when a light source IFR_LC is mounted in the first hole H 1 , the light IFR reflected from the user&#39;s eyeball EB may be incident on the first lens LS 1 . The light IFR incident on the first lens LS 1  may be refracted and/or reflected by the first lens LS 1  to enter the first transmission area TR 1 . 
     Referring again to  FIG. 5 , the light IFR incident on the first transmission area TR 1  may pass through the first transmission area TR 1  to enter the first sensor lens SL. The light IFR incident on the first sensor lens SL may be refracted and/or reflected by the first sensor lens SL, and thus focused on the first camera sensor IFR_C 1 . 
     Referring to  FIG. 10 , the light IFR incident on the first transmission area TR 1  may be refracted and/or reflected by a plurality of microlenses, and thus focused on the first camera sensor IFR_C 1 . 
     Referring again to  FIG. 5 , the first camera sensor IFR_C 1  may obtain an image for calculating the user&#39;s gaze direction by photographing the incident light IFR. The captured image may be processed to efficiently capture feature points of the user&#39;s eyeball EB and correct distortion due to a lens or the like. 
     After the image is processed, it may be determined whether the processed image contains the user&#39;s pupil PP. When the processed image contains the user&#39;s pupil PP, the position of the user&#39;s pupil PP can be identified. The position of the user&#39;s pupil PP may be identified by applying various eye tracking methods such as, for example, a pupil tracking method in which the contrast of the user&#39;s pupil PP and its surroundings is used, and a limbus tracking method in which the contrast of the user&#39;s iris and its surroundings is used. For example, the position of the user&#39;s pupil PP and the user&#39;s gaze direction may be determined using a vector value calculated based on a difference between relative positions of a center point of the user&#39;s pupil PP and light or glint reflected from the user&#39;s pupil PP. 
     When the user&#39;s gaze direction is determined, an output image corresponding to the user&#39;s gaze direction may be displayed on the front surface DP_FS of the display unit DP. Through repetition of the above operation, an output image corresponding to the user&#39;s gaze direction may be displayed on the display unit DP. As described above, since the output image includes partial images or partial videos that form one VR image, the display device  1  can provide the VR image corresponding to the user&#39;s gaze direction to the user. 
     While the present invention has been particularly shown and described with reference to the embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention as defined by the following claims.