Patent ID: 12216816

DETAILED DESCRIPTION

For clearer descriptions of the objectives, technical solutions, and advantages of the present disclosure, embodiments of the present disclosure are described in detail hereinafter with reference to the accompanying drawings.

In the related art, a VR device includes a display panel, a camera, a processor, and a drive circuit. The camera is configured to capture an eye image of the user. The processor is configured to determine a gaze position of the user on the display panel according to the eye image, and partially render the image to be displayed according to the gaze position. The drive circuit is configured to drive, based on a received partially rendered display image, the display panel display. Because the processor may only partially render the region of a gaze point in the display image and does not need to globally render the to-be-displayed image, not only a load of the processor may be reduced, but also a display effect of the display panel may be ensured.

However, in the related art, the processor has a low efficiency in determining the gaze position according to the eyes image taken by the camera, thereby resulting in a lower display efficiency of the display panel.

The terms used in the detailed description of the present disclosure are merely for interpreting, instead of limiting, the embodiments of the present disclosure. It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present disclosure shall have ordinary meanings understandable by persons of ordinary skill in the art to which the disclosure belongs. The terms “first,” “second,” and the like used in the embodiments of the present disclosure are not intended to indicate any order, quantity or importance, but are merely used to distinguish the different components. The terms “comprise,” “include,” and derivatives or variations thereof are used to indicate that the element or object preceding the terms covers the element or object following the terms and its equivalents, and shall not be understood as excluding other elements or objects. The terms “connect,” “contact,” and the like are not intended to be limited to physical or mechanical connections, but may include electrical connections, either direct or indirect connection. The terms “on,” “under,” “left,” and “right” are only used to indicate the relative positional relationship. When the absolute position of the described object changes, the relative positional relationship may change accordingly.

FIG.1is a schematic structural diagram of a wearable display device according to an embodiment of the present disclosure. As illustrated inFIG.1, the wearable display device10may include: a display panel101, a plurality of light-emitting elements102, a plurality of photoelectric sensor assemblies103, and an optical structure104.

Light emitted by the plurality of light-emitting elements102is configured to be irradiated to eyes of a user, and the eyes of the user may reflect the light emitted by the plurality of light-emitting elements102. In this way, each of the photoelectric sensor assemblies103is configured to receive an optical signal, reflected via the eyes of the user, of each of the light-emitting elements102, and convert the optical signal into an electric signal. The electric signal is configured to determine a gaze position of the eyes of the user on the display panel101.

In general, an amount of data of the electric signals is small, and the amount of data of images is large. Therefore, an efficiency of the wearable display device10in processing the electric signals is higher than an efficiency of the wearable device10in processing the images. In the embodiments of the present disclosure, the wearable display device10has a high efficiency in processing an electric signal transmitted by each of the photoelectric sensor assemblies103, such that the gaze position of the eyes of the user on display panel101is quickly determined. In this way, an efficiency of displaying images by the display panel101is improved, and thus a higher refresh rate of the display panel is achieved.

FIG.2is a top view of a display panel according to an embodiment of the present disclosure. As illustrated inFIG.2, the display panel101may include a display region101aand a peripheral region101bsurrounding the display region101a. The plurality of photoelectric sensor assemblies103are disposed in the peripheral region101b. The plurality of photoelectric sensor assemblies103may not affect a normal display of the display panel101, and a display effect of display panel101is better.

Referring toFIG.1andFIG.2together, the optical structure104is disposed on a side, distal from the display panel101, of the photoelectric sensor assembly103. An orthographic projection of the optical structure104on the display panel101is within the peripheral region101b. The optical structure103includes a light shielding region104aand a plurality of light transmitting regions104b. The plurality of light transmitting regions104bare in one-to-one correspondence to the plurality of photoelectric sensor assemblies103, and each of the light transmitting regions104bis at least configured to transmit the optical signal to a corresponding photoelectric sensor assembly103.

Optionally, each of the light transmitting regions104bmay only transmit the optical signal to the corresponding photoelectric sensor assembly103. Alternatively, each of the light transmitting regions104bmay also transmit the optical signal to photoelectric sensor assemblies103which are adjacent to the photoelectric sensor assembly corresponding to the light transmitting region104bin addition to transmitting the optical signal to the corresponding photoelectric sensor assembly103. Moreover, the optical signal transmitted to the corresponding photoelectric sensor assembly103by the light transmitting region104bmay be strong, and the optical signal transmitted to the adjacent photoelectric sensor assemblies103may be weak.

Because the orthographic projection of the optical structure104on the display panel101is within the peripheral region101b, a normal display of the display region101aof the display panel101is not affected by the optical structure104. Moreover, because the optical structure104may be divided into the light shielding region104aand the plurality of light transmitting regions104b, and each of the light transmitting regions104bis at least configured to transmit the optical signal to the corresponding photoelectric sensor assembly103, regions of the eyes of the user corresponding to the optical signals received by different photoelectric sensor assemblies103are different. In this way, the wearable display device10may determine the gaze position based on electric signals converted from optical signals reflected via different regions of the eyes of the user.

In summary, the embodiments of the present disclosure provide a wearable display device. Because the wearable display device has a high efficiency in processing the electric signal transmitted by each of the photoelectric sensor assemblies, the wearable display device may quickly determine the gaze position of the eyes of the user on the display panel based on the electric signal transmitted by each of photoelectric sensor assemblies. In this way, the efficiency of displaying the images by the display panel is improved, and thus a higher refresh rate of the display panel is achieved.

Optionally, the light-emitting element102may be an infrared light-emitting diode. Because reflectivities of a pupil, a sclera, and an iris of the eye of the user against infrared light are greatly different (e.g., a reflectivity of the pupil is from 3% to 5%, a reflectivity of the sclera is from 70% to 80%, and a reflectivity of the iris is from 10% to 20%), by designing the light-emitting element102as the infrared light-emitting diode, an optical signal, reflected via the pupil, of the infrared light, an optical signal, reflected via the sclera, of the infrared light, and an optical signal, reflected via the iris, of the infrared light that are received by the photoelectric sensor assembly103are greatly different. In this way, it is convenient for a processor of the wearable display device10to determine the gaze position of the eyes (the pupils) of the user on the display panel101. Exemplarily, a wavelength of the light emitted by the light-emitting element102may range from 850 nm (nanometer) to 940 nm.

As illustrated inFIG.2, the peripheral region101bof the display panel101includes: a first region101b1extending along a first direction X and a second region101b2extending along a second direction Y. The first direction X is intersected with the second direction Y.

Referring toFIG.2andFIG.3together, the plurality of photoelectric sensor assemblies103may include a plurality of first photoelectric sensor assemblies103aand a plurality of second photoelectric sensor assemblies103b. The plurality of first photoelectric sensor assemblies103aare disposed in the first region101b1and arranged along the first direction X. The plurality of second photoelectric sensor assemblies104bare disposed in the second region101b2and arranged along the second direction Y.

Optionally, the plurality of first photoelectric sensor assemblies103aare arranged evenly along the first direction X, and the plurality of second photoelectric sensor assemblies103bare arranged evenly along the second direction Y.

Referring toFIG.2andFIG.4together, the optical structure104may include a first branch structure1041extending along the first direction X and a second branch structure1042extending along the second direction Y.

The first branch structure1041is provided with a plurality of first light transmitting regions104b1in one-to-one correspondence to the plurality of first photoelectric sensor assemblies103a. An orthographic projection of each of the first light transmitting regions104b1on the display panel101is at least overlapped with an orthographic projection of a corresponding first photoelectric sensor assembly103aon the display panel101. In this way, the first light transmitting region104b1may transmit the optical signal to the corresponding first photoelectric sensor assembly103a.

The second branch structure1042is provided with a plurality of second light transmitting regions104b2in one-to-one correspondence to the plurality of second photoelectric sensor assemblies103b. An orthographic projection of each of the second light transmitting regions104b2on the display panel101is at least overlapped with an orthographic projection of a corresponding second photoelectric sensor assembly103bon the display panel101. In this way, the second light transmitting region104b2may transmit the optical signal to the corresponding second photoelectric sensor assembly103b.

In the embodiments of the present disclosure, the processor included in the wearable display device10may receive an electric signal transmitted by each first photoelectric sensor assembly103ain the plurality of first photoelectric sensor assemblies103a, and may determine at least one target first photoelectric sensor assembly from the plurality of first photoelectric sensor assemblies103a. The processor may also receive an electric signal transmitted by each second photoelectric sensor assembly103bin the plurality of second photoelectric sensor assemblies103b, and may determine at least one target second photoelectric sensor assembly from the plurality of second photoelectric sensor assemblies103b. Finally, the processor may determine the gaze position of the eyes of the user on the display panel101based on a position of the at least one target first photoelectric sensor assembly and a position of the at least one target second photoelectric sensor assembly.

A signal value of an electric signal transmitted by the target first photoelectric sensor assembly may be less than or equal to a first threshold, and a signal value of an electric signal transmitted by the target second photoelectric sensor assembly may be less than or equal to a second threshold. The first threshold and the second threshold may be equal, or not, which is not limited herein.

Because reflectivities of different regions of the eyes of the user against the light emitted by the light-emitting element102are different, optical signals, collected by the first photoelectric sensor assemblies103aat different positions and the second photoelectric assemblies103bat different positions, are different. Due to a deepest color of the pupil, a signal value of an optical signal reflected via the pupil is minimum. Furthermore, a signal value of an electric signal converted from the optical signal reflected via the pupil is minimum. The signal value of the optical signal indicates an intensity of the light.

Optionally, the processor may pre-store positions of the first photoelectric sensor assemblies013aand positions of the second photoelectric sensor assemblies103b. For any photoelectric sensor assembly103in the first photoelectric sensor assemblies103aand the second photoelectric sensor assemblies103b, the position, stored in the processor, of the photoelectric sensor assembly103may refer to coordinate values of the photoelectric sensor assembly013in a two-dimensional coordinate system. The two-dimensional coordinate system may refer to a coordinate system based on a plane where the display panel101is disposed.

Exemplarily,FIG.5shows that signal values, collected by each of the first photoelectric sensor assemblies103a, of the optical signals reflected via different regions of the eyes of the user.FIG.6shows signal values, collected by each of the second photoelectric sensor assemblies103b, of the optical signals reflected via different regions of the eyes of the user. InFIG.5andFIG.6, an abscissa x0 refers to a position where each of the photoelectric sensor assemblies103is disposed, and an ordinate y0 refers to a signal value of the received electric signal. In conjunction withFIG.5andFIG.6, due to the deepest color of the pupil, the optical signal reflected via the pupil is minimum. Correspondingly, the signal value of the electric signal converted from the optical signal reflected via the pupil is minimum. Therefore, in the embodiments of the present disclosure, the processor may reliably determine the gaze position of the eye of the user on the display panel101, based on a position of a target first photoelectric sensor assembly transmitting an electric signal with a small signal value and a position of a target second photoelectric sensor assembly transmitting an electric signal with a small signal value.

In the embodiments of the present disclosure, the first threshold and the second threshold may be fixed values pre-stored in the processor. Alternatively, the first threshold may be determined, by the processor, based on the received signal values of the electric signals of the plurality of first photoelectric sensor assemblies103a. The second threshold may be determined, by the processor, based on the received signal values of the electric signals of the plurality of second photoelectric sensor assemblies103b.

Exemplarily, the processor may rank signal values of N electric signals transmitted by N first photoelectric sensor assemblies103ain an ascending order, and determine a signal value in the nthposition as the first threshold. N is an integer greater than 1 and n is an integer greater than 1 and less than N/2. The processor may rank signal values of M electric signals transmitted by M second photoelectric sensor assemblies103bin the ascending order, and determine a signal value in the mthposition as the second threshold. M is an integer greater than 1 and n is an integer greater than 1 and less than M/2.

Alternatively, the processor determines a minimum of signal values of the received electric signals of the plurality of first photoelectric sensor assemblies103aas the first threshold, and determines a minimum of signal values of the received electric signals of the plurality of second photoelectric sensor assemblies103bas the second threshold.

In the embodiments of the present disclosure, the processor may determine first coordinate values of a target first photoelectric sensor assembly, transmitting an electric signal with a minimum signal value, in the plurality of first photoelectric sensor assemblies103a, and may determine second coordinate values of a target second photoelectric sensor assembly, transmitting an electric signal with a minimal signal value, in the plurality of second photoelectric sensor assemblies103b. The processor may determine the gaze position of the eyes of the user on the display panel101based on the first coordinate values and the second coordinate values.

As illustrated inFIG.2, the first direction X is perpendicular to the second direction Y. The first direction X may be a pixel row direction of the display panel101, and the second direction Y may be a pixel column direction of the display panel101.

As illustrated inFIG.2, the peripheral region101bmay include two first regions101b1and two second regions101b2. The two first regions101b1may be arranged along the second direction Y and respectively disposed on two sides of the display region101a. The two second regions101b2may be arranged along the first direction X and respectively disposed on the two sides of the display region101a. Accordingly, as illustrated inFIG.7, in the plurality of first photoelectric sensor assemblies103aof the plurality of the photoelectric sensor assemblies103, one portion of the first photoelectric sensor assemblies103aare disposed in one of the first regions101b1, and the other portion of the first photoelectric sensor assemblies104aare disposed in the other of the first regions101b1. In the plurality of second photoelectric sensor assemblies103bof the plurality of the photoelectric sensor assemblies103, one portion of the second photoelectric sensor assemblies103bare disposed in one of the second regions101b2, and the other portion of the second photoelectric sensor assemblies103bare disposed in the other of the second regions101b2. In addition, as illustrated inFIG.8, the optical structure104includes: two first branch structures1041in one-to-one correspondence to the two first regions101b1and two second branch structures1042in one-to-one correspondence to the two second regions101b2.

In this way, the processor may determine the gaze position of the eyes of the user on the display panel101based on the first photoelectric sensor assemblies103ain the two first regions101b1and the second photoelectric sensor assemblies103bin the two second regions101b2, thereby improving an accuracy of the determined gaze position.

In the embodiments of the present disclosure, a detection width, along a target direction, of the photoelectric sensor assembly103is positively correlated with a width, along the target direction, of the light transmitting region104b. That is, the larger the width, along the target direction, of the light transmitting region104b, the larger the detection width, along the target direction, of the photoelectric sensor assembly103; and the smaller the width, along the target direction, of the light transmitting region104b, the smaller the detection width, along the target direction, of the photoelectric sensor assembly103.

In addition, the detection width, along the target direction, of the photoelectric sensor assembly103is positively correlated with a width, along the target direction, of the photoelectric sensor assembly103. That is, the larger the width, along the target direction, of the photoelectric sensor assembly103, the larger the detection width, along the target direction, of the photoelectric sensor assembly103; and the smaller the width, along the target direction, of the photoelectric sensor assembly103, the smaller the detection width, along the target direction, of the photoelectric sensor assembly103.

The detection width, along the target direction, of the photoelectric sensor assembly103may refer to a width of a region of the eyes of the user corresponding to an optical signal that is detected by the photoelectric sensor assembly103. That is, an optical signal, reflected via a region, a width of which along the target direction is the detection width, of the eyes of the user, may be determined by the photoelectric sensor assembly103. The target direction may be the first direction X or the second direction Y.

Optionally, the detection width h, along the target direction, of the photoelectric sensor assembly103satisfies:

h=2⁢u*tan(arcsin(n1n2*sin(arctan(p+s2⁢v))))+sformula⁢(1)

As illustrated inFIG.9, u represents a distance between the eyes of the user and the optical structure104, v represents a distance between the optical structure104and the plurality of photoelectric sensor assemblies103, n1 represents a refractive index of a medium between the optical structure104and the plurality of photoelectric sensor assemblies103, n2 represents a refractive index of a medium between the eyes of the user and optical structure104, p represents the width, along the target direction, of the photoelectric sensor assembly103, and s represents the width, along the target direction, of the light transmitting region104b.

In the embodiments of the present disclosure, formula (1) may be derived from the following process.

As illustrated inFIG.9, an included angle between two first connect lines is a (the included angel α may be referred to as a light receiving angle), wherein the two first connect lines respectively connect two sides of the detection region of the photoelectric sensor assembly103to the light transmitting regions104bof the optical structure104. An included angle between two second connect lines is β, wherein the two second connect lines respectively connect two sides of the light transmitting regions104bof the optical structure104to two sides of the photoelectric sensor assembly103. The two second connect lines may be lines of refract light in the case that light of the two first connect lines is incident from the light transmitting region104b.

Assuming that a distance between an intersection of the two first connect lines and the photoelectric sensor assembly103is x, then referring toFIG.9,

tan⁢α2=h/2u+(v-x)formula⁢(2)

According to formula (2),

h=2*tan⁡(α2)*u+2⁢tan⁡(α2)*(v-x)formula⁢(3)

According to the refraction law,

α2=arcsin(n⁢1n⁢2*sin(β2))formula⁢(4)

Assuming that a distance between an intersection of the two second connect lines and the photoelectric sensor assembly103is x, then referring toFIG.9,

tan⁢β2=p/2yformula⁢(5)

According to the triangle similarity

sp=v-yyformula⁢(6)

According to formula (5) and formula (6),

β2=arctan(s+p2⁢v)formula⁢(7)

According toFIG.9,

tan⁢α2=s/2v-xformula⁢(8)

In the case that formula (7) is substituted into formula (4),

α2
may be obtained, and then the obtained

α2
is substituted into the

2*tan⁡(α2)*u
in formula (3). Afterwards, formula (8) is substituted into

2⁢tan⁡(α2)*(v-x)
in formula (3). In this way, formula (1) may be derived.

Optionally, the detection width h, along the target direction, of the photoelectric sensor assembly103is correlated with an accuracy of determining the gaze position by the wearable display device10. The larger the detection width h, along the target direction, of the photoelectric sensor assembly103, the smaller the accuracy of determining the gaze position by the wearable display device10; and the smaller the detection width h, along the target direction, of the photoelectric sensor assembly103, the larger the accuracy of determining the gaze position by the wearable display device10.

Before designing the wearable display device10, an appropriate width p, along the target direction, of the photoelectric sensor assembly103and an appropriate width s, along the target direction, of the light transmitting region103bmay be determined to obtain an appropriate detection width h, along the target direction, of the photoelectric sensor assembly103. In this way, the accuracy of determining the gaze position by the wearable display device10may meet an accuracy requirement.

Exemplarily, referring toFIG.10andFIG.11together, the width s, along the target direction, of the light transmitting region104binFIG.11is enlarged relative to that inFIG.10. The width p, along the target direction, of the photoelectric assembly103inFIG.11may be less than the width p, along the target direction, of the photoelectric assembly103inFIG.10, such that the detection width d, along the target direction, of the photoelectric sensor assembly103inFIG.11is equal to the detection width, along the target direction, of the photoelectric sensor assembly103inFIG.10.

In the embodiments of the present disclosure, as illustrated inFIG.4andFIG.8, each of the light transmitting regions104bis a circular through hole. Alternatively, as illustrated inFIG.12andFIG.13, each of the light transmitting regions104bis a rectangular through hole. The rectangular through hole may be a slit.

As illustrated inFIG.13, a first edge of each rectangular through hole in the plurality of rectangular through holes104bis parallel to the first direction X, and a second edge is parallel to the second direction Y. The plurality of rectangular through holes104binclude: a plurality of first rectangular through holes104b1and a plurality of second rectangular through holes104b2. The first rectangular through hole104b1is disposed in the first region101b1extending along the first direction X in the peripheral region101b, and a length of a first edge of the first rectangular through hole104b1is less than a length of a second edge. The second rectangular through hole104b2is disposed in the second region101b2extending along the second direction Y in the peripheral region101b, and a length of a first edge of the second rectangular through hole104b2is greater than a length of a second edge.

Relative to the circular through hole, the slit only has a light-receiving limit in the target direction. For example, the first rectangular through hole104b1disposed in the first region101b1only has the light-receiving limit in the first direction X (the target direction is the first direction X). The second rectangular through hole104b2disposed in the second region101b2only has the light-receiving limit in the second direction Y (the target direction is the second direction Y).

Because the slit only has a light-receiving limit in the target direction, but does not have a light-receiving limit in another direction perpendicular to the target direction, relative to the circular through-hole, the slit transmits more optical signals. In this way, the photoelectric sensor assembly103may receive strong optical signals, and requirements on capabilities of the photoelectric sensor assembly103to receive the optical signals are lowered.

FIG.14is a schematic structural diagram of still another wearable display device according to an embodiment of the present disclosure. As illustrated inFIG.14, the wearable display device10may further include a light transmitting layer105. The light transmitting layer105may be disposed between the plurality of photoelectric sensor assemblies103and the optical structure104.

By disposing the light transmitting layer105between the photoelectric sensor assembly103and the optical structure104, a distance between the photoelectric sensor assembly103and the optical structure104may be enlarged, and the region of the eyes of the user corresponding to the optical signal that may be received by each of the photoelectric sensor assemblies103may be reduced. In this way, an accuracy of the optical signal received by the photoelectric sensor assembly103is increased, thereby increasing the accuracy of the determined gaze position.

As illustrated inFIG.14, the wearable display device10may further include a filter106. The filter106may be disposed on a side, distal from the display panel101, of the plurality of photoelectric sensor assemblies103, and an orthographic projection of the filter106on the display panel101covers orthographic projections of the plurality of photoelectric sensor assemblies103on the display panel101. The filters106may be configured to transmit the infrared light and absorb visible light.

By disposing the filter106on the side, distal from the display panel101, of the photoelectric sensor assembly103to filter out the visible light, the light signal received by photoelectric sensor assembly103may be prevented from being affected by light emitted by the display panel101, thereby ensuring the accuracy of the determined gaze position.

As illustrated inFIGS.1,2, and14, the wearable display device10may further include a lens107and a lens frame108. The lens107may be disposed on a display side of the display panel101through which a user may view the images displayed by the display panel101. The lens frame108may be disposed on an edge of the lens107for supporting and securing the lens107.

As illustrated inFIGS.1,12, and14, the plurality of light-emitting elements102may be disposed on a side, distal from the display panel101, of the lens frame108, and connected to the lens frame108. That is, the plurality of light-emitting elements102may be fixed to the side, distal from the display panel101, of the lens frame108.

Optionally, the plurality of light-emitting elements102may be arranged evenly on the side, distal from the display panel101, of the lens frame108. In this way, a uniformity of the optical signals received by each region of the eyes of the user is improved, and thus the accuracy of the determined gaze position is ensured.

In the embodiments of the present disclosure, as illustrated inFIG.14, the wearable display device10may further include: a first polarizer layer109and a second polarizer layer110. The first polarizer layer109may be disposed on a light-exiting side of the light-emitting element102, and the second polarizer layer110may be disposed on the side, distal from the display panel101, of the plurality of photoelectric sensor assemblies103. A polarization direction of the second polarizer layer110may be intersected with a polarization direction of the first polarizer layer109.

The light emitted by the light-emitting element102may first pass through the first polarizer layer109, and then be irradiated to the eyes of the user. Moreover, light reflected via the eyes of the user may first pass through the second polarizer layer110, and then be irradiated to the photoelectric sensor assembly103.

The light emitted by the light-emitting element102is converted into polarized light upon passing through the first polarizer layer109. The polarized light is irradiated to the eyes of the user, and specular and diffuse reflection occurs at the eyes of the user. The light specularly reflected and diffusely reflected via the eyes of the user may be transmitted to the second polarizer layer110.

Because the specularly reflected light is the polarized light which is reflected and then emitted along a parallel direction, and the polarization direction of the second polarizer layer110is intersected with the polarization direction of the first polarizer layer109, the light specularly reflected via the eyes of the user may not be transmitted through the second polarizer layer110. Because the diffusely reflected light is the polarized light which is reflected and then emitted along each direction, the light diffusely reflected via the eyes of the user may be transmitted through the second polarizer layer110even in the case that the polarization direction of the second polarizer layer110is intersected with the polarization direction of the first polarizer layer109.

Accordingly, the photoelectric sensor assembly103may not receive the light which is specularly reflected via the eyes of the user, but may only receive the light which is diffusely reflected via the eyes of the user. That is, in the embodiments of the present disclosure, the optical signal, reflected via the eyes of the user, of the light-emitting element102received by the photoelectric sensor assembly103is the optical signal, diffusely reflected via the eyes of the user, of the light-emitting element102.

The photoelectric sensor assembly103converts the diffusely reflected optical signal of the light-emitting element102into the electric signal, and the wearable display device10determines the gaze position of the eyes of the user on the display panel101based on the electric signal. Because the solutions according to the embodiments of the present disclosure suppress a specular reflection, by the eyes of the user, of the light emitted by the light-emitting element102, it is possible to avoid an effect of the specularly reflected light on determining the gaze position, thereby ensuring the accuracy of the determined gaze position.

Optionally, the polarization direction of the second polarizer layer110is perpendicular to the polarization direction of the first polarizer layer109. By configuring the polarization direction of the second polarizer layer110to be perpendicular to the polarization direction of the first polarizer layer109, the light, specularly reflected via the eyes of the user, of the first light-emitting element102may not be transmitted through the second polarizer layer110. In this way, the photoelectric sensor assembly103is prevented from receiving the specularly reflected light, and thus a determination of the gaze position is not affected.

In summary, the embodiments of the present disclosure provide a wearable display device. Because the wearable display device has a high efficiency in processing the electric signal transmitted by each of the photoelectric sensor assemblies, the wearable display device may quickly determine the gaze position of the eyes of the user on the display panel based on the electric signal transmitted by each of photoelectric sensor assemblies. In this way, the efficiency of displaying the images by the display panel is improved, and thus a higher refresh rate of the display panel is achieved.

FIG.15is a flowchart of a method for determining gaze positions according to an embodiment of the present disclosure. The method is applicable to the wearable display device10according to the embodiments described above. As illustrated inFIG.15, the method may include the following steps.

In step201, optical signals, reflected via eyes of a user, of a plurality of light-emitting elements are received.

In the embodiments of the present disclosure, the wearable display device10includes a display panel101, a plurality of light-emitting elements102, and a plurality of photoelectric sensor assemblies103. Light emitted by the plurality of light-emitting elements102is configured to be irradiated to the eyes of the user. The display panel101includes a display region101aand a peripheral region101bsurrounding the display region101a. The plurality of photoelectric sensor assemblies103may be disposed in the peripheral region101b, each of the photoelectric sensor assemblies103being configured to receive the optical signals, reflected via the eyes of the user, of the plurality of light-emitting elements102.

In step202, the optical signals are converted into electric signals.

In the embodiments of the present disclosure, in the case that each of the photoelectric sensor assemblies103receives the optical signal, reflected via the eyes of the user, the received optical signal may be converted into the electric signal.

In step203, the gaze position of the eyes of the user on the display panel is determined based on signal values of the electric signals and of a position of at least one photoelectric sensor assembly.

In the embodiments of the present disclosure, the wearable display device10further includes a processor connected to each of the photoelectric sensor assemblies103and capable of receiving electric signal transmitted by each of the photoelectric sensor assemblies103. In the case that the processor receives the electric signal transmitted by each of the photoelectric sensor assemblies103, the gaze position of the eyes of the user on the display panel101may be determined based on a signal value of the electric signal transmitted by each of the photoelectric sensor assemblies103and the position of the at least one photoelectric sensor assembly103.

In the embodiments of the present disclosure, a position of each of the photoelectric sensor assemblies103may be pre-stored in the processor. Because different regions of human eyes are different in reflectivity to light (e.g., infrared light), the optical signals, reflected via different regions of the human eyes, received by the photoelectric sensor assemblies103are different. Signal values of electric signals, converted by the photoelectric sensor assembly103based on different optical signals, are different. In this way, the processor may determine the gaze position of the eyes of the user on display panel101based on the signal values of the electric signals and the position of the photoelectric sensor assembly103.

In general, an amount of data of the electric signals is small, and an amount of data of images is large. Therefore, an efficiency of the processor in processing the electric signals is higher than an efficiency of the processor in processing the images. In the embodiments of the present disclosure, the processor has a high efficiency in processing the electric signal transmitted by each of the photoelectric sensor assemblies103, such that the gaze position of the eyes of the user on display panel101is quickly determined. In this way, an efficiency of displaying images by the display panel101is improved, and thus a higher refresh rate of the display panel is achieved.

In summary, the embodiments of the present disclosure provide a wearable display device. The wearable display device has a high efficiency in processing the electric signal transmitted by each of the photoelectric sensor assemblies, such that the wearable display device may quickly determine the gaze position of the eyes of the user on the display panel based on the electric signal transmitted by each of the photoelectric sensor assemblies. In this way, the efficiency of displaying the images by the display panel is improved, and thus a higher refresh rate of the display panel is achieved.

FIG.16is a flowchart of another method for determining gaze positions according to an embodiment of the present disclosure. The method is applicable to the wearable display device10according to the embodiments described above. As illustrated inFIG.16, the method may include the following steps.

In step301, a plurality of first photoelectric sensor assemblies, and a plurality of second photoelectric sensor assemblies receive optical signals reflected via eyes of a user.

In the embodiments of the present disclosure, the wearable display device10includes a display panel101and a plurality of photoelectric sensor assemblies103. The display panel101includes a display region101aand a peripheral region101bsurrounding the display region101a. A user is typically disposed on a display side of the display panel101to view images displayed in the display panel101. Moreover, the plurality of photoelectric sensor assemblies103may be disposed on the display side of the display panel101and disposed in the peripheral region101b.

The display side of the display panel101is provided with a plurality of light-emitting elements102. Light emitted by the plurality of light-emitting elements102may be irradiated to the eyes of the user, and the eyes of the user may reflect the light emitted by the plurality of light-emitting elements102. Moreover, the light emitted by the plurality of light-emitting elements102may be irradiated to the plurality of photoelectric sensor assemblies103upon being reflected via the eyes of the user. In this way, the plurality of photoelectric sensor assemblies103may receive the optical signals reflected via the eyes of the user.

Optionally, the plurality of photoelectric sensor assemblies103include a plurality of first photoelectric sensor assemblies103aarranged along a first direction X and a plurality of second photoelectric sensor assemblies103barranged along a second direction Y. Either the plurality of first photoelectric sensor assemblies103aor the plurality of second photoelectric sensor assemblies103bare capable of receiving the optical signals reflected via the eyes of the user.

In step302, each photoelectric sensor assembly of the plurality of first photoelectric sensor assemblies and the plurality of second photoelectric sensor assemblies converts the received optical signal into an electric signal.

In the embodiments of the present disclosure, in the case that the plurality of first photoelectric sensor assemblies103aand the plurality of second photoelectric sensor assemblies103breceive the optical signals, each of the photoelectric sensor assemblies103may convert the received optical signal into the electric signal.

Moreover, a signal value of the electric signal converted by the photoelectric sensor assembly103is positively correlated with a signal value of the optical signal received by the photoelectric sensor assembly103. That is, the larger the signal value of the optical signal received by the photoelectric sensor assembly103, the larger the signal value of the electric signal converted, by the photoelectric sensor assembly103, from the optical signal received; and the smaller the signal value of the optical signal received by the photoelectric sensor assembly103, the smaller the signal value of the electric signal converted, by the photoelectric sensor assembly103, from the optical signal received. The signal value of the optical signal may indicate an intensity of the light.

In step303, each of the photoelectric sensor assemblies transmits the electric signal to a processor.

In the embodiments of the present disclosure, the processor in the wearable display device10may be connected to each of the photoelectric sensor assemblies103. Each of the photoelectric sensor assemblies103may transmit the electric signal to the processor.

In step304, the processor determines at least one target first photoelectric sensor assembly from the plurality of first photoelectric sensor assemblies.

In the embodiments of the present disclosure, in the case that the processor receives electric signals transmitted by the plurality of first photoelectric sensor assemblies103a, at least one target first photoelectric sensor assembly may be determined from the plurality of first photoelectric sensor assemblies103a. Furthermore, the processor may also determine a position of each of the at least one target first photoelectric sensor assembly, e.g., determine coordinate values of each of the at least one target first photoelectric sensor assembly.

A signal value of an electric signal transmitted, by the target first photoelectric sensor assembly, to the processor is less than or equal to a first threshold. The first threshold may be a fixed value pre-stored in the processor. Alternatively, the first threshold may be determined, by the processor, based on signal values of the received electric signals of the plurality of first photoelectric sensor assemblies103a.

Exemplarily, the processor may sort the signal values of N electric signals transmitted by N first photoelectric sensor assemblies103ain an ascending order, and determine the signal value in the nthposition as the first threshold. N is an integer greater than 1 and n is an integer greater than 1 and less than N/2. Optionally, the processor may determine a minimum of the signal values of the received electric signals of the plurality of first photoelectric sensor assemblies103aas the first threshold

In the case that the first threshold value is the minimum of the signal values of the electric signals transmitted by the plurality of the first photoelectric sensor assemblies103a, the processor may determine one target first photoelectric sensor assembly from the plurality of first photoelectric sensor assemblies103a. In this way, the processor may determine first coordinate values of the target first photoelectric sensor assembly transmitting an electric signal with the minimum signal value in the plurality of first photoelectric sensor assemblies103a.

Optionally, the first coordinate values may be expressed in terms of (a first abscissa value, a first ordinate value). The first abscissa value may be a coordinate value, in the first direction X, of the target first photoelectric sensor assembly, and the first ordinate value may be a coordinate value, in the second direction Y, of the target first photoelectric sensor assembly. Because the plurality of first photoelectric sensor assemblies103aare arranged along the first direction X, the coordinate value of each of the first photoelectric sensor assemblies103ain the second direction Y may be 0. That is, the first ordinate value of the target first photoelectric sensor assembly may be 0.

FIG.17is a graph of an optical signal of a first photoelectric sensor assembly according to an embodiment of the present disclosure.FIG.18is a graph of an optical signal of another first photoelectric sensor assembly according to an embodiment of the present disclosure. As illustrated inFIG.17andFIG.18, the first photoelectric sensor assemblies103aare respectively disposed at positions: −4, −3, −2, −1, 0, 1, 2, 3, and 4, from left to right. InFIG.17andFIG.18, an abscissa indicates the position of each of the first photoelectric sensor assemblies103a, and an ordinate indicates the signal value of the electric signal. E-k inFIG.17andFIG.18indicates 10 to the power of minus k, e.g., 5E-8 indicates 5 multiplied by 10 to the power of minus 8.

Assuming that the processor determines one target first photoelectric sensor assembly from the plurality of first photoelectric sensor assemblies103a, then inFIG.17, a signal value of an electric signal transmitted by a first photoelectric sensor assembly103a, which is disposed at a position 0, is minimum. In this way, the first photoelectric sensor assembly103a, which is disposed at the position 0, may be determined as the target first photoelectric sensor assembly. Alternatively, inFIG.18, a signal value of an electric signal transmitted by a first photoelectric sensor assembly103a, which is disposed at a position −0.1, is minimum. In this way, the first photoelectric sensor assembly103a, which is disposed at the position −1, may be determined as the target first photoelectric sensor assembly. As illustrated inFIG.19, totally nine first photoelectric sensor assemblies103aare disposed in a first region101b1of the peripheral region101bof the display panel101.

In step305, the processor determines at least one target second photoelectric sensor assembly from the plurality of second photoelectric sensor assemblies.

In the embodiments of the present disclosure, in the case that the processor receives electric signals transmitted by the plurality of second photoelectric sensor assemblies103b, at least one target second photoelectric sensor assembly may be determined from the plurality of second photoelectric sensor assembly103b. Furthermore, the processor may also determine a position of each of at least one target second photoelectric sensor assembly, e.g., determine coordinate values of each of the at least one target second photoelectric sensor assembly.

A signal value of an electric signal transmitted, by the target second photoelectric sensor assembly, to the processor is less than or equal to a second threshold value. The second threshold may be a fixed value pre-stored in the processor. Alternatively, the second threshold may be determined, by the processor, based on signal values of the received electric signals of the plurality of second photoelectric sensor assemblies103b.

Exemplarily, the processor may sort signal values of M electric signals transmitted by M second photoelectric sensor assemblies103bin the ascending order, and determine the signal value in the mthposition as the second threshold. M is an integer greater than 1 and n is an integer greater than 1 and less than M/2. Optionally, the processor may determine a minimum of the signal values of the received electric signals of the plurality of second photoelectric sensor assemblies104bas the second threshold.

In the case that the second threshold value is the minimum of the signal values of the electric signals transmitted by the plurality of the second photoelectric sensor assemblies103b, the processor may determine one target second photoelectric sensor assembly from the plurality of second photoelectric sensor assemblies103b. In this way, the processor may determine second coordinate values of the target second photoelectric sensor assembly transmitting an electric signal with the minimum signal value in the plurality of second photoelectric sensor assemblies103b.

Optionally, the second coordinate values may be expressed in terms of (a second abscissa value and a second ordinate value). The second abscissa value may be a coordinate value, in the first direction X, of the target second photoelectric sensor assembly, and the second ordinate value may be a coordinate value, in the second direction Y, of the target second photoelectric sensor assembly. Because the plurality of second photoelectric sensor assemblies103bare arranged along the second direction Y, the coordinate value of each of the second photoelectric sensor assemblies103bin the first direction X may be 0. That is, the second abscissa value of the target second photoelectric sensor assembly may be 0.

FIG.20is a graph of an optical signal of a second photoelectric sensor assembly according to an embodiment of the present disclosure.FIG.21is a graph of an optical signal of another second photoelectric sensor assembly according to an embodiment of the present disclosure. As illustrated inFIG.20andFIG.21, the second photoelectric sensor assemblies103bare respectively disposed at positions: −4, −3, −2, −1, 0, 1, 2, 3, and 4, from top to bottom. InFIG.20andFIG.21, an abscissa indicates the position of each of the second photoelectric sensor assemblies103b, and an ordinate indicates the signal value of the electric signal. E-k inFIG.20andFIG.21indicates 10 to the power of minus k, e.g., 5E-8 indicates 5 multiplied by 10 to the power of minus 8.

Assuming that the processor determines one target second photoelectric sensor assembly, from the plurality of second photoelectric sensor assemblies103b, then in MG.20, a signal value of an electric signal transmitted by a second photoelectric sensor assembly103b, which is disposed at a position −2, is minimum. In this way, the second photoelectric sensor assembly103b, which is disposed at the position −2, may be determined as the target second photoelectric sensor assembly. Alternatively, inFIG.21, a signal value of an electric signal transmitted by a second photoelectric sensor assembly103b, which is disposed at a position −3, is minimum. In this way, the second photoelectric sensor assembly103b, which is disposed at the position −3, may be determined as the target second photoelectric sensor assembly. As illustrated inFIG.19, 9 second photoelectric sensor assemblies103bare disposed in a second region101b2of the peripheral region101bof the display panel101.

It should be noted that, the number of first photoelectric sensor assemblies103adisposed in the first region101b1of the peripheral region101bof the display panel101may be equal to the number of second photoelectric sensor assemblies103bdisposed in the second region101b2. The number of first photoelectric sensor assemblies103adisposed in the first region101b1may not be equal to the number of second photoelectric sensor assemblies103bdisposed in the second region101b2. For example, in general, a length, along the first direction X, of the display panel101is greater than a length along the second direction Y, and thus the number of first photoelectric sensor assemblies103adisposed in the first region101b1may be greater than the number of second photoelectric sensor assemblies103bdisposed in the second region101b2.

In step306, the processor determines a gaze position of the eyes of the user on the display panel according to the position of each of the target first photoelectric sensor assemblies and the position of each of the target second photoelectric sensor assemblies.

In the embodiments of the present disclosure, in the case that the processor determines the position of each of the target first photoelectric sensor assemblies and the position of each of the target second photoelectric sensor assemblies, the gaze position of the eyes of the user on the display panel101may be determined according to the position of each of the target first photoelectric sensor assemblies and the position of each of the target second photoelectric sensor assemblies.

In some embodiments, assuming that the processor determines a plurality of target first photoelectric sensor assemblies, then the processor may determine first coordinate values of each target first photoelectric sensor assembly in the plurality of target first photoelectric sensor assemblies. The first coordinate values of each target first photoelectric sensor assembly may be expressed in terms of (a first abscissa value and a first ordinate value). Afterwards, the processor may determine a first abscissa average value of the first abscissa values of the plurality of target first photoelectric sensor assemblies and a first ordinate average value of the first ordinate values of the plurality of target first photoelectric sensor assemblies.

Because the first ordinate value of each target first photoelectric sensor assembly is 0, the first ordinate average value of the first ordinate values of the plurality of target first photoelectric sensor assemblies is also 0.

Accordingly, assuming that the processor determines a plurality of target second photoelectric sensor assemblies, then the processor may determine second coordinate values of each target second photoelectric sensor assembly in the plurality of target second photoelectric sensor assemblies. The second coordinate values of each target second photoelectric sensor assembly may be expressed in terms of (a second abscissa value, a second ordinate value). Afterwards, the processor may determine a second abscissa average value of the second abscissa values of the plurality of target second photoelectric sensor assemblies, and a second ordinate average value of the second ordinate values of the plurality of target second photoelectric sensor assemblies.

Because the second abscissa value of each target second photoelectric sensor assembly is 0, the second abscissa average value of the second abscissa values of the plurality of target second photoelectric sensor assemblies is also 0.

Afterwards, the processor may determine the gaze position of the eyes of the user on the display panel101based on the first abscissa average value and the second ordinate average value. For example, the gaze position may be expressed by coordinates of the gaze position (the first abscissa average value, the second ordinate average value).

In some embodiments, assuming that the processor determines one target first photoelectric sensor assembly, then the processor may determine first coordinate values of the target first photoelectric sensor assembly. The first coordinate values of the target first photoelectric sensor assembly may be expressed in terms of (a first abscissa value and a first ordinate value).

Accordingly, assuming that the processor determines one target second photoelectric sensor assembly, then the processor may determine second coordinate values of the target second photoelectric sensor assembly. The second coordinate values of the target second photoelectric sensor assembly may be expressed in terms of (a second abscissa value and a second ordinate value).

Afterwards, the processor may determine the gaze position of the eyes of the user on the display panel101based on the first abscissa value and the second ordinate value. For example, the gaze position may be expressed by coordinates of the gaze point (the first abscissa value and the second ordinate value).

Exemplarily, referring toFIG.17andFIG.20together, a gaze position a may be expressed by coordinates of the gaze point (0, −2). Alternatively, a gaze position b may be expressed by coordinates of the gaze point (−1, −3).

It is noted that, upon determining the gaze position, the processor may render an image to be displayed in the display panel101based on the gaze position and transmit the rendered image to be displayed to the drive circuit of the display panel101, such that the drive circuit drives the display panel101to display the rendered image to be displayed. Alternatively, the processor may transmit the gaze position to another processor upon determining the gaze position. The another processor renders the image to be displayed in the display panel101based on the gaze position and transmits the rendered image to be displayed to the drive circuit of the display panel101, such that the drive circuit drives the display panel101to display the rendered image to be displayed.

The rendering of the to-be-displayed image may be to partially render a region where the gaze point is disposed in the image to be displayed. The region of the gaze point refers to a target region centered on the gaze position. A shape of the target region may be circular, rectangular, or the like, and a size of the target region may be a pre-stored size in the processor.

It should be noted that a sequence of steps of the method for determining the gaze positions according to the embodiments of the present disclosure may be adjusted appropriately, and the steps may be scaled accordingly. For example, step305and step304may be performed concurrently, or step305may be performed before step304. Any variations within the scope of the technology disclosed in this disclosure made by persons of ordinary skill in the art fall within the protection scope of the present disclosure, and are therefore not repeated herein.

In summary, the embodiments of the present disclosure provide a method for determining the gaze positions. The wearable display device has a high efficiency in processing the electric signal transmitted by each of the photoelectric sensor assemblies, such that the gaze position of the eyes of the user on display panel is quickly determined based on the electric signal transmitted by each of the photoelectric sensor assemblies. In this way, an efficiency of displaying images by the display panel is improved, and thus a higher refresh rate of the display panel is achieved.

An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores one or more instructions therein, wherein the one or more instructions, when loaded and executed by a wearable display device, cause the wearable display device to perform the method as described above.

An embodiment of the present disclosure further provides a computer program product storing one or more instructions therein, wherein the one or more instructions, when loaded and executed by a computer, cause the computer to perform the method as described above.

Described above are merely exemplary embodiments of the present disclosure and are not intended to limit the present disclosure. Within the spirit and principles of the present disclosure, any modifications, equivalent substitutions, improvements, and the like fall within the protection scope of the present disclosure.