Electronic device and screen adjusting method based on condition of user's eye

A method for adjusting a display screen of an electronic device is disclosed. The method includes obtaining a pupil image of a user of the electronic device. An eye condition of the user is determined by analyzing the pupil image. The proportions between a red channel, a green channel, and a blue channel (RGB) of a display screen of the electronic device is adjusted according to the eye condition.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Taiwanese Patent Application No. 105110676 filed on Apr. 5, 2016, the contents of which are incorporated by reference herein.

FIELD

The subject matter herein generally relates to managing technology, and particularly to an electronic device and a method for adjusting a display screen of the electronic device.

BACKGROUND

Improper or extended use of an electronic device may cause deterioration of a user's eyesight.

DETAILED DESCRIPTION

FIG. 1is a block diagram of one embodiment of an electronic device. Depending on the embodiment, an electronic device1can include, but is not limited to, a screen adjusting system10, a display screen11, an image gathering device12, a storage device13, at least one processor14. In at least one exemplary embodiment, the electronic device1can be a mobile phone, a tablet computer, a personal digital assistant, or any other suitable device such as a smart television.FIG. 1illustrates only one example of the electronic device1, other examples can include more or fewer components than illustrated, or have a different configuration of the various components in other exemplary embodiments.

In at least one exemplary embodiment, the screen adjusting system10adjusts the display screen11according to an eye condition of a user of the electronic device1. For example, when the user of the electronic device1is determined to be old, because of age-related cataracts for example, the screen adjusting system10can adjust proportions between a red channel, a green channel, and a blue channel (hereinafter “proportions between RGB”) of the display screen11by decreasing the blue channel.

In at least one exemplary embodiment, the image gathering device12can be a camera device or a human face scanner. The image gathering device12can be used to capture images of the user of the electronic device1. In at least one exemplary embodiment, the image gathering device12can be integrated in the electronic device1. For example, the image gathering device12can be a built-in front-facing camera of the electronic device1. In other exemplary embodiments, the image gathering device12can be externally connected with the electronic device1. For example, the image gathering device12can connect with the electronic device1using a universal serial bus (USB).

In at least one exemplary embodiment, the storage device13can be a memory of the electronic device1. In other exemplary embodiments, the storage device13can be a secure digital card, or other external storage device such as a smart media card. The storage device13can be used to store data of the electronic device1.

The at least one processor14can be a central processing unit (CPU), a microprocessor, or other data processor chip that performs functions of the electronic device1.

FIG. 2illustrates a block diagram of one exemplary embodiment of modules of the screen adjusting system10. In at least one exemplary embodiment, the screen adjusting system10can include an obtaining module101, an analyzing module102, and a processing module103. The modules101-103include computerized codes in the form of one or more programs that may be stored in the storage device13. The computerized codes include instructions that can be executed by the at least one processor14.

The obtaining module101can obtain an image of pupil (hereinafter “pupil image”) of the user of the electronic device1.

In at least one exemplary embodiment, the obtaining module101can control the image gathering device12to capture an image of the user. The obtaining module101can recognize a face area from the image of the user using a preset face recognition method. The obtaining module101can recognize an eye area from the face area using a preset eye recognition method. The obtaining module101can recognize a pupil area from the eye area, and clip or extract the pupil area from the eye area to obtain the pupil image.

In at least one exemplary embodiment, the preset face recognition method can include, but is not limited to, a face template matching method, or a skin template matching method. The preset eye recognition method can include, but is not limited to, a Hough Transform method, or a template-deformation method. In at least one exemplary embodiment, the obtaining module101can recognize the pupil area from the eye area by comparing the eye area with a preset pupil image template. When an area of the eye area matches with the preset pupil image template, the obtaining module101can determine the area is the pupil area.

The analyzing module102can determine an eye condition of the user by processing and analyzing the pupil image.

In at least one exemplary embodiment, the analyzing module102can process the pupil image using a high-contrast and low-light image processing method and obtain a processed pupil image. The analyzing module102can analyze the processed pupil image by analyzing a grayscale and chrominance variances of the processed pupil image.

In at least one exemplary embodiment, as illustrated inFIG. 3A, when the analyzing module102determines that the grayscale of a middle portion21of a pupil image2is within a first preset range such as [50 to 150], the analyzing module102can determine that the eye of the user has cataract, i.e., the user has an eye disease. The analyzing module102can determine that the eye of the user is not normal.

In at least one exemplary embodiment, as illustrated inFIG. 3B, when the analyzing module102determines that the grayscale of the middle portion21is within a second preset range around zero, the analyzing module102can determine that the eye of the user is in a normal state, i.e., the user does not have an eye disease.

In at least one exemplary embodiment, as illustrated inFIG. 3C, when the analyzing module102determines that a red chroma of a side portion22is within a third preset range such as [50 to 150], the analyzing module102can determine that the eye of the user is pinkeye, i.e., the user has an eye disease. The analyzing module102can determine that the eye of the user is not normal.

In at least one exemplary embodiment, as illustrated inFIG. 3D, when the analyzing module102determines that a blue chroma of the pupil image2is within a fourth preset range such as [50 to 150], the analyzing module102can determine that the user is wearing contact lens(es). The analyzing module102can determine that the eye of the user is not normal.

The processing module103can adjust the display screen11according to the determined eye condition.

In at least one exemplary embodiment, when the eye of the user is in the normal state, the processing module103makes no change to the display screen11, i.e., the processing module103does not adjust the display screen11.

In at least one exemplary embodiment, the processing module103can adjust the display screen11by adjusting the proportions between RGB when the eye of the user is deemed not normal.

For example, when the analyzing module102determines that the user has an eye disease, the processing module103can decrease the blue channel of the display screen11to allow the user to look at the display screen11more comfortably.

In at least one exemplary embodiment, when the analyzing module102determines that the user is wearing contact lens(es), the processing module103can record a start time when the user begins to watch the display screen11while wearing the contact lens(es). The processing module103can calculate a time length of the user wears the contact lens(es) based on the start time. When the time length is equal to or greater than a preset time length (for example, 4 hours), the processing module103can transmit a prompt. For example, the processing module103can display a preset message such as “please remove your contact lens(es)”, and/or play a preset warning audio message.

In at least one exemplary embodiment, when the analyzing module102determines that the user is wearing contact lens(es), the processing module103can obtain a current time of the electronic device1, and set the current time as the start time.

In other exemplary embodiments, the processing module103can provide a user interface and record the start time in response to a user input.

In at least one exemplary embodiment, the obtaining module101obtains pupil images of the user from the start time. When the user is determined to be wearing contact lens(es) in each pupil image obtained during the preset time length, the processing module103can determine that the user is wearing contact lens(es) for the preset time length. The processing module103can transmit the prompt.

In other exemplary embodiments, the obtaining module101can obtain a pupil image of the user when a time length from the start time equals the preset time length. When the analyzing module102determines that the user is wearing contact lens(es) in the pupil image, the analyzing module102determines that the user is wearing contact lens(es) for the preset time length, and the processing module103can transmit the prompt. That is, in the other exemplary embodiments, only one pupil image is used for determining whether the user is wearing contact lens(es) for the preset time length.

In other exemplary embodiments, the obtaining module101can obtain a first pupil image of the user when the start time is recorded. The obtaining module101can obtain a second pupil image of the user when a time length calculated from the start time equals the preset time length. When the analyzing module102determines that the user is wearing contact lens(es) in the first pupil image and the second pupil image, the analyzing module102determines that the user has been wearing contact lens(es) for the preset time length, and the processing module103can transmit the prompt. That is, only two pupil images are used for determining whether the user is wearing contact lens(es) for the preset time length.

In at least one exemplary embodiments, the processing module103can adjust the display screen11according to changes in a diameter of the pupil (hereinafter “pupil diameter”) of the user in the pupil image.

In at least one exemplary embodiments, the analyzing module102can determine the pupil diameter of the user in the pupil image using an image recognition algorithm. The processing module103can adjust a brightness value of a backlight of the display screen11. For example, the processing module103can adjust the brightness value of the backlight of the display screen11when the pupil diameter of the user is determined to be in different preset ranges.

The pupil diameter of the user generally changes according to a brightness value of a surrounding environment. The pupil diameter of the user decreases when the brightness increases. Conversely, the pupil diameter of the user increases when the brightness decreases. In other words, the value of the pupil diameter of the user can represent the current brightness value of the surrounding environment. That is, the processing module103can adjust the brightness value of the backlight of the display screen11according to the pupil diameter of the user.

For a first example, when the analyzing module102determines that the value of pupil diameter of the user is greater than or equal to 2 mm and less than or equal to 3 mm (i.e., [2 to 3 mm]), the processing module103can determine that the user is in an environment with strong or bright light. The processing module103can increase a current brightness value of the backlight of the display screen11. For a second example, when the analyzing module102determines that the value of pupil diameter of the user is greater than 3 mm and less than or equal to 5 mm (i.e., (3 to 5 mm]), the processing module103can determine that the user is in a comfortable environment. The processing module103can keep the current brightness value of the backlight of the display screen11or slightly adjust the current brightness value of the display screen11. For a third example, when the analyzing module102determines that the value of pupil diameter of the user is greater than 5 mm and less than or equal to 8 mm (i.e., (5 to 8 mm]), the processing module103can determine that the user is in an environment with low light. The processing module103can decrease the current brightness value of the backlight of the display screen11to make it dimmer.

In at least one exemplary embodiment, to enable comfortable viewing, the processing module103can further adjust the display screen11according to eye actions. In at least one exemplary embodiment, the eye actions can include, but are not limited to, frowning, narrowing of the eyes, and closing the eyes.

In at least one exemplary embodiment, the obtaining module101can obtain eye images of the user in real-time.

It should be noted that the method of obtaining the eye image of the user is similar to the method of obtaining the pupil image. When the eye area is recognized from the face area, the obtaining module101can clip or extract the eye area from the face area, thus the eye image is obtained.

In at least one exemplary embodiment, the analyzing module102can determine the eye action by analyzing the eye image.

In at least one exemplary embodiment, the analyzing module102can pre-store a number of image templates. The number of image templates includes frown images, eye narrowing images, and eye closing images. The frown image can be defined as an image of eye area in which the user frowns. The eye narrowing image can be defined as an image of eye area in which the user narrows his eyes. The eye closing image can be defined as an image of eye area in which the user closes at least one eye. In at least one exemplary embodiment, the analyzing module102can determine that the user frowns in the eye image when the eye image matches one of the frown image templates. Similarly, the analyzing module102can determine that the user narrows eyes in the eye image when the eye image matches one of the eye narrowing image templates. The analyzing module102can determine that the user closes eyes in the eye image when the eye image matches one of the eye closing image templates.

In at least one exemplary embodiment, the processing module103can adjust the display screen11by adjusting the brightness value of the backlight of the display screen11and/or the proportions between RGB of the display screen11according to the determined eye action. For example, when the user is determined to be closing his/her eyes, the processing module103can inactivate the display screen11to save power.

In other exemplary embodiments, the analyzing module102can analyze whether the user has eye fatigue, and the processing module103can control the electronic device1accordingly to reduce eye fatigue.

In at least one exemplary embodiment, the analyzing module102can calculate a ratio between a first total number of red pixels of the eye image and a second total number of all pixels of the eye image. The analyzing module102can determine that the user has eye fatigue when such ratio is greater than a preset ratio (for example, 70%). The analyzing module102can determine that the user does not have eye fatigue when the ratio is less than or equal to the preset ratio.

In at least one exemplary embodiment, the analyzing module102can first find the red pixels from the eye image according to the RGB value of each of pixels of the eye image. The analyzing module102can calculate the first total number of the red pixels and the second total number of all pixels of the eye image. The analyzing module102can divide the first total number by the second total number to obtain the ratio.

In at least one exemplary embodiment, when the user is determined to be suffering from eye fatigue, the processing module103can further transmit a prompt. For example, the processing module103can display a preset message on the display screen11or can play a predetermined audio message such as “your eyes are fatigued, please have a rest”.

It should be noted that when the user is lack of sleep, the eye area is bloodshot. Thus, the ratio can be used to determine whether the user has eye fatigue.

In at least one exemplary embodiment, when the user has eye fatigue, the processing module103can transmit the prompt to prompt the user to stop using the electronic device1. The processing module103can control the electronic device1to enter a sleep state when the user has eye fatigue. In other exemplary embodiments, the processing module103can further wake up the electronic device1from the sleep state when the electronic device1is kept in the sleep state for a predetermined time length (for example, 10 minutes).

FIG. 4illustrates a flowchart which is presented in accordance with an example embodiment. The exemplary method400is provided by way of example, as there are a variety of ways to carry out the method. The method400described below can be carried out using the configurations illustrated inFIG. 1, for example, and various elements of these figures are referenced in explaining exemplary method400. Each block shown inFIG. 4represents one or more processes, methods, or subroutines, carried out in the exemplary method400. Additionally, the illustrated order of blocks is by example only and the order of the blocks can be changed according to the present disclosure. The exemplary method400can begin at block401. Depending on the embodiment, additional steps can be added, others removed, and the ordering of the steps can be changed.

At block401, the obtaining module101can obtain a pupil image of the user of the electronic device1.

In at least one exemplary embodiment, the obtaining module101can control the image gathering device12to capture an image of the user. The obtaining module101can recognize a face area from the image of the user using a preset face recognition method. The obtaining module101can recognize an eye area from the face area using a preset eye recognition method. The obtaining module101can recognize a pupil area from the eye area, and clip or extract the pupil area from the eye area to obtain the pupil image.

In at least one exemplary embodiment, the preset face recognition method can include, but is not limited to, a face template matching method, or a skin template matching method. The preset eye recognition method can include, but is not limited to, a Hough Transform method, or a template-deformation method. In at least one exemplary embodiment, the obtaining module101can recognize the pupil area from the eye area by comparing the eye area with a preset pupil image template. When an area of the eye area matches with the preset pupil image template, the obtaining module101can determine the area is the pupil area.

At block402, the analyzing module102can determine an eye condition of the user by processing the pupil image and analyzing the processed pupil image.

In at least one exemplary embodiment, the analyzing module102can process the pupil image using a high-contrast and low-light image processing method and obtain a processed pupil image. The analyzing module102can analyze the processed pupil image by analyzing a grayscale and chrominance variances of the processed pupil image.

In at least one exemplary embodiment, as illustrated inFIG. 3A, when the analyzing module102determines that the grayscale of a middle portion21of a pupil image2is within a first preset range such as [50 to 150], the analyzing module102can determine that the eye of the user has cataract, i.e., the user has an eye disease. The analyzing module102can determine that the eye of the user is not normal.

In at least one exemplary embodiment, as illustrated inFIG. 3B, when the analyzing module102determines that the grayscale of the middle portion21is within a second preset range around zero, the analyzing module102can determine that the eye of the user is in a normal state, i.e., the user does not have an eye disease.

In at least one exemplary embodiment, as illustrated inFIG. 3C, when the analyzing module102determines that a red chroma of a side portion22is within a third preset range such as [50 to 150], the analyzing module102can determine that the eye of the user is pinkeye, i.e., the user has eye disease. The analyzing module102can determine that the eye of the user is not normal.

In at least one exemplary embodiment, as illustrated inFIG. 3D, when the analyzing module102determines that a blue chroma of the pupil image2is within a fourth preset range such as [50 to 150], the analyzing module102can determine that the user is wearing contact lens(es)es. The analyzing module102can determine that the eye of the user is not normal.

At block403, the processing module103can adjust the display screen11according to the determined eye condition.

In at least one exemplary embodiment, when the eye of the user is in the normal state, the processing module103can make no change to the display screen11, i.e., the processing module103does not adjust the display screen11.

In at least one exemplary embodiment, the processing module103can adjust the display screen11by adjusting the proportions between RGB when the eye of the user is deemed not normal.

For example, when the analyzing module102determines that the user has an eye disease such as cataract or pinkeye, the processing module103can decrease the blue channel of the display screen11to enable the user to look at the display screen11more comfortably.

In at least one exemplary embodiment, when the analyzing module102determines that the user is wearing contact lens(es), the processing module103can record a start time when the user begins to watch the display screen11while wearing the contact lens(es). The processing module103can calculate a time length of the user wears the contact lens(es) based on the start time. When the time length is equal to or greater than a preset time length (for example, 4 hours), the processing module103can transmit a prompt. For example, the processing module103can display a preset message such as “please remove your contact lens(es)”, or play a preset warning audio message.

In at least one exemplary embodiment, when the analyzing module102determines that the user is wearing contact lens(es), the processing module103can obtain a current time of the electronic device1, and set the current time as the start time. In other exemplary embodiments, the processing module103can provide a user interface and record the start time in response to a user input.

In at least one exemplary embodiment, the obtaining module101obtains pupil images of the user from the start time. When the user is determined to be wearing contact lens(es) in each pupil image obtained during the preset time length, the processing module103can determine that the user is wearing contact lens(es) for the preset time length. The processing module103can transmit the prompt.

In other exemplary embodiments, the obtaining module101can obtain a pupil image of the user when a time length calculated from the start time equals the preset time length. When the analyzing module102determines that the user is wearing contact lens(es) in the pupil image, the analyzing module102determines that the user wears contact lens(es) for the preset time length, and the processing module103can transmit the prompt. That is, in the other exemplary embodiments, only one pupil image is used for determining whether the user wears contact lens(es) for the preset time length.

In other exemplary embodiments, the obtaining module101can obtain a first pupil image of the user when the start time is recorded. The obtaining module101can obtain a second pupil image of the user when a time length calculated from the start time equals the preset time length. When the analyzing module102determines that the user is wearing contact lens(es) in the first pupil image and the second pupil image, the analyzing module102determines that the user has been wearing contact lens(es) for the preset time length, and the processing module103can transmit the prompt. That is, only two pupil images are used for determining whether the user is wearing contact lens(es) for the preset time length.

In at least one exemplary embodiments, the processing module103can adjust the display screen11according to a pupil diameter of the user in the pupil image.

In at least one exemplary embodiments, the analyzing module102can determine the pupil diameter of the user in the pupil image using an image recognition algorithm. The processing module103can adjust a brightness value of the backlight of the display screen11. For example, the processing module103can adjust the brightness value of the backlight of the display screen11when the pupil diameter of the user is determined to be in different preset ranges.

The pupil diameter of the user generally changes according to a brightness value of a surrounding environment. The pupil diameter of the user decreases when the brightness value of the surrounding environment increases. Conversely, the pupil diameter of the user increases when the brightness value of the surrounding environment decreases. In other words, the value of the pupil diameter of the user can represent the current brightness value of the surrounding environment. That is, the processing module103can adjust the brightness value of the backlight of the display screen11according to the pupil diameter of the user.

For a first example, when the analyzing module102determines that the value of pupil diameter of the user is greater than or equal to 2 mm and less than or equal to 3 mm (i.e., [2 to 3 mm]), the processing module103can determine that the user is in an environment with strong or bright light. The processing module103can increase a current brightness value of the backlight of the display screen11. For a second example, when the analyzing module102determines that the value of pupil diameter of the user is greater than 3 mm and less than or equal to 5 mm (i.e., (3 to 5 mm]), the processing module103can determine that the user is in a comfortable environment. The processing module103can keep the current brightness value of the backlight of the display screen11or slightly adjust the current brightness value of the backlight of the display screen11. For a third example, when the analyzing module102determines that the value of pupil diameter of the user is greater than 5 mm and less than or equal to 8 mm (i.e., (5 to 8 mm]), the processing module103can determine that the user is in an environment with low light. The processing module103can decrease the current brightness value of the backlight of the display screen11.

After the blocks401-403are executed, to determine whether the user is adapted to the display screen11that has been adjusted, the method of adjusting the display screen11can further include blocks404-406. Blocks404-406illustrate a first exemplary embodiment of further adjusting the display screen11according to an eye action. A second exemplary embodiment of further adjusting the display screen11according to a determining result of whether the user has eye fatigue is illustrated inFIG. 5.

At block404, the obtaining module101can obtain an eye image of the user.

It should be noted that the method of obtaining the eye image of the user is similar to the method of obtaining the pupil image. When the eye area is recognized from the face area, the obtaining module101can clip or extract the eye area from the face area, thus the eye image is obtained.

At block405, the analyzing module102can determine an eye action by analyzing the eye image. In at least one exemplary embodiment, the eye action can include, but is not limited to, frowning, narrowing eye, and closing eye.

In at least one exemplary embodiment, the analyzing module102can pre-store a number of image templates. The number of image templates includes frown images, eye narrowing images, and eye closing images. The frown image can be defined as an image of eye area in which the user frowns. The eye narrowing image can be defined as an image of eye area in which the user narrows his/her eyes. The eye closing image can be defined as an image of eye area in which the user closes at least one eye. In at least one exemplary embodiment, the analyzing module102can determine that the user frowns in the eye image when the eye image matches one of the frown images templates. Similarly, the analyzing module102can determine that the user narrows eyes in the eye image when the eye image matches one of the eye narrowing images templates. The analyzing module102can determine that the user closes eyes in the eye image when the eye image matches one of the eye closing images templates.

At block406, the processing module103can adjust the display screen11by adjusting the brightness value of the backlight of the display screen11and/or the proportions between RGB of the display screen11according to the determined eye action. For example, when the user is determined to be closing his/her eye, the processing module103can inactivate the display screen11to save power for the electronic device1.

FIG. 5illustrates the second exemplary embodiment of further adjusting the display screen11according to the determining result of whether the user has eye fatigue. The example method500is provided by way of example, as there are a variety of ways to carry out the method. The method500described below can be carried out using the configurations illustrated inFIG. 1, for example, and various elements of these figures are referenced in explaining exemplary method500. Each block shown inFIG. 5represents one or more processes, methods, or subroutines, carried out in the exemplary method500. Additionally, the illustrated order of blocks is by example only and the order of the blocks can be changed according to the present disclosure. The exemplary method500can begin at block501. Depending on the embodiment, additional steps can be added, others removed, and the ordering of the steps can be changed.

At block501, the obtaining module101can obtain an eye image of the user.

At block502, the analyzing module102can calculate a ratio between a first total number of red pixels of the eye image and a second total number of all pixels of the eye image.

In at least one exemplary embodiment, the analyzing module102can first find the red pixels from the eye image according to the RGB value of each of the pixels of the eye image. The analyzing module102can calculate the first total number of the red pixels and the second total number of all pixels of the eye image. Thus, the analyzing module102can calculate the ratio using the first total number and the second total number. The analyzing module102can divide the first total number by the second total number to obtain the ratio.

At block503, the analyzing module102can determine whether the user has eye fatigue according to the ratio. In at least one exemplary embodiment, the analyzing module102can determine that the user has eye fatigue when the ratio is greater than a preset ratio (for example, 70%). The analyzing module102can determine that the user does not have eye fatigue when the ratio is less than or equal to the preset ratio. When the user has eye fatigue, the process goes to block504. When the user does not have eye fatigue, the process goes to block501.

It should be noted that when the user has eye fatigue due to lack of sleep, the eye area is bloodshot. Thus the ratio can be used to determine whether the user has eye fatigue.

At block504, when the user has eye fatigue, the processing module103can transmit a prompt to prompt the user to stop using the electronic device1. For example, the processing module103can display a preset message on the display screen11or can play a predetermined audio message such as “you are suffering eye fatigue, please have a rest”. The processing module103can control the electronic device1to enter a sleep state when the user has eye fatigue. In at least one exemplary embodiment, the processing module103can further wake up the electronic device1from the sleep state when the electronic device1is kept in the sleep state for a predetermined time length (for example, 10 minutes).