Patent Description:
With the development of mobile intelligent terminals, the technology of the screens of mobile phones has become increasingly mature. Also, a wave of full-screen phones has been set off in the mobile phone market in recent years. In order to increase the screen-to-body ratio of a display screen of a mobile phone, the arrangement of a front camera of the mobile phone is a key issue. In the existing products and technical schemes, there are also many design schemes found intended to increase the screen-to-body ratio of mobile phones, for example, by reducing the width of the forehead of a mobile phone, applying a pop-up camera, etc..

<CIT> discloses an optical isolation system used in a display to reduce light that is transmitted from one or more light sources to a camera.

<CIT> discloses a display panel comprising a display area having a transparent display area and a non-transparent display area surrounding the transparentdisplay area, wherein the display panel further comprises a first substrate layer, a light emitting layer, a liquid crystal layer, and first and second driving layers.

<CIT> discloses a mobile device with a light sensor overlaid under a display , the mobile device can identify a command to capture an image, the mobile device can adjust at least a target portion of an opaqueness adjustable region of the display directly over the light sensor.

<CIT> discloses an electronic device, which includes a display screen including a first display area and a second display area, the first display area having a resolution smaller than that of the second display area, and agap exists between at least two adjacent pixel units in the first display area.

The invention is defined in the independent claims, while advantageous embodiments are defined in the dependent claims.

The present disclosure provides an electronic device and method for performing photographing and displaying with an under-screen camera and a computer-readable storage medium to enable a mobile phone to automatically adjust the size of an area with no display content around the camera according to the screen brightness when a user takes a picture with the mobile phone.

In an embodiment, the present disclosure provides an electronic device, including a display screen, a camera, a memory and a processor, where the camera is provided under the display screen, program instructions are stored in the memory, and the processor is configured to process the program instructions to implement the following steps: generating a first area and a second area in the display screen, wherein the first area is centered on a center of the lens of the camera; determining a preset area in the display screen according to user selection, wherein the first area is located within the preset area; and adjusting the size of the first area according to the screen brightness of the display screen, wherein the size of the first area is increased with the increase of the screen brightness of the display screen within the preset area; when the camera is in the photographing state, the first area has no display content while the second area displays a preview image of a photographed object that is acquired by the camera; when the camera is not in the photographing state, both the first area and the second area display content.

In another embodiment, the present disclosure provides a method for performing photographing and displaying with an under-screen camera, including: generating a first area and a second area in the display screen, wherein the first area is centered on a center of the lens of the camera; determining a preset area in the display screen according to user selection, wherein the first area is located within the preset area; and adjusting the size of the first area according to the screen brightness of the display screen, wherein the size of the first area is increased with the increase of the screen brightness of the display screen within the preset area; when the camera is in the photographing state, the first area has no display content while the second area displays a preview image of a photographed object that is acquired by the camera; when the camera is not in the photographing state, both the first area and the second area display content.

In yet another embodiment, the present disclosure provides a computer program which, when executed by a processor, causes the processor to implement the method for performing photographing and displaying with an under-screen camera as described above.

The accompanying drawings are used to provide a further understanding of the technical schemes of the present disclosure and constitute a part of the description. The accompanying drawings are used to explain the technical schemes of the present disclosure together with the embodiments of the present disclosure and do not constitute a restriction on the technical schemes of the present disclosure.

In order to make the technical schemes and advantages of the present disclosure clear, the present disclosure will be further described in detail with reference to the accompanying drawings and by way of example. It should be noted that any combinations of embodiments and features of the embodiments of the present disclosure without conflict are possible.

A hardware architecture diagram of an electronic device <NUM> provided by an embodiment of the present disclosure is shown in <FIG>. The electronic device <NUM> may be a mobile phone, a tablet, a computer, etc. As shown in <FIG>, the electronic device <NUM> includes a camera <NUM> and a display screen <NUM>. There may be one or more cameras <NUM> provided. In this embodiment, an electronic device <NUM> including only one camera <NUM> is taken as an example to explain the present disclosure. The camera <NUM> is configured to acquire preview image data of a photographed object. When the camera <NUM> is in a photographing state, external light rays may enter the camera <NUM> through the display screen <NUM>, so as to image the photographed object. The camera <NUM> is provided under the display screen <NUM> and parallel to the display screen <NUM>. There may be a certain distance between the camera <NUM> and the display screen <NUM>, or they may be closely attached to each other. In this embodiment, the display screen <NUM> may be a transparent display screen, including but not limited to an active-matrix organic light emitting diode (AMOLED) display screen. An area with no display content and a normal area with display contents are displayed on the display screen <NUM> when the camera <NUM> is in a photographing state, or the area with no display content of the display screen <NUM> is displayed as a normal area with display contents when the camera <NUM> is not in the photographing state. The electronic device <NUM> further includes a processor <NUM> and a memory <NUM>. The memory <NUM> may be configured to store computer programs, for example, software programs and modules of application software. For example, in the embodiment of the present disclosure, the memory <NUM> stores a photographing and displaying unit <NUM>. The photographing and displaying unit <NUM> processes a computer program corresponding to a method for performing photographing and displaying with an under-screen camera for the electronic device <NUM>. The processor <NUM> runs the computer program stored in the memory <NUM> to perform various functional applications and data processing, that is, to implement the method as described above. The memory <NUM> may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage apparatuses, a flash memory, or other non-volatile solid-state memories. In some examples, the memory <NUM> may include memories remotely located with respect to the processor <NUM>, and these remote memories may be connected to the electronic device <NUM> through a network. Examples of the above-mentioned network include the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The photographing and displaying unit <NUM> is configured to generate an area with no display content according to a screen brightness of the display screen <NUM>, so as to synthesize a picture of a display layer which corresponds to the area with no display content and a preview image of a photographed object that is acquired by the camera <NUM> into a display image of the photographed object, and display the display image on the display screen <NUM>.

A diagram of functional modules of a photographing and displaying unit <NUM> of an embodiment of the present disclosure is shown in <FIG>. The photographing and displaying unit <NUM> includes the following modules.

A receiving module <NUM> is configured to receive the preview image of the photographed object that is acquired by the camera <NUM>. When the camera <NUM> is in the photographing state, external light rays may enter a lens of the camera <NUM> through the display screen <NUM> to image the photographed object. After light is emitted from screen display pixels of the display screen <NUM>, some of the emitted light rays are reflected by a glass cover plate of the display screen <NUM>, and the reflected light rays will enter the lens of the camera <NUM> and become noise light rays of the camera <NUM>. The intensity of the noise light rays decreases as the distance between the light-emitting screen display pixels and the lens of the camera <NUM> increases, until the light rays will no longer become the noise light rays affecting the imaging quality of the camera <NUM>. As shown in <FIG>, when the screen display pixels <NUM> emit light, some of the light rays <NUM> emitted thereby are reflected into the lens of the camera <NUM> through the glass cover plate <NUM> of the display screen <NUM>, and become noise light rays affecting the imaging quality of the camera <NUM>. As shown in <FIG>, when the screen display pixels <NUM> emit light and some of the light rays <NUM> emitted thereby are not reflected into the lens of the camera <NUM> through the glass cover plate <NUM> of the display screen <NUM>, the emitted light rays <NUM> do not constitute noise light rays affecting the imaging quality of the camera <NUM>.

A monitoring module <NUM> is configured to monitor a magnitude of electric current supplied to the display screen <NUM>, so as determine the screen brightness of the display screen <NUM> according to the magnitude of the electric current. The magnitude of the electric current supplied to the display screen <NUM> is directly proportional to the screen brightness of the display screen <NUM>. The larger the magnitude of the electric current supplied to the display screen <NUM>, the higher the screen brightness of the display screen <NUM>. The smaller the magnitude of the electric current supplied to the display screen <NUM>, the lower the screen brightness of the display screen <NUM>.

A generation module <NUM> is configured to generate an area with no display content within a preset area according to the screen brightness of the display screen <NUM>. The preset area is an area preset by a manufacturer when the electronic device <NUM> leaves the factory. Alternatively, the preset area may also be an area determined according to user selection. The area with no display content may be of any shape, such as a circle or a rectangle, which is not limited here. In this embodiment, taking a circular area with no display content as an example, as shown in <FIG>, there is a circular area <NUM> with no display content on the display screen <NUM>, and the area <NUM> with no display content is centered on a center <NUM> of the lens of the camera <NUM>. As shown in <FIG>, there are two circles with different radii on the display screen <NUM>, both of which are centered on the center <NUM> of the lens of the camera <NUM>. When the camera <NUM> is in the photographing state, an area inside the circle is the area with no display content. When the screen brightness of the display screen <NUM> is lower, the area with no display content is the inner circle <NUM>. When the screen brightness of the display screen <NUM> is higher, the area with no display content is the outer circle <NUM>'. The circle radius of the area with no display content increases with the increase of the screen brightness of the display screen <NUM>, that is, the range of the area with no display content can be adjusted according to the screen brightness of the display screen <NUM>. In this embodiment, a certain threshold may also be set for the area <NUM> with no display content. When the circle radius of the area <NUM> with no display content reaches a certain threshold, the circle radius of the area <NUM> with no display content will no longer increase with the increase of the screen brightness of the display screen <NUM>.

In another embodiment, the area <NUM> with no display content may also be a fixed area around the camera <NUM>, and the area of the area <NUM> with no display content does not change when the screen brightness of the display screen <NUM> has changed. The area <NUM> with no display content may be of any shape, which is not limited here. In this embodiment, a rectangular area <NUM> with no display content is taken as an example. As shown in <FIG>, the area <NUM> with no display content is a rectangular area in the upper half of the display screen <NUM>, and the area of the rectangular area does not change when the screen brightness of the display screen <NUM> has changed, while the remaining area of the display screen <NUM> is a normal area <NUM> with display contents.

When the camera <NUM> is in the photographing state, the display screen <NUM> is in the form of a photographing preview interface. An addition module <NUM> is configured to add a display layer which corresponds to the area <NUM> with no display content to the photographing preview interface. The display layer is a picture generated according to the screen brightness of the display screen <NUM>, and the picture of the display layer may be of any shape, such as a circle or a rectangle. The picture of the display layer is configured to overlay the preview image of the photographed object that corresponds to the area with no display content <NUM>. The picture of the display layer which corresponds to the area <NUM> with no display content and the preview image of the photographed object that is acquired by the camera <NUM> are synthesized into a display image of the photographed object. Here, there are no restrictions on the display pixels of the display layer, as long as, when the camera <NUM> is in the photographing state, external light rays can enter the camera <NUM> through the display screen <NUM> so as to image the photographed object. As shown in <FIG>, when the camera <NUM> is in the photographing state, the display screen <NUM> is in the form of the photographing preview interface, and a display layer corresponding to the area <NUM> with no display content is added to the photographing preview interface. The display layer is a circular picture with a certain radius generated according to the screen brightness of the display screen <NUM>. The circular picture of the display layer is configured to overlay the preview image of the photographed object that corresponds to the area with no display content <NUM>. The display pixels (R, G, B) of all pixels on the circular picture of the display layer are (<NUM>, <NUM>, <NUM>), that is, the circular picture of the display layer is a black circular picture, and the area where the black circular picture is located is the area <NUM> with no display content. Since the display pixels (R, G, B) of all the pixels on the black circular picture are (<NUM>, <NUM>, <NUM>), when the display screen <NUM> is in the form of the photographing preview interface, the area <NUM> with no display content is a transparent area with no display content, and the remaining area of the display screen <NUM> is a normal area <NUM> with display contents in which there are display contents, as shown in <FIG>. When the camera <NUM> is not in the photographing state, the display screen <NUM> is not in the form of the photographing preview interface, the area <NUM> with no display content is filled with display contents and becomes a non-transparent area, and the whole screen of the display screen <NUM> is normally displayed with no circular transparent area.

An image synthesizing module <NUM> is configured to synthesize the picture of the display layer which corresponds to the area <NUM> with no display content and the preview image of the photographed object that is acquired by the camera <NUM> into a display image of the photographed object.

A display module <NUM> is configured to send the display image to the display screen <NUM> for display. The display image is an image presented when the display screen <NUM> is in the form of the photographing preview interface. As shown in <FIG>, when the display screen <NUM> is in the form of the photographing preview interface, the display screen <NUM> displays the area <NUM> with no display content and the normal area <NUM> with display contents. The area <NUM> with no display content is a transparent area with no display content, and the normal area <NUM> with display contents is a non-transparent area with display contents.

A flowchart of a method of performing photographing and displaying with an under-screen camera for the electronic device <NUM> in an embodiment of the present disclosure is shown in <FIG>.

At step S50, a receiving module <NUM> receives a preview image of a photographed object that is acquired by the camera <NUM>. When the camera <NUM> is in the photographing state, external light rays may enter a lens of the camera <NUM> through the display screen <NUM> to image the photographed object. After light is emitted from screen display pixels of the display screen <NUM>, some of the emitted light rays are reflected by a glass cover plate of the display screen <NUM>, and the reflected light rays will enter the lens of the camera <NUM> and become noise light rays of the camera <NUM>. The intensity of the noise light rays decreases as the distance between the light-emitting screen display pixels and the lens of the camera <NUM> increases, until the light rays will no longer become the noise light rays affecting the imaging quality of the camera <NUM>. As shown in <FIG>, when the screen display pixels <NUM> emit light, some of the light rays <NUM> emitted thereby are reflected into the lens of the camera <NUM> through the glass cover plate <NUM> of the display screen <NUM>, and become noise light rays affecting the imaging quality of the camera <NUM>. As shown in <FIG>, when the screen display pixels <NUM> emit light and some of the light rays <NUM> emitted thereby are not reflected into the lens of the camera <NUM> through the glass cover plate <NUM> of the display screen <NUM>, the emitted light rays <NUM> do not constitute noise light rays affecting the imaging quality of the camera <NUM>.

At step S51, a monitoring module <NUM> monitors a magnitude of electric current supplied to the display screen <NUM>, so as to determine the screen brightness of the display screen <NUM> according to the magnitude of the electric current. The magnitude of the electric current supplied to the display screen <NUM> is directly proportional to the screen brightness of the display screen <NUM>. The larger the magnitude of the electric current supplied to the display screen <NUM>, the higher the screen brightness of the display screen <NUM>. The smaller the magnitude of the electric current supplied to the display screen <NUM>, the lower the screen brightness of the display screen <NUM>.

At step S52, a generation module <NUM> generates an area with no display content within a preset area according to the screen brightness of the display screen <NUM>. The preset area is an area preset by a manufacturer when the electronic device <NUM> leaves the factory. Alternatively, the preset area may also be an area determined according to user selection. The area with no display content may be of any shape, such as a circle or a rectangle, which is not limited here. In this embodiment, taking a circular area with no display content as an example, as shown in <FIG>, there is a circular area <NUM> with no display content on the display screen <NUM>, and the area <NUM> with no display content is centered on a center <NUM> of the lens of the camera <NUM>. As shown in <FIG>, there are two circles with different radii on the display screen <NUM>, both of which are centered on the center <NUM> of the lens of the camera <NUM>. When the camera <NUM> is in the photographing state, an area inside the circle is the area with no display content. When the screen brightness of the display screen <NUM> is lower, the area with no display content is the inner circle <NUM>. When the screen brightness of the display screen <NUM> is higher, the area with no display content is the outer circle <NUM>'. The circle radius of the area with no display content increases with the increase of the screen brightness of the display screen <NUM>, that is, the range of the area with no display content can be adjusted according to the screen brightness of the display screen <NUM>. In this embodiment, a certain threshold may also be set for the area <NUM> with no display content. When the circle radius of the area <NUM> with no display content reaches a certain threshold, the circle radius of the area <NUM> with no display content will no longer increase with the increase of the screen brightness of the display screen <NUM>.

At step S53, when the camera <NUM> is in the photographing state, the display screen <NUM> is in the form of a photographing preview interface. An addition module <NUM> adds a display layer which corresponds to the area <NUM> with no display content to the photographing preview interface. The display layer is a picture generated according to the screen brightness of the display screen <NUM>, and the picture of the display layer may be of any shape, such as a circle or a rectangle. The picture of the display layer is configured to overlay the preview image of the photographed object that corresponds to the area with no display content <NUM>. The picture of the display layer which corresponds to the area <NUM> with no display content and the preview image of the photographed object that is acquired by the camera <NUM> are synthesized into a display image of the photographed object. Here, there are no restrictions on the display pixels of the display layer, as long as, when the camera <NUM> is in the photographing state, external light rays can enter the camera <NUM> through the display screen <NUM> so as to image the photographed object. As shown in <FIG>, when the camera <NUM> is in the photographing state, the display screen <NUM> is in the form of the photographing preview interface, and a display layer corresponding to the area <NUM> with no display content is added to the photographing preview interface. The display layer is a circular picture with a certain radius generated according to the screen brightness of the display screen <NUM>. The circular picture of the display layer is configured to overlay the preview image of the photographed object that corresponds to the area with no display content <NUM>. The display pixels (R, G, B) of all pixels on the circular picture of the display layer are (<NUM>, <NUM>, <NUM>), that is, the circular picture of the display layer is a black circular picture, and the area where the black circular picture is located is the area <NUM> with no display content. Since the display pixels (R, G, B) of all the pixels on the black circular picture are (<NUM>, <NUM>, <NUM>), when the display screen <NUM> is in the form of the photographing preview interface, the area <NUM> with no display content is a transparent area with no display content, and the remaining area of the display screen <NUM> is a normal area <NUM> with display contents in which there are display contents, as shown in <FIG>. When the camera <NUM> is not in the photographing state, the display screen <NUM> is not in the form of the photographing preview interface, the area <NUM> with no display content is filled with display contents and becomes a non-transparent area, and the whole screen of the display screen <NUM> is normally displayed with no circular transparent area.

At step S54, an image synthesizing module <NUM> synthesizes the picture of the display layer which corresponds to the area <NUM> with no display content and the preview image of the photographed object that is acquired by the camera <NUM> into a display image of the photographed object.

At step S55, a display module <NUM> sends the display image to the display screen <NUM> for display. The display image is an image presented when the display screen <NUM> is in the form of the photographing preview interface. As shown in <FIG>, when the display screen <NUM> is in the form of the photographing preview interface, the display screen <NUM> displays the area <NUM> with no display content and the normal area <NUM> with display contents. The area <NUM> with no display content is a transparent area with no display content, and the normal area <NUM> with display contents is a non-transparent area with display contents.

At step S60, an area <NUM> with no display content is generated within a preset area according to a screen brightness of the display screen <NUM>.

At step S61, a picture of a display layer which corresponds to the area <NUM> without display content and a preview image of a photographed object that is acquired by the camera <NUM> are synthesized into a display image of the photographed object.

In another embodiment, a computer program is stored in a computer-readable storage medium, where the computer program is configured to implement the method for performing photographing and displaying with an under-screen camera as described above when the computer program is executed, for example, by a processor.

According to the technical schemes of the present disclosure, by means of generating an area with no display content within a preset area according to a screen brightness of a display screen, and synthesizing a picture of a display layer which corresponds to the area without display content and a preview image of a photographed object that is acquired by the camera into a display image of the photographed object, when an electronic device enters a photographing state, the area with no display content around the camera can be automatically adjusted according to the screen brightness of the display screen, thereby reducing light noise caused by screen pixels of the display screen and improving the imaging effect when the under-screen camera takes a picture.

Claim 1:
An electronic device, comprising a display screen (<NUM>), a camera (<NUM>), a memory (<NUM>) and a processor (<NUM>), wherein the camera (<NUM>) is provided under the display screen (<NUM>), and the memory (<NUM>) is configured to store program instructions, wherein the processor (<NUM>) is configured to process the program instructions to execute the following steps:
generating a first area (<NUM>, <NUM>') and a second area (<NUM>) in the display screen (<NUM>), wherein the first area (<NUM>, <NUM>') is centered on a center (<NUM>) of the lens of the camera (<NUM>);
determining a preset area in the display screen (<NUM>) according to user selection, wherein the first area (<NUM>, <NUM>') is located within the preset area; and
adjusting the size of the first area (<NUM>, <NUM>') according to the screen brightness of the display screen (<NUM>), wherein the size of the first area (<NUM>, <NUM>') is increased with the increase of the screen brightness of the display screen (<NUM>) within the preset area;
when the camera (<NUM>) is in the photographing state, the first area (<NUM>, <NUM>') has no display content while the second area (<NUM>) displays a preview image of a photographed object that is acquired by the camera (<NUM>);
when the camera (<NUM>) is not in the photographing state, both the first area (<NUM>, <NUM>') and the second area (<NUM>) display content.