Patent Description:
With the development of intelligent terminals, the requirement on the screen-to-body ratio of intelligent terminals is getting higher and higher. The screen-to-body ratio of an intelligent terminal can be maximized by arranging an under-panel camera under a preset area of a display screen of the intelligent terminal, which has become an important development direction of intelligent terminals. When a user turns on the under-panel camera for photographing, the area corresponding to the under-panel camera is set not to display contents, so that light can enter the camera through this area, thus achieving the photographing function of the camera. The area corresponding to the under-panel camera is usually black when no content is displayed.

The inventor discovered that in some cases, when photographing is performed using the under-panel camera, the area corresponding to the under-panel camera is black, and if the preset area and the under-panel camera are arranged in the central area of the display screen, a black area will appear at the center of a displayed picture during photographing, seriously affecting the display effect of the picture. Therefore, the under-panel camera is usually arranged in an edge area of the display screen, which leads to an inflexible arrangement of the under-panel camera of the intelligent terminal.

<CIT> discloses a method and an apparatus for taking an image and conducting video communication. A synchronization controller is used to control the front camera and the transparent display in such a manner that image capturing of the front camera alternates with screen output of the transparent display, that is, the front camera captures an image while the transparent display does not output a screen and the transparent display outputs a screen while the front camera does not capture an image, in each video frame, so that the front camera may be aligned with the position of a preview screen. Therefore, the user may look at the preview screen to monitor his or her appearance and thus the user is shown as looking straight ahead in the displayed preview screen.

<CIT> discloses a mobile device with a front camera, which includes a main body and a display screen arranged on the main body, and further includes a front camera. The front camera is arranged at a rear side of the display screen and located at an inner side of the main body to align with the display screen, and the front camera is used for capturing images through the display screen. The mobile device further includes a timing unit and a control unit. The timing unit is used for setting the period and alternating time of the operations of the display screen and the front camera. The control unit is used for alternately controlling the operations of the display screen and the front camera according to the alternating time within the period set by the timing unit.

<CIT> discloses a mobile device with a light sensor overlaid under a display (e.g., a touchscreen). The mobile device can identify a command to capture an image. The mobile device can adjust at least a target portion of an opaqueness adjustable region of the display directly over the light sensor. The opaqueness adjustable region is capable of transforming from substantially opaque to substantially transparent. The mobile device can capture the image utilizing at least the light sensor while the target portion of the opaqueness adjustable region is substantially transparent.

While the above patent applications may achieve their intended purposes, there is still a need for a new and improved photographing method.

Embodiments of the disclosure provide a photographing method, an electronic device, and a storage medium, which enables an under-panel camera to be flexibly arranged at any position under a display screen.

The features of the photographing method, the electronic device and the storage medium according to embodiments of the present invention are set out in the appended set of claims.

Exemplary description will be made for one or more embodiments with reference to the figures in the corresponding accompanying drawings without constituting a limitation to the embodiments. Elements with the same reference numeral in the accompanying drawings represent similar elements, and unless stated otherwise, the figures in the accompanying drawings do not constitute a scale limitation.

In order to make the object, technical schemes and advantages of the embodiments of the disclosure clearer, each embodiment of the disclosure will be set forth in detail hereinafter with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that many technical details are put forward in each embodiment of the disclosure in order for readers to better understand the disclosure.

<FIG> is a schematic diagram of a display device to which a photographing method is applied, according to an embodiment of the disclosure. As shown in <FIG>, this scenario includes a display screen <NUM>, a shading material <NUM>, and an under-panel camera <NUM>. The shading material <NUM> is attached under the display screen <NUM>, and a preset area of the shading material <NUM> is grooved to obtain a partially grooved shading material <NUM>. The under-panel camera <NUM> is arranged under the preset area of the shading material <NUM>. When no content is displayed on an area of the display screen <NUM> corresponding to the preset area of the shading material <NUM>, light can enter the under-panel camera <NUM> through the display screen, so that the under-panel camera <NUM> can perform photographing.

When photographing is performed using the under-panel camera <NUM>, the area of the display screen <NUM> not displaying contents usually appears black, that is, the area of the display screen <NUM> corresponding to the preset area of the shading material <NUM> appears black. If the preset area is arranged in a central area of the display screen, a black area will appear at the center of a displayed picture during photographing, seriously affecting the display effect of the picture. Therefore, the under-panel camera is typically arranged in an edge area of the display screen, which leads to an inflexible arrangement of the under-panel camera.

Alternatively, the display screen is an active-matrix organic light-emitting diode (AMOLED). As shown in <FIG>, the AMOLED may include:.

The shading material <NUM>, which is attached under the transparent substrate <NUM>, may be shading foam, and a light-transmitting area is formed by partially perforating the shading foam, so that light can pass through the cathode <NUM>, the organic light-emitting layer <NUM>, the anode <NUM>, the substrate <NUM> and the perforated area of the shading material <NUM> to get into the under-panel camera <NUM>.

<FIG> is a schematic flowchart of a photographing method in an embodiment. In this embodiment, when a camera startup instruction is received, an electronic device performs alternated steps of displaying an image on the display screen and acquiring an image with the camera, so that no black area will appear on the display screen when an image is displayed and the arrangement of the under-panel camera in any area of the display screen will not affect the display effect of a picture. The details of implementation of a subfield division method of the present embodiment will be described below. The following contents are merely the details of implementation provided to facilitate understanding, and are not necessary for the implementation of this scheme. As shown in <FIG>, the method includes steps S101 to S <NUM>.

At S101, after a camera startup instruction is received, alternated steps of displaying an image on a display screen and acquiring an image with a camera are executed.

The camera startup instruction may be used to instruct the electronic device to start photographing by means of the under-panel camera. The camera startup instruction may be at least one of a voice instruction, a touch instruction and a text instruction, which is not limited herein. When receiving the camera startup instruction, the electronic device may perform photographing by executing the alternated steps of displaying an image on the display screen and acquiring an image with the camera. The electronic device may immediately execute the aforementioned alternated steps when receiving the camera startup instruction, or execute the aforementioned alternated steps after displaying a preview image through the display screen first, which is not limited herein. The alternated steps include the following steps.

At S102, after displaying an image, the display screen is set into a transparent state.

When the display screen displays images, multiple frames of images are typically displayed per second according to a display refresh rate of the display screen. For example, if the display refresh rate of the display screen indicates that the display screen displays <NUM> frames of images per second, the duration occupied by each frame of image is <NUM>/<NUM> second, that is, the display screen can display a frame of image in <NUM>/<NUM> second. After completing the display of an image, the display screen may be set into the transparent state. When the display screen is set into the transparent state, light can enter the under-panel camera through the display screen, to allow the under-panel camera to perform photographing. It should be noted that when the display screen is in the transparent state, it may mean that part of the display screen or the whole display screen is in a state of not displaying contents, which is not limited herein. For example, when the display screen is in the transparent state, it may mean that the area of the display screen corresponding to the under-panel camera does not display content, and usually appears black. It should be noted that the electronic device may set the display screen into the transparent state through a display screen integrated circuit (IC) or a processor, which is not limited herein.

At S103, the under-panel camera is controlled to acquire an image, and the acquired image is sent to the display screen for display.

When the display screen is set into the transparent state, light can enter the under-panel camera through the display screen, to allow the under-panel camera to perform photographing to acquire an image. In this case, the under-panel camera is controlled to acquire a frame of image, and then the image acquired by the under-panel camera is sent to the display screen for the display screen to display the acquired preview image. It should be noted that when the image acquired by the under-panel camera is sent to the display screen for display, the acquired image may be directly sent to the display screen for display through the under-panel camera, or the under-panel camera sends the acquired image to a processor of the electronic device and the processor sends the image to the display screen for display, which is not limited herein.

At S <NUM>, the alternated steps are executed until an end instruction indicating completion of photographing is received.

After the image acquired by the under-panel camera is sent to the display screen for display, the aforementioned alternated steps are executed, that is, the aforementioned steps of displaying an image on the display screen and acquiring an image with the camera are alternately executed. Typically, it will take a very short time (e.g. <NUM>/<NUM> second) for the display screen to display a frame of image and for the under-panel camera to acquire an image. By executing the alternated steps of displaying an image on the display screen and acquiring an image with the camera, e.g. displaying an image in the <NUM>/<NUM>th second, acquiring an image with the under-panel camera in the <NUM>/<NUM>th second, displaying the image in the <NUM>/<NUM>th second, acquiring an image with the under-panel camera in the <NUM>/<NUM>th second and so on, i.e. quick switching between the image displaying by the display screen and image acquiring by the camera, the user's eyes cannot notice the switching of contents displayed by the display screen when observing a photographing interface. The end instruction is used to indicate the completion of photographing. The end instruction may be at least one of a voice instruction, a touch instruction and a text instruction, which is not limited herein. When the end instruction is received, the electronic device stops executing the aforementioned alternated steps. The electronic device may immediately stop executing the aforementioned alternated steps when receiving the end instruction, or stop executing the aforementioned alternated steps after displaying a frame of end preview image through the display screen first, which is not limited herein.

Generally, if the number of frames of images displayed by the display screen within one second is greater than a threshold, the human eyes will not perceive the flicker of the display screen when observing the display screen. By alternately executing the steps of displaying an image on the display screen and acquiring an image with the camera, the number of frames of images displayed by the display screen will not be less than the aforementioned threshold, achieving the function of displaying images on the display screen in the process of photographing. For example, if the display refresh rate of the display screen indicates that the display screen displays <NUM> frames of images per second, by alternately executing the steps of displaying an image on the display screen and acquiring an image with the camera, the number of frames of images displayed by the display screen within one second is <NUM>, the number of frames of images acquired by the camera within one second is <NUM>, and the aforementioned threshold is generally <NUM> frames, that is, the number (<NUM>) of frames of images displayed by the display screen within one second is greater than the threshold (<NUM>), and in this case, the human eyes will not perceive the flicker of the display screen.

In this embodiment, compared with the existing technology, during photographing by the under-panel camera, after a frame of image is displayed by the display screen, the display screen is set into the transparent state, to allow the under-panel camera to acquire an image by means of light transmitted through the display screen. Then, the acquired image is sent to the display screen for display. Meanwhile, during photographing, the steps of displaying an image on the display screen and acquiring an image with the camera are alternately executed, so that no black area will appear on the display screen when an image is displayed. Therefore, the arrangement of the under-panel camera in any area of the display screen will not affect the display effect of a picture, that is, the flexibility of the arrangement of the under-panel camera can be increased.

In an embodiment, when the display screen needs to be set into the transparent state, the display screen may be set into the transparent state by a display screen integrated circuit (IC). After the display screen is set into the transparent state by the display screen IC, a confirmation signal used to indicate that the display screen is in the transparent state may be sent to the processor. After receiving the aforementioned confirmation signal, the processor controls the under-panel camera to acquire data. The embodiment shown in <FIG> will be described in detail below.

As shown in <FIG>, the method further includes steps S201 to S202.

At S201, the display screen is set into the transparent state by the display screen IC.

The display screen IC may be an integrated circuit arranged on the display screen. The display screen IC may set the display screen to the transparent state by setting a level of signals of a signal source of the display screen to be less than a preset low-level threshold and setting a level of gate signals of the display screen to be greater than a preset high-level threshold. For example, the display screen IC may set the level of source signals to 0V and the level of gate signals to 5V, so that the display screen can be in the transparent state.

At S202, after receiving a confirmation signal sent by the display screen IC, the under-panel camera is controlled to acquire an image, where the confirmation signal is used to indicate that the display screen is in the transparent state.

The confirmation signal is used to indicate that the display screen is in the transparent state. The display screen IC may generate the aforementioned confirmation signal after confirming that the display screen has completed image display. When the display screen completes the display of a frame of image, all pixels on the display screen are lit up, i.e. activated. The display screen IC can detect whether or not pixels on the display screen are activated, and when the pixels on the display screen are activated, the display screen IC can generate a confirmation signal which is used to indicate that the display screen has completed the display of a frame of image, and send the confirmation signal to the processor of the electronic device, so that the processor of the electronic device can set the display screen into the transparent state after receiving the confirmation signal. Alternatively, the aforementioned confirmation signal is generated when the display screen IC detects that target pixels on the display screen are activated. The target pixels may be all or part of the pixels on the display screen, which is not limited herein. Alternatively, the target pixels are a last row of pixels on the display screen.

According to the photographing method, after the confirmation signal sent by the display screen IC is received, the display screen is set into the transparent state, and therefore the under-panel camera can only set the display screen into the transparent state after a determination is made that the display screen has completed the display of a current frame of image, thus avoiding black screen of the display screen as a result of the display screen being set into a black transparent state when the display screen has not complete the display of the image, increasing the reliability of the display screen in displaying images. For example, the display screen IC sends a confirmation signal to the processor when the target pixels are activated, that is, the confirmation signal can more accurately reflect that the display screen has completed the display of an image, further avoiding the situation that the under-panel camera sets the display screen into the black transparent state when the display screen has not complete the display of an image, thereby further increasing the reliability of the display screen in displaying images. For example, when the target pixels are the last row of pixels on the display screen, i.e. part of the pixels on the display screen, the number of pixels to be detected is small in determination of whether or not the target pixels on the display screen are activated, thus increasing the efficiency of determining the confirmation signal.

In an embodiment not covered by the appended claims, , the executing body is the processor of the electronic device, and the aforementioned alternated steps may be alternately executed at a preset acquisition time and a preset display time. Alternatively, at the preset acquisition time, the under-panel camera is controlled to acquire a frame of image, and at the preset display time, the acquired frame of image is sent to the display screen for display. The preset acquisition time alternates with the preset display time.

The alternation of the preset acquisition time and the preset display time may be the alternation of one preset acquisition time and one preset display time, the alternation of multiple preset acquisition times and multiple preset display times, or the alternation of one preset acquisition time and multiple preset display times, which is not limited herein. For example, if the alternation of the preset acquisition time and the preset display time is the alternation of one preset acquisition time and one preset display time, a time interval between a preset acquisition time and an adjacent preset display time is the duration of displaying a frame of image. It is assumed that the duration taken by a frame of image is <NUM>/<NUM> second, the preset acquisition time is <NUM>/<NUM>th second, <NUM>/<NUM>th second, <NUM>/<NUM>th second and so on, and the preset display time is <NUM>/<NUM>th second, <NUM>/<NUM>th second, <NUM>/<NUM>th second and so on. A frame of image acquired by the under-panel camera at the <NUM>/<NUM>th second is displayed by the display screen at the <NUM>/<NUM>th second; a frame of image acquired by the under-panel camera at the <NUM>/<NUM>th second is displayed by the display screen at the <NUM>/<NUM>th second; a frame of image acquired by the under-panel camera at the <NUM>/<NUM>th second is displayed by the display screen at the <NUM>/<NUM>th second, and so on. For another example, if the alternation of the preset acquisition time and the preset display time is the alternation of one preset acquisition time and multiple preset display times and the duration taken by a frame of image is <NUM>/<NUM> second, the preset acquisition time is <NUM>/<NUM>th second, <NUM>/<NUM>th second, <NUM>/<NUM>th second and so on, and the preset display times are <NUM>/<NUM>th second and <NUM>/<NUM>th second, <NUM>/<NUM>th second and <NUM>/<NUM>th second, <NUM>/<NUM>th second and <NUM>/<NUM>th second and so on. A frame of image acquired by the under-panel camera at the <NUM>/<NUM>th second is displayed by the display screen at the <NUM>/<NUM>th second and the <NUM>/<NUM>th second respectively; a frame of image acquired by the under-panel camera at the <NUM>/<NUM>th second is displayed by the display screen at the <NUM>/<NUM>th second and the <NUM>/<NUM>th second respectively; a frame of image acquired by the under-panel camera at the <NUM>/<NUM>th second is displayed by the display screen at the <NUM>/<NUM>th second and the <NUM>/<NUM>th second respectively, and so on.

In the aforementioned photographing method not covered by the appended claims, by directly setting the preset display time and the preset acquisition time to be alternated, in the process of photographing, the under-panel camera is controlled to acquire an image at the preset acquisition time and the acquired image is displayed by the display screen at the preset display time, thus avoiding the signal interaction between the processor and the display screen IC, increasing the convenience of photographing.

In an example, after receiving a camera startup instruction, the electronic device may also increase the display refresh rate of the display screen, so that the number of frames of images displayed by the display screen will not sharply drop due to the number of frames occupied by the under-panel camera acquiring images. Alternatively, the display refresh rate of the display screen is increased to obtain an updated display refresh rate. The display refresh rate is used to indicate the number of frames of images displayed per unit time.

The updated display refresh rate is higher than the initial display refresh rate, so when the alternated steps of displaying an image on the display screen and acquiring an image with the camera are executed, the number of frames occupied by the display screen displaying images will not decrease sharply. Generally, the electronic device increases the display refresh rate of the display screen by means of a timing controller. In a possible case, alternatively, the updated display refresh rate is twice the initial display refresh rate. In this case, the number of frames occupied by the display screen displaying images is the same as the number of frames occupied by displaying images when the aforementioned alternated steps are not executed.

In the aforementioned photographing method, by increasing the display refresh rate of the display screen, the number of frames occupied by the display screen displaying images will not drop sharply due to the number of frames occupied by the under-panel camera acquiring images, thus avoiding the poor display effect caused by insufficient number of image frames displayed on the display screen.

It should be noted that the aforementioned examples are illustrative for easy understanding rather than constitutes a limitation to the technical scheme of the disclosure.

Another embodiment of the disclosure relates to an electronic device. As shown in <FIG>, the electronic device includes: at least one processor <NUM>; a memory <NUM> in communicative connection with the at least one processor <NUM>; a display screen <NUM>; and an under-panel camera <NUM>. The memory <NUM> stores instructions executable by the at least one processor <NUM> which, when executed by the at least one processor <NUM>, cause the at least one processor <NUM> to execute the method steps shown in the aforementioned method embodiments. The display screen <NUM> is configured for displaying images. The under-panel camera <NUM> is configured for acquiring images.

The memory <NUM> and the processor <NUM> are connected via a bus. Any number of interconnected buses and bridges may be included. The bus connects various circuits of the one or more processors <NUM> and the memory <NUM> together. The bus may also connect various other circuits such as a peripheral device, a voltage regulator, a power management circuit, etc., which are well-known in the art and therefore will not be further described herein. A bus interface provides an interface between the bus and a transceiver. The transceiver may be one component or multiple components (such as multiple receivers and transmitters), providing a unit for communicating with various other apparatuses on a transmission medium. Data processed by the processor <NUM> are transmitted on a wireless medium through an antenna. For example, the antenna also receives data and transmits the data to the processor <NUM>.

The processor <NUM> is responsible for managing the bus and conventional processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory <NUM> may be configured to store data which is used by the processor 401when performing operations.

In an embodiment, the aforementioned under-panel camera <NUM> is arranged under a preset area of the display screen <NUM>, and a distance between the preset area and an edge of the display screen is not less than a preset threshold.

According to the electronic device, when the under-panel camera is arranged in a central area which is at a distance from the edge of the display screen not less than the preset threshold, the user's line of sight can face the central area of the display screen when the user is making a video call or taking a selfie, thus improving the interactive experience in taking a video call or taking a selfie.

Another embodiment of the disclosure relates to a computer-readable storage medium, storing a computer program which, when executed by the processor, causes the processor to implement the aforementioned method embodiments.

That is, those having ordinary skill in the art may understand that all or part of the steps in the methods for implementing the aforementioned embodiments may be carried out by related hardware instructed by a program. The program is stored in a storage medium and includes a plurality of instructions to cause a device (such as a single chip, a chip or the like) or a processor to execute all or part of the steps of the method described in each embodiment of the disclosure. The aforementioned storage medium includes a USB flash disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or various other media capable of storing program codes.

Claim 1:
A photographing method, comprising:
in response to reception of a camera startup instruction, executing alternated steps of displaying an image on a display screen and acquiring an image with an under-panel camera (S101);
the alternated steps comprising:
setting the display screen into a transparent state after the display screen displays an image (S102), wherein light is allowed to enter the under-panel camera through the display screen in case that the display screen is in the transparent state;
controlling the under-panel camera to acquire an image, and sending the acquired image to the display screen for display (S103); and
returning to execute the alternated steps until an end instruction indicating completion of photographing is received (S104);
wherein the setting the display screen into a transparent state comprises:
in response to reception of a confirmation signal sent by a display screen integrated circuit, IC, setting the display screen into a transparent state, the confirmation signal being used to indicate that the display screen has completed display of the image.