Patent Description:
Photographing functions (for example, photo and video) of a camera app in a mobile phone have been widely used in daily life of a user. However, in a handheld condition, a specific shake phenomenon may occur in a photo or video captured by the user through the camera app, affecting definition of a photographing image.

Currently, a stabilization algorithm may be preset in some mobile phones, to reduce an image blur problem caused by a shake during photographing. Usually, a stabilization angle in the stabilization algorithm is positively correlated with a field of view (field of view, FOV) of a photographing image. For example, when the FOV of the photographing image is <NUM>°, a corresponding stabilization angle is <NUM>°; or when the FOV of the photographing image is <NUM>°, a corresponding stabilization angle is <NUM>°.

In this case, in a photographing process, if the user enlarges the photographing image by N times in a digital zoom manner, the FOV of the photographing image is also reduced by N times accordingly, and correspondingly, the stabilization angle of the photographing image is also reduced by N times accordingly. Consequently, a stabilization effect of the mobile phone is significantly reduced, and definition of the photographing image obtained through high-magnification zoom photographing is poor. <CIT> discloses a n image pickup apparatus that includes an optical zooming unit, an electronic zooming unit, and a controller, wherein when the optical zooming unit moves from a first zoom region to a second zoom region that is closer to a wide-angle end than the first zoom region, the controller discretely moves the electronic zooming unit along with the operation of the optical zooming unit at the first change rate, and when the optical zooming unit moves from the first zoom region to a third zoom region that is closer to a telephoto end than the first zoom region, the controller discretely operates the electronic zooming unit along with the operation of the optical zooming unit at the second change rate. <CIT> discloses an electronic device. The electronic device includes a processor. The processor is configured to obtain a plurality of images. The processor is also configured to obtain global motion information indicating global motion between at least two of the plurality of images. The processor is further configured to obtain object tracking information indicating motion of a tracked object between the at least two of the plurality of images. The processor is additionally configured to perform automatic zoom based on the global motion information and the object tracking information. Performing automatic zoom produces a zoom region including the tracked object. The processor is configured to determine a motion response speed for the zoom region based on a location of the tracked object within the zoom region.

This application provides a photographing method and an electronic device as defined in the appended claims, to ensure stabilization performance existing during photographing in a zoom scenario, improve definition of a photographing image, and improve photographing experience of a user.

The following describes implementations of the embodiments of this application in detail with reference to the accompanying drawings.

For example, a photographing method provided in the embodiments of this application may be applied to an electronic device such as a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a handheld computer, a netbook, a personal digital assistant (personal digital assistant, PDA), a wearable electronic device, or a virtual reality device. This is not limited in the embodiments of this application.

For example, <FIG> is a schematic diagram of a structure of an electronic device <NUM>.

The electronic device <NUM> may include a processor <NUM>, an external memory interface <NUM>, an internal memory <NUM>, a universal serial bus (universal serial bus, USB) port <NUM>, a charging management module <NUM>, a power management module <NUM>, a battery <NUM>, an antenna <NUM>, an antenna <NUM>, a mobile communications module <NUM>, a wireless communications module <NUM>, an audio module <NUM>, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module <NUM>, a button <NUM>, a motor <NUM>, an indicator <NUM>, a camera <NUM>, a display <NUM>, a subscriber identity module (subscriber identification module, SIM) card interface <NUM>, and the like.

It can be understood that the structure shown in this embodiment of the present invention does not constitute a specific limitation on the electronic device <NUM>. In other embodiments of this application, the electronic device <NUM> may include more or fewer components than those shown in the figure, combine some components, split some components, or have different component arrangements. The components shown in the figure may be implemented by using hardware, software, or a combination of software and hardware.

The processor <NUM> may include one or more processing units. For example, the processor <NUM> may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processing unit (neural-network processing unit, NPU). Different processing units may be independent components, or may be integrated into one or more processors.

A memory may be further disposed in the processor <NUM>, and is configured to store instructions and data. In some embodiments, the memory in the processor <NUM> is a cache. The memory may store instructions or data just used or cyclically used by the processor <NUM>. If the processor <NUM> needs to use the instructions or the data again, the processor may directly invoke the instructions or the data from the memory. This avoids repeated access and reduces waiting time of the processor <NUM>. Therefore, system efficiency is improved.

The interface may include an inter-integrated circuit (inter-integrated circuit, I2C) interface, an inter-integrated circuit sound (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver/transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (general-purpose input/output, GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, a universal serial bus (universal serial bus, USB) port, and/or the like.

The charging management module <NUM> is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments in which wired charging is used, the charging management module <NUM> may receive a charging input from the wired charger through the USB port <NUM>. In some embodiments in which wireless charging is used, the charging management module <NUM> may receive a wireless charging input through a wireless charging coil of the electronic device <NUM>. The charging management module <NUM> may further supply power to the electronic device by using the power management module <NUM> when the battery <NUM> is charged.

The power management module <NUM> is configured to connect the battery <NUM> and the charging management module <NUM> to the processor <NUM>. The power management module <NUM> may receive an input of the battery <NUM> and/or an input of the charging management module <NUM>, and supply power to the processor <NUM>, the internal memory <NUM>, the display <NUM>, the camera <NUM>, the wireless communications module <NUM>, and the like.

The power management module <NUM> may be configured to monitor a performance parameter such as a battery capacity, a quantity of battery cycles, a battery charging voltage, a battery discharging voltage, and a battery health status (for example, electric leakage or impedance). In some other embodiments, the power management module <NUM> may alternatively be disposed in the processor <NUM>. In some other embodiments, the power management module <NUM> and the charging management module <NUM> may alternatively be disposed in a same device.

The antenna <NUM> and the antenna <NUM> are configured to: transmit and receive electromagnetic wave signals. Each antenna in the electronic device <NUM> may be configured to cover one or more communication bands. Different antennas may be further multiplexed, to improve antenna utilization. For example, the antenna <NUM> may be multiplexed as a diversity antenna in a wireless local area network. In some other embodiments, the antenna may be used in combination with a tuning switch.

The mobile communications module <NUM> may provide a wireless communication solution that includes <NUM>/<NUM>/<NUM>/<NUM> or the like and that is applied to the electronic device <NUM>. The mobile communications module <NUM> may include one or more filters, one or more switches, one or more power amplifiers, one or more low noise amplifiers (low noise amplifier, LNA), and the like. The mobile communications module <NUM> may receive an electromagnetic wave through an antenna <NUM>, perform processing such as filtering and amplification on the received electromagnetic wave, and transfer a processed electromagnetic wave to the modem processor for demodulation. The mobile communications module <NUM> may further amplify a signal modulated by the modem processor, and convert the signal into an electromagnetic wave for radiation through the antenna <NUM>. In some embodiments, at least some function modules in the mobile communications module <NUM> may be disposed in the processor <NUM>. In some embodiments, at least some function modules in the mobile communications module <NUM> and at least some modules in the processor <NUM> may be disposed in a same device.

The wireless communications module <NUM> may provide a solution for wireless communications including a wireless local area network (wireless local area networks, WLAN) (such as a wireless fidelity (wireless fidelity, Wi-Fi) network), Bluetooth (Bluetooth, BT), a global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), a near field communication (near field communication, NFC) technology, an infrared (infrared, IR) technology, and the like applied to the electronic device <NUM>. The wireless communications module <NUM> may one or more components integrated with one or more communication processing modules. The wireless communications module <NUM> receives an electromagnetic wave through an antenna <NUM>, performs frequency modulation and filtering processing on the electromagnetic wave signal, and sends a processed signal to the processor <NUM>. The wireless communications module <NUM> may further receive a to-be-sent signal from the processor <NUM>, perform frequency modulation and amplification on the signal, and convert the signal into an electromagnetic wave for radiation through the antenna <NUM>.

In some embodiments, in the electronic device <NUM>, the antenna <NUM> and the mobile communications module <NUM> are coupled, and the antenna <NUM> and the wireless communications module <NUM> are coupled, so that the electronic device <NUM> can communicate with a network and another device by using a wireless communications technology. The wireless communications technology may include a global system for mobile communications (global system for mobile communications, GSM), a general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, a GNSS, a WLAN, NFC, FM, an IR technology, and/or the like. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a BeiDou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi-zenith satellite system, QZSS), and/or a satellite based augmentation system (satellite based augmentation systems, SBAS).

The electronic device <NUM> implements a display function by using the GPU, the display <NUM>, the application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display <NUM> and the application processor. The GPU is configured to: perform mathematical and geometric calculation, and render an image. The processor <NUM> may include one or more GPUs that execute program instructions to generate or change display information.

The display <NUM> is configured to display an image, a video, and the like. The display <NUM> includes a display panel. The display panel may be a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (organic light-emitting diode, OLED), an active-matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), a flexible light-emitting diode (flex light-emitting diode, FLED), a mini-LED, a micro-LED, a micro-OLED, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), or the like. In some embodiments, the electronic device <NUM> may include one or N displays <NUM>, where N is a positive integer greater than <NUM>.

The electronic device <NUM> may implement a photographing function through the ISP, the camera <NUM>, the video codec, the GPU, the display <NUM>, the application processor, and the like.

The ISP is configured to process data fed back by the camera <NUM>. For example, during photographing, a shutter is pressed, light is transmitted to a photosensitive element of the camera through a lens, an optical signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, to convert the electrical signal into a visible image. The ISP may further perform algorithm optimization on noise, brightness, and complexion of the image. The ISP may further optimize parameters such as exposure and a color temperature of a photographing scenario. In some embodiments, the ISP may be disposed in the camera <NUM>.

The camera <NUM> is configured to capture a static image or a video. In some embodiments, the mobile phone <NUM> may include one or N cameras, where N is a positive integer greater than <NUM>. The camera <NUM> may be a front-facing camera or a rear-facing camera. As shown in <FIG>, the camera <NUM> usually includes a lens (lens) and a photosensitive element (sensor). The photosensitive element may be any photosensitive element such as a CCD (charge-coupled device, charge-coupled device) or a CMOS (complementary metal oxide semiconductor, complementary metal oxide semiconductor).

Still as shown in <FIG>, in a process of capturing a photo or a video, an optical image may be generated after reflected light of a to-be-photographed object passes through a lens, the optical image is projected onto a photosensitive element, and the photosensitive element converts a received optical signal into an electrical signal. Further, the camera <NUM> sends the obtained electrical signal to a DSP (Digital Signal Processing, digital signal processing) module for digital signal processing, to finally obtain each frame of digital image.

The image or the video captured by the camera <NUM> may be output on the mobile phone <NUM> through the display <NUM>, or the digital image may be stored in the internal memory <NUM>. This is not limited in this embodiment of this application.

The digital signal processor is configured to process a digital signal, and may process another digital signal in addition to the digital image signal. For example, when the electronic device <NUM> selects a frequency, the digital signal processor is configured to perform Fourier transform on frequency energy and the like.

The video codec is configured to compress or decompress a digital video. The electronic device <NUM> may support one or more video codecs. Therefore, the electronic device <NUM> may play or record videos in a plurality of coding formats, for example, moving picture experts group (moving picture experts group, MPEG)-<NUM>, MPEG-<NUM>, MPEG-<NUM>, and MPEG-<NUM>.

The external memory interface <NUM> may be configured to connect to an external storage card, for example, a micro SD card, to extend a storage capability of the electronic device <NUM>. The external storage card communicates with the processor <NUM> through the external memory interface <NUM>, to implement a data storage function. For example, files such as music and videos are stored in the external storage card.

The internal memory <NUM> may be configured to store one or more computer programs, and the one or more computer programs include instructions. The processor <NUM> may run the instructions stored in the internal memory <NUM>, to enable the electronic device <NUM> to perform a method for intelligently recommending a contact provided in some embodiments of this application, various function applications, data processing, and the like. The internal memory <NUM> may include a program storage area and a data storage area. The program storage area may store an operating system. The program storage area may further store one or more applications (for example, Gallery and Contacts), and the like. The data storage area may store data (such as photos and contacts) created during the use of the electronic device <NUM>, and the like. In addition, the internal memory <NUM> may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more disk storage components, a flash component, or a universal flash storage (universal flash storage, UFS). In some other embodiments, the processor <NUM> runs the instructions stored in the internal memory <NUM> and/or instructions stored in a memory disposed in the processor, to enable the electronic device <NUM> to perform a method for intelligently recommending a number provided in the embodiments of this application, various function applications, and data processing.

The electronic device <NUM> may implement audio functions, for example, music playing and recording, by using the audio module <NUM>, the speaker 170A, the receiver 170B, the microphone 170C, the headset jack 170D, the application processor, and the like.

The audio module <NUM> is configured to convert digital audio information into an analog audio signal for output, and is also configured to convert an analog audio input into a digital audio signal. The audio module <NUM> may be further configured to encode and decode an audio signal. In some embodiments, the audio module <NUM> may be disposed in the processor <NUM>, or some function modules in the audio module <NUM> are disposed in the processor <NUM>.

The speaker 170A, also referred to as a "horn", is configured to convert an audio electrical signal into a sound signal. The electronic device <NUM> may be configured to listen to music or answer a call in a hands-free mode over the speaker 170A.

The receiver 170B, also referred to as an "earpiece", is configured to convert an audio electrical signal into a sound signal. When a call is answered or audio information is listened to by using the electronic device <NUM>, the receiver 170B may be put close to a human ear to listen to a voice.

The microphone 170C, also referred to as a "mike" or a "microphone", is configured to convert a sound signal into an electrical signal. When making a call or sending voice information, a user may make a sound by moving a human mouth close to the microphone 170C to input a sound signal to the microphone 170C. One or more microphones 170C may be disposed in the electronic device <NUM>. In some other embodiments, two microphones 170C may be disposed in the electronic device <NUM>, to collect a sound signal and implement a noise reduction function. In some other embodiments, three, four, or more microphones 170C may alternatively be disposed in the electronic device <NUM>, to collect a sound signal, implement noise reduction, and identify a sound source, so as to implement a directional recording function and the like.

The headset jack 170D is configured to connect to a wired headset. The headset jack 170D may be the USB port <NUM>, or may be a <NUM> open mobile terminal platform (open mobile terminal platform, OMTP) standard interface or a cellular telecommunications industry association of the USA (cellular telecommunications industry association of the USA, CTIA) standard interface.

The sensor module <NUM> may include a pressure sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, an optical proximity sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like. This is not limited in this embodiment of this application.

Certainly, the electronic device <NUM> provided in this embodiment of this application may further include one or more components such as a button <NUM>, a motor <NUM>, an indicator <NUM>, and an SIM card interface <NUM>. This is not limited in this embodiment of this application.

To clearly understand the related conventional technology and the embodiments of this application, a brief description of the related technology is first provided.

A field of view (field of view, FOV) may also be referred to as an angle of view, and a value of the field of view determines a field of vision of a photographing image captured by an optical instrument (for example, a camera). A larger FOV of a camera means a larger field of vision of a photographing picture, and a smaller FOV of a camera means a smaller field of vision of a photographing picture.

Digital zoom (digital zoom) is to increase an area of each pixel in the photographing image by using a component such as a DSP or a processor in an electronic device, to implement enlargement. That a zoom ratio is <NUM> (namely, 4x zoom) is used as an example. As shown in <FIG>, when a camera whose FOV is <NUM>° is used to collect a photographing image <NUM>, if a zoom ratio of <NUM> is set for the electronic device, it indicates that a user wants to enlarge the photographing image <NUM> by four times for display. In this case, the electronic device may use a central point A of the photographing image <NUM> as a center, crop the photographing image <NUM>, and retain a photographing image <NUM> whose FOV is <NUM>° (namely, <NUM>°/<NUM>). Further, the electronic device may display the cropped photographing image <NUM> in a preview box <NUM> of the electronic device in an enlargement manner. In this case, the user may view, in the preview box <NUM>, a display effect obtained after the photographing image <NUM> is enlarged by <NUM> times.

In the conventional technology, that the electronic device is a mobile phone is used as an example. After obtaining each frame of photographing image through a camera, the mobile phone may first crop the photographing image based on a current zoom ratio. Still as shown in <FIG>, after the mobile phone obtains the photographing image <NUM> whose FOV is <NUM>°, if the current zoom ratio is <NUM>, the mobile phone may crop the photographing image <NUM> to obtain the photographing image <NUM> whose FOV is <NUM>°. In addition, a fixed crop ratio (init ratio) is preset in the mobile phone, to eliminate an image shake. For example, the crop ratio = <NUM>%. In other words, the mobile phone may further crop the photographing image <NUM> by <NUM>%, to reduce a shake phenomenon of the photographing image <NUM>.

As shown in <FIG>, the mobile phone may determine a first shake amount D1 on an x-axis of the photographing image <NUM> and a second shake amount D2 on a y-axis of the photographing image <NUM> based on a preset stabilization algorithm. That the first shake amount D1 is <NUM>° and the second shake amount D2 is <NUM>° is used as an example. It indicates that a shake of <NUM>° is generated along a positive direction of the x-axis of the currently obtained photographing image <NUM>, and a shake of <NUM>° is generated along a positive direction of the y-axis of the photographing image <NUM>.

Because the FOV of the photographing image <NUM> is <NUM>°, when the crop ratio is <NUM>%, a corresponding stabilization angle is <NUM>° (namely, <NUM>° x <NUM>%). Still as shown in <FIG>, after the photographing image <NUM> is cropped at the crop ratio of <NUM>%, an FOV of <NUM>° (namely, <NUM>° - <NUM>° x <NUM>% x <NUM>) remains in a crop box <NUM>. As shown in <FIG>, if an initial location of the crop box <NUM> is located in a center of the photographing image <NUM>, the mobile phone may move the crop box <NUM> by <NUM>° in a negative direction of the x-axis, and move the crop box <NUM> by <NUM>° in a negative direction of the y-axis. Further, the mobile phone may crop and output the photographing image <NUM> based on a current location of the crop box <NUM>, to compensate for the shake that is generated in the positive direction of the x-axis and the positive direction of the y-axis of the photographing image <NUM>.

It can be learned that when the zoom ratio of the photographing image is 4x zoom, a stabilization angle of the photographing image is only <NUM>°. Because the crop ratio used to compensate for the image shake is fixed, a higher zoom ratio of the photographing image indicates a smaller stabilization angle of the photographing image. That an FOV of an initial photographing image is <NUM>° and the crop ratio is <NUM>% is still used as an example. When a zoom ratio of the photographing image is 10x zoom, an FOV of a photographing image obtained after zooming is performed is <NUM>° (namely, <NUM>°/<NUM>). In this case, the stabilization angle of the photographing image is only <NUM>° (namely, <NUM>° x <NUM>%). However, a shake of a photographing image captured by the user in a walk is approximately <NUM>° to <NUM>°, and a shake of a photographing image captured by the user during running may reach more than <NUM>°. It is clear that a stabilization processing method in the conventional technology cannot meet a stabilization requirement of the user in a high-magnification zoom scenario. Consequently, both photographing quality of the photographing image and photographing experience of the user are reduced.

Therefore, an embodiment of this application provides a photographing method. That the electronic device is a mobile phone is still used as an example. As shown in <FIG>, the method includes steps S501 to S506.

S501: The mobile phone obtains a first photographing image through a camera, where a zoom ratio of the first photographing image is a first zoom ratio.

Usually, one or more photographing modes such as a photo mode, a video mode, a panorama mode, a slow motion mode, or a time-lapse mode are set in a camera application of the mobile phone. After detecting that the user opens the camera application, the mobile phone may invoke the camera application to enter a photographing mode and open the camera. In this case, the camera may collect each frame of photographing image at a specific operating frequency. Before storing or displaying the photographing image, the mobile phone may perform anti-shake processing on each frame of photographing image in real time, to reduce a shake phenomenon of the photographing image.

For example, as shown in <FIG>, after the camera application is opened, the mobile phone may display a preview interface <NUM> in the photo mode. The preview interface <NUM> includes a framing window <NUM>, and the framing window <NUM> may be used to display in real time a preview image existing before a photo is captured. In addition, the preview interface <NUM> may further include a zoom option <NUM>. The user may select a zoom ratio for current photographing from the zoom option <NUM>, for example, 2x zoom, 4x zoom, or 10x zoom. The mobile phone may display, in an enlargement manner or a shrinking manner based on the current zoom ratio, a photographing image collected by the camera. It should be noted that the zoom option <NUM> may alternatively be hidden in the preview interface <NUM>. For example, the mobile phone may correspondingly adjust the current zoom ratio based on a pinch operation performed by the user in the framing window <NUM>. Certainly, the mobile phone may alternatively display, in an enlargement manner or a shrinking manner based on a default zoom ratio, the photographing image collected by the camera. This is not limited in this embodiment of this application.

That the current zoom ratio is 2x zoom (namely, the first zoom ratio) is used as an example. An FOV of each frame of photographing image collected by the camera after the camera operates is fixed. For example, when an FOV of the camera is <NUM>°, the FOV of the photographing image collected by the camera is also <NUM>°. For example, as shown in <FIG>, after the camera application of the mobile phone is opened, the mobile phone may collect, through the camera, a first photographing image <NUM> whose FOV is <NUM>°. In this case, if the current zoom ratio of the mobile phone is 2x zoom, it indicates that the user wants to enlarge the first photographing image <NUM> by two times and display the enlarged first photographing image <NUM> in the framing window <NUM>.

It should be noted that, that the mobile phone obtains the first photographing image <NUM> in a preview scenario in the photo mode is used as an example for description in <FIG>. It can be understood that, the mobile phone may alternatively obtain the first photographing image in another photographing mode (for example, the video mode, the panorama mode, the slow motion mode, or the time-lapse mode). For example, the mobile phone may obtain the first photographing image in a preview scenario in the video mode, or may obtain the first photographing image in a photographing scenario in the video mode. This is not limited in this embodiment of this application.

S502: The mobile phone determines a first crop ratio of the first photographing image based on the first zoom ratio.

In this embodiment of this application, after obtaining the first photographing image <NUM>, the mobile phone may set a corresponding crop ratio for the first photographing image <NUM> based on a current real-time zoom ratio, to crop the first photographing image <NUM>. If the current zoom ratio is large, the mobile phone may set a large crop ratio for the first photographing image <NUM>; and correspondingly, if the current zoom ratio is small, the mobile phone may set a small crop ratio for the first photographing image <NUM>.

For example, after obtaining each frame of photographing image, the mobile phone may calculate a crop ratio (crop_ratio) of a current photographing image according to the following formula (<NUM>).

Herein, zoom_ratio is the current zoom ratio, and init ratio is a preset initial crop ratio. For example, init_ratio may be a constant <NUM>%.

In this case, that the first zoom ratio is 2x zoom is still used as an example. After obtaining the first photographing image <NUM>, the mobile phone may calculate the first crop ratio of the first photographing image <NUM> according to the formula (<NUM>): <MAT>. In other words, in a 2x zoom scenario, the first photographing image <NUM> whose FOV is <NUM>° needs to be cropped by <NUM>%, to meet a zoom requirement for 2x zoom.

In an example not covered by the claims, as shown in Table <NUM>, correspondences that are between different zoom ratios and different crop ratios and that exist when there is a specific FOV (for example, <NUM>°) may be further prestored in the mobile phone. Therefore, after obtaining the first photographing image <NUM>, the mobile phone may query a current first zoom ratio. Further, the mobile phone may determine, based on the correspondences shown in Table <NUM>, that a first crop ratio that is of the first photographing image <NUM> and that corresponds to the first zoom ratio is <NUM>%.

It can be learned that in this example, a crop ratio that is set for each frame of photographing image dynamically changes with reference to the current zoom ratio. However, in the conventional technology, the crop ratio that is set for each frame of photographing image is fixed, for example, a fixed crop ratio of <NUM>%. Therefore, in the conventional technology, after each frame of photographing image is obtained, cropping needs to be first performed for one time based on the current zoom ratio, to meet a current zoom requirement, and then cropping is performed for the second time based on the fixed crop ratio of <NUM>%, to compensate for the image shake that occurs in the photographing image.

The mobile phone may determine a corresponding crop ratio (namely, crop_ratio) for the photographing image at one time with reference to the current zoom ratio, and subsequently, the mobile phone only needs to crop the photographing image for one time based on the crop ratio. Therefore, the mobile phone may perform cropping for one time based on the entire photographing image (the first photographing image <NUM>), and a stabilization angle that may be used to compensate for the image shake in a cropping process is increased, so that a stable photographing effect of a photographing image captured by the mobile phone can still be obtained when there is a large shake.

That a crop ratio (crop_ratio) of the first photographing image <NUM> is <NUM>% is used as an example. As shown in <FIG>, an FOV of <NUM>° remains after the first photographing image <NUM> whose FOV is <NUM>° is cropped by <NUM>%. Therefore, the mobile phone may determine, in the first photographing image <NUM>, that a value of an FOV of a crop box <NUM> is <NUM>°. When the crop box <NUM> is located in a center of the first photographing image <NUM>, an FOV of <NUM>° remains between each boundary of the crop box <NUM> and a corresponding boundary of the first photographing image <NUM>. Subsequently, the mobile phone may determine a specific crop location by moving the crop box <NUM>, to compensate for an image offset generated due to a shake of the mobile phone. However, each remaining FOV of <NUM>° may be used to compensate for the image offset generated due to the shake of the mobile phone. In other words, in a 2x zoom scenario, the stabilization angle of the photographing image may reach <NUM>°, to improve stabilization performance existing during photographing.

That the FOV of the photographing image is <NUM>° is still used as an example. As shown in Table <NUM>, different stabilization angles of the mobile phone may be obtained in the foregoing method in different zoom scenarios. Compared with the conventional technology in which the stabilization angle decreases with an increase in the zoom ratio, in the photographing method provided in this embodiment, the stabilization angle of the photographing image may increase with the increase in the zoom ratio, and therefore, a good stabilization effect can still be obtained in a high-magnification zoom scenario.

S503: The mobile phone crops the first photographing image based on the first crop ratio.

The first photographing image <NUM> is still used as an example. Still as shown in <FIG>, when the crop ratio of the first photographing image <NUM> is <NUM>%, the mobile phone may determine that the value of the FOV of the crop box <NUM> in the first photographing image <NUM> is <NUM>°. Further, the mobile phone may determine a specific location of the crop box <NUM> in the first photographing image <NUM> based on a shake status of the first photographing image <NUM>, and perform cropping, to compensate for an image shake that occurs in the first photographing image <NUM>.

For example, if it is detected that the first photographing image <NUM> shakes leftwards by <NUM>°, the mobile phone may move the crop frame <NUM> rightwards by <NUM>° from the center of the first photographing image <NUM>, to compensate for an image offset generated when the first photographing image <NUM> shakes leftwards by <NUM>°. For another example, if it is detected that the first photographing image <NUM> shakes upwards by <NUM>°, the mobile phone may move the crop frame <NUM> downwards by <NUM>° from the center of the first photographing image <NUM>, to compensate for an image offset generated when the first photographing image <NUM> shakes upwards by <NUM>°.

For example, when collecting the first photographing image <NUM> through the camera, the mobile phone may further enable a sensor such as a gyroscope to detect an actual displacement of the mobile phone on the x-axis and the y-axis, to be specific, an actual displacement S1 on the x-axis of the first photographing image <NUM> and an actual displacement S2 on the y-axis of the first photographing image <NUM>. Alternatively, the mobile phone may obtain a previous frame of photographing image adjacent to the first photographing image <NUM>, and further calculate the actual displacement S1 on the x-axis of the first photographing image <NUM> and the actual displacement S2 on the y-axis of the first photographing image <NUM> based on an OIS (optical image stabilization, optical image stabilization) algorithm or an optical flow algorithm.

Usually, a part of actual displacements (namely, S1 and S2) on the x-axis and the y-axis of the photographing image is generated due to actual movement of the user, and the other part is generated due to the shake of the mobile phone. To determine a shake amount that is generated on the x-axis and the y-axis of the first photographing image <NUM> due to the shake of the mobile phone, after the mobile phone obtains the first photographing image <NUM>, the mobile phone may predict, with reference to latest N (N > <NUM>) frames of photographing images, a motion displacement Y1 of the user on the x-axis and a motion displacement Y2 of the user on the y-axis during capturing of the first photographing image <NUM>. Therefore, the mobile phone may calculate the first shake amount (namely, D1 = S1 - Y <NUM>) on the x-axis of the first photographing image <NUM> and the second shake amount (namely, D2 = S2 - Y2) on the y-axis of the first photographing image <NUM>.

That the first shake amount D1 on the x-axis of the first photographing image <NUM> is <NUM>° and the second shake amount D2 on the y-axis of the first photographing image <NUM> is <NUM>° is used as an example. It indicates that a shake of <NUM>° is generated along a positive direction of the x-axis of the first photographing image <NUM>, and a shake of <NUM>° is generated along a positive direction of the y-axis. In this case, as shown in <FIG>, the mobile phone may move the crop box <NUM> in the first photographing image <NUM> by <NUM>° in a negative direction of the x-axis, to compensate for an image offset that is generated due to a shake of <NUM>° in the negative direction of the x-axis of the first photographing image <NUM>. In addition, the mobile phone may move the crop box <NUM> in the first photographing image <NUM> by <NUM>° in the negative direction of the y-axis, to compensate for an image offset that is generated due to the shake of <NUM>° in the positive direction of the y-axis of the first photographing image <NUM>.

Because a stabilization angle of the first photographing image <NUM> is <NUM>°, in other words, when a shake of an angle within <NUM>° is generated in the positive direction (or the negative direction) of the x-axis of the first photographing image <NUM>, and/or when a shake of an angle within <NUM>° is generated in the positive direction (or the negative direction) of the y-axis of the first photographing image <NUM>, the mobile phone may adjust the crop box <NUM>, to compensate for an image offset generated due to an shake, so that a stabilization effect of the mobile phone is significantly improved.

In addition, when the mobile phone captures the first photographing image <NUM>, a shake may be further generated in a z-axis direction. The shake that is generated in the z-axis direction of the first photographing image <NUM> causes image distortion in the first photographing image <NUM>. Therefore, in addition to adjusting the crop box <NUM> to eliminate the shake phenomenon of the first photographing image <NUM>, the mobile phone may further eliminate the image distortion in the first photographing image <NUM> by using a preset warp (distortion) algorithm, to further improve a stabilization effect of the mobile phone.

As shown in <FIG>, after determining the specific location of the crop box <NUM> based on shake amounts on the x-axis and the y-axis of the first photographing image <NUM>, the mobile phone may crop the first photographing image <NUM> based on the location of the crop box <NUM>, to obtain a photographing image <NUM> obtained after the first photographing image <NUM> is cropped. Further, the mobile phone may display, in the framing window <NUM> of the preview interface <NUM> in an enlargement manner, the photographing image <NUM> obtained after cropping. In this case, not only the photographing image <NUM> meets a current zoom requirement for 2x zoom, but also the image offset generated due to the shake of the mobile phone is compensated for, so that the user obtains good photographing experience.

In some embodiments, after cropping the first photographing image <NUM> to obtain the photographing image <NUM>, the mobile phone may first store the photographing image <NUM> in a cache of the mobile phone. For example, the photographing image <NUM> may be stored in a preview cache or a video cache. In a preview scenario of the photo mode or the video mode, the mobile phone may obtain, from the preview buffer in real time, each photographing image that is obtained through cropping, and output the photographing image in the framing window <NUM> of the preview interface <NUM>. In a video capturing process, the mobile phone may obtain, from the video buffer in real time, each photographing image that is obtained through cropping, and encode the photographing image into a currently recorded video for storage.

In the foregoing embodiment, that the first photographing image <NUM> is cropped based on the current zoom ratio after the mobile phone obtains the first photographing image <NUM> is used as an example for description. It can be understood that, when the zoom ratio is the first zoom ratio (for example, the 2x zoom), after obtaining each frame of photographing image, the mobile phone may crop the photographing image in the method described in S501 to S503, to implement zoom and stabilization functions of the photographing image.

S504: The mobile phone obtains a second photographing image through the camera in response to a zoom operation of the user, where a zoom ratio of the second photographing image is a second zoom ratio.

The user may alternatively manually adjust the current zoom ratio in a video preview or capturing process. For example, as shown in <FIG>, the user may slide the zoom option <NUM> in the preview interface <NUM>, to adjust the photographing image from 2x zoom (namely, the first zoom ratio) to 4x zoom (namely, the second zoom ratio). After detecting the zoom operation entered by the user, the mobile phone may update the current zoom ratio to 4x zoom.

In addition, similar to step S501, the mobile phone may further obtain the second photographing image through the camera in response to the zoom operation. As shown in <FIG>, an FOV of a second photographing image <NUM> is also <NUM>°. Different from that of the first photographing image <NUM>, the current zoom ratio of the mobile phone is 4x zoom. It indicates that the user wants to enlarge the second photographing image <NUM> by four times and display the enlarged second photographing image <NUM> in the framing window <NUM>.

In some embodiments, after detecting the zoom operation entered by the user, the mobile phone may further replace a camera that is being used. For example, when it is detected that the user adjusts the photographing image from 2x zoom to 4x zoom, the mobile phone may switch, to a long-focus lens whose FOV is <NUM>°, a pantoscopic lens whose FOV is <NUM>° and that is being used. In this case, an FOV of the second photographing image obtained by the mobile phone is <NUM>°.

S505: The mobile phone determines a second crop ratio of the second photographing image based on the second zoom ratio.

Similar to step S502, after obtaining the second photographing image <NUM>, the mobile phone may further determine a crop ratio (namely, the second crop ratio) of the second photographing image <NUM> according to the formula (<NUM>) or based on the correspondences shown in Table <NUM>.

That the second zoom ratio is 4x zoom is still used as an example. After obtaining the second photographing image <NUM>, the mobile phone may calculate the second crop ratio of the second photographing image <NUM> according to the formula (<NUM>): <MAT>. In other words, in a 4x zoom scenario, the second photographing image <NUM> whose FOV is <NUM>° needs to be cropped by <NUM>%, to meet a zoom requirement for 4x zoom.

When the second crop ratio of the second photographing image <NUM> is <NUM>%, as shown in <FIG>, an FOV of <NUM>° remains after the second photographing image <NUM> whose FOV is <NUM>° is cropped by <NUM>%. Therefore, the mobile phone may determine, in the second photographing image <NUM>, that a value of an FOV of a crop box <NUM> is <NUM>°. When the crop box <NUM> is located in a center of the second photographing image <NUM>, an FOV of <NUM>° remains between each boundary of the crop box <NUM> and a corresponding boundary of the second photographing image <NUM>. The mobile phone may move the crop box <NUM>, to compensate for the image offset generated due to the shake of the mobile phone, and each remaining FOV of <NUM>° may be used to compensate for the image offset generated due to the shake of the mobile phone. In other words, in the 4x zoom scenario, the stabilization angle of the photographing image may reach <NUM>°, and stabilization performance existing during photographing is significantly improved.

It should be noted that, in the foregoing embodiment, that a camera whose FOV is <NUM>° is still used for photographing after zooming is performed on the mobile phone is used as an example for description. It can be understood that, if the camera with another FOV (for example, the FOV is <NUM>°) is replaced for the mobile phone in response to the zoom operation of the user, the mobile phone may further store correspondences between different zoom ratios and different zoom ratios when the FOV is <NUM>°, and further, the mobile phone may determine the second crop ratio of the second photographing image <NUM> based on the second zoom ratio.

S506: The mobile phone crops the second photographing image based on the second crop ratio.

The second photographing image <NUM> is still used as an example. Still as shown in <FIG>, when the crop ratio of the second photographing image <NUM> is <NUM>%, the mobile phone may determine that the value of the FOV of the crop box <NUM> in the second photographing image <NUM> is <NUM>°. Further, similar to step S503, the mobile phone may determine a specific location of the crop box <NUM> in the second photographing image <NUM> based on a first shake amount D1 on an x-axis of the second photographing image <NUM> and a second shake amount D2 on a y-axis of the second photographing image <NUM>, and perform cropping, to compensate for an image shake that occurs in the second photographing image <NUM>.

That the first shake amount D1 on the x-axis of the second photographing image <NUM> is <NUM>° and the second shake amount D2 on the y-axis of the second photographing image <NUM> is <NUM>° is used as an example. It indicates that a shake of <NUM>° is generated along a positive direction of the x-axis of the first photographing image <NUM>, and a shake of <NUM>° is generated along a positive direction of the y-axis. In this case, as shown in <FIG>, the mobile phone may move the crop box <NUM> in the second photographing image <NUM> by <NUM>° in a negative direction of the x-axis, to compensate for an image offset that is generated due to the shake of <NUM>° in the positive direction of the x-axis of the second photographing image <NUM>. In addition, the mobile phone may move the crop box <NUM> in the second photographing image <NUM> by <NUM>° in a negative direction of the y-axis, to compensate for an image offset that is generated due to the shake of <NUM>° in the positive direction of the y-axis of the second photographing image <NUM>.

Because a stabilization angle of the second photographing image <NUM> is <NUM>°, in other words, when a shake of an angle within <NUM>° generated in the positive direction (or the negative direction) of the x-axis of the second photographing image <NUM>, and/or when a shake of an angle within <NUM>° is generated in the positive direction (or the negative direction) of the y-axis of the second photographing image <NUM>, the mobile phone may adjust the crop box <NUM>, to compensate for an image offset generated due to a shake, so that a stabilization effect of the mobile phone is significantly improved.

Further, as shown in <FIG>, after determining the specific location of the crop box <NUM> based on shake amounts on the x-axis and the y-axis of the second photographing image <NUM>, the mobile phone may crop the second photographing image <NUM> based on the crop box <NUM>, to obtain a photographing image <NUM> obtained after the second photographing image <NUM> is cropped. Further, the mobile phone may display, in the framing window <NUM> of the preview interface <NUM> in an enlargement manner, the photographing image <NUM> obtained after cropping. In this case, not only the photographing image <NUM> meets a current zoom requirement for 4x zoom, but also the image offset generated due to the shake of the mobile phone is compensated for, so that the user obtains good photographing experience.

For example, when the mobile phone displays the photographing image <NUM> in the framing window <NUM> of the preview interface <NUM>, if it is detected that the user taps a photo button in the preview interface <NUM>, the mobile phone may use, as a current photographing image, the photographing image <NUM> currently displayed in the framing window <NUM>, and store the photographing image <NUM> in an album of the mobile phone.

Alternatively, when the mobile phone displays the photographing image <NUM> in the framing window <NUM> of the preview interface <NUM>, if it is detected that the user taps a photo button in the preview interface <NUM>, the mobile phone may obtain a photographing image currently collected by the camera, perform zooming and anti-shake processing on the collected photographing image in the cropping method in the steps S505 and S506, use, as the current photographing image, a photographing image obtained after cropping, and store, in an album of the mobile phone, the photographing image obtained after cropping.

It can be learned that, in any one of the photo capturing scenario, the video capturing scenario, the preview scenario, and a recording scenario, after obtaining each frame of photographing image, the mobile phone may dynamically set the crop ratio of the current photographing image based on the current zoom ratio, and the crop ratio may meet the current zoom requirement. In addition, the mobile phone may determine the specific crop location in the photographing image based on the crop ratio, to compensate for the image offset generated due to the shake of the mobile phone. A larger zoom ratio indicates that the photographing image corresponds to a larger crop ratio. Therefore, a larger zoom ratio indicates a larger stabilization angle that is used to compensate for an image shake and that remains when the mobile phone crops the photographing image. Therefore, in a high-magnification zoom photographing scenario, the mobile phone may still calibrate a photographing image with a large shake angle, to improve stabilization performance existing during capturing of an image and photographing experience of the user.

In addition, when photographing a moving object, the mobile phone may also crop a photographing image in the foregoing method, to satisfy a current zoom ratio and reduce a shake phenomenon of the photographing image.

For example, as shown in <FIG>, after obtaining a photographing image A, the mobile phone may identify a moving object <NUM> in the photographing image Abased on a preset image recognition algorithm. In other words, a current to-be-photographed target is the moving object <NUM>. In addition, if the current zoom ratio is 2x zoom, the mobile phone may determine, in the method in the foregoing embodiment, that a crop ratio of the photographing image A is <NUM>%, and further, the mobile phone may determine a size of a crop box <NUM> in the photographing image A. Further, the mobile phone may determine a specific location of the crop box <NUM> based on a shake amount of the moving object <NUM> in the photographing image A, so that the moving object <NUM> still occupies a main location in the photographing image after cropping is performed.

When the user adjusts the zoom ratio from 2x zoom to 4x zoom, as shown in <FIG>, the mobile phone may obtain a photographing image B, and identify the moving object <NUM> in the photographing image B. If the current zoom ratio is 4x zoom, the mobile phone may determine, in the method in the foregoing embodiment, that a crop ratio of the photographing image B is <NUM>%, and further, the mobile phone may determine a size of a crop box <NUM> in the photographing image B. Further, the mobile phone may determine a specific location of the crop box <NUM> based on a shake amount of the moving object <NUM> in the photographing image B, so that the moving object <NUM> still occupies a main location in the photographing image after cropping is performed.

When the user adjusts the zoom ratio from 4x zoom to 10x zoom, as shown in <FIG>, the mobile phone may obtain a photographing image C, and identify the moving object <NUM> in the photographing image C. If the current zoom ratio is 10x zoom, the mobile phone may determine, in the method in the foregoing embodiment, that a crop ratio of the photographing image C is <NUM>%, and further, the mobile phone may determine a size of a crop box <NUM> in the photographing image C. Further, the mobile phone may determine a specific location of the crop box <NUM> based on a shake amount of the moving object <NUM> in the photographing image C, so that the moving object <NUM> still occupies a main location in the photographing image after cropping is performed.

It can be learned that, during photographing of a moving object, the moving object may randomly appear on different locations in a photographing image. Therefore, after obtaining each frame of photographing image, the mobile phone may determine a crop ratio of a current photographing image based on a current zoom ratio, and retain the moving object in the photographing image based on the crop ratio, so that the moving object can smoothly appear on a main location in the photographing image, to reduce composition difficulty existing when the user performs photographing.

The embodiments of this application disclose an electronic device, including a processor, and a memory, an input device, and an output device that are connected to the processor. The input device and the output device may be integrated into one device. For example, a touch sensor may be used as the input device, a display may be used as the output device, and the touch sensor and the display are integrated into a touchscreen.

In this case, as shown in <FIG>, the electronic device may include a touchscreen <NUM>. The touchscreen <NUM> includes a touch sensor <NUM> and a display <NUM>, one or more processors <NUM>, one or more cameras <NUM>, a memory <NUM>, one or more applications (not shown), and one or more computer programs <NUM>. The foregoing components may be connected through one or more communication buses <NUM>. The one or more computer programs <NUM> are stored in the memory <NUM>, and are configured to be executed by the one or more processors <NUM>. The one or more computer programs <NUM> include instructions, and the instructions may be used to perform the steps in the foregoing embodiments. All related content of the steps in the foregoing method embodiments may be cited in function descriptions of corresponding physical devices.

For example, the processor <NUM> may be specifically the processor <NUM> shown in <FIG>, the memory <NUM> may be specifically the internal memory <NUM> shown in <FIG>, and the camera <NUM> may be specifically the camera <NUM> shown in <FIG>. The display <NUM> may be specifically the display <NUM> shown in <FIG>, and the touch sensor <NUM> may be specifically the touch sensor in the sensor module <NUM> shown in <FIG>. This is not limited in this embodiment of this application.

The foregoing descriptions about implementations allow a person skilled in the art to understand that, for the purpose of convenient and brief description, division into the foregoing function modules is used as an example for illustration. In actual application, the foregoing functions can be allocated to different function modules and implemented according to a requirement, that is, an inner structure of an apparatus is divided into different function modules to implement all or some of the functions described above. For detailed working processes of the foregoing system, apparatus, and unit, refer to corresponding processes in the foregoing method embodiments.

Functional units in the embodiments of this application may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.

Claim 1:
A photographing method, comprising:
obtaining, by an electronic device (<NUM>), a first photographing image (<NUM>) through a first camera, wherein a zoom ratio of the first photographing image (<NUM>) is a first zoom ratio;
determining, by the electronic device (<NUM>), a first crop ratio of the first photographing image (<NUM>) based on the first zoom ratio, wherein a stabilization angle of the first photographing image (<NUM>) is a product of a field of view, FOV, of the first camera and the first crop ratio;
cropping, by the electronic device (<NUM>), the first photographing image (<NUM>) based on the first crop ratio, to obtain and output a first cropped image;
obtaining, by the electronic device (<NUM>), a second photographing image (<NUM>) through the first camera in response to a first zoom operation entered by a user, wherein a zoom ratio of the second photographing image (<NUM>) is a second zoom ratio, and the second zoom ratio is greater than the first zoom ratio;
determining, by the electronic device (<NUM>), a second crop ratio of the second photographing image (<NUM>) based on the second zoom ratio, wherein the second crop ratio is greater than the first crop ratio, and a stabilization angle of the second photographing image (<NUM>) is a product of the FOV of the first camera and the second crop ratio; and
cropping, by the electronic device (<NUM>), the second photographing image (<NUM>) based on the second crop ratio, to obtain and output a second cropped image
characterized in that
the determining, by the electronic device (<NUM>), a first crop ratio of the first photographing image (<NUM>) based on the first zoom ratio comprises:
calculating, by the electronic device (<NUM>), the first crop ratio based on the first zoom ratio according to a preset formula; and
the determining, by the electronic device (<NUM>), a second crop ratio of the second photographing image (<NUM>) based on the second zoom ratio comprises:
calculating, by the electronic device (<NUM>), the second crop ratio based on the second zoom ratio according to the preset formula, wherein
the preset formula is: <MAT> wherein
zoom ratio is a current zoom ratio, and init ratio is a constant.