Patent Description:
Recently, electronic devices have been providing more diversified services and additional functions. Various applications executable on electronic devices are being developed to meet the diverse demands of users and to raise the utility of electronic devices.

Among these are camera applications, whereby the user may take a selfie or foreground picture using the camera of their electronic device. The electronic device may include a camera module for capturing images. The camera module may typically include a lens for collecting light, a photodiode for converting the collected light into an electrical signal, and an analog-to-digital converter (ADC) for converting the electrical signal. The process of a camera module converting electrical signals from multiple photodiodes into digital electrical signals and outputting the digital electrical signals may be called "read-out. " A camera module of an electronic device may provide an autofocus or manual focus function.

From <CIT>, a video editing method is known using contextual data and content discovery using clusters, wherein different video portions may be merged.

<CIT> describes a mobile terminal with a touch screen and an associated method of capturing images, wherein when a camera function is initiated, a preview image is displayed on the touch screen, and if a touch input is received ona specific position of the preview image on the touch screen, focusing of the image is performed on the specific position where the touch input has been received.

<CIT> relates to a photographic device, a photographing method, and a playback method, wherein a photographing parameter set prior to capturing a motion picture may be changed during the recording of the motion picture.

When focusing is performed while capturing video in super slow motion, a read-out by the image sensor may be done before the calculation for focusing is finished, and the resulting super slow motion video may end up with images not properly focused.

According to an embodiment of the disclosure, an electronic device is provided according to any of claims <NUM>-<NUM>.

According to an embodiment of the disclosure, a method for operating an electronic device configured to record an image is provided according to any of claims <NUM>-<NUM>.

According to an embodiment of the disclosure, non-transitory computer readable recording medium storing instructions for executing operations of an electronic device including a camera, a display, a memory, and at least one processor, is provided according to claim <NUM>.

According to various embodiments, an electronic device may lock focusing before initiating super slow motion video recording and record super slow motion video using the focusing-locked camera, obtaining a super slow motion video with the user's desired focus quality. According to various embodiments, an electronic device may provide a focusing method appropriate for super slow motion video recording.

The processor <NUM> may execute, e.g., software (e.g., a program <NUM>) to control at least one other component (e.g., a hardware or software component) of the electronic device <NUM> connected with the processor <NUM> and may process or compute various data.

The auxiliary processor <NUM> may control at least some of functions or states related to at least one (e.g., the display device <NUM>, the sensor module <NUM>, or the communication module <NUM>) of the components of the electronic device <NUM>, instead of the main processor <NUM> while the main processor <NUM> is in an inactive (e.g., sleep) state or along with the main processor <NUM> while the main processor <NUM> is an active state (e.g., executing an application).

According to an embodiment, the audio module <NUM> may obtain a sound through the input device <NUM> or output a sound through the sound output device <NUM> or an external electronic device (e.g., an electronic device <NUM> (e.g., a speaker or a headphone) directly or wirelessly connected with the electronic device <NUM>.

The communication module <NUM> may support establishing a direct (e.g., wired) communication channel or wireless communication channel between the electronic device <NUM> and an external electronic device (e.g., the electronic device <NUM>, the electronic device <NUM>, or the server <NUM>) and performing communication through the established communication channel.

The first and second external electronic devices <NUM> and <NUM> each may be a device of the same or a different type from the electronic device <NUM>.

<FIG> is a block diagram <NUM> illustrating the camera module <NUM> (e.g., the camera module <NUM> of <FIG>) according to various embodiments. Referring to <FIG>, the camera module 280includes a lens assembly <NUM> and may further include a flash <NUM>, an image sensor <NUM>, an image stabilizer <NUM>, memory <NUM> (e.g., buffer memory), or an image signal processor <NUM>. The lens assembly <NUM> may collect light emitted or reflected from an object whose image is to be taken. The lens assembly <NUM> may include one or more lenses. According to an embodiment, the camera module <NUM> may include a plurality of lens assemblies <NUM>. In such a case, the camera module <NUM> may form, for example, a dual camera, a <NUM>-degree camera, or a spherical camera. Some of the plurality of lens assemblies <NUM> may have the same lens attribute (e.g., view angle, focal length, auto-focusing, f number, or optical zoom), or at least one lens assembly may have one or more lens attributes different from those of another lens assembly. The lens assembly <NUM> may include, for example, a wide-angle lens or a telephoto lens.

The flash <NUM> may emit light that is used to reinforce light reflected from an object. According to an embodiment, the flash <NUM> may include one or more light emitting diodes (LEDs) (e.g., a red-green-blue (RGB) LED, a white LED, an infrared (IR) LED, or an ultraviolet (UV) LED) or a xenon lamp. The image sensor <NUM> may obtain an image corresponding to an object by converting light emitted or reflected from the object and transmitted via the lens assembly <NUM> into an electrical signal. According to an embodiment, the image sensor <NUM> may include one selected from image sensors having different attributes, such as a RGB sensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same attribute, or a plurality of image sensors having different attributes. Each image sensor included in the image sensor <NUM> may be implemented using, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

The image stabilizer <NUM> may move the image sensor <NUM> or at least one lens included in the lens assembly <NUM> in a particular direction, or control an operational attribute (e.g., adjust the read-out timing) of the image sensor <NUM> in response to the movement of the camera module <NUM> or the electronic device <NUM> including the camera module <NUM>. This allows compensating for at least part of a negative effect (e.g., image blurring) by the movement on an image being captured. According to an embodiment, the image stabilizer <NUM> may sense such a movement by the camera module <NUM> or the electronic device <NUM> using a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module <NUM>. According to an embodiment, the image stabilizer <NUM> may be implemented, for example, as an optical image stabilizer. The memory <NUM> may store, at least temporarily, at least part of an image obtained via the image sensor <NUM> for a subsequent image processing task. For example, if image capturing is delayed due to shutter lag or multiple images are quickly captured, a raw image obtained (e.g., a Bayer-patterned image, a high-resolution image) may be stored in the memory <NUM>, and its corresponding copy image (e.g., a low-resolution image) may be previewed via the display device <NUM>. Thereafter, if a specified condition is met (e.g., by a user's input or system command), at least part of the raw image stored in the memory <NUM> may be obtained and processed, for example, by the image signal processor <NUM>. According to an embodiment, the memory <NUM> may be configured as at least part of the memory <NUM> or as a separate memory that is operated independently from the memory <NUM>.

The image signal processor <NUM> may perform one or more image processing with respect to an image obtained via the image sensor <NUM> or an image stored in the memory <NUM>. The one or more image processing may include, for example, depth map generation, three-dimensional (3D) modeling, panorama generation, feature point extraction, image synthesizing, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor <NUM> may perform control (e.g., exposure time control or read-out timing control) with respect to at least one (e.g., the image sensor <NUM>) of the components included in the camera module <NUM>. An image processed by the image signal processor <NUM> may be stored back in the memory <NUM> for further processing, or may be transferred to an external component (e.g., the memory <NUM>, the display device <NUM>, the electronic device <NUM>, the electronic device <NUM>, or the server <NUM>) outside the camera module <NUM>. According to an embodiment, the image signal processor <NUM> may distributively process at least some of operations performed by the processor <NUM>. According to an embodiment, the image signal processor <NUM> may be configured as at least part of the processor <NUM>, or as a separate processor that is operated independently from the processor <NUM>. If the image signal processor <NUM> is configured as a separate processor from the processor <NUM>, at least one image processed by the image signal processor <NUM> may be displayed, by the processor <NUM>, via the display device <NUM> as it is or after being further processed.

According to an embodiment, the electronic device <NUM> may include a plurality of camera modules <NUM> having different attributes or functions. In such a case, at least one of the plurality of camera modules <NUM> may form, for example, a wide-angle camera and at least another of the plurality of camera modules180 may form a telephoto camera. Similarly, at least one of the plurality of camera modules <NUM> may form, for example, a front camera and at least another of the plurality of camera modules180 may form a rear camera.

<FIG> is a diagram illustrating a structure of an image sensor, according to an embodiment of the disclosure.

An image sensor <NUM> may be a component of a camera module in an electronic device.

Referring to <FIG>, the image sensor <NUM> may include at least one of a pixel array <NUM>, a row-driver <NUM>, a column-readout circuit <NUM>, a controller <NUM>, a memory <NUM>, or an interface <NUM>.

The pixel array <NUM> may include a plurality of pixels <NUM> to <NUM>. The pixel array <NUM> may have a structure in which the plurality of pixels <NUM> to <NUM> are arrayed in an MxN matrix pattern (where M and N are positive integers). The pixel array <NUM> where the plurality of pixels <NUM> to <NUM> are arrayed in a two-dimensional (2D) MxN pattern may have M rows and N columns. The pixel array <NUM> may include a plurality of photosensitive elements (e.g., photodiodes or pinned photodiodes). The pixel array <NUM> may detect light using the plurality of photosensitive elements and convert the light into an analog electrical signal to generate an image signal.

The row-driver <NUM> may drive the pixel array <NUM> for each row. The row-driver <NUM> may output transmission control signals to the transmission transistors of the plurality of pixels <NUM> to <NUM> in the pixel array <NUM>, reset control signals to control reset transistors, or select control signals to control selection transistors to the pixel array <NUM>. The row-driver <NUM> may determine a row to be read out.

The column-readout circuit <NUM> may receive analog electrical signals generated by the pixel array <NUM>. The column-readout circuit <NUM> may receive an analog electrical signal from a column line selected from among the plurality of columns constituting the pixel array <NUM>. The column-readout circuit <NUM> may include an ADC <NUM> that may convert the analog electrical signal received from the selected column line into pixel data (or a digital signal) and output the pixel data. Meanwhile, the column-readout circuit <NUM> that receives an analog electrical signal from the pixel array <NUM>, converts the received analog electrical signal into pixel data using the ADC <NUM>, and outputs the pixel data may be referred to as read-out. The column-readout circuit <NUM> and the ADC <NUM> may determine a column to be read out.

The column-readout circuit <NUM> of the image sensor <NUM> that supports of high speed photography may include a plurality of ADCs <NUM>. Each of the plurality of ADCs <NUM> may be connected in parallel with a respective one of the plurality of photodiodes in the pixel array <NUM>, and analog electrical signals simultaneously received from the plurality of photodiodes may quickly be converted into pixel data based on the parallel structure. The column-readout circuit <NUM> of the image sensor <NUM> that supports of high speed photography may perform a read-out at a high frame rate (e.g., <NUM> frames per second (fps)). Reading out at <NUM> fps means that receiving an analog electrical signal from the pixel array <NUM>, converting the received analog electrical signal into pixel data using the ADC <NUM>, and outputting the pixel data are performed once every <NUM>/<NUM> seconds. In other words, reading out at 960fps may mean that <NUM> image frames are outputted per second.

The controller <NUM> may obtain an image frame based on the pixel data received from the column-readout circuit <NUM>. The controller <NUM> may output the image frame through the interface <NUM> to an external circuit <NUM> (e.g., an image signal processor (ISP), processor, communication circuit, or external server). The controller <NUM> may generate transmission control signals to control the transmission transistors of the plurality of pixels <NUM> to <NUM>, reset control signals to control reset transistors, or select control signals to control selection transistors and provide the generated signals to the row-driver <NUM>. The controller <NUM> may generate a selection control signal to select at least one column line from among the plurality of column lines constituting the pixel array <NUM> and provide the generated signal to the column-readout circuit <NUM>. The column-readout circuit <NUM> may enable some of the plurality of column lines and disable the other column lines based on selection control signals provided from the controller <NUM>. The controller <NUM> may distributively process at least some of operations performed by the processor <NUM>. The controller <NUM> may be implemented in a processor <NUM> including a CPU or AP, as a sort of block or module. When the controller <NUM> is implemented as a block, the controller <NUM> may include a subtractor for detecting a difference between images, or a comparator for comparing images. The controller <NUM> may downsize read-out images and compare the plurality of downsized images to detect differences between the images.

The memory <NUM> may include a volatile and/or non-volatile memory. The memory <NUM> is a storage device inside the image sensor <NUM>. The memory <NUM> may include a buffer memory. The memory <NUM> may temporarily store digital signals output from the column-readout circuit <NUM> or the controller <NUM>. The memory <NUM> may include at least one image frame obtained based on light received by the pixel array <NUM>. The memory <NUM> may store at least one digital signal received from the external circuit <NUM> through the interface <NUM>.

The memory <NUM> may store at least one image frame read out at an Nth frame rate (e.g., <NUM> fps) or an Mth frame rate (e.g., 120fps) from the column-readout circuit <NUM> and deliver at least one image frame stored through the interface <NUM> to the external circuit <NUM> (e.g., an IPS, processor, communication circuit, or external server). In other words, the memory <NUM> may store at least one image frame read out once every <NUM>/<NUM> seconds or every <NUM>/<NUM> seconds from the column-readout circuit <NUM>, and the memory <NUM> may deliver at least one image frame stored through the interface <NUM> to the external circuit <NUM>. The speed at which the image frame is transferred to the external circuit <NUM> is not limited thereto. The electronic device <NUM> may transfer immediately, without storing, the read-out image frame through the interface <NUM> to the external circuit <NUM>.

Meanwhile, the controller <NUM> may store only some of N image frames read out through the column-readout circuit <NUM> at the Nth frame rate (e.g., <NUM> fps) in the memory <NUM>, allowing for substantially the same effect as if M image frames were obtained which are read out at the Mth frame rate (e.g., <NUM> fps). For example, the controller <NUM> may store only one of eight image frames read out at 960fps for <NUM>/<NUM> seconds in the memory <NUM>. When from among a plurality of image frames read out at 960fps, only image frames selected in the ratio of <NUM>:<NUM> are stored in the memory <NUM>, the image frames stored in the memory <NUM> may be substantially the same image frames as those read out at 120fps through the column-readout circuit <NUM>. When a video constituted of only image frames obtained at the cycle of <NUM>/<NUM> seconds is defined as '<NUM> fps video,' a video constituted of only image frames selected in the ratio of <NUM>:<NUM> from among the plurality of image frames read out at 960fps may be defined as a 120fps video. A video constituted of only image frames read out at 120fps through the column-readout circuit <NUM> may also be defined as a 120fps video.

The interface <NUM> may include the interface <NUM> or the communication module <NUM>. The interface <NUM> may connect components of the image sensor <NUM> (e.g., the controller <NUM> or the memory <NUM>) with the external circuit <NUM> in a wireless or wired scheme. The interface <NUM> may deliver at least one image frame stored in the memory <NUM> of the image sensor <NUM> to the external circuit <NUM>. (e.g., the memory 130of the electronic device <NUM>). The interface <NUM> may also deliver control signals from the processor <NUM> to the controller <NUM> of the image sensor <NUM>.

The image sensor <NUM> may communicate with the external circuit <NUM> through the interface <NUM> (e.g., in a serial communication scheme). The memory <NUM> of the image sensor <NUM> may communicate with the processor <NUM> in an inter-integrated circuit (I<NUM>C) scheme.

The image sensor <NUM> may connect with the external circuit <NUM> through the interface <NUM> (e.g., an interface as defined as per the mobile industry processor interface (MIPI) protocol). The memory <NUM> of the image sensor <NUM> may communicate with the processor <NUM> as per the interface defined in the MIPI protocol. The interface <NUM> (e.g., the interface defined as per the MIPI protocol) may deliver pixel data corresponding to the image frames stored in the memory <NUM> to the external circuit <NUM> at the cycle of <NUM>/<NUM> seconds.

Meanwhile, while the image frames stored in the memory <NUM> are delivered through the interface <NUM> having an output speed of 240fps to the external circuit <NUM> once every <NUM>/<NUM> seconds, at least some of the image frames read out in real-time through the column-readout circuit <NUM> may be delivered to the external circuit <NUM> as preview images once every <NUM>/<NUM> seconds. The processor <NUM> in the external circuit <NUM> may display, through the display, all or some of the image frames output as preview images from the image sensor <NUM> at 30fps or 60fps.

All or some of the above-described components <NUM> to <NUM> may be included in the image sensor <NUM> as necessary, and each component may be configured in a single unit or multiple units. The frame rates 120fps, 240fps, and 960fps, may be varied depending on the settings of the electronic device or the performance of the interface.

<FIG> is a diagram illustrating a process for obtaining an image frame through an image sensor, according to an embodiment. An image sensor <NUM> may be a component of a camera module in an electronic device.

Referring to <FIG>, the image sensor <NUM> may include at least one a pixel array <NUM>, a memory <NUM>, and an interface <NUM>. The image sensor <NUM> may include the whole or part of the image sensor <NUM> of <FIG>.

The pixel array <NUM> of the image sensor <NUM> may output an electrical signal corresponding to light received from the outside. The pixel array <NUM> may include a plurality of pixels constituted of photodiodes. The photodiodes may receive light and generate analog electrical signals corresponding to the received light. Analog electrical signals generated from the plurality of photodiodes constituting the plurality of pixels may be converted into a plurality of pieces of pixel data through a column-readout circuit. In this case, each piece of pixel data may mean a pixel value corresponding to its respective pixel. A set of a plurality of pieces of pixel data obtained at a particular time may constitute at least one image frame.

The pixel array <NUM> of the image sensor <NUM> may output a plurality of image frames <NUM> to <NUM> at a preset read-out speed. When the read-out speed is set to 960fps, the image sensor <NUM> may read-out <NUM> image frames per second based on light received by the pixel array <NUM>. The electronic device <NUM> may detect an event of slow motion recording while recording in a normal mode in which case the electronic device <NUM> may turn the read-out rate from 120fps to 950fps.

The plurality of image frames <NUM> to <NUM> read out may be stored in a memory <NUM> inside the image sensor <NUM>. The memory <NUM> of the image sensor <NUM> may include a buffer memory <NUM>. Some of the plurality of image frames <NUM> to <NUM> read out at 960fps may be stored in the buffer memory <NUM>. From among a plurality of image frames continuously read out, a designated number of image frames may be stored in the buffer memory <NUM>. The processor may repeat the operations of deleting the image frame stored earliest from among the image frames stored in the buffer memory <NUM> and storing the image frame latest from among the image frames.

At least one image frame stored in the memory <NUM> of the image sensor <NUM> may be delivered to an external circuit <NUM> through the interface <NUM>. The processor may control the interface <NUM> to deliver at least one image frame stored in the memory <NUM> to the external circuit <NUM>.

The image sensor <NUM> may exclude the buffer <NUM>. The image frames <NUM>, <NUM>, <NUM>, and <NUM> from the pixel array <NUM> may directly be transferred through the interface <NUM> to the external circuit <NUM>.

<FIG> is a flowchart of a method for locking the focusing of a lens assembly <NUM> of an electronic device <NUM> and capturing an image in slow motion, according to an embodiment. <FIG> is a diagram illustrating an operation in which an electronic device <NUM> performs the focusing of a lens assembly <NUM> in response to a user's touch input, according to an embodiment. <FIG> is a diagram illustrating an operation in which an electronic device <NUM> performs the focusing of a lens assembly <NUM> based on user input to select the degree of focus, according to an embodiment. <FIG> is a diagram illustrating an example in which an electronic device <NUM> initiates slow motion recording based on a movement of an object in a designated region, according to an embodiment.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> of <FIG> or the controller <NUM> of <FIG>) may obtain a plurality of first images as per a first frame rate on one or more objects using a camera <NUM> based on a signal related to image recording. For example, upon receiving an input regarding running a camera application or an input to start recording video from the user, the electronic device <NUM> may obtain the plurality of first images as per the first frame rate (e.g., 30fps or 60fps).

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera to perform a focusing of a lens assembly <NUM> included in the camera <NUM> on at least one object from among one or more objects while obtaining the plurality of first images. The electronic device <NUM> may focus on the at least one object included in a designated region in the plurality of first images. The electronic device <NUM> may focus on an object included in the region of interest (ROI) included in the plurality of first images. The electronic device <NUM> may focus on the plurality of first images using continuous autofocus-type contrast auto-focusing. The electronic device <NUM> may perform the focusing using an optical triangulation system or a phase-difference detection system, but focusing is not limited thereto.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera to perform the focusing of the lens assembly <NUM> for the objects corresponding to a region where the user's touch input is obtained. As shown in <FIG>, the electronic device <NUM> may receive the user's touch input through the display <NUM> and control the camera to perform the focusing of the lens assembly <NUM> onto the object corresponding to the region <NUM> where the touch input is received. The electronic device <NUM> may control the camera to perform the focusing of the lens assembly <NUM> based on user input to select the degree of focus. As shown in <FIG>, the electronic device <NUM> may display a user interface <NUM> for selecting the degree of focus on the plurality of first images through the display <NUM>. The electronic device <NUM> may control the camera to perform the focusing of the lens assembly <NUM> based on the user input to select the degree of focus through the user interface <NUM>. The above-described focusing methods amount to a mere example, and embodiments of the disclosure are not limited thereto.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera to perform adjusting exposure of the camera <NUM> on at least one object from among one or more objects while obtaining the plurality of first images. The electronic device <NUM> may adjust its exposure by adjusting at least one of the aperture or shutter speed of the camera <NUM> or sensitivity (e.g., the International Organization of Standardization (ISO)) of the image sensor <NUM>. The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera to perform adjusting exposure of the camera <NUM> on at least one object. The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera to perform adjusting exposure of the camera <NUM> on at least one object while locking the focusing.

At step <NUM>, the electronic device <NUM> (e.g., the processor <NUM>) may provide a first portion of the plurality of first images as a preview through the display <NUM>. The electronic device <NUM> may display all or some of the image frames, which have been output in the preview from the image sensor <NUM>, through the display <NUM> at 30fps or 60fps.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera to lock the focusing using the camera <NUM> to prevent a change of the focusing on at least one object. The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera <NUM> to lock the adjusting the exposure to prevent a change of the exposure on the at least one object. The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera <NUM> to lock the adjusting the exposure to prevent a change of the exposure on the at least one object while locking the focusing.

At step <NUM>, the electronic device <NUM> (e.g., the processor <NUM>) may identify a designated event in relation to obtaining the plurality of first images. The electronic device <NUM> (e.g., the processor <NUM>) may identify a user input related to initiating slow motion recording as part of identifying the designated event. As shown in <FIG>, the electronic device <NUM> may display a user interface <NUM> related to initiating slow motion recording through the display <NUM> and identify the reception of a user input through the user interface <NUM>. In response to receiving a user interface through the user interface <NUM> related to initiating slow motion recording, the electronic device <NUM> may lock the focusing to prevent defocusing on at least one object using the camera. In response to receiving a user interface through the user interface <NUM> related to initiating slow motion recording, the electronic device <NUM> may lock the adjustment of exposure to prevent a change in exposure from occurring on at least one object using the camera <NUM>.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may obtain a plurality of third images as per a third frame rate through the image sensor <NUM> included in the camera <NUM> based on a movement of the electronic device <NUM>, as part of identifying the designated event. As shown in <NUM> of <FIG>, upon detecting a movement of the electronic device <NUM>, the electronic device <NUM> may change the read-out speed from the first frame rate, 30fps or 60fps, to a third frame rate of 120fps, through the image sensor <NUM> and obtain a plurality of third images as per the third frame rate of 120fps. The third frame rate may be higher than the first frame rate and be equal to or lower than a second frame rate. The electronic device <NUM> may identify a movement of an object in a region <NUM> of the plurality of third images. As shown in <NUM>, the electronic device <NUM> may display the preview on the display <NUM> using at least one image of the plurality of third images and may identify a movement of an object (or a variation in pixel value in the ROI <NUM>) using a designated region (e.g., the ROI <NUM>) of the preview displayed. When the variation in pixel value in the designated region <NUM> of the displayed preview is not less than a threshold, the electronic device <NUM> may determine that an object movement is identified. The object movement may refer to various factors to vary the pixel value, such as, but not limited to, an object moving from outside the region <NUM> to the inside of the region <NUM>, an object moving out of the region <NUM>, or an object inside the region <NUM> reshaping. The electronic device <NUM> may identify whether there is a movement of the electronic device <NUM> within a designated time, and when no movement of the electronic device <NUM> is identified within the time, the electronic device <NUM> may identify a movement of an object in the designated region <NUM> of the preview. The electronic device <NUM> may identify a movement of the object in the designated region <NUM> of the preview using the camera <NUM> of which the focusing has been locked. The electronic device <NUM> may identify a movement of the object in the designated region <NUM> of the preview using the exposure adjustment-locked camera <NUM>. In response to identifying a movement of the object in the designated region <NUM> of the preview, the electronic device <NUM> may lock the focusing to prevent defocusing on at least one object using the camera <NUM>. In response to identifying a movement of the object in the designated region <NUM> of the preview, the electronic device <NUM> may lock the exposure adjustment to prevent the exposure from changing on at least one object using the camera <NUM>.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may obtain a plurality of second images at the second frame rate, which is higher than the first frame rate, using the focusing-locked camera <NUM> based at least in part on the designated event.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may obtain the plurality of second images as per the second frame rate using the focusing-locked camera <NUM> based on identifying a user input related to initiating slow motion recording. As shown in <FIG>, based on identifying the reception of the user input through the user interface <NUM>, the electronic device <NUM> may obtain the plurality of second images at a frame rate of 960fps using the focusing-locked camera <NUM>. The electronic device <NUM> may obtain the plurality of second images as per the second frame rate using the exposure adjustment-locked camera <NUM> in response to identifying a user input related to initiating slow motion recording. Based at least in part on the designated event, the electronic device <NUM> may obtain the plurality of second images as per the second frame rate using a camera <NUM> which locks at least one of the focusing or the adjusting the exposure.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may obtain the plurality of second images as per the second frame rate using the focusing-locked camera <NUM> based on identifying a movement of an object in a designated region of at least one of the plurality of the first images. As shown in <NUM>, based on identifying the movement of an object in the designated region <NUM> of at least one of the plurality of the first images, the electronic device <NUM> may obtain the plurality of second images at a frame rate of 960fps using the focusing-locked camera <NUM>. The electronic device <NUM> may obtain the plurality of second images as per the second frame rate using the exposure adjustment-locked camera <NUM> based on identifying the movement of an object in the designated region of the at least one of the plurality of the first images.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may provide a video related to at least one object of the one or more objects using a second portion of the plurality of first images and at least one of the plurality of second images. The electronic device <NUM> may provide a video related to an object by merging at least one of the plurality of first images obtained at a frame rate of 30fps or 60fps and at least one of the plurality of second images obtained at a frame rate of 960fps. The electronic device <NUM> may play the provided video through the display <NUM> as per a fourth frame rate. The fourth frame rate may be the same as or different from the first frame rate. The electronic device <NUM> may play the provided video at a frame rate of 30fps or 60fps through the display <NUM>.

<FIG> is a diagram illustrating an example in which an electronic device <NUM> resumes focusing after slow motion recording, according to an embodiment. <FIG> is a flowchart of a method for capturing video on an electronic device <NUM> in slow motion in response to identifying the movement of an object in a designated range of a preview, according to an embodiment. The embodiment of <FIG> is described in detail with reference to <FIG>, <FIG>.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM> in the image sensor <NUM>) may obtain a plurality of first images as per a first frame rate on one or more objects using a camera <NUM> based on a signal related to image recording. The electronic device <NUM> may obtain a plurality of first images as per a frame rate of 30fps or 60fps on objects using the camera <NUM> based on the image recording-related signal. The electronic device <NUM> may perform the focusing of a lens assembly <NUM> included in the camera <NUM> for at least one object from among one or more objects while obtaining the plurality of first images. The electronic device <NUM> may perform the focusing using the focusing step <NUM> of <FIG>.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may store one or more images of the plurality of first images for background recording in the memory <NUM>. When the electronic device <NUM> meets a designated condition for deleting at least one image from the memory, the electronic device <NUM> may delete at least one image of the one or more images of the plurality of first images stored in the memory <NUM>. Based on a designated time being elapsed since the recording-related signal is provided, the electronic device <NUM> may identify that the designated condition is met, and based on the designated condition being met, sequentially delete at least one image of the one or more images of the plurality of first images stored in the order of the storage. Based on one or more images of the plurality of first images for background recording being stored in the memory <NUM> by a predetermined range or capacity since the recording-related signal is provided, the electronic device <NUM> may identify that the designated condition is met, and based on the designated condition being met, sequentially delete at least one image of the one or more images of the plurality of first images stored in the order of the storage.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may identify whether there is movement of the electronic device <NUM>. The electronic device <NUM> may identify whether there is movement of the electronic device <NUM> within a predetermined time using sensor module <NUM>. As shown in <NUM> of <FIG>, the electronic device <NUM> may identify sensor data through the sensor module <NUM> during the predetermined time, and when the sensor data is less than a threshold, the electronic device <NUM> may determine that there is no movement of the electronic device <NUM>. The electronic device <NUM> may sense the movement of the electronic device <NUM> based on at least one of a gyro sensor or an acceleration sensor. As a sensor to detect the movement of the electronic device <NUM>, the gyro sensor or the acceleration sensor is merely an example, but is not limited thereto.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may lock the focusing of the lens assembly <NUM> included in the camera <NUM> on at least one object of one or more objects. The electronic device <NUM> may lock the focusing based on no movement of the electronic device <NUM>.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may identify the movement of an object in a designated region of at least one image as a preview. Based on no movement of the electronic device <NUM> being identified, the electronic device <NUM> may operate in slow motion recording standby mode. In the slow motion recording standby mode, the electronic device <NUM> may change the read-out speed of the image sensor <NUM> from the first frame rate corresponding to the plurality of first images to the third frame rate corresponding to the plurality of third images and object the plurality of third images as per the third frame rate. For example, in the slow motion recording standby mode, the electronic device <NUM> may change a read-out speed of 30fps or 60fps to a read-out speed of 120fps and obtain a plurality of images at the read-out speed of 120fps. The third frame rate may be higher than the first frame rate and be equal to or lower than the second frame rate.

Based on identifying no movement of the electronic device <NUM>, the electronic device <NUM> (at least one of the processor <NUM> or the controller <NUM>) may identify the movement of an object in the designated region <NUM> of at least one of the plurality of first images or in the designated region <NUM> of at least one of the plurality of third images. As shown in <NUM> of <FIG>, based on identifying no movement of the electronic device <NUM>, the electronic device <NUM> may identify whether there is movement of the object in the designated region <NUM>. When the pixel variation in the designated region <NUM> of the plurality of images is larger than a threshold related to the movement of the object, the electronic device <NUM> may determine that there is movement of the object. The electronic device <NUM> may identify the movement of the object in the designated region <NUM> of the plurality of images using the focusing-locked camera. In response to identifying the movement of the object in the designated region <NUM> of the plurality of images, the electronic device <NUM> may lock the focusing on the object.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> of <FIG> or the controller <NUM> of <FIG>) may obtain a plurality of second images at the second frame rate, which is higher than the first frame rate, using the focusing-locked camera <NUM>. As shown in <NUM> of <FIG>, upon identifying the movement of an object in the designated region <NUM>, the electronic device <NUM> may obtain the plurality of second images at a frame rate of 960fps, which is higher than the frame rate of 30fps or 60fps, using the focusing-locked camera <NUM>. While obtaining the plurality of second images, the electronic device <NUM> may read out the plurality of second images as per the second frame rate through the image sensor <NUM> and store the plurality of second images in the memory <NUM> as per the second frame rate.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera <NUM> to perform focusing based on the driving speed of the lens assembly <NUM> and the read-out speed of the image sensor <NUM>. When the time for the focusing of the lens assembly <NUM> is equal to or shorter than the read-out time, the electronic device <NUM> may control the camera to perform the focusing of the lens assembly <NUM>. When the time for the focusing of the lens assembly <NUM> is <NUM>/<NUM> seconds, and the read-out time of the image sensor <NUM> operating at a frame rate of 120fps is <NUM>/<NUM> seconds, the electronic device <NUM> may perform focusing.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera to perform focusing based on the moving distance of the object for the read-out time of the image sensor <NUM>. The electronic device <NUM> may identify the moving distance of the object along the optical axis through the sensor module <NUM> (e.g., an IR sensor). The electronic device <NUM> may identify the moving distance of the object based on at least one image frame obtained by performing the focusing. Based on the moving distance of the object along the optical axis with respect to the read-out time of the image sensor <NUM> being not less than the depth of focus, the electronic device <NUM> may control the camera <NUM> to perform focusing. Based on the read-out time of the image sensor <NUM> operating at a frame rate of 120fps being <NUM>/<NUM> seconds, and the moving distance of the object along the optical axis during the time being not less than the depth of focus, the electronic device <NUM> may control the camera <NUM> to perform focusing.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera <NUM> to resume the focusing of the lens assembly <NUM> based on stopping obtaining the plurality of second images. When a slow motion recording termination event occurs, the electronic device <NUM> may stop obtaining the plurality of second images. As the slow motion recording termination event, the electronic device <NUM> may determine that the slow motion recording termination event has occurred when a designated time elapses after the time of obtaining the plurality of second images. As the slow motion recording termination event, the electronic device <NUM> may determine that the slow motion recording termination event has occurred when the plurality of second images are stored in the memory <NUM> by a designated number of image frames or by a designated capacity. As the slow motion recording termination event, the electronic device <NUM> may determine that the slow motion recording termination event has occurred when user input is identified through a user interface related to the termination of the slow motion recording. When the slow motion recording termination event occurs, the electronic device <NUM> may control the camera <NUM> to resume the focusing using the focusing step <NUM> of <FIG>.

The electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera <NUM> to resume the focusing of the lens assembly <NUM> by moving the lens assembly <NUM> from a first position at the time of resuming the focusing to a second position away from where the at least one object is positioned. When the distance between the electronic device <NUM> and an object <NUM> increases as shown in <FIG>, the electronic device <NUM> may control the camera <NUM> to resume the focusing of the lens assembly <NUM> by moving the position <NUM> of the lens assembly <NUM> at the time of resuming the focusing away from where the object <NUM> is positioned.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may identify whether a video recording termination event has occurred. As shown in <FIG>, upon identifying a user input related to the termination of video recording through the user interface <NUM>, the electronic device <NUM> may identify that the video recording termination event has occurred.

Upon identifying no video recording termination event, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may return to step <NUM>, obtaining a plurality of first images for one or more objects as per the first frame rate. From the time of stopping obtaining the plurality of second images or the time of obtaining the plurality of first images, the electronic device <NUM> may perform part of background recording at step <NUM>. From the time of stopping obtaining the plurality of second images or the time of obtaining the plurality of first images, the electronic device <NUM> may store at least one image of the plurality of first images in the memory <NUM>, and when a designated condition is met at step <NUM>, the electronic device <NUM> may delete at least some of the plurality of first images stored in the memory <NUM>.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> of <FIG> or the controller <NUM> of <FIG>) may provide a video related to an object using at least one image of the plurality of first images and at least one image of the plurality of second images. The electronic device <NUM> may provide a video related to the objects by merging at least one image of the plurality of first images obtained at a frame rate of 30fps or 60fps and at least one image of the plurality of second images obtained at a frame rate of 960fps. The electronic device <NUM> may store the video related to the objects in the memory <NUM>. Based on stopping obtaining the plurality of second images, the electronic device <NUM> may provide the video related to the objects using at least one image of the plurality of first images and at least one image of the plurality of second images. The electronic device <NUM> may play the provided video through the display <NUM> as per a fourth frame rate. The fourth frame rate may be the same as or different from the first frame rate. The electronic device <NUM> may play the provided video at a frame rate of 30fps or 60fps through the display <NUM>.

<FIG> is a flowchart of a method for capturing video on an electronic device <NUM> in slow motion in response to identifying a user input related to initiating slow motion recording, according to an embodiment.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM> in the image sensor <NUM>) may obtain a plurality of first images as per a first frame rate on one or more objects using a camera <NUM> based on a signal related to image recording. The electronic device <NUM> may obtain a plurality of first images as per a frame rate of 30fps or 60fps on objects using the camera <NUM> based on the image recording-related signal. The electronic device <NUM> may perform the focusing of a lens assembly <NUM> included in the camera <NUM> on at least one object from among one or more objects while obtaining the plurality of first images. The electronic device <NUM> may control the camera <NUM> to perform the focusing using the focusing step <NUM> of <FIG>.

The electronic device <NUM> (e.g., the processor <NUM>) may store at least one image of the plurality of first images for background recording in the memory <NUM>. The electronic device <NUM> may perform the background recording using the background recording step <NUM> of <FIG>.

At step <NUM>, the electronic device <NUM> (e.g., the processor <NUM>) may identify user input related to initiating slow motion recording. As shown in <FIG>, the electronic device <NUM> may display a user interface <NUM> related to initiating slow motion recording through the display <NUM> and identify the reception of the user input, which is related to initiating slow motion recording, through the user interface <NUM>.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera <NUM> to lock the focusing of the lens assembly <NUM> included in the camera <NUM> on at least one object of one or more objects. Based on obtaining a user interface through the user interface <NUM> related to initiating slow motion recording, the electronic device <NUM> may control the camera <NUM> to lock the focusing to prevent a change of the focusing on at least one object using the camera <NUM>.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> of <FIG> or the controller <NUM> of <FIG>) may obtain a plurality of second images at the second frame rate, which is higher than the first frame rate, using the focusing-locked camera <NUM>. As shown in <FIG>, based on a user input being received through the user interface <NUM> related to initiating slow motion recording, the electronic device <NUM> may obtain the plurality of second images at a frame rate of 960fps, which is higher than the frame rate of 30fps or 60fps, using the focusing-locked camera <NUM>. While obtaining the plurality of second images, the electronic device <NUM> may read out the plurality of second images as per the second frame rate through the image sensor <NUM> and store the plurality of second images in the memory <NUM> as per the second frame rate.

At step <NUM>, the electronic device <NUM> (e.g., at least one of the processor <NUM> or the controller <NUM>) may control the camera <NUM> to resume the focusing of the lens assembly <NUM> based on stopping obtaining the plurality of second images. The electronic device <NUM> may control the camera <NUM> to resume the focusing using the step <NUM> of <FIG> of resuming focusing.

At step <NUM>, as shown in <FIG>, based on identifying user input related to the termination of video recording through the user interface <NUM>, the electronic device <NUM> (e.g., the processor <NUM>) may identify that the video recording termination event has occurred.

Upon identifying no video recording termination event, the electronic device <NUM> may return to step <NUM>, obtaining a plurality of first images for one or more objects as per the first frame rate. From the time of stopping obtaining the plurality of second images or the time of obtaining the plurality of first images, the electronic device <NUM> may perform part of background recording at step <NUM>. From the time of stopping obtaining the plurality of second images or the time of obtaining the plurality of first images, the electronic device <NUM> may store at least one image of the plurality of first images in the memory <NUM>, and when a designated condition is met at step <NUM>, the electronic device <NUM> may delete one or more images of the plurality of first images stored in the memory <NUM>.

At step <NUM>, the electronic device <NUM> may provide a video related to an object using at least one image of the plurality of first images and at least one image of the plurality of second images. The electronic device <NUM> may provide a video related to the objects by merging at least one image of the plurality of first images obtained at a frame rate of 30fps or 60fps and at least one image of the plurality of second images obtained at a frame rate of 960fps. The electronic device <NUM> may store the video related to the objects in the memory <NUM>. Based on stopping obtaining the plurality of second images, the electronic device <NUM> may provide the video related to the objects using at least one image of the plurality of first images and at least one image of the plurality of second images. The electronic device <NUM> may play the provided video through the display <NUM> as per a fourth frame rate. The fourth frame rate may be the same as or different from the first frame rate. The electronic device <NUM> may play the provided video at a frame rate of 30fps or 60fps through the display <NUM>.

The electronic device may be one of various types of electronic devices. The electronic device is not limited to the above-listed embodiments.

It should be appreciated that various embodiments of the disclosure and the terms used therein are not intended to limit the technological features set forth.

According to an embodiment, a method may be provided in a computer program product.

Claim 1:
An electronic device (<NUM>), comprising:
a camera (<NUM>) including a lens assembly (<NUM>);
a display (<NUM>);
a memory (<NUM>); and
at least one processor (<NUM>) configured to:
obtain (<NUM>) a plurality of first images for one or more objects as per a first frame rate using the camera based on a signal related to image recording and control the camera to perform focusing of the lens assembly (<NUM>) on at least one of the one or more objects corresponding to a region where a user's touch is obtained;
provide (<NUM>) a first portion of the plurality of first images as a preview through the display (<NUM>);
control (<NUM>) the camera (<NUM>) to lock the focusing;
identify (<NUM>) movement of at least one of the one or more objects corresponding to said region;
based on the identifying of movement, obtain (<NUM>) a plurality of second images as per a second frame rate higher than the first frame rate using the focusing-locked camera (<NUM>); and
provide (<NUM>) a video related to the at least one of the one or more objects corresponding to said region by merging at least one of the plurality of first images and at least one of the plurality of second images.