Patent Description:
With the development of information communication technology and semiconductor technology, various electronic devices are improving as multimedia devices that are capable of providing various multimedia services. The multimedia service may include at least one of a voice call service, a messaging service, a broadcast service, a wireless Internet service, a camera service, an electronic payment service, and/or a music playback service.

The electronic device may provide various services utilized by the user by way of application programs. The application programs may include for example music applications, games, a call application program, and a camera application program, etc..

<CIT> discloses a system including a computing device that includes a memory for storing content of a sequence of frames, and a frame predictor. The frame predictor determines a difference between content of a first frame and content of a second frame, in which the first and second frames are adjacent in the sequence of frames. The difference between the content of the first and second frames is representative of image changes included in the content of first and second frames. The frame predictor is configured to produce an intermediate frame using at least one of the first and second frames, and the determined difference. The frame predictor is further configured to insert the intermediate frame into the frame sequence.

The above information is presented as background information only to assist with an understanding of the invention. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with regard to the invention.

The electronic device may support a frame interpolation (FI) method to improve graphic performance (e.g., frames per second or "FPS") of an application program. For example, in the frame interpolation method, an interpolated image is generated by mixing the (n-<NUM>)th image (or frame) and the nth image (or frame), and inserting and rendering the interpolated image between the images (e.g., the (n-<NUM>)th image and the nth image) to double the FPS of the application program.

However, the frame interpolation method may generate higher graphic performance than necessary when adding the interpolated image between images (or frames) associated with an application program. For example, the frame interpolation method can improve the graphic performance of an application program supporting <NUM> FPS to <NUM> FPS. However, when the electronic device supports a maximum of <NUM> FPS, computing resources are wasted in generation <NUM> FPS, which produces a waste of <NUM> unnecessary images (or frames).

In addition, when the frame interpolation method is applied to the electronic device, an afterimage may occur in the interpolated image due to the difference between the images referred to generate the interpolated image.

Certain embodiments of the invention disclose a device and a method for optimizing the graphic performance of an application program in an electronic device.

Certain embodiments of the invention disclose a device and method for reducing the occurrence of an afterimage caused by application of a frame interpolation method in an electronic device.

According to certain embodiments not covered by the invention, an electronic device may include a display device, and a processor operatively connected to the display device, wherein the processor is configured to: execute an application program, based on detecting a frame drop, identify an insertion position of an interpolation image for a plurality of images related to execution of the application program, generate an interpolation image corresponding to the identified insertion position of the interpolation image, and insert the interpolation image into at least a portion of the plurality of images according to the identified insertion position.

According to certain embodiments not covered by the invention, an operation method of an electronic device may include: executing an application program, based on detecting a frame drop, identifying an insertion position of an interpolation image based on a plurality of images related to execution of the application program, generating an interpolation image corresponding to the identified insertion position of the interpolation image, and inserting the interpolation image into at least a portion of the plurality of images according to the identified insertion position.

According to the invention, an electronic device includes a display device, and a processor operatively connected to the display device, wherein the processor is configured to execute an application program, generate a plurality of images for output to the display device according to the execution of the application program, identify a difference between images having adjacent playback time points among the plurality of images, identify an insertion position of an interpolation image as being between the images based on the identified difference between the images, wherein a plurality of reference ranges is set for the plurality of images, each reference range indicating a count of frames for which a single interpolation image is to be inserted, and wherein the insertion position of the interpolation image identified within each reference range is based on the smallest identified difference of respective images included in each reference range, generate the interpolation image corresponding to the identified insertion position of the interpolation image, and insert the interpolation image into the insertion position between the images.

According to certain embodiments not covered by the invention, the electronic device selectively generates and inserts an interpolation image on the basis of the difference between the images, thereby reducing the occurrence of an afterimage caused by the frame interpolation method.

According to certain embodiments not covered by the invention, when a frame drop occurs, the electronic device can maintain the graphic performance of the electronic device by selectively applying the clock control or frame interpolation method of the internal circuit on the basis of the result of machine learning (e.g., Q-learning) based on the difference of the images (or frames).

For a more complete understanding of the invention and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which like reference numerals represent like parts:.

Hereinafter, certain embodiments will be described in detail with reference to the accompanying drawings. It should be understood that the examples and terms used herein are not intended to limit the technology described in this document to specific embodiments, but include various modifications, equivalents, and/or substitutes of the examples. In connection with the description of the drawings, similar reference numerals may be used for similar components. Singular expressions may include plural expressions unless the context clearly indicates otherwise.

<FIG> is a block diagram illustrating an electronic device <NUM> in a network environment <NUM> according to certain embodiments not covered by the invention but which are described for illustrative purpose only.

The input device <NUM> may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., stylus pen).

A corresponding one of these communication modules may communicate with the external electronic device via the first network <NUM> (e.g., a short-range communication network, such as BluetoothTM , wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA)) or the second network <NUM> (e.g., a long-range communication network, such as a cellular network, the Internet, or a computer network (e.g., LAN or wide area network (WAN)).

According to an embodiment, the antenna module <NUM> may include an antenna including a radiating element implemented using a conductive material or a conductive pattern formed in or on a substrate (e.g., PCB).

According to an embodiment not covered by the invention, commands or data may be transmitted or received between the electronic device <NUM> and the external electronic device <NUM> via the server <NUM> coupled with the second network <NUM>. Each of the external electronic devices <NUM> and <NUM> may be a device of a same type as, or a different type, from the electronic device <NUM>.

The electronic device according to certain embodiments may be one of various types of electronic devices.

It should be appreciated that certain embodiments of the present disclosure and the terms used therein are not intended to limit the technological features set forth herein to particular embodiments and include various changes, equivalents, or replacements for a corresponding embodiment.

Certain embodiments as set forth herein may be implemented as software (e.g., the program <NUM>) including one or more instructions that are stored in a storage medium (e.g., internal memory <NUM> or external memory <NUM>) that is readable by a machine (e.g., the electronic device <NUM>). The term "non-transitory" simply means that the storage medium is a tangible device, and does not include a signal (e.g., an electromagnetic wave), but this term does not differentiate between where data is semi-permanently stored in the storage medium and where the data is temporarily stored in the storage medium.

According to an embodiment not covered by the invention, a method according to certain embodiments of the disclosure may be included and provided in a computer program product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded) online via an application store (e.g., Play StoreTM ), or between two user devices (e.g., smart phones) directly.

According to certain embodiments not covered by the invention, each component (e.g., a module or a program) of the above-described components may include a single entity or multiple entities. According to certain embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. In such a case, according to certain embodiments, the integrated component may still perform one or more functions of each of the plurality of components in the same or similar manner as they are performed by a corresponding one of the plurality of components before the integration. According to certain embodiments, operations performed by the module, the program, or another component may be carried out sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be executed in a different order or omitted, or one or more other operations may be added.

<FIG> is a block diagram of an electronic device <NUM> for selectively inserting an interpolation image according to certain embodiments not covered by the invention but which are described for illustrative purpose only. As an example, at least some components of <FIG> will be described with reference to <FIG>, <FIG>, and <FIG>. <FIG> illustrates a configuration in which interpolation images according to certain embodiments not covered by the invention are inserted, <FIG> illustrates a configuration in which interpolation images according to certain embodiments not covered by the invention are inserted, and <FIG> illustrates a configuration in which interpolation images according to certain embodiments not covered by the invention are inserted.

Referring to <FIG>, according certain embodiments, a processor <NUM> may include a control module <NUM>, an application execution module <NUM>, an interpolation image generation module <NUM>, and/or a machine learning module <NUM>. For example, the modules <NUM>, <NUM>, <NUM>, and <NUM> included in the processor <NUM> may be implemented using software, so that instructions related to the processor <NUM> may be located and executed from a memory (e.g., the memory <NUM> of <FIG>).

According to certain embodiments, the control module <NUM> may control the execution of an application program. According to an embodiment, the control module <NUM> may control the application execution module <NUM> to execute an application program (e.g., a game) corresponding to an icon selected by a user input among a plurality of icons displayed on the display device <NUM>. According to an embodiment, the control module <NUM> may control the interpolation image generation module <NUM> to generate an interpolation image to improve graphic performance (e.g., frames per second (FPS)) of the application program executed via the application execution module <NUM>. For example, the control module <NUM> may configure the insertion position of the interpolation image based on the target graphic performance (e.g., FPS) of the application program executed in the application execution module <NUM> (e.g., positioning an insertion frame at regular intervals - referred to herein as reference ranges - in order to boost FPS from a <NUM> FPS to <NUM> FPS, which would require inserting <NUM> interpolation frame for every <NUM> existing frames). For another example, the control module <NUM> may configure the insertion position of the interpolation image for each reference range based on the difference (or difference value) of adjacent images (or frames) related to the execution of the application program. According to the invention, the control module <NUM> configures the insertion position of an interpolation image within a reference range based on two images (or frames) having the smallest difference between adjacent images (or frames) within the reference range. A reference range indicates a number of frames in which one interpolation image is to be inserted. As an example, the size of the reference range may be configured based on at least one of target graphic performance (e.g., FPS) of the application program or status information of the electronic device <NUM>. As another example, the control module <NUM> may configure the insertion position of the interpolation image on the basis of two images (or frames) in which a difference between images (or frames) satisfies a reference difference among a plurality of images (or frames) related to execution of an application program. For example, the control module <NUM> may configure an insertion position of the interpolation image on the basis of a Q-learning result on the basis of a difference between images performed through the machine learning module <NUM>.

According to certain embodiments, the control module <NUM> may identify differences between adjacent frames/images among the plurality of images (or frames) related to the execution of the application program executed in the application execution module <NUM>. According to an embodiment, the control module <NUM> may configure an image segmentation method on the basis of target graphic performance (e.g., FPS) of the application program executed in the application execution module <NUM>. The control module <NUM> may compare the images by dividing each image into parts (e.g., cells) using image segmentation, and then compare corresponding cells between the two images (e.g., being disposed in a same part of the image) to detect differences between the images. For example, the control module <NUM> may configure the granularity of an image segmentation method by selection of a number of parts or segments (e.g., <NUM> x <NUM> segmentation has less detail than <NUM> x <NUM> segmentation) so that images can be compared with more detail when the target program's target graphic performance is relatively below a certain threshold (e.g., <NUM> FPS or less). According to an embodiment, the control module <NUM> may change an image segmentation method for identifying a difference between images on the basis of the number of times the difference between the images does not satisfy the reference difference. For example, the control module <NUM> may change the image segmentation method so that images can be relatively closely compared when the number of times that the difference between the images does not satisfy the reference difference exceeds the reference number. For example, the difference between the images is a change amount of at least one object included in the image, and it can be determined that the difference between the images is relatively large as the change amount of the object is relatively large.

According to certain embodiments not covered by the invention, when the occurrence of frame reduction is detected or predicted, the control module <NUM> may control the graphic performance of the application program generated by the application execution module <NUM> to be improved. According to an embodiment, when the occurrence of frame reduction is detected or predicted, the control module <NUM> may control the interpolation image generation module <NUM> to apply a frame interpolation (FI). For example, the control module <NUM> may control the interpolation image generation module <NUM> to selectively generate an interpolation image on the basis of at least one of target graphic performance or image differences of the application program generated by the application execution module <NUM>. According to an embodiment, when the occurrence of frame reduction is detected or predicted, the control module <NUM> may identify the Q-learning result on the basis of the difference between the images performed through the machine learning module <NUM>. When it is determined that the application of the frame interpolation method is appropriate on the basis of the Q-learning result, the control module <NUM> may control the interpolation image generation module <NUM> to selectively interpolate based on at least one of target graphic performance or differences images (or frames) of the application program generated by the application execution module <NUM>. When it is determined that the application of the clock control method is appropriate on the basis of the Q-learning result, the control module <NUM> may control the clock of the internal circuit (e.g., CPU or GPU) related to the graphic of the application program. For example, frame reduction may include a phenomenon in which the generation of a frame (or image) related to execution of an application program for rendering is limited or lost at a time point based on a state of the electronic device <NUM>.

According to certain embodiments, the control module <NUM> may render an image (or frame) generated by at least one of the application execution module <NUM> or the interpolation image generation module <NUM> so that the image (or frame) can be displayed on the display device <NUM>. According to an embodiment, the control module <NUM> may insert and render the interpolation image generated by the interpolation image generation module <NUM> into a plurality of images (or frames) related to execution of the application program generated by the application execution module <NUM>. For example, the control module <NUM> may insert and render an interpolation image <NUM> for each image (or frame) <NUM> related to the execution of the application program generated by the application execution module <NUM>, as shown in <FIG>. By inserting one interpolation image for each existing frame, the graphic performance (FPS) may be mathematically doubled from, for example, <NUM> FPS to <NUM> FPS, which improves upon the native <NUM> FPS performance originally supported by the application program, as shown in <FIG>. For another example, the control module <NUM> may insert and render the interpolation image <NUM> for every two images (or frames) on the basis of the target graphic performance (e.g., <NUM> FPS to be increased to <NUM> FPS) of the application program, as shown in <FIG>. For example, the control module <NUM> may insert and render the interpolation image <NUM> after the first frame <NUM> related to the execution of the application program, but contrary to the previous example, neglect insertion of another interpolation image in position <NUM>. This results in an insertion of <NUM> interpolation image for every <NUM> existing frames, resulting in an approximate <NUM>% increase in frame rate (e.g., <NUM> FPS to <NUM> FPS). In another example, the control module <NUM> may selectively insert and render an interpolation image on the basis of a difference between images (or frames) related to the execution of an application program generated by the application execution module <NUM>, as shown in <FIG>. For example, if the difference between the fifth image (or frame) <NUM> and the sixth image (frame) <NUM> is greater than or equal to a reference difference (e.g., <NUM> %), the control module <NUM> may limit the insertion of the interpolation image between the fifth image (or frame) <NUM> and the sixth image (or frame) <NUM> (<NUM>). If the difference between the sixth image (or frame) <NUM> and the seventh image (or frame) <NUM> is less than the reference difference (e.g., <NUM> %), the control module <NUM> may insert and render an interpolation image <NUM> between the sixth image (or frame) <NUM> and the seventh image (or frame) <NUM>. The sum of this methodology is that positions <NUM>, <NUM> and <NUM> result in an absence of an interpolation image for frame pairs having sufficient differences equal or greater than the threshold, whereas interpolation images <NUM>, <NUM> and <NUM> are generated for frames having insufficient differences less than the threshold.

According to certain embodiments, the application execution module <NUM> may execute an application program. According to an embodiment, the application execution module <NUM> may generate a plurality of images (or frames) related to the execution of the application program. For example, when executing a game application, the application execution module <NUM> may generate a plurality of images (or frames) corresponding to the progress of the game.

According to certain embodiments not covered by the invention, the interpolation image generation module <NUM> may generate at least one interpolation image for improving the graphic performance of the application program executed in the application execution module <NUM> on the basis of the control of the control module <NUM>. According to an embodiment, if an application program is executed in the application execution module <NUM>, the interpolation image generation module <NUM> may generate the interpolation image corresponding to an insertion position configured by the control module <NUM>. For example, the interpolation image may be generated by referring to two adjacent (or contiguous) images (or frames) of a plurality of images (or frames) related to the execution of an application program to be inserted between two adjacent (or contiguous) images (or frames). For example, the reference ratio of two adjacent images for generating an interpolation image may be fixed (e.g., <NUM>), or may be varied based on an insertion position of an interpolation image or an image difference of a reference image. According to an embodiment, when the occurrence of frame reduction associated with the application program executed in the application execution module <NUM> is detected or predicted, the interpolation image generation module <NUM> may generate an interpolation image corresponding to the insertion position configured by the control unit <NUM>.

According to certain embodiments not covered by the invention, the machine learning module <NUM> may perform machine learning on the basis of at least one of status information of the electronic device <NUM> or images (or frames) provided in an application program. According to an embodiment, the machine learning module <NUM> may determine optimal graphic performance (e.g., FPS) of an application program executed in the application execution module <NUM> through machine learning. According to an embodiment, when the application program is executed by the application execution module <NUM>, the machine learning module <NUM> may predict a frame drop associated with the application program through machine learning. According to an embodiment, the machine learning module <NUM> may configure (or recommend) an optimal method among at least one method for solving frame reduction through learning. As an example, a method for solving the frame reduction may include at least one of a CPU clock control scheme, a GPU clock control scheme, or a frame interpolation scheme. In an example, machine learning may include Q-learning. For example, the status information of the electronic device <NUM> may include at least one of a clock of an internal element (e.g., GPU and/or CPU), a load of the internal element, and a power value of the electronic device <NUM> (e.g., remaining battery capacity), or temperature changes (internal and/or external) of the electronic device <NUM>.

According to certain embodiments not covered by the invention, the control module <NUM> may adjust a ratio between an image (or frame) related to the execution of the application program generated by the application execution module <NUM> and an interpolation image generated by the interpolation image generation module <NUM>. According to an embodiment, the control module <NUM> may control the application execution module <NUM> and/or the interpolation image generation module <NUM> in order to replace at least some of the multiple images (or frames) related to the execution of the application program to be generated by the application execution module <NUM> with the interpolation image generated by the interpolation image generation module <NUM>. For example, if the control module <NUM> executes a game application program that supports a first graphic performance (e.g., <NUM> FPS), the control module <NUM> may control the application execution module <NUM> to generate a plurality of images (or frames) related to the game application program on the basis of a second graphic performance (e.g., <NUM> FPS) lower than the first graphic performance. The control module <NUM> may control the interpolation image generation module <NUM> to generate at least one interpolation image corresponding to a difference (e.g., <NUM> FPS) between the first graphic performance (e.g., <NUM> FPS) and the second graphic performance (e.g., <NUM> FPS) so that the game application program is executed with the first graphic performance. For example, the interpolation image generation module <NUM> may generate at least one interpolation image with reference to the rendered images in connection with the execution of the game application program. In this case, the electronic device <NUM> may use at least one interpolation image corresponding to the difference (e.g., <NUM> FFS) between the first graphic performance (e.g., <NUM> FPS) and the second graphic performance (e.g., <NUM> FPS), so that it is possible to reduce resource consumption for the execution of the game application program rather than rendering the entire image (or frame) related to the execution of the game application program.

According to certain embodiments not covered by the invention, an electronic device (e.g., the electronic device <NUM> of <FIG>) may include a display device (e.g., the display device <NUM> of <FIG>) and a processor (e.g., the processor <NUM> of <FIG>) operatively connected to the display device, and the processor may execute an application program, and when a frame drop is detected, identify an insertion position of an interpolation image on the basis of a plurality of images related to execution of the application program, generate an interpolation image corresponding to the insertion position of the interpolation image, and insert the interpolation image into at least a part of the plurality of images on the basis of the insertion position of the interpolation image.

According to certain embodiments, the processor may identify a difference between adjacent images having a playback time point among the plurality of images, and identify an interpolation image insertion position on the basis of the difference between the images.

According to certain embodiments, the processor may identify the insertion position of the interpolation image within the reference range on the basis of a difference of images for each reference range.

According to certain embodiments of the invention, the processor configures the insertion position of the interpolation image on the basis of images having the smallest difference in images within the reference range.

According to certain embodiments, the processor may identify a difference between a first image and a second image adjacent to a playback time point among a plurality of images related to the execution of the application program, and may configure an insertion position of the interpolation image on the basis of the first image and the second image when the difference between the first image and the second image satisfies a designated first condition.

According to certain embodiments, when configuring an insertion position of the interpolation image on the basis of the first image and the second image, the processor may generate an interpolation image between the first image and the second image through frame interpolation on the basis of the first image and the second image.

According to certain embodiments, when the difference between the first image and the second image does not satisfy the designate first condition, the processor may identify the difference between the second image and a third image, which are adjacent to each other in a playback time point among the plurality of frames.

According to certain embodiments, the processor may divide the first image and the second image into a first number of regions on the basis of a first segmentation method to identify differences in images, may identify the number of consecutive failures of the designated first condition when the difference between the first image and the second image does not satisfy the designated first condition, and may identify the difference between the images by dividing the second image and the third image into a second number of regions on the basis of a second segmentation method different from the first segmentation method when the number of consecutive failures of the designated first condition satisfies the designated second condition.

According to certain embodiments, the processor may identify the insertion position of the interpolation image on the basis of the Q-learning result based on the difference between images having adjacent playback time points among the plurality of images related to the execution of the application program.

According to certain embodiments, an electronic device (e.g., the electronic device <NUM> of <FIG>) may include a display device (e.g., the display device <NUM> of <FIG>) and a processor (e.g., the processor <NUM> of <FIG>) operatively connected to the display device, and the processor may execute an application program, identify the difference between images adjacent to each other in a playback time point among the plurality of images related to the execution of the application program, identify an insertion position of an interpolation image on the basis of the difference of the images, generate an interpolation image corresponding to the insertion position of the interpolation image, and insert the interpolation image into at least a part of the plurality of images on the basis of the insertion position of the interpolation image.

<FIG> is a flowchart illustrating insertion of an interpolation image on the basis of differences in images in an electronic device according to the invention. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. As an example, the electronic device may be the electronic device <NUM> of <FIG>.

Referring to <FIG>, the electronic device (e.g., the processor <NUM> of <FIG>, the control module <NUM> or the application execution module <NUM> of <FIG>), in operation <NUM>, executes at least one application program related to the electronic device (e.g., the electronic device <NUM>). According to an embodiment, the processor <NUM> (or the control module <NUM>) may execute an application program associated with an icon corresponding to a selection input when the selection input (or a user input) for any one of at least one icon displayed on the display device <NUM> is detected. For example, at least one icon displayed on the display device <NUM> may correspond to at least one application program installed in the electronic device <NUM>.

In operation <NUM>, the electronic device (e.g., the processor <NUM> or the application execution module <NUM>) identifies a plurality of images (or frames) related to the execution of the application program. According to an embodiment, the processor <NUM> (or the application execution module <NUM>) generates a plurality of images (or frames) for output to the display device <NUM> according to the execution of the application program. For example, the plurality of images (or frames) may be generated based on a time point at which the images are displayed on the display device <NUM> on the basis of the execution of an application program.

In operation <NUM>, the electronic device (e.g., the processor <NUM> or the application execution module <NUM>) identifies a difference between two adjacent (or continuous) images among a plurality of images (or frames) related to the execution of the application program. According to an embodiment, the processor <NUM> (or the control module <NUM>) may divide the (n-<NUM>)th image (or frame) and the nth image (or frame) generated in connection with the execution of the application program into multiple regions on the basis of an image segmentation method (e.g., 5x5 method). The processor <NUM> (or the control module <NUM>) may compare corresponding partitions with each other in the (n-<NUM>)th image (or frame) and the nth image (or frame) to identify the difference between the respective partitions. The processor <NUM> (or the control module <NUM>) may determine the average of the difference between the partition included in the (n-<NUM>)th image (or frame) and the nth image (or frame) as the difference between the (n-<NUM>)th image (or frame) and the nth image (or frame). As an example, the difference between the images (or frames) is the amount of change of at least one object included in the image (or frame), and may be determined that the difference between the images (or frames) is relatively large as the amount of change of the object is relatively large.

In operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) identifies insertion positions of interpolation images for each reference range based on the differences between two adjacent (or continuous) images (or frames within each range/set). According to an embodiment, the processor <NUM> (or the control module <NUM>) may configure the insertion position of the interpolation image on the basis of a difference between adjacent (or continuous) images (or frames) related to the execution of an application program. According to the invention, the insertion position of the interpolation image within the reference range may be configured based on two images (or frames) having the smallest difference between adjacent images (or frames) within the reference range. As an example, the size of the reference range may be configured based on at least one of target graphic performance (e.g., FPS) of the application program or status information of the electronic device <NUM>. According to an embodiment, the processor <NUM> (or the control module <NUM>) may configure insertion position of an interpolation image on the basis of two images (or frames) in which a difference satisfies a reference difference (e.g., a designated first condition) among a plurality of images (or frames) related to execution of an application program.

In operation <NUM>, the electronic device (e.g., the processor <NUM>, the control module <NUM>, or the interpolation generation module <NUM>) generates and inserts an interpolation image corresponding to the insertion position of the interpolation image on the basis of the difference between the images (or frames). In an embodiment, as illustrated in <FIG>, when the difference between a third image (or frame) <NUM> and a fourth image (or frame) <NUM> satisfies a reference difference, the processor <NUM> (or the control module <NUM>) may determine to insert the interpolation image <NUM> between the third image (or frame) <NUM> and the fourth image (or frame) <NUM>. In this case, the processor <NUM> (or the interpolation image generation module <NUM>) may generate the interpolation image <NUM> through interpolation based on the third image (or frame) <NUM> and the fourth image (or frame) <NUM>. The processor <NUM> (or the control module <NUM>) may render by inserting the interpolation image <NUM> between the third image (or frame) <NUM> and the fourth image (or frame) <NUM>.

<FIG> is a flowchart illustrating configuration of an insertion position of an interpolation image on the basis of a reference range in an electronic device according to certain embodiments of the invention. According to an embodiment, the operations of <FIG> may be an example of operations <NUM> and <NUM> of <FIG>. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. As an example, the electronic device may be the electronic device <NUM> of <FIG>.

Referring to <FIG>, according certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> of <FIG> or the control module <NUM> of <FIG>) may identify differences between sets of images (or frames) included in a reference range. According to an embodiment, the processor <NUM> (or the control module <NUM>) may insert an interpolation image for each reference range configured based on target graphic performance (e.g., FPS) of an application program. For example, when the target graphic performance of a game application supporting <NUM> FPS is set for <NUM> FPS, the processor <NUM> (or the control module <NUM>) may determine to insert one interpolation image at intervals of two images (or frames). Accordingly, the three images (or frames) including <NUM> existing images may be set as a single reference range. For example, the single reference range may include the third image (or frame) <NUM>, the fourth image (or frame) <NUM>, and the fifth image (or frame) <NUM> of <FIG>. In this case, the processor <NUM> (or the control module <NUM>) may identify the difference between the third image (or frame) <NUM> and the fourth image (or frame) <NUM>, and the difference between the fourth image (or frame) <NUM> and the fifth image (or frame) <NUM>. For example, the next reference range may include the fifth image (or frame) <NUM>, the sixth image (or frame) <NUM>, and the seventh image (or frame) <NUM> of <FIG>. In this case, the processor <NUM> (or the control module <NUM>) may identify the difference between the fifth image (or frame) <NUM> and the sixth image (or frame) <NUM>, and the difference between the sixth image (or frame) <NUM> and the seventh image (or frame) <NUM>.

According to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may identify the insertion position of the interpolation image within a reference range, based the images having the lowest difference within the reference range. According to an embodiment, in the case of <FIG>, when the difference between a third image (or frame) <NUM> and a fourth image (or frame) <NUM> is less than the difference between the fourth image (or frame) <NUM> and a fifth image (or frame) <NUM>, the processor <NUM> (or the control module <NUM>) may determine to insert an interpolation image <NUM> between the third image (or frame) <NUM> and the fourth image (or frame) <NUM>. In this case, the processor <NUM> (or the interpolation image generation module <NUM>) may generate an interpolation image <NUM> via the interpolation method on the basis of the third image (or frame) <NUM> and the fourth image (or frame) <NUM>. According to an embodiment, in the case of <FIG>, when the difference between the fifth image (or frame) <NUM> and a sixth image (or frame) <NUM> is greater than the difference between the sixth image (or frame) <NUM> and a seventh image (or frame) <NUM>, the processor <NUM> (or the control module <NUM>) may determine to insert an interpolation image <NUM> between the sixth image (or frame) <NUM> and a seventh image (or frame) <NUM>. In this case, the processor <NUM> (or the interpolation image generation module <NUM>) may generate an interpolation image <NUM> via the interpolation method on the basis of the sixth image (or frame) <NUM> and the seventh image (or frame) <NUM>.

<FIG> is a flowchart illustrating configuration of an insertion position of an interpolation image on the basis of differences in images in an electronic device according to certain embodiments of the invention. According to an embodiment, the operations of <FIG> may be an example of operation <NUM> and operation <NUM> of <FIG>. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. As an example, the electronic device may be the electronic device <NUM> of <FIG>. Hereinafter, at least some operations of <FIG> will be described with reference to <FIG> illustrates a configuration of an image segmentation method according to certain embodiments of the invention, and <FIG> illustrates a configuration of an image segmentation method according to certain embodiments of the invention.

Referring to <FIG>, according to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> of <FIG> or the control module <NUM> of <FIG>) may identify the difference between the (n-<NUM>)th image (or frame) and the nth image (or frame) among a plurality of images (or frames) related to an application program executed in the electronic device (e.g., the electronic device <NUM>). According to an embodiment, the processor <NUM> (or the control module <NUM>) may divide the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM> into a plurality of regions (e.g., <NUM>) on the basis of the first image segmentation method (e.g., <NUM> x <NUM> segmentation method), as shown in <FIG>. The processor <NUM> (or the control module <NUM>) may identify the difference between the divided regions corresponding to each other in the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM>. The processor <NUM> (or the control module <NUM>) may determine the average of the differences between the respective partitions as the difference between the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM>. As an example, n is identification information of an image (or frame) related to the execution of an application program and may be configured based on a time point when the corresponding image (or frame) is displayed on the display device <NUM>.

According to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may identify whether the difference of images is equal to or less than a reference difference (e.g., a designated first condition). As an example, the reference difference may be predefined as a reference for determining whether to insert an interpolation image or may include a designated first condition configured (or updated) based on graphic configuration information of the application program or graphic configuration information of the display device <NUM> or user input. As an example, the processor <NUM> (or the control module <NUM>) may determine that the designated first condition is satisfied, when the difference between the images exceeds the reference difference.

According to certain embodiments, in operation <NUM>, when the difference of the images exceeds the reference difference (e.g., "No" of operation <NUM>), the electronic device (e.g., the processor <NUM> or the control module <NUM>) may update the count of instances in which an interpolation is not generated. According to an embodiment, when the difference between the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM> of <FIG> exceeds the reference difference, the processor <NUM> (or the control module <NUM>) may determine not to insert an interpolation image between the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM>. In this case, the non-generation count related to the interpolation image may be increased. For example, the non-generation count related to the interpolation image may include the number of times it is determined that the interpolation image is not continuously inserted.

According to certain embodiments, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may identify whether the non-generation count exceeds the reference number (e.g., the specified second condition) in operation <NUM>. As an example, the reference number of times may include a designated second condition for determining when to change the image segmentation method. As an example, the processor <NUM> (or the control module <NUM>) may determine that the designated second condition is satisfied when the non-generation count related to the interpolation image exceeds the reference number.

According to certain embodiments, in operation <NUM>, when the non-generation count related to the interpolation image exceeds the reference number (e.g., "Yes" of operation <NUM>), the electronic device (e.g., the processor <NUM> or the control module <NUM>) may update the image segmentation method used to identify differences in images (or frames). According to an embodiment, the processor <NUM> (e.g., the control module <NUM>) may change the image segmentation method so that images can be compared relatively closely when the non-generation count related to the interpolated image exceeds the reference number. For example, the processor <NUM> (or the control module <NUM>) may change the image segmentation method to the second image segmentation method (e.g., 5x5 division method) of <FIG>, which can compare images relatively more closely than the first image segmentation method (e.g., <NUM> x <NUM> segmentation method) of <FIG>.

According to certain embodiments, when the non-generation count related to the interpolated image equal to or less than the reference number (e.g., "No" of operation <NUM>) or the image segmentation method is updated (e.g., operation <NUM>), in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may update n, which is identification information of an image (or frame). For example, the image identification n may be updated to identify the difference between the next images (or frames) in operation <NUM> (e.g., n++).

According to certain embodiments, when the difference of images is equal to or less than a reference difference (e.g., "Yes" of operation <NUM>), in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may configure or set the insertion position of an interpolation image on the basis of the (n-<NUM>)th image (or frame) and the nth image (or frame). According to an embodiment, when the difference between the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM> of <FIG> is equal to or less than the reference difference, the processor <NUM> (or the control module <NUM>) may determine to insert an interpolation image between the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM>. As an example, the insertion position of the interpolation image may be configured between the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM>.

According to certain embodiments, the electronic device (e.g., the processor <NUM> of <FIG> or the control module <NUM> of <FIG>) may configure the insertion position of an interpolation image on the basis of the machine learning (e.g., the Q-learning) on the basis of the difference of images. According to an embodiment, the processor <NUM> (or the control module <NUM>) may determine that the difference between the corresponding images does not satisfy the reference difference when the reward related to frame interpolation is less than or equal to the reference value on the basis of the result of Q-learning based on the difference between the images. Accordingly, the processor <NUM> (or the control module <NUM>) may determine that an interpolation image is not inserted between the corresponding images. According to an embodiment, when a reward related to frame interpolation exceeds a reference value on the basis of a result of Q-learning based on a difference between images, the processor <NUM> (or the control module <NUM>) may determine that the difference between the corresponding images satisfies the reference difference. Accordingly, the processor <NUM> (or the control module <NUM>) may determine to insert an interpolation image between the corresponding images.

According to certain embodiments, the electronic device (e.g., the processor <NUM> of <FIG> or the control module <NUM> of <FIG>) may configure the insertion position of the interpolation image on the basis of the number of divided regions that do not satisfy the reference difference. According to an embodiment, the processor <NUM> (or the control module <NUM>) may identify the difference the corresponding divided regions in the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM>, divided based on the first image segmentation method (e.g., 3x3 segmentation method), as shown in <FIG>. The processor <NUM> (or the control module <NUM>) may identify the number of the divided regions having a difference exceeding a reference difference among the divided regions. For example, when the number of divided regions exceeding the reference difference exceeds the reference number, the processor <NUM> (or the control module <NUM>) may determine not to insert an interpolation image between the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM>. For example, when the number of divided regions exceeding the reference difference is less than or equal to the reference number, the processor <NUM> (or the control module <NUM>) may determine to insert an interpolation image between the (n-<NUM>)th image (or frame) <NUM> and the nth image (or frame) <NUM>.

<FIG> is a flowchart illustrating identification of differences in images in an electronic device according to certain embodiments of the invention. According to an embodiment, the operations of <FIG> may be examples of the operation <NUM> of <FIG>. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. As an example, the electronic device may be the electronic device <NUM> of <FIG>.

Referring to <FIG>, according to certain embodiments, when identifying multiple images (or frames) related to the execution of an application program (e.g., operation <NUM> of <FIG>), in operation <NUM>, the electronic device (e.g., the processor <NUM> of <FIG> or the control module <NUM> of <FIG>) may identify a target graphic performance (e.g., FPS) of the application program. For example, the target graphic performance may be configured based on a graphic performance supported by the display device <NUM> or a result of user input or machine learning (e.g., Q-learning).

According to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may identify the image segmentation method corresponding to the target graphic performance (e.g., FPS) of the application program. Selection of the image segmentation method may be facilitated by a table association methods, granularities, etc. with a present FPS performance and target FPS performances, etc. For example, a processor may identify that an increase from <NUM> FPS to <NUM> FPS requires an image segmentation of <NUM> x <NUM>. According to an embodiment, executing of frame interpolation may result in a severe afterimage effect when the graphic performance of the application program is below a certain threshold. Accordingly, the processor <NUM> (or the control module <NUM>) may select (e.g., by the table) an image segmentation method (e.g., <NUM> x <NUM> segmentation) having a sufficient granularity to compare images in a detail level sufficient for target graphic performance (e.g., <NUM> FPS or less) of an application program to avoid negative image effects such as a distracting afterimage effect.

According to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may identify the difference of images (or frames) on the basis of the image segmentation method corresponding to the target graphic performance (e.g., FPS) of the application program. According to an embodiment, the processor <NUM> (or the control module <NUM>) may identify the difference between the divided regions corresponding to each other of the divided images (or frames) on the basis of the image segmentation method corresponding to the target graphic performance (e.g., FPS) of the application program. The processor <NUM> (or the control module <NUM>) may determine the average of the differences of the divided regions as the difference of the images. For example, the average of the differences of the divided regions may include an average value of the differences of all the divided regions included in the image (or frame). For another example, the average of the differences of the divided regions may include an average value of at least one divided region having a relatively large difference among the divided regions included in the image (or frame).

<FIG> is a flowchart illustrating control of a clock of an internal circuit in an electronic device according to certain embodiments not covered by the invention. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. As an example, the electronic device may be the electronic device <NUM> of <FIG>.

Referring to <FIG>, according to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> of <FIG>, the control module <NUM> or the application execution module <NUM> of <FIG>) may execute at least one application program among a plurality of application programs installed in the electronic device (e.g., the electronic device <NUM>). According to an embodiment, the processor <NUM> (or the application execution module <NUM>) may execute at least one application program corresponding to a selection input detected through the input device <NUM> among the plurality of application programs installed in the electronic device <NUM>.

According to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may selectively generate and insert an interpolation image on the basis of the difference between adjacent images (or frames) among the plurality of images (or frames) related to the application program. According to an embodiment, when the difference between images among the plurality of images related to the application program satisfies a reference difference, the processor <NUM> (or the control module <NUM>) may configure an insertion position of an interpolation image on the basis of the images, as operation <NUM> to operation <NUM> of <FIG>. The processor <NUM> (or the control module <NUM>) may generate and insert an interpolation image corresponding to the insertion position of the interpolation image.

According to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may identify whether a frame drop is detected (or predicted to occur). According to an embodiment, the processor <NUM> (or the control module <NUM>) may selectively generate and insert an interpolation image on the basis of a difference between images (or frames), thereby identifying whether it is predicted that the target graphic performance of the application program (e.g., FPS) cannot be satisfied. The processor <NUM> (or the control module <NUM>) may determine that frame reduction will occur when it is predicted that the target graphic performance of the application program (e.g., FPS) cannot be satisfied.

According to certain embodiments, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may maintain the execution of the application program when the occurrence of frame reduction is not detected and is not predicted (e.g., "No" in operation <NUM>). According to an embodiment, the processor <NUM> (or the control module <NUM>) may selectively generate and insert an interpolation image on the basis of a difference between images (or frames) related to the execution of the application program when the application program is running.

According to certain embodiments, when occurrence of frame reduction is detected or predicted (e.g., "Yes" of operation <NUM>), in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may control the clock of an internal circuit on the basis of the frame reduction. According to an embodiment, when the frame reduction occurs, the processor <NUM> (or the control module <NUM>) may increase the clock of a CPU or GPU related to the execution of the application program.

According to certain embodiments, when occurrence of frame reduction is detected or predicted, the electronic device (e.g., the processor <NUM> of <FIG> or the control module <NUM> of <FIG>) may determine a method for responding to frame reduction on the basis of a result of machine learning (e.g., Q-learning) based on differences in images. According to an embodiment, the frame reduction may be caused by selectively generating and inserting the interpolation images based on differences in images. Accordingly, the processor <NUM> (or the control module <NUM>) may perform machine learning by configuring the weight of the CPU clock control scheme and/or the GPU clock control scheme relatively higher than the frame interpolation scheme among the operations for responding to the frame reduction. In one example, the processor <NUM> (or the control module <NUM>) may increase the CPU clock in response to the frame reduction when the reward of the CPU's clock control scheme is relatively high on the basis of a result of machine learning based on differences in images. For example, the processor <NUM> (or the control module <NUM>) may generate and insert an interpolation image when the compensation of the frame interpolation method is relatively high on the basis of the machine learning result based on the difference of the images. In this case, the insertion position of the interpolated image may be configured based on the target graphics performance of the application program regardless of differences in images, or may be configure on the basis of the differences in images.

<FIG> is a flowchart illustrating insertion of an interpolation image on the basis of frame reduction in an electronic device according to certain embodiments not covered by the invention. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. As an example, the electronic device may be the electronic device <NUM> of <FIG>. Hereinafter, at least some operations of <FIG> will be described with reference to <FIG> is a performance change graph according to the insertion of an interpolation image according to certain embodiments not covered by the invention.

Referring to <FIG>, according to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> of <FIG>, the control module <NUM> or the application execution module <NUM> of <FIG>) may execute at least one application program among a plurality of application programs installed in the electronic device (e.g., the electronic device <NUM>). According to an embodiment, the display device <NUM> may display a plurality of icons related to a plurality of application programs installed in the electronic device <NUM>. The processor <NUM> (or the control module <NUM>) may execute an application program related to an icon in which a selection input (or user input) among the plurality of icons displayed on the display device <NUM> is detected. The processor <NUM> (or the control module <NUM>) may render a plurality of images (or frames) related to execution of the application program generated by the application execution module <NUM>. The display device <NUM> may output a plurality of images (or frames) rendered by the processor <NUM> (or the control module <NUM>).

According to certain embodiments, in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may identify whether the frame reduction is detected (or predicted). According to an embodiment, when the image (or frame) related to the execution of the application program is not identified at a predetermined time related to the execution of the application program, the processor <NUM> (or the control module <NUM>) may determine that frame reduction has occurred. According to an embodiment, the processor <NUM> (or the control module <NUM>) may predict the occurrence of frame reduction on the basis of a result of machine learning based on differences in images.

According to certain embodiments, when the occurrence of frame reduction is not detected or not predicted (e.g., "No" in operation <NUM>), the electronic device (e.g., the processor <NUM> or the control module <NUM>) may maintain the execution of at least one application program. According to an embodiment, the processor <NUM> (or the control module <NUM>) may control the display device <NUM> to display a plurality of images (or frames) related to the execution of the application program generated by the application execution module <NUM>.

According to certain embodiments, when the occurrence of frame reduction is detected or predicted (e.g., "Yes" in operation <NUM>), in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may generate and insert at least one interpolation image associated with the application program. According to an embodiment, when the occurrence of frame reduction is detected or predicted (<NUM>) as in <FIG>, the processor <NUM> (or the control module <NUM>) may selectively generate and insert an interpolation image on the basis of a difference between adjacent images (or frames) among a plurality of images (or frames) related to the execution of an application program (<NUM>). For example, the processor <NUM> (or the control module <NUM>) may configure a position between images (or frames) having the lowest difference for each reference range as an insertion position of an interpolation image, as in operation <NUM> to operation <NUM> of <FIG>. For another example, the processor <NUM> (or the control module <NUM>) may configure the position between images (or frames) in which the difference in images satisfies a reference difference as the insertion position of the interpolation image, as in operation <NUM> to operation <NUM> of <FIG>.

According to certain embodiments, when the occurrence of frame reduction is detected or predicted, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may render by inserting at least one interpolation image in at least a part of a plurality of images (or frames) related to the execution of an application program. Accordingly, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may reduce frame reduction.

<FIG> is a flowchart illustrating selective insertion of an interpolation image based on frame reduction in an electronic device according to certain embodiments not covered by the invention. According to an embodiment, the operations of <FIG> may be examples of operation <NUM> of <FIG>. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. As an example, the electronic device may be the electronic device <NUM> of <FIG>. Hereinafter, at least some operations of <FIG> will be described with reference to <FIG> is a graph of a performance change according to selectively inserting an interpolation image according to certain embodiments not covered by the invention.

Referring to <FIG>, according to certain embodiments, when the occurrence of frame reduction is detected or predicted (e.g., "Yes" of operation <NUM> of <FIG>), in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM> of <FIG>) may determine whether to apply interpolation image insertion. According to an embodiment, the processor <NUM> (or the control module <NUM>) may identify whether a frame interpolation method is recommended (or selected) as an optimal response method in response to a frame reduction on the basis of a Q-learning result performed through the machine learning module <NUM>. For example, Q-learning uses the Q function as shown in Math <FIG> (e.g., Equation <NUM>) below to determine the CPU clock, GPU clock, CPU load, GPU load, graphic performance (e.g., FPS), stability related to application program execution, or learning based on at least one of differences in images.

For example, in [Math <FIG>], Qt+<NUM>(st,at) denotes learning information at the present time, Qt(st,at) denotes learning information at the previous time, and s denotes the state of the electronic device <NUM> related to the execution of the application program, a denotes an action performed in the state of the electronic device <NUM> related to execution of the application program, and R denotes a reward of each method. For example, the processor <NUM> (or the control module <NUM>) may select a response method to be performed in response to the frame reduction on the basis of the compensation of each method corresponding to the state of the frame reduction on the basis of the Q-learning result performed through the machine learning module <NUM>. For example, when the compensation of the CPU clock control method is predicted with a probability of <NUM>, the compensation of the GPU clock control method is predicted with a probability of <NUM>, and the compensation of the frame interpolation method is predicted with a probability of <NUM> on the basis of the results of machine learning, the processor <NUM> (or the control module <NUM>) may determine that the frame interpolation method of inserting an interpolation image can produce the best results for frame reduction. According to an embodiment, the processor <NUM> (or the control module <NUM>) may derive the best result for frame reduction through Q-learning additionally applying pattern information of a previously used corresponding method. For example, when detecting a method for responding to frame reduction, the processor <NUM> (or the control module <NUM>) may identify pattern information corresponding to the current state. When the CPU's clock control method is used to cope with frame reduction based on the pattern information corresponding to the current state, the processor <NUM> (or the control module <NUM>) may identify compensation in case of using each control method and compensation in case of using the clock control method of the CPU together. For example, when the compensation of the CPU clock control method is predicted with a probability of <NUM>, the compensation of the frame interpolation method is predicted with a probability of <NUM>, and the compensation of a case that the clock control method and the frame interpolation method are applied together is predicted with a probability of <NUM> on the basis of the results of machine learning, the processor <NUM> (or the control module <NUM>) may determine that the method of applying the frame interpolation method and the CPU clock control method together can produce the best results for frame reduction.

According to certain embodiments, when it is determined that the interpolation image insertion is applied (e.g., "Yes" in operation <NUM>), in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may selectively generate and insert an interpolation image on the basis of differences in images (or frames) related to the execution of the application program. For example, when the occurrence of frame reduction is detected or predicted, as in <FIG>, the processor <NUM> (or the control module <NUM>) may select a response method for responding to the state of frame reduction on the basis of the result of machine learning (e.g., Q-learning) (<NUM>). When the frame interpolation method is selected based on the machine learning result, as in <FIG>, the processor <NUM> (or the control module <NUM>) may prevent frame reduction by selectively inserting (<NUM>) an interpolation image between images (or frames) related to the execution of an application program. For example, the insertion position of the interpolation image may be configured based on images (or frames) having the lowest difference for each reference range, as operation <NUM> to operation <NUM> of <FIG>. For another example, the insertion position of the interpolation image may be configured based on images (or frames) in which the difference between the images (or frames) satisfies a reference difference, as operation <NUM> to operation <NUM> of <FIG>.

According to certain embodiments, when the interpolation image insertion is not applied (e.g., "No" in operation <NUM>), in operation <NUM>, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may adjust the clock of the internal circuit (e.g., CPU or GPU). According to an embodiment, when the occurrence of frame reduction is detected or predicted (<NUM>), as in <FIG>, the processor <NUM> (or the control module <NUM>) may select a clock control method of a CPU as a method for responding to a state of frame reduction on the basis of a result of machine learning (e.g., Q-learning). In this case, the processor <NUM> (or the control module <NUM>) can prevent the frame reduction by increasing the CPU clock (<NUM>), as in <FIG>.

According to certain embodiments, when the occurrence of frame reduction is detected or predicted, the electronic device (e.g., the processor <NUM> or the control module <NUM>) may select a plurality of response methods on the basis of the result of machine learning. According to an embodiment, the processor <NUM> (or the control module <NUM>) may select at least two response methods among a CPU clock control method, a GPU clock control method, or a frame interpolation method on the basis of the result of machine learning (e.g., Q learning).

According to certain embodiments not covered by the invention, an operation method of an electronic device (e.g., the electronic device <NUM> of <FIG>) may include executing an application program, identifying an insertion position of an interpolation image on the basis of a plurality of images related to the execution of the application program, if a frame drop is detected, generating an interpolation image corresponding to the insertion position of the interpolation image, and inserting the interpolation image into at least a portion of the plurality of images on the basis of the insertion position of the interpolation image.

According to certain embodiments, the identifying the insertion position of the interpolation image may include identifying the insertion position of the interpolation image within a reference range on the basis of a difference of the images for each reference range.

According to the invention, the identifying the insertion position of the interpolation image includes configuring the insertion position of the interpolation image on the basis of images having a smallest difference in images within the reference range.

According to certain embodiments, the identifying the insertion position of the interpolation image may include identifying a difference between images having adjacent playback time points among the plurality of images, and identifying the insertion position of the interpolation image on the basis of the difference of the images.

According to certain embodiments, the identifying the difference between the images may include: dividing a first image and a second image having adjacent playback time point among the plurality of image related to the execution of the application program into a first number of regions on the basis of a first segmentation method; identifying a difference between regions corresponding to each other in the first image and the second image; and identifying the difference between the first image and the second image on the basis of the difference between the divided regions.

According to certain embodiments, the identifying the insertion position of the interpolation image may include configuring the insertion position of the interpolation image on the basis of the first image and the second image, if the difference between the first image and the second image satisfies a designated first condition.

According to certain embodiments, the generating the interpolation image may include generating the interpolation image between the first image and the second image via a frame interpolation based on the first image and the second image, if the insertion position of the interpolation is configured based on the first image and the second image.

According to certain embodiments, the operation method may further include identifying a difference between the second image and a third image having adjacent playback time points among the plurality of frames, if the difference between the first image and the second image does not satisfy the designated first condition.

According to certain embodiments, the identifying the difference between the second image and the third image may include: identifying the number of consecutive failures of the designated first condition, if the difference between the first image and the second image does not satisfy the designated first condition; dividing the second image and the third image into a second number of regions on the basis of a second segmentation method different from the first segmentation method, if the number of the consecutive failures of the designated first condition satisfies a designated second condition; and identifying a difference between the second image and a third image on the basis of a difference in the second number of regions.

According to certain embodiments, the identifying the insertion position of the interpolation image may include identifying the insertion position of the interpolation image on the basis of a Q-learning result based on the difference of the images having adjacent playback time points among the plurality of images related to the execution of the application program.

Claim 1:
An electronic device (<NUM>), comprising:
a display device (<NUM>); and
a processor (<NUM>) operatively connected to the display device,
wherein the processor (<NUM>) is configured to:
execute (<NUM>) an application program;
generate (<NUM>) a plurality of images for output to the display device (<NUM>) according to the execution of the application program;
identify (<NUM>) a difference between images having adjacent playback time points from among the plurality of images;
characterized in that the processor (<NUM>) is configured to:
identify (<NUM>) an insertion position of an interpolation image as being between the images based on the identified difference between the images,
wherein a plurality of reference ranges is set for the plurality of images, each reference range indicating a count of frames for which a single interpolation image is to be inserted, and wherein the insertion position of the interpolation image identified within each reference range is based on the smallest identified difference of respective images included in each reference range;
generate (<NUM>) the interpolation image corresponding to the identified insertion position of the interpolation image; and
insert (<NUM>) the interpolation image into the identified insertion position between the images.