Patent Description:
Machine learning, which is a field of artificial intelligence, refers to a technology of studying and constructing a system that collects and analyzes large-scale big data to predict the future and improves its own performance and an algorithm therefor.

Recently, due to the development of hardware technology, big data may be collected and stored, and as a computing ability and technology for analyzing big data has become sophisticated and faster, research into machine learning which is an algorithm capable of recognizing objects and understanding information like humans has been actively conducted. In particular, in the field of machine learning technology, self-learning type deep learning using a neural network has been actively researched.

A neural network, which is an algorithm that determines a final output by comparing an active function with a specific boundary value for a sum acquired by multiplying a plurality of inputs by a weight based on an intention to actively mimic functions of human brains, includes multiple layers. Typical examples thereof include a convolutional neural network (CNN), which is commonly used for image recognition, and a recurrent neural network (RNN), which is commonly used for speech recognition.

Research into improvement of image quality to increase resolution using such a deep learning network has been actively conducted. However, the existing image quality improvement method has a problem of artifacts that several lines occur in a mosaic format or at an edge when resolution increases.

Accordingly, a need for a technology of increasing resolution of an image, while reducing artifacts, has emerged. <NPL>"; XP055539761 describes a CNN structure for improved super-resolution. <NPL>"; XP055545653 describes linear methods for interpolation. <CIT> describes an image processing method comprising: filtering a first real image to obtain a first feature map; upscaling the obtained first feature map, the improved first feature map forming a second feature map; and constructing a second real image. <NPL>"; XP055768593 describes a CNN used for real-time super-resolution of video.

The disclosure provides an electronic device, an image processing method, and a computer-readable recording medium for generating a high-quality image using an upscaling filter in a form of a function which is bilaterally symmetrical and nonlinearly decreases.

According to an embodiment of the disclosure, an electronic device includes: a memory configured to store a learned artificial intelligence model; and a processor configured to input an input image to the artificial intelligence model and to output an enlarged image with increased resolution, wherein the learned artificial intelligence model includes an upscaling module configured to acquire a pixel value of an interpolated pixel near an original pixel corresponding to a pixel of the input image in the enlarged image based on a function in a form which is bilaterally symmetrical and nonlinearly decreases with respect to the original pixel, wherein the upscaling module calculates the pixel value of the interpolated pixel near a plurality of original pixels based on a ratio at which the plurality of original pixel values are reflected in the pixel value of the interpolated pixel, and the ratio is identified according to distances between the plurality of original pixels and the interpolated pixel, on a plurality of Gaussian functions based on the plurality of original pixels, wherein, a variance of the Gaussian function is calculated based on a linear function for bilinear interpolation of an upscaling factor, wherein the upscaling factor corresponds to a magnification of the enlarged image compared to the input image.

In this case, a variance of the function is acquired based on a linear function for bilinear interpolation of an upscaling factor.

In this case, a variance σd of the function may be acquired by <MAT>, where s may be the upscaling factor, d may be an x coordinate of a contact point, and t(s) may be a value acquired by adding <NUM> at a distance between x intercepts of the function.

In this case, the function (f(x;s)) may be <MAT> and σd(s)-s*<NUM> ≤ σ(s) ≤ σd(s)+s*<NUM>.

Meanwhile, the upscaling module may further include a convolution filter configured to acquire a feature of the input image, wherein the processor may acquire the enlarged image using the feature of the input image acquired using the convolution filter.

According to another embodiment of the disclosure, an image processing method includes: receiving an image; and inputting an input image to a learned artificial intelligence model and outputting an enlarged image with increased resolution, wherein the learned artificial intelligence model includes an upscaling module configured to acquire a pixel value of an interpolated pixel near an original pixel corresponding to a pixel of the input image in the enlarged image based on a function in a form which is bilaterally symmetrical and nonlinearly decreases with respect to the original pixel, wherein the upscaling module calculates the pixel value of the interpolated pixel near a plurality of original pixels based on a ratio at which the plurality of original pixel values are reflected in the pixel value of the interpolated pixel, and the ratio is identified according to distances between the plurality of original pixels and the interpolated pixel, on a plurality of Gaussian functions based on the plurality of original pixels, wherein, a variance of the Gaussian function is calculated based on a linear function for bilinear interpolation of an upscaling factor, wherein the upscaling factor corresponds to a magnification of the enlarged image compared to the input image.

In this case, the plurality of original pixels may correspond to one pixel of the input image in the enlarged image, at least one of a plurality of pixels adjacent to the one pixel based on the one pixel, and a pixel corresponding to at least one of a plurality of pixels which are spaced apart to the one pixel but are adjacent to the plurality of pixels.

According to another embodiment of the disclosure, a computer-readable recording medium including a program for executing an image processing method, wherein the image processing method includes: receiving an image; and inputting an input image to a learned artificial intelligence model and outputting an enlarged image with increased resolution, wherein the learned artificial intelligence model includes an upscaling module configured to acquire a pixel value of an interpolated pixel near an original pixel corresponding to a pixel of the input image in the enlarged image based on a function in a form which is bilaterally symmetrical and nonlinearly decreases with respect to the original pixel, wherein the upscaling module calculates the pixel value of the interpolated pixel near a plurality of original pixels based on a ratio at which the plurality of original pixel values are reflected in the pixel value of the interpolated pixel, and the ratio is identified according to distances between the plurality of original pixels and the interpolated pixel, on a plurality of Gaussian functions based on the plurality of original pixels, wherein, a variance of the Gaussian function is calculated based on a linear function for bilinear interpolation of an upscaling factor, wherein the upscaling factor corresponds to a magnification of the enlarged image compared to the input image.

Terms used in the description of the various example embodiments of the disclosure are briefly described and then the various example embodiments of the disclosure will be described in greater detail.

The terms used in the example embodiments of the disclosure are general terms which are widely used now and selected considering the functions of the disclosure. However, the terms may vary depending on the intention of a person skilled in the art, a precedent, or the advent of new technology. In addition, in a specified case, the term may be arbitrarily selected. In this case, the meaning of the term will be explained in the corresponding description. Therefore, terms used in the disclosure may be defined based on a meaning of the terms and contents described in the disclosure, not simply based on names of the terms.

In the disclosure, terms including an ordinal number such as 'first', 'second', etc. may be used to describe various components, but the components are not to be construed as being limited to the terms. The terms are only used to differentiate one component from other components.

As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be understood that the terms "comprising", "including", "having" and variants thereof specify the presence of stated features, numbers, steps, operations, elements, components, and/or groups thereof, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, elements, components, and/or groups thereof.

In the description, the word "module" or "unit" refers to a software component, a hardware component, or a combination thereof, which is capable of carrying out at least one function or operation. A plurality of modules or units may be integrated into at least one module and realized using at least one processor except for those modules or units that need to be realized in specific hardware.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings such that they can be easily practiced by those skilled in the art to which the disclosure pertains. In the accompanying drawings, a portion irrelevant to description of the disclosure will be omitted for clarity.

Hereinafter, the disclosure will be described in more detail with reference to the drawings.

<FIG> is a view schematically illustrating an image processing process of an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, when an input image <NUM> is input to the electronic device <NUM>, the electronic device <NUM> may sequentially perform a series of image processing processes and output an enlarged image <NUM>. In this case, the input image <NUM> being input may be a low-resolution image acquired by processing an original image.

Here, the electronic device <NUM> may be a device capable of performing artificial intelligence learning. For example, the electronic device <NUM> may be a desktop PC, a notebook computer, a smartphone, a tablet PC, a server, or the like. Alternatively, the electronic device <NUM> may refer to a system in which a cloud computing environment is built. However, the disclosure is not limited thereto, and the electronic device <NUM> may be any device capable of performing artificial intelligence learning.

Specifically, the electronic device <NUM> may include a plurality of layers <NUM> extracting features of the input image <NUM> and an upscaling module <NUM> upscaling the input image <NUM> using the extracted features.

Here, the plurality of layers <NUM> may extract features of the input image <NUM> using a plurality of filters trained by a neural network. That is, the plurality of layers <NUM> may perform pre-processing before upscaling.

Here, the filters are masks having weights and is defined as a matrix of weights. The filters are also referred to as windows or kernels. The weights configuring the matrix in the filters may include <NUM> (zero value) or a zero element that may be approximated to <NUM> and a non-zero element having a certain value between <NUM> and <NUM> and may have various patterns according to functions thereof.

For example, when the neural network is realized as a convolution neural network (CNN) for recognizing an image, the electronic device <NUM> may put the filters having weights on the input image <NUM> and determine the sum (convolution operation) of values acquired by multiplying the image by each of the weights of the filters, as a pixel value of the output image, to extract a feature map. The input image may be extracted as a plurality of input images through multiple filters to extract robust features, and a plurality of feature maps may be extracted according to the number of filters. Such a convolutional image may be repeated by multiple layers. Here, the filters to be trained vary depending on a learning target of the CNN and patterns of selected filters vary. In other words, the trained filters and the selected filters vary depending on what a learning target of the CNN is cat, dog, pig, cow, and the like.

In this manner, the electronic device <NUM> may determine what type of features the input original data has by combining the plurality of layers <NUM> from which different features may be extracted and applying a combination of the plurality of layers to the CNN.

The electronic device <NUM> may output an enlarged image by inputting a feature map of the input image <NUM> extracted from the plurality of layers <NUM> to the upscaling module <NUM>.

Meanwhile, the upscaling module <NUM> may optionally further include convolutional layers <NUM>-<NUM> and <NUM>-<NUM> ahead and behind. In this case, the upscaling module <NUM> may be referred to by including the convolutional layers <NUM>-<NUM> and <NUM>-<NUM>. In this case, the convolutional layers <NUM>-<NUM> and <NUM>-<NUM> may be a convolutional layer or a combination of a convolutional layer and a ReLu layer.

Further, the electronic device <NUM> may learn parameters of the plurality of layers <NUM> or the convolutional layers <NUM>-<NUM> and <NUM>-<NUM> by comparing the output enlarged image <NUM> and the original image.

Meanwhile, the upscaling module <NUM> increases the resolution of an image using a filter in the form of a function that is bilaterally symmetrical and decreases nonlinearly. The upscaling module is in the form of a Gaussian function. Details of the upscaling module <NUM> will be described with reference to the accompanying drawings.

<FIG> is a block diagram illustrating a simplified configuration of an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, the electronic device <NUM> includes a memory <NUM> and a processor <NUM>.

The memory <NUM> may be realized as a memory of various formats such as a hard disk drive (HDD), a solid state drive (SSD), a DRAM memory, an SRAM memory, an FRAM memory, or a flash memory.

Specifically, an artificial intelligence model is stored in the memory <NUM>. Here, the artificial intelligence model may be learned. In addition, the artificial intelligence model includes an upscaling module for increasing the resolution of the input image.

Specifically, the upscaling module is a module for acquiring a pixel value of an original pixel corresponding to a pixel of the input image in the enlarged image and a pixel value of an interpolated pixel near the original pixel. Here, the upscaling module acquires the pixel value of the interpolated pixel near the original pixel according to a function in a form which is bilaterally symmetrical and decreases nonlinearly with respect to the original pixel under the control of the processor <NUM>. The upscaling module is in the form of a Gaussian function based on the original pixel.

The processor <NUM> generally controls the operation of the electronic device <NUM>.

According to an embodiment, the processor <NUM> may be realized as a digital signal processor (DSP), a microprocessor, or a time controller (TCON). However, without being limited thereto, the processor <NUM> may include at least one of a central processing unit (CPU), a microcontroller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), or an ARM processor or may be defined by the corresponding term. In addition, the processor <NUM> may be realized as a system on chip (SoC) or large scale integration (LSI) with a built-in processing algorithm or may be realized in the form of a field programmable gate array (FPGA).

The processor <NUM> outputs the enlarged image of the input image using the upscaling module included in the artificial intelligence model stored in the memory <NUM>. Specifically, the processor <NUM> acquires the pixel value of the interpolated pixel near the original pixel according to the function which is bilaterally symmetrical and nonlinearly decreases with respect to the original pixel corresponding to the pixel of the input image using the upscaling module stored in the memory <NUM>, and output the enlarged image based on the acquired pixel values. The upscaling module is in the form of a Gaussian function based on the original pixel. Details of the original pixel and the interpolated pixel will be described with reference to <FIG> below.

Specifically, the upscaling module used by the processor <NUM> acquires the pixel value of interpolated pixel near a plurality of original pixels based on each ratio of the plurality of original pixel values. That is, the processor <NUM> acquires a pixel value of one interpolated pixel using the upscaling module, and to this end, the processor <NUM> uses pixel values of the plurality of original pixels around the interpolated pixel. Meanwhile, the processor <NUM> acquires the pixel value of the interpolated pixel using the plurality of pixel values respectively corresponding to the plurality of original pixels in the input image.

Specifically, the processor <NUM> identifies a reflection ratio of the pixel value of the original pixel according to distances between the interpolated pixel and the plurality of original pixels around the interpolated pixel. In this case, the plurality of original pixels may be pixels corresponding to a first pixel of the input image, a second pixel which is at least one of a plurality of pixels adjacent to the first pixel based on the first pixel, and a third pixel which is at least one of a plurality of pixels which are spaced apart from the first pixel but adjacent to the second pixel, in the enlarged image.

Specifically, the processor <NUM> identifies a ratio reflecting the pixel value of the original pixel on the Gaussian function with respect to the original pixel according to the distance of the interpolated pixel to the original pixel. Here, variance of the Gaussian function may be acquired based on an upscaling factor. Specifically, the variance of the Gaussian function may be acquired based on a slope of a linear function for bilinear interpolation of the upscaling factor. A process of acquiring the variance of the Gaussian function will be described in detail below with reference to <FIG> and <FIG>.

Meanwhile, a reflection ratio of a pixel value of an original pixel other than the original pixel adjacent to the interpolated pixel may be identified on the Gaussian function with respect to the other original pixel according to a distance between the interpolated pixel and the other original pixel.

The pixel value reflection ratio as described above may also be applied between the original pixels. Specifically, a pixel value of a first original pixel in the enlarged image will affect a pixel value of an adjacent second original pixel, and thus, when the pixel value of the second original pixel is acquired, the processor <NUM> may identify the ratio at which the pixel value of the first original pixel is reflected on the Gaussian function based on the first original pixel according to a distance between the first original pixel and the second original pixel and acquire the pixel value of the second original pixel using the identified ratio.

The method of acquiring the pixel value of the enlarged image as described above will be described in detail below with reference to <FIG> and <FIG>.

<FIG> is a block diagram illustrating a specific configuration of the electronic device disclosed in <FIG>.

Referring to <FIG>, the electronic device <NUM> may include a memory <NUM>, a processor <NUM>, a communication unit <NUM>, a display <NUM>, a button <NUM>, a video processor <NUM>, an audio processor <NUM>, a microphone <NUM>, an imaging unit <NUM>, and an audio output unit <NUM>.

Here, the memory <NUM> and the processor <NUM> are the same as those shown in <FIG>, and redundant descriptions are omitted.

The memory <NUM> may store various programs and data necessary for the operation of the electronic device <NUM>. Specifically, a parameter for processing the input image may be stored in the memory <NUM>. Here, the stored parameter may be machine-learned based on a previously input low-quality image and a high-quality image corresponding thereto.

In addition, the memory <NUM> may store a reduction ratio for use in reducing the input image. Here, the stored reduction ratio, which is calculated by a manufacturer through machine learning, may be previously stored at the factory or may be updated through periodic firmware upgrading. Meanwhile, the memory <NUM> may store an algorithm for deriving the reduction ratio.

In addition, the memory <NUM> may store a plurality of low-quality images to be upscaled to high-quality images. The processor <NUM> may generate a high-quality image for a low-quality image selected by a user from among the plurality of stored low-quality images.

In addition, the memory <NUM> may store information on a reduction ratio corresponding to the degree of deterioration of an image. Here, the reduction ratio based on the degree of deterioration may be stored in the form of a lookup table.

In addition, the memory <NUM> may store programs and data for upscaling a low-quality image. Accordingly, the processor <NUM> may generate a high-quality image from the input low-quality image using the program and data stored in the memory <NUM>, and in some cases, the processor <NUM> may determine a reduction ratio used in a parameter updating process or an upscaling process.

The communication unit <NUM> is a component for performing communication with various types of external devices according to various types of communication methods. Specifically, the communication unit <NUM> may receive a low-quality image from an external device and transmit a high-quality image generated by the processor <NUM> to an external device such as a separate display device. In addition, the communication unit <NUM> may also receive an original image which is a high-quality image corresponding to the low-quality image.

Specifically, the communication unit <NUM> may receive an image from an external device through a wired method such as an antenna, a cable, or a port or may receive an image through a wireless method such as Wi-Fi and Bluetooth. Meanwhile, in actual realization, the electronic device <NUM> may receive an image selected by the user from among a plurality of images stored in a storage unit (not shown) provided in the electronic device <NUM> and process the image.

When the electronic device <NUM> is capable of performing wireless communication, the communication unit <NUM> may include a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, and an NFC chip. Specifically, the Wi-Fi chip and the Bluetooth chip perform communication in a Wi-Fi method and a Bluetooth method, respectively. In case of using the Wi-Fi chip or the Bluetooth chip, various connection information such as an SSID and a session key may be first transmitted and received, and various types of information may be transmitted and received, connection may be established using the various connection information, and various information may then be transmitted and received. The wireless communication chip refers to a chip that performs communication according to various communication standards such as IEEE, Zigbee, 3rd generation (<NUM>), 3rd generation partnership project (3GPP), and long term evolution (LTE). The NFC chip refers to a chip that operates in a near field communication (NFC) method using a <NUM> band among various RF-ID frequency bands such as <NUM>, <NUM>, <NUM>, <NUM> to <NUM>, and <NUM>.

The display <NUM> may display an image acquired by processing the input image using an adjusted parameter. Here, the processed image displayed by the display <NUM> may be an image generated by improving image quality of the input image with the adjusted parameter. The display <NUM> may be realized as various types of displays such as a liquid crystal display (LCD), an organic light emitting diodes (OLED) display, and a plasma display panel (PDP). The display <NUM> may include a driving circuit, a backlight unit, and the like, which may be realized in the form of an a-si TFT, a low temperature polysilicon (LTPS) TFT, an organic TFT (OTFT), or the like. Also, the display <NUM> may be realized as a flexible display.

In addition, the display <NUM> may include a touch sensor for detecting a user's touch gesture. The touch sensor may be realized as various types of sensors such as capacitive, resistive, and piezoelectric sensors. The capacitive type is a method of calculating touch coordinates by sensing micro-electricity excited to the user's body when a part of the user's body touches a surface of the display <NUM> using a dielectric coated on the surface of the display. The resistive type, which includes two electrode plates embedded in the display <NUM>, is a method of calculating touch coordinates by sensing a current flowing as upper and lower plates at a touched point are in contact with each other when the user touches a screen. In addition, if the electronic device <NUM> supports a pen input function, the display <NUM> may detect a user's gesture using an input unit such as a pen in addition to the user's finger. If the input unit is a stylus pen including a coil therein, the electronic device <NUM> may include a magnetic field detection sensor capable of detecting a magnetic field changed by a coil inside the stylus pen. Accordingly, the display <NUM> may detect even a proximity gesture, i.e., hovering, as well as a touch gesture.

Meanwhile, it has been described that the display function and the gesture detection function are performed in the same component, but the display function and the gesture detection function may be performed in different components. In addition, according to various embodiments, the display <NUM> may not be provided in the electronic device <NUM>.

The processor <NUM> may include a RAM <NUM>, a ROM <NUM>, a CPU <NUM>, a graphic processing unit (GPU) <NUM>, and a bus <NUM>. The RAM <NUM>, the ROM <NUM>, the CPU <NUM>, the GPU <NUM>, and the like may be connected to each other through the bus <NUM>.

The CPU <NUM> accesses the memory <NUM> and performs booting using an operating system (O/S) stored in the memory <NUM>. In addition, the CPU <NUM> performs various operations using various programs, contents, data, and the like stored in the memory <NUM>.

The ROM <NUM> stores an instruction set for system booting. When a turn-on command is input and power is supplied, the CPU <NUM> copies the O/S stored in the storage unit <NUM> to the RAM <NUM> according to a command stored in the ROM <NUM> and executes the O/S to boot the system. When booting is completed, the CPU <NUM> copies various programs stored in the storage unit <NUM> to the RAM <NUM> and executes the programs copied to the RAM <NUM> to perform various operations.

When the booting of the electronic device <NUM> is completed, the GPU <NUM> displays a user interface (UI) on the display <NUM>. Specifically, the GPU <NUM> may generate a screen including various objects such as an icon, an image, text, and the like using a calculation unit (not shown) and a rendering unit (not shown). The calculation unit calculates attribute values such as coordinate values where each object is to be displayed, shapes, sizes, colors, and the like of each object according to a layout of the screen. The rendering unit generates screens of various layouts including objects based on the attribute values calculated by the calculation unit. The screen (or a UI window) generated by the rendering unit is provided to the display <NUM> and displayed in each of a main display area and a sub-display area.

The button <NUM> may be various types of buttons such as a mechanical button, a touch pad, a wheel, and the like formed in a certain area such as a front portion, a side portion, or a rear portion of the exterior of a main body of the electronic device <NUM>.

The video processor <NUM> is a component for processing content received through the communication unit <NUM> or video data included in the content stored in the memory <NUM>. The video processor <NUM> may perform various image processing such as decoding, scaling, noise filtering, frame rate conversion, resolution conversion, and the like on video data.

The audio processor <NUM> is a component for processing the content received through the communication unit <NUM> or audio data included in the content stored in the memory <NUM>. The audio processor <NUM> may perform various processing such as decoding, amplification, noise filtering, or the like on audio data.

When a playback application for multimedia content is executed, the processor <NUM> may drive the video processor <NUM> and the audio processor <NUM> to play the corresponding content. Here, the display <NUM> may display an image frame generated by the video processor <NUM> on at least one of the main display area or the sub-display area.

The audio output unit <NUM> outputs audio data generated by the audio processor <NUM>.

The microphone <NUM> is a component for receiving a user's voice or other sound and converting the received user's voice or the sound into audio data. The processor <NUM> may use the user's voice input through the microphone <NUM> during a call process or convert the user's voice into audio data and store the converted audio data in the memory <NUM>. Meanwhile, the microphone <NUM> may be configured as a stereo microphone that receives sound input at a plurality of locations.

The imaging unit <NUM> is a component for capturing a still image or a video according to the user's control. The imaging unit <NUM> may be provided in plurality such as a front camera and a rear camera. As described above, the imaging unit <NUM> may be used as a unit for acquiring an image of the user in an embodiment for tracking a user's gaze.

When the imaging unit <NUM> and the microphone <NUM> are provided, the processor <NUM> may perform a control operation according to the user's voice input through the microphone <NUM> or the user's motion recognized by the imaging unit <NUM>. That is, the electronic device <NUM> may operate in a motion control mode or a voice control mode. When operating in the motion control mode, the processor <NUM> activates the imaging unit <NUM> to capture an image of the user, tracks a change in the user's motion, and performs a corresponding control operation. When operating in the voice control mode, the processor <NUM> may operate in a voice recognition mode to analyze the user's voice input through the microphone <NUM> and perform a control operation according to the analyzed user's voice.

In the electronic device <NUM> supporting the motion control mode or the voice control mode, a voice recognition technology or a motion recognition technology may be used in various embodiments described above. For example, when the user takes a motion as if selecting an object displayed on a home screen or utters a voice command corresponding to the object, it is determined that the object is selected and a control operation matched to the object may be performed.

In addition, although not shown in <FIG>, according to an embodiment, the electronic device <NUM> may further include a USB port to which a USB connector may be connected, external input port to be connected to various external terminals such as a headset, a mouse, a LAN, and the like, a digital multimedia broadcasting (DMB) chip that receives and processes a DMB signal, various sensors, and the like.

<FIG> is a view illustrating an image processing method of increasing resolution of an image. Specifically, in <FIG>, as an example, it is assumed that an input image <NUM> of <NUM> by <NUM> is input to an upscaling module having a scaling factor of <NUM> and an enlarged image <NUM> of <NUM> by <NUM> is acquired.

Referring to <FIG>, a pixel of (<NUM>,<NUM>) of the input image <NUM> is referred to as a first pixel <NUM>-<NUM> and a pixel of (<NUM>,<NUM>) is referred to as a second pixel <NUM>-<NUM>.

Here, if a pixel corresponding to a pixel of the input image <NUM> among a plurality of pixels of the enlarged image <NUM> is referred to as an original pixel, a pixel of (<NUM>,<NUM>) in the enlarged image <NUM> may be a first original pixel <NUM>-<NUM> to which the first pixel <NUM>-<NUM> corresponds. Also, a second original pixel <NUM>-<NUM> of the enlarged image <NUM> corresponding to the second pixel <NUM>-<NUM> adjacent to the first pixel <NUM>-<NUM> of the input image <NUM> may be a pixel of (<NUM>,<NUM>).

Meanwhile, in addition to the original pixel corresponding to the pixels of the input image <NUM> among the plurality of pixels included in the enlarged image <NUM>, a pixel near the original pixel may be referred to as an interpolated pixel. Specifically, among the pixels between the first original pixel <NUM>-<NUM> and the second original pixel <NUM>-<NUM> corresponding to the pixels of the input image <NUM> in the enlarged image <NUM>, an interpolated pixel adjacent to the first original pixel <NUM>-<NUM> may be referred to as a first interpolated pixel <NUM>-<NUM> and an interpolated pixel adjacent to the first interpolated pixel <NUM>-<NUM> may be referred to as a second interpolated pixel <NUM>-<NUM>, and here, the second interpolated pixel <NUM>-<NUM> may be adjacent to the second original pixel <NUM>-<NUM>.

In <FIG>, a center pixel among the plurality of pixels included in the region in which one pixel of the input image <NUM> is upscaled is illustrated as the original pixel, but another pixel other than the center pixel in the upscaled region may also be set as the original pixel.

Meanwhile, the electronic device <NUM> acquires pixel values of the original pixel and interpolated pixels of the enlarged image <NUM> by using the pixel values of the pixels of the input image <NUM>. Specifically, the electronic device <NUM> acquires pixel values of the original pixel, the interpolated pixel near the original pixel, and another original pixel according to a Gaussian function based on the original pixel in the enlarged image <NUM>. Specifically, the electronic device <NUM> acquires a pixel value of each pixel by identifying a reflection ratio of the pixel value of the original pixel according to a distance to the original pixel on a Gaussian function.

A specific method of acquiring pixel values of a plurality of pixels configuring the enlarged image <NUM> will be described in detail below with reference to <FIG> and <FIG>.

<FIG> is a view illustrating an interpolation method in an image processing method of the related art.

Specifically, <FIG> illustrates a filter of a nearest neighbor method and <FIG> illustrates a filter of a bilinear interpolation method. In both methods, it is assumed that a scaling factor is <NUM>.

Referring to <FIG>, the nearest neighbor method is a method of acquiring a pixel value of an interpolated pixel adjacent to an original pixel to be equal to a pixel value of the original pixel. Specifically, according to the nearest neighbor method, a pixel value of the first interpolated pixel <NUM>-<NUM> adjacent to the first original pixel <NUM>-<NUM> may be acquired as a pixel value of the first original pixel <NUM>-<NUM>. Also, a pixel value of the second interpolated pixel <NUM>-<NUM> adjacent to the second original pixel <NUM>-<NUM> may be acquired as a pixel value of the second original pixel <NUM>-<NUM>.

In other words, according to the nearest neighbor method, the pixel value of the second original pixel <NUM>-<NUM> which is not adjacent is not considered in acquiring the pixel value of the first interpolated pixel <NUM>-<NUM>. In addition, there is a problem in that mosaic-shaped checkboard artifacts may be formed in the enlarged image due to the boundary between the first interpolated pixel <NUM>-<NUM> and the second interpolated pixel <NUM>-<NUM> based on a difference between the pixel value of the first interpolated pixel <NUM>-<NUM> and the pixel value of the second interpolated pixel <NUM>-<NUM>.

Also, referring to <FIG>, the bilinear interpolation method is a method of acquiring a pixel value of an interpolated pixel using values of a plurality of original pixels around the interpolated pixel and determining a reflection ratio of the pixel values of the plurality of original pixels according to a linear function. Here, a y-intercept of the linear function may be <NUM> (which means that all pixel values of the original pixels are reflected) and a slope may be a reciprocal of a scaling factor. Accordingly, a slope of the linear function of the right area may be -<NUM>/<NUM> and a slope of the linear function of the left area may be <NUM>/<NUM> based on the first original pixel. Because the slope of the linear function is identified by the scaling factor, the slope of the linear function may also vary if the scaling factor is different.

Specifically, according to the bilinear interpolation method, the pixel value of the first interpolated pixel <NUM>-<NUM> adjacent to the first original pixel <NUM>-<NUM> may be acquired as a pixel value of the first original pixel <NUM>-<NUM> and as a pixel value of the second original pixel <NUM>-<NUM>. Specifically, the pixel value of the first interpolated pixel <NUM>-<NUM> may be acquired based on a reflection ratio of the pixel value of the first original pixel <NUM>-<NUM> and the pixel value of the second original pixel <NUM>-<NUM> based on the distance between the first original pixel <NUM>-<NUM> and the second original pixel <NUM>-<NUM>.

For example, referring to <FIG>, a distance between the first interpolated pixel <NUM>-<NUM> and the first original pixel <NUM>-<NUM> is one pixel. Accordingly, the electronic device may acquire the pixel value of the first interpolated pixel <NUM>-<NUM> by reflecting <NUM>/<NUM> of the pixel value of the first original pixel <NUM>-<NUM> according to a linear function based on the first original pixel <NUM>-<NUM> shown in <FIG>.

Although not shown, the electronic device may acquire a ratio in which the pixel value of the second original pixel <NUM>-<NUM> is reflected on the pixel value of the first interpolated pixel <NUM>-<NUM> in the same manner. Specifically, a distance between the first interpolated pixel <NUM>-<NUM> and the second original pixel <NUM>-<NUM> is two pixels. Accordingly, the electronic device may acquire the pixel value of the first interpolated pixel <NUM>-<NUM> by reflecting <NUM>/<NUM> of the pixel value of the second original pixel <NUM>-<NUM> according to a linear function based on the second original pixel <NUM>-<NUM>.

In conclusion, the electronic device may acquire the pixel value of the first interpolated pixel <NUM>-<NUM> using <NUM>/<NUM> of the pixel value of the first original pixel <NUM>-<NUM> and <NUM>/<NUM> of the pixel value of the second original pixel <NUM>-<NUM>.

The electronic device may acquire the pixel value of the second interpolated pixel <NUM>-<NUM> using <NUM>/<NUM> of the pixel value of the first original pixel <NUM>-<NUM> and <NUM>/<NUM> of the pixel value of the second original pixel <NUM>-<NUM> in the same manner.

In other words, according to the bilinear interpolation method, in acquiring the pixel value of the first interpolated pixel <NUM>-<NUM>, the pixel values of the two closest original pixels <NUM>-<NUM> and <NUM>-<NUM> are considered but the pixel value of the original pixel which is farther is not considered. In addition, in case of the bilinear interpolation method, there is a problem in that ringing artifacts occur in a region with a high frequency and an edge region in which a pixel value changes rapidly is not clear.

<FIG> is a view illustrating an interpolation method in an image processing method according to an embodiment of the disclosure. Specifically, <FIG> shows a Gaussian function based on the first original pixel <NUM>-<NUM> in the enlarged image among a plurality of Gaussian functions included in the upscaling module. Here, it is assumed that the upscaling factor is <NUM>.

Referring to <FIG>, a Gaussian function <NUM> based on the first original pixel <NUM>-<NUM> defines a ratio in which the pixel value of the first original pixel <NUM>-<NUM> is defined according to a distance to the first original pixel <NUM>-<NUM>. In other words, the x axis of the Gaussian function <NUM> refers to a distance between one pixel in the enlarged image and the first original pixel <NUM>-<NUM>, which is a reference pixel, and the y-axis represents a ratio in which the pixel value of the first original pixel <NUM>-<NUM> is reflected according to distances. Here, because the Gaussian function <NUM> is two-dimensional, the pixels expressed in the Gaussian function <NUM> may be in the same row or the same column. Here, the Gaussian function <NUM> is bilaterally symmetrical with respect to the first original pixel <NUM>-<NUM> and may have a shape that decreases nonlinearly away from the first original pixel <NUM>-<NUM>.

Meanwhile, variance of the Gaussian function <NUM> according to the disclosure is acquired based on the upscaling factor. Specifically, the variance of the Gaussian function <NUM> according to the disclosure is acquired based on a linear function <NUM> for bilinear interpolation of the same upscaling factor. Specifically, the variance of the Gaussian function <NUM> according to the disclosure may be acquired to form a point of contact with the linear function <NUM> for bilinear interpolation. Here, an absolute value of a slope of the linear function <NUM> may be a reciprocal (<NUM>/<NUM> or -<NUM>/<NUM>) of the upscaling factor.

Accordingly, the variance of the Gaussian function <NUM> according to the disclosure may be acquired based on Equation <NUM> as follows.

Here, σd is the variance of the Gaussian function <NUM>, s is the upscaling factor, d is the x-coordinate of the point of contact of the linear function <NUM> and the Gaussian function <NUM>, and t(s) may be a value acquired by adding <NUM> to the distance between x intercepts of the Gaussian function.

Here, t(s) may refer to a size of the Gaussian function <NUM> and may be acquired based on Equation <NUM> below.

Here, s denotes the upscaling factor, and Equation <NUM> may be based on a user's setting. In other words, the size of the Gaussian function <NUM> is not limited to Equation <NUM> described above and may be adjusted according to a user's setting.

As described above, according to the disclosure, by identifying the pixel value reflection ratio of the reference pixel according to the Gaussian function, the pixel value of the reference pixel may be reflected even when acquiring the pixel value of the pixel at a distance greater than that of the related art. That is, compared with the related art in which an enlarged image is generated by reflecting only pixel values of adjacent pixels, pixel values of pixels in a wider range including separated pixels are reflected, thereby generating a more improved enlarged image.

<FIG> is a view illustrating a range of variance of a Gaussian function. Specifically, <FIG> is a view illustrating a variable range of the variance of the Gaussian function <NUM> acquired based on the linear function <NUM> for bilinear interpolation as shown in <FIG>.

Referring to <FIG>, the Gaussian function <NUM> may be acquired to have a point of contact with the linear function <NUM> for bilinear interpolation. Hereinafter, for convenience of description, the Gaussian function having a point of contact with the linear function <NUM> will be referred to as a first Gaussian function <NUM>. Here, the variance σd of the first Gaussian function <NUM> may be acquired based on Equation <NUM> described above.

Meanwhile, the electronic device may change the variance of the Gaussian function based on the variance σd of the first Gaussian function <NUM>. Specifically, the electronic device may set a range of variance so that a full width at half maximum (FWHM) of the Gaussian function does not deviate significantly compared to a FWHM of the linear function <NUM> for bilinear interpolation. Here, the FWHM, a term representing a width of a function, may refer to a difference between two variables in which a function value becomes half of a maximum value of the function.

Specifically, the Gaussian function may be acquired by Equation <NUM> below.

Here, the range of the variance may be σd(s)-s*<NUM> ≤ σ(s) ≤ σd(s)+s*<NUM>.

Based on the range of the variance described above, the Gaussian function having a minimum variance value may be in the form of a second Gaussian function <NUM>, and the Gaussian function having a maximum variance value may be in the form of a third Gaussian function <NUM>.

Meanwhile, as shown in <FIG> below, the range of the variance described above may be a range set so that artifacts do not occur in a high frequency region of a frequency domain.

<FIG> and <FIG> are views illustrating a difference between the related art and the disclosure. Specifically, <FIG> illustrates a difference between the related art and the disclosure in an image domain, and <FIG> illustrates a difference between the related art and the disclosure in a frequency domain. Here, it is assumed that the scaling factor is <NUM> in both the related art and the disclosure. In addition, for convenience of explanation, only the right area of the reference pixel in which x is <NUM> will be described. Because the filters are bilaterally symmetrical, a description of the left area of the reference pixel is the same as the description of the right area of the reference pixel.

In the image domain of <FIG>, a filter <NUM> of a nearest neighbor method, a filter <NUM> of a bilinear interpolation method, and a filter <NUM> in the form of a Gaussian function of the disclosure are illustrated.

In the nearest neighbor method, a value of an interpolated pixel is acquired as a pixel value of an adjacent original pixel. Referring to <FIG>, the filter <NUM> of the nearest neighbor method may be a filter based on an original pixel whose x is <NUM> and may acquire a pixel value of an interpolated pixel whose x is <NUM> as a pixel value of an original pixel whose x is <NUM> which is a pixel adjacent to the pixel whose x is <NUM>.

In other words, the filter <NUM> of the nearest neighbor method uses the pixel value of the reference pixel whose x is <NUM> only to obtain the pixel value of the adjacent pixel whose x is <NUM>. Based on this, a size of the filter <NUM> of the nearest neighbor method may be <NUM> (including a reference pixel whose x is <NUM>, an interpolated pixel whose x is <NUM>, and an interpolated pixel whose x is -<NUM>).

Meanwhile, the filter of the bilinear interpolation method acquires a reflection ratio of a pixel value of an original pixel in accordance with a linear function based on a distance between an interpolated pixel and the original pixel and obtain a pixel value of the interpolated pixel based on the acquired reflection ratio. Referring to <FIG>, the filter <NUM> of the bilinear interpolation method is a filter based on the original pixel whose x is <NUM>, and the ratio of reflecting the pixel value of the original pixel linearly decreases according to a distance between the interpolated pixel and the original pixel whose x is <NUM>. Accordingly, the pixel value of the interpolated pixel whose x is <NUM> may be acquired using <NUM>/<NUM> of the pixel value of the original pixel whose x is <NUM>, and the pixel value of the interpolated pixel whose x is <NUM> may be acquired using <NUM>/<NUM> of the pixel value of the original pixel whose x is <NUM>. Meanwhile, this is based on measurement of a distance based on a middle point of the pixel, and precisely, because the distance varies within the pixel, an average of reflection ratios that vary within one pixel may be used. In case of the pixel whose x is <NUM> and the pixel whose x is <NUM>, the ratio based on the distance linearly decreases, and thus there is no problem even if the reflection ratio corresponding to the middle point of the pixel is used. However, in case of a pixel whose x is <NUM>, the reflection ratio linearly decreases from a starting point to a middle point but is <NUM> from the middle point, and thus an average of reflection ratios in accordance with the distance from the starting point to the middle point of the pixel whose x is <NUM> with respect to the pixel whose x is <NUM> may be used as a reflection ratio of the pixel whose x is <NUM>.

That is, a size of the filter <NUM> of the bilinear interpolation method may be <NUM> (including a reference pixel whose x is <NUM>, interpolated pixels whose x is <NUM>, <NUM>, and <NUM>, and interpolated pixels whose x is -<NUM>, -<NUM>, and -<NUM>).

Meanwhile, the filter <NUM> in the form of the Gaussian function of the disclosure may be acquired based on a scaling factor. Specifically, the Gaussian function type filter <NUM> may have a variance acquired based on the bilinear interpolation type filter <NUM> having the same scaling factor. Here, the Gaussian function type filter <NUM> may use the pixel value of the reference pixel whose x is <NUM> up to a pixel whose x is <NUM> due to the shape characteristic of the filter. Accordingly, the size of the Gaussian function type filter <NUM> may be <NUM> (including the reference pixel whose x is <NUM>, the interpolated pixels whose x is <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>, and the interpolated pixels whose x is -<NUM>, -<NUM>, -<NUM>, -<NUM>, -<NUM>, and -<NUM>).

As described above, when the Gaussian function type filter of the disclosure is used, the pixel values of the interpolated pixels may be acquired using the pixel values of the original pixels in a wider range. In addition, the reflection ratio of the pixel value of the reference pixel gradually decreases according to distances to the neighboring pixels, and compared with the bilinear interpolation type filter, the pixel value of the pixel which is closer is used more frequently and the pixel value of a pixel which is away is used less frequently. Therefore, according to the disclosure, a more improved high-quality image may be generated.

Meanwhile, <FIG> shows a result of analyzing an enlarged image acquired using the nearest neighbor type filter <NUM>, the bilinear interpolation type filter <NUM>, and the Gaussian function type filter <NUM> of the disclosure in the frequency domain.

Referring to <FIG>, it can be seen that boosting occurs in a high frequency region when the nearest neighbor type filter <NUM> and the bilinear interpolation type filter <NUM> are used. As a result, artifacts occur in the enlarged image.

In contrast, in case of using the Gaussian function type filter <NUM>, boosting does not occur even in the high frequency region, and thus it can be seen that artifacts in the enlarged image are reduced.

<FIG> is a view illustrating an interpolation method using a plurality of pixels of an input image according to an embodiment of the disclosure.

Referring to <FIG>, pixel values of a plurality of pixels in an enlarged image may be acquired using pixel values of a plurality of original pixels <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>. Specifically, reflection ratios of the pixel values of the original pixels may be identified according to distances to the respective original pixels on a first Gaussian function <NUM>-<NUM> based on the first original pixel <NUM>-<NUM>, a second Gaussian function <NUM>-<NUM> based on the second original pixel <NUM>-<NUM>, and a third Gaussian function <NUM>-<NUM> based on the third original pixel <NUM>-<NUM>.

That is, the original pixels or interpolated pixels configuring the enlarged image may be acquired by overlapping the reflection ratio of the pixel value of the first original pixel <NUM>-<NUM> identified according to the distance to the first original pixel <NUM>-<NUM> on the first Gaussian function <NUM>-<NUM>, the reflection ratio of the pixel value of the second original pixel <NUM>-<NUM> identified according to the distance to the second original pixel <NUM>-<NUM> on the second Gaussian function <NUM>-<NUM>, and the reflection ratio of the pixel value of the third original pixel <NUM>-<NUM> identified according to the distance to the third original pixel <NUM>-<NUM> on the third Gaussian function <NUM>-<NUM>.

Meanwhile, in <FIG>, for convenience of explanation, Gaussian functions corresponding to original pixels other than the first to third original pixels <NUM>-<NUM> to <NUM>-<NUM> are not shown, but in actual realization, pixel values of the original pixels and the interpolated pixels of the enlarged image may be acquired based on Gaussian functions respectively corresponding to all the original pixels of an input image.

<FIG> is a view illustrating the interpolation method of <FIG> in a 3D domain. Specifically, <FIG> shows that the pixel value of the reference original pixel <NUM>-<NUM> is used only for the pixel values of the left and right interpolated pixels and the original pixel, but in actuality, as shown in <FIG>, the pixel value of the reference original pixel <NUM>-<NUM> may also be used for pixel values of upper and lower interpolated pixels and the original pixel and pixel values of diagonal interpolated pixels and the original pixel.

Referring to <FIG>, a Gaussian function <NUM>-<NUM> may be in the form of a 3D Gaussian function based on the reference original pixel <NUM>-<NUM>. Pixel values of interpolated pixels around the reference original pixel <NUM>-<NUM> and other original pixels may be acquired by reflecting the pixel value of the reference original pixel <NUM>-<NUM> by a ratio identified according to the distance to the reference original pixel <NUM>-<NUM>.

Meanwhile, in <FIG>, for convenience of explanation, the pixels of the enlarged image are shown as <NUM> by <NUM>, but a range of pixels in which the pixel value of the reference original pixel <NUM>-<NUM> is used may be <NUM> by <NUM> with respect to the reference original pixel <NUM>-<NUM> as shown in <FIG>.

<FIG> is a view illustrating the interpolation method of <FIG> in a 3D domain. Specifically, in <FIG>, it is illustrated that pixel values of a plurality of original pixels included in the same row or column are used to acquire a pixel value of an original pixel or an interpolated pixel of an enlarged image, but in actuality, pixel values of a plurality of original pixels included in different rows or columns may also be used. Here, the Gaussian function based on each original pixel may have a 3D form.

Specifically, the original pixels or interpolated pixels configuring the enlarged image may be acquired by overlapping a reflection ratio of a pixel value of a first original pixel <NUM>-<NUM> identified according to a distance to the first original pixel <NUM>-<NUM> on the first Gaussian function <NUM>-<NUM>, a reflection ratio of a pixel value of a second original pixel <NUM>-<NUM> identified according to a distance to the second original pixel <NUM>-<NUM> on the second Gaussian function <NUM>-<NUM>, a reflection ratio of a pixel value of a fourth original pixel <NUM>-<NUM> identified according to a distance to the fourth original pixel <NUM>-<NUM> on the fourth Gaussian function <NUM>-<NUM>, and a reflection ratio of a pixel value of a fifth original pixel <NUM>-<NUM> identified according to a distance to the fifth original pixel <NUM>-<NUM> on the fifth Gaussian function <NUM>-<NUM>.

<FIG> is a flowchart schematically illustrating an image processing method according to an embodiment of the disclosure.

First, an electronic device receives an image (S1310). Specifically, the electronic device may receive an image from an external device or may receive an image stored in a memory of the electronic device.

Next, the electronic device does input the input image to a learned artificial intelligence model and output an enlarged image having increased resolution (S1320). Specifically, the artificial intelligence model includes an upscaling module, and the upscaling module acquires a pixel value of an interpolated pixel near an original pixel according to a Gaussian function based on the original pixel corresponding to a pixel of the input image.

Here, the upscaling module identifies a reflection ratio of the pixel value of the original pixel according to a distance between the original pixel and the interpolated pixel on the Gaussian function based on the original pixel. In addition, the upscaling module acquires the pixel value of the interpolated pixel using the identified reflection ratio.

<FIG> is a view comparing enlarged images acquired according to the related art and an image processing method according to an embodiment of the disclosure.

<FIG> shows results of analyzing frequencies of the images in the x direction from the boundary of the enlarged image to the black line, which is the center of the images, as shown in <FIG>.

Specifically, <FIG> show the enlarged image acquired according to the related art and the analysis results and <FIG> shows the enlarged image acquired according to the disclosure and the analysis results. Here, <FIG> shows an enlarged image acquired by a deconvolution method, <FIG> shows an enlarged image acquired by a nearest neighbor method, and <FIG> shows an enlarged image acquired by a bilinear interpolation method.

Referring to <FIG>, it can be seen that the frequency of the enlarged image forms a wave shape at a certain period and mosaic-shaped artifacts occurred. Referring to <FIG>, it can be seen that the frequency fluctuates in an edge region between a gray surface and a black line, and thus ringing artifacts occurred in the edge region.

Meanwhile, referring to <FIG>, it can be seen that the frequency is even in the gray area and there is no fluctuation of the frequency in the edge area between the gray surface and the black line, and thus the edge in the enlarged image is clear. In other words, it can be seen that an improved high-quality image is acquired, compared to the related art.

According to the various embodiments described above, when the Gaussian function type filter is used, pixel values of interpolated pixels may be acquired using a pixel value of an original pixel of a wider range. In addition, the reflection ratio of the pixel value of the reference pixel gradually decreases according to distances to the neighboring pixels, and compared with the bilinear interpolation type filter, the pixel values of the pixels which are closer are used more frequently and the pixel values of the pixels which are away are used less frequently. Therefore, according to the disclosure, an improved high-quality image may be generated.

Meanwhile, various embodiments described above may be realized in a computer or similar device-readable recording medium using software, hardware, or a combination thereof. In case of implementation by hardware, embodiments described in the disclosure may be realized using at least one of application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or electronic units performing other functions. In some cases, embodiments described in the disclosure may be realized by the processor <NUM> itself. In case of software implementation, embodiments such as procedures and functions described in the disclosure may be realized as separate software modules. Each of the software modules may perform one or more functions and operations described in the disclosure.

Meanwhile, the image processing method according to various embodiments described above may be stored in a non-transitory readable medium. Such a non-transitory readable medium may be installed and used in a variety of devices.

Such a non-transitory readable medium is not a medium for storing data for a short time such as a register, cache or memory, but refers to a medium that semi-permanently stores data and may be read by a device. Specifically, programs for performing various methods described above may be stored in the non-transitory readable medium may include a CD, DVD, hard disk, Blu-ray disc, USB, memory card, ROM, and the like, and provided.

According to embodiments, the methods according to various embodiments disclosed in this document may be included in a computer program product and provided. The computer program product may be traded as goods between a seller and a purchaser. The computer program product may be distributed as a device-readable storage medium (e.g., compact disk read only memory (CD-ROM)) or online through an application store (e.g., play store™). In case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a storage medium such as a server of a manufacturer, a server of an application store, or a memory of a relay server.

Claim 1:
An electronic device (<NUM>) comprising:
a memory (<NUM>) storing a learned artificial intelligence model; and
a processor (<NUM>) configured to input an input image to the artificial intelligence model and to output an enlarged image with increased resolution,
wherein the learned artificial intelligence model includes an upscaling module (<NUM>) configured to calculate a pixel value of an interpolated pixel near an original pixel corresponding to a pixel of the input image in the enlarged image based on a Gaussian function in a form which is bilaterally symmetrical and nonlinearly decreases with respect to the original pixel, wherein the upscaling module (<NUM>) calculates the pixel value of the interpolated pixel near a plurality of original pixels based on a ratio at which the plurality of original pixel values are reflected in the pixel value of the interpolated pixel, and the ratio is identified according to distances between the plurality of original pixels and the interpolated pixel, on a plurality of Gaussian functions based on the plurality of original pixels,
wherein, a variance of the Gaussian function is calculated based on a linear function for bilinear interpolation of an upscaling factor, wherein the upscaling factor corresponds to a magnification of the enlarged image compared to the input image.