Patent Description:
An artificial intelligence (Al) system is a computer system that realizes human-level intelligence, and enables machines to become smart by learning and making decisions on their own, unlike existing rule-based smart systems. An Al system may improve its recognition rates and is capable of understanding a user's preferences more accurately through experience, and thus, existing rule-based smart systems are increasingly being replaced with deep learning-based Al systems.

Al technology consists of machine learning (deep learning) and element technologies using machine learning.

Machine learning is an algorithmic technology for autonomously classifying/learning features of input data, and element technologies are technologies for simulating functions of a human brain such as cognition and decision-making by using machine learning algorithms such as deep learning and include technical fields such as linguistic understanding, visual understanding, reasoning/prediction, knowledge representation, motion control, etc..

Various technical fields to which Al technology is applied are as follows. Linguistic understanding is a technology for recognizing and applying/processing human language/characters and includes natural language processing, machine translation, a dialog system, question answering, speech recognition/synthesis, etc. Visual understanding is a technology for recognizing and processing an object in the same way as performed by a human visual system, and includes object recognition, object tracking, image retrieval, person recognition, scene understanding, spatial understanding, image enhancement, etc. Reasoning/prediction is a technology for judging information and logically inferring and predicting new information and includes knowledge/probability-based inferring, optimization prediction, preference-based planning, recommendations, etc. Knowledge representation is a technology for automatically processing information about human experience as knowledge data and includes knowledge construction (data generation/classification), knowledge management (data utilization), etc. Motion control is a technology for controlling autonomous driving of a vehicle and motion of a robot and includes movement control (navigation, collision avoidance, and travelling), manipulation control (action control), etc..

Moreover, with the widespread use of mobile devices and social network service (SNS), users have been capturing images of objects by using a camera of a mobile device anytime, anywhere, and uploading the captured images to an SNS server. Mobile device users have been attempting to edit images through various image editing programs in order to improve the quality or atmosphere of an image, but it may be somewhat difficult for users having no specialized knowledge to perform such an editing process. Thus, there is a need for a method of transforming a style of an image to match a style desired by a user even when the user has no specialized knowledge about it.

<CIT>discusses image enhancement of night scenes. The paper "<NPL>, discusses the visual sentiment task of mapping an image to an adjective noun pair.

A technical objective of the disclosure is to provide an image processing apparatus and method according to embodiments, which are capable of easily transforming a style of an image without needing specialized knowledge.

Another technical objective is to provide an image processing apparatus and method according to embodiments, which are capable of reducing an amount of data to be stored in an apparatus for performing style transformation.

An image processing apparatus and method according to embodiments may easily transform a style of an image even without special knowledge.

Furthermore, the image processing apparatus and method according to the embodiments may reduce an amount of data to be stored in a device that performs style transformation.

However, effects that can be achieved by an image processing apparatus and method according to embodiments are not limited to those mentioned above, and other effects not mentioned may be clearly understood from the following description by those of ordinary skill in the art to which the present disclosure belongs.

A brief description of the accompanying drawings is provided in order to more fully understand the drawings recited in the present specification.

An image processing method according to the present invention includes the features of claim <NUM>.

In an exemplary embodiment, the selecting of the target reference image may include: displaying, on a display, at least one thumbnail image corresponding to the at least one candidate reference image in an order based on the degree of relevance; and selecting, as the target reference image, a candidate reference image corresponding to a thumbnail image selected by a user from among the at least one thumbnail image.

In an exemplary embodiment, the obtaining of the reference style data for the target reference image may include selecting, based on a frequency of selection by the user, the target reference image from among the at least one candidate reference image to which the same visual sentiment label as the label of the first image is assigned.

In an exemplary embodiment, the obtaining of the label of the first image may include obtaining a label of a region of interest (ROI) in the first image by inputting the ROI to the recognition model, and the obtaining of the reference style data for the target reference image may include obtaining the reference style data for the target reference image to which the same vertical sentiment label as the label of the ROI is assigned.

In an exemplary embodiment, the obtaining of the reference style data for the target reference image may include: identifying at least one candidate reference image to which the same vertical sentiment label as the label of the ROI is assigned; calculating a degree of relevance to the first image for each of the at least one candidate reference image; and selecting, based on the calculated degree of relevance, the target reference image from among the at least one candidate reference image.

In an exemplary embodiment, the obtaining of the label of the first image may include obtaining a plurality of labels of the first image and probability values respectively corresponding to the plurality of labels, and the calculating of the degree of the relevance may include, for each of at least one candidate reference image to which the same one or more visual sentiment labels as one or more of the plurality of labels of the first image are assigned, calculating an average of probability values respectively corresponding to the one or more labels as the degree of relevance, wherein a probability value corresponding to the label of the ROI is set to a preset value.

In an exemplary embodiment, the image processing method may further include: changing internal parameters of the recognition model to internal parameters corresponding to a reference image group selected by the user from among a plurality of reference image groups; and inputting the first image to the recognition model that has the changed internal parameters.

In an exemplary embodiment, the plurality of reference image groups are classified according to a creator of reference images.

In an exemplary embodiment, the image processing method may further include: connecting to a server and receiving a reference image group list from the server; receiving a selection, from a user, of at least one reference image group from the reference image group list; and receiving, from the server, internal parameters of the recognition model trained based on reference images included in the selected at least one reference image group, reference style data for the reference images included in the at least one reference image group, and visual sentiment labels assigned to the reference images included in the at least one reference image group.

In an exemplary embodiment, the image processing method may further include: obtaining the first style data for the first image by inputting the first image to a feature extraction model; and generating a second image by inputting the generated second style data to a feature synthesis model.

An image processing apparatus according to the present invention includes the features of claim <NUM>.

In an exemplary embodiment, the processor may display, on a display, at least one thumbnail image corresponding to the at least one candidate reference image in an order based on the degree of relevance, and select, as the target reference image, a candidate reference image corresponding to a thumbnail image selected by the user from among the at least one thumbnail image.

In an exemplary embodiment, the processor may select, based on a frequency of selection by the user, the target reference image from among the at least one candidate reference image to which the same visual sentiment label as the label of the first image is assigned.

In an exemplary embodiment, the processor may obtain a label of an ROI in the first image by inputting the ROI to the recognition model, and obtain the reference style data for the target reference image to which the same vertical sentiment label as the label of the ROI is assigned.

In the present disclosure, various changes may be made, and numerous embodiments may be provided. Particular embodiments are illustrated in the drawings and will be described in detail in the detailed description.

In describing embodiments, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the essence of the disclosure, the descriptions thereof will be omitted. Furthermore, numbers (e.g., a first, a second, etc.) used in the description of the embodiments are merely identifying symbols for distinguishing one element from another.

Furthermore, throughout the present specification, it should be understood that when an element is referred to as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, but may be connected or coupled to the other element with an intervening element interposed therebetween unless there is a particular description contrary thereto.

Furthermore, in the present specification, for an element expressed as a "unit", a "module", or the like, two or more elements may be combined into a single element, or a single element may be divided into two or more elements according to subdivided functions. Furthermore, each element to be described below may further perform, in addition to its main functions, some or all of functions performed by another element, and some of the main functions of each element may be performed entirely by another component.

Furthermore, in the present specification, a 'first image' refers to an image on which style transformation is to be performed, and a 'second image' refers to an image generated by performing the style transformation.

Hereinafter, embodiments based on a technical idea of the present disclosure will be sequentially described in detail.

<FIG> is a schematic diagram for describing an operation of an image processing apparatus <NUM> according to an embodiment.

The image processing apparatus <NUM> generates a second image <NUM> by transforming a style of the first image <NUM>. The first image <NUM> may be an image stored in the image processing apparatus <NUM>, or may be a preview image recognized via a camera of the image processing apparatus <NUM>.

The image processing apparatus <NUM> generates the second image <NUM> by transforming the style of the first image <NUM> according to a style of a reference image having a high relevance to the first image <NUM> among several reference images. The same objects are included in the first and second images <NUM> and <NUM>, but features of the second image <NUM> (brightness, contrast, viewpoint, tone, white balance, sharpness, histogram, etc.) may be different from features of the first image <NUM>.

A 'reference image' may include, for example, an image captured and/or edited by a professional creator. Because it is difficult for the user himself or herself to capture an image having the same style as a reference image or edit an existing image, according to an embodiment, the second image <NUM> generated by performing transformation according to a style of the reference image may be easily obtained using a method such as the user selecting a reference image.

The image processing apparatus <NUM> may be implemented as a user device or a server. For example, the user device may include various types of devices such as a smartphone, a desktop computer, a notebook , a tablet PC, etc. An operation when the image processing apparatus <NUM> is implemented as a user device or a server will be described below.

<FIG> is a flowchart of an image processing method according to an embodiment.

In operation S210, the image processing apparatus <NUM> inputs the first image <NUM> to the recognition model and obtains a label of the first image <NUM> output from the recognition model.

In an embodiment, the label may be information describing content of the first image <NUM>. For example, when the first image <NUM> includes an ocean, the recognition model may output a label such as a 'cool ocean'.

The recognition model may be trained based on reference images to which visual sentiment labels are pre-assigned, and analyze the first image <NUM> to output a label corresponding to the first image <NUM> from among several visual sentiment labels. The recognition model may include, for example, an object recognition deep neural network (DNN).

In operation S220, the image processing apparatus <NUM> obtains reference style data for a target reference image to which the same visual sentiment label as the label of the first image <NUM> is assigned from among reference images to which visual sentiment labels are pre-assigned.

When a plurality of labels related to the first image <NUM> are output from the recognition model, the image processing apparatus <NUM> may identify at least one candidate reference image to which the same visual sentiment labels as one or more of the plurality of labels are assigned and select a target reference image from among the at least one candidate reference image. A method of selecting a target reference image from among at least one candidate reference image will be described below with reference to <FIG>.

Reference style data for a target reference image may include at least one of brightness data, contrast data, viewpoint data, tone data, white balance data, sharpness data, histogram data, and feature map of the target reference image. Pieces of reference style data for reference images may be prestored in the image processing apparatus <NUM>, and when a target reference image is selected from among the reference images, the image processing apparatus <NUM> obtains reference style data corresponding to the target reference image.

In operation S230, the image processing apparatus <NUM> generates second style data based on first style data for the first image <NUM> and the reference style data for the target reference image. In this case, the second style data may be different from the first style data and may be generated by transforming the first style data or be newly generated based on the first style data and the reference style data.

For example, the image processing apparatus <NUM> may generate the second style data by transforming at least one of brightness, contrast, viewpoint, tone, white balance, sharpness, histogram, and feature map of the first image <NUM> based on at least one of brightness, contrast, viewpoint, tone, white balance, sharpness, histogram, and feature map of the target reference image.

As an example, the image processing apparatus <NUM> may generate second style data by transforming at least one of brightness, contrast, viewpoint, tone, white balance, sharpness, histogram, and feature map of the first image <NUM> to be similar to or the same as corresponding at least one of brightness, contrast, viewpoint, tone, white balance, sharpness, histogram, and feature map of the target reference image.

In addition, as an example, the image processing apparatus <NUM> may generate a new feature map based on a feature map of the first image <NUM> and a reference feature map of the target reference image.

In operation S240, the image processing apparatus <NUM> generates a second image <NUM> based on the second style data. The image processing apparatus <NUM> may generate the second image <NUM> by changing the first image <NUM> according to the second style data or inputting the second style data (e.g., a feature map) to a DNN.

<FIG> is an exemplary diagram showing labels <NUM> obtained by inputting the first image <NUM> to a recognition model <NUM> and probability values <NUM> of the labels <NUM>.

As described above, the image processing apparatus <NUM> inputs the first image <NUM> to the recognition model <NUM> and obtains the labels <NUM> for the first image <NUM> output from the recognition model <NUM>. Referring to <FIG>, the labels <NUM> corresponding to the first image <NUM> may include 'beautiful sunset', 'blue sky', 'misty cloud', and 'dashing ocean wave'. Furthermore, the recognition model <NUM> may output the probability values <NUM> respectively corresponding to the labels <NUM>. Each of the probability values <NUM> may be a value indicating to what extent the first image <NUM> is relevant to a corresponding one of the labels <NUM>. Because the probability value <NUM> of 'beautiful sunset' which is one of the labels <NUM> for the first image <NUM> is <NUM>%, it can be seen that the first image <NUM> is highly relevant to 'beautiful sunset'.

The image processing apparatus <NUM> selects a target reference image to which the same visual sentiment label as a label of the first image <NUM> is assigned, and as shown in <FIG>, each reference image may be assigned some of visual sentiment labels <NUM> for describing the corresponding reference image.

For example, 'beautiful sunset', 'blue sky', and 'amazing beach' among the visual sentiment labels <NUM> may be assigned to reference image <NUM>, and 'beautiful sunrise' and 'large mountain' thereamong may be assigned to reference image <NUM>.

Hereinafter, a method of selecting a target reference image from among reference images will be described with reference to <FIG>.

Referring to <FIG>, when labels A, B, and C are obtained as labels of the first image <NUM>, and the labels A, B, and C respectively have probability values of <NUM>%, <NUM>%, and <NUM>%, the image processing apparatus <NUM> selects, as a candidate reference image, a reference image to which the same visual sentiment labels as one or more of the labels A, B, and C are assigned.

It can be seen that visual sentiment labels of candidate reference image <NUM> are labels A and B which are the same as the labels A and B of the first image <NUM> while visual sentiment labels of candidate reference image <NUM> are labels C and D, of which the label C is the same as the label C of the first image <NUM>. In addition, it can be seen that visual sentiment labels of candidate reference image <NUM> are labels B, C, and E, of which the labels B and C are the same as the labels B and C of the first image <NUM>.

In an embodiment, the image processing apparatus <NUM> may select, as a target reference image, a candidate reference image having a highest frequency of selection by the user from among at least one candidate reference image.

Furthermore, in the present invention, the image processing apparatus <NUM> calculates a degree of relevance between each of the at least one candidate reference image and the first image <NUM>, and select a target reference image based on the calculated degree of relevance. For example, the image processing apparatus <NUM> may select a candidate reference image with a highest degree of relevance as a target reference image.

The degree of relevance is a value indicating how similar content of each candidate reference image is to that of the first image <NUM>, and is calculated based on probability values of labels of the first image <NUM>.

In the present invention, for each candidate reference image that is assigned the same visual sentiment labels as one or more of the labels of the first image <NUM>, the degree of relevance is calculated as an average of probability values of the one or more labels.

Referring to <FIG>, labels A and B are assigned as common labels to candidate reference image <NUM>, and because probability values of the labels A and B of the first image <NUM> are respectively <NUM>% and <NUM>%, <NUM>% that is an average of the probability values may be calculated as a degree of relevance of the candidate reference image <NUM>. Furthermore, label C is assigned as a common label to candidate reference image <NUM>, and because a probability value of the label C of the first image <NUM> is <NUM>%, <NUM>% may be calculated as a degree of relevance of the candidate reference image <NUM>. In addition, labels C and D are assigned as common labels to the candidate reference image <NUM>, and because probability values of the labels B and C of the first image <NUM> are respectively <NUM>% and <NUM>%, <NUM>% that is an average of the probability values may be calculated as a degree of relevance of the candidate reference image <NUM>.

The image processing apparatus <NUM> may select, as a target reference image, a candidate reference image (the candidate reference image <NUM> in <FIG>) with a highest degree of relevance from among candidate reference images.

In another example, the degree of relevance of each candidate reference image may be calculated based on the number of visual sentiment labels that are the same as labels of the first image <NUM> from among visual sentiment labels of the corresponding candidate reference image. In this case, referring to <FIG>, because two visual sentiment labels that are the same as labels of the first image <NUM> are assigned to each of the candidate reference image <NUM> and candidate reference image <NUM> while one visual sentiment label that is the same as a label of the first image <NUM> is assigned to the candidate reference image <NUM>, a lowest degree of relevance may be calculated for the candidate reference image <NUM>. In addition, a higher degree of relevance may be calculated for a candidate reference image which has a higher average of probability values corresponding to the same visual sentiment labels as labels of the first image <NUM> between the candidate reference image <NUM> and the candidate reference image <NUM>.

In an embodiment, when the user selects a region of interest (ROI) from the first image <NUM>, the image processing apparatus <NUM> may select a target reference image by further taking a label of the ROI into account. For example, when the user selects a portion of the first image <NUM> displayed on a display via touching, etc., the image processing apparatus <NUM> may crop the ROI selected by the user from the first image <NUM> and input the cropped ROI to the recognition model <NUM>. In addition, the image processing apparatus <NUM> may obtain a label of the ROI and a probability value of the label which are output from the recognition model <NUM>.

Referring to <FIG>, label A (<NUM>%), label B (<NUM>%), and label C (<NUM>%) are output from the recognition model <NUM> as labels of the first image <NUM>, and label B (<NUM>%) may be output from the recognition model <NUM> as a label of the ROI. The image processing apparatus <NUM> may obtain labels and probability values of the labels by inputting both the first image <NUM> and the ROI to the recognition model <NUM>.

The image processing apparatus <NUM> may select a target reference image from among candidate reference images to which the same visual sentiment label as the label of the ROI is assigned. Because an ROI is a region on which the user's attention is particularly focused, reference images to which the same visual sentiment label as a label of the ROI is essentially assigned are identified as candidate reference images.

Referring to <FIG>, when the label B is obtained as the label of the ROI, the candidate reference image <NUM> to which the labels B and A are assigned, the candidate reference image <NUM> to which the labels B, A, and C are assigned, and the candidate reference image <NUM> to which the labels B and C are assigned may be identified.

As described above, the image processing apparatus <NUM> may select, as a target reference image, a candidate reference image having a highest frequency of selection by the user from among at least one candidate reference image.

In the present invention, the image processing apparatus <NUM> calculates a degree of relevance between each of the at least one candidate reference image and the first image <NUM>, and select a target reference image based on the calculated degree of relevance. For example, the image processing apparatus <NUM> may select a candidate reference image with a highest degree of relevance as a target reference image.

The image processing apparatus <NUM> may set, to a preset value (e.g., <NUM>%), a probability value corresponding to a label of an ROI among labels of the first image <NUM> and then calculate, for each candidate reference image that is assigned the same visual sentiment labels as one or more of the labels of the first image <NUM>, an average of probability values of the one or more labels as the degree of relevance.

For example, in this case, because visual sentiment labels that are the same as the labels of the first image <NUM> from among visual sentiment labels assigned to the candidate reference image <NUM> are labels A and B, the degree of relevance of the candidate reference image <NUM> may be calculated as <NUM>% that is an average of <NUM>% and <NUM>%. Furthermore, because visual sentiment labels that are the same as the labels of the first image <NUM> from among visual sentiment labels assigned to the candidate reference image <NUM> are labels A, B, and C, the degree of relevance of the candidate reference image <NUM> may be calculated as <NUM>% that is an average of <NUM>%, <NUM>%, and <NUM>%. In addition, because visual sentiment labels that are the same as the labels of the first image <NUM> from among visual sentiment labels assigned to the candidate reference image <NUM> are labels B and C, the degree of relevance of the candidate reference image <NUM> may be calculated as <NUM>% that is an average of <NUM>% and <NUM>%. For example, if the degree of relevance may be calculated as <NUM>% for a candidate reference image to which the same visual sentiment label as only the label of the ROI is assigned.

In an embodiment, when a degree of relevance is calculated for each candidate reference image as described with reference to <FIG> and <FIG>, as shown in <FIG>, the image processing apparatus <NUM> may display, on a display <NUM>, thumbnail images <NUM>, <NUM>, and <NUM> respectively corresponding to candidate reference images in an order based on the degree of relevance. For example, the thumbnail images <NUM>, <NUM>, and <NUM> respectively corresponding to the candidate reference images may be displayed on the display <NUM> in order from highest to lowest degrees of relevance. In addition, the image processing apparatus <NUM> may select, as a target reference image, a candidate reference image corresponding to a thumbnail image selected by the user from among the thumbnail images <NUM>, <NUM>, and <NUM> displayed on the display <NUM>.

As an example, when the image processing apparatus <NUM> is implemented as a user device, the image processing apparatus <NUM> may display the thumbnail images <NUM>, <NUM>, and <NUM> on its own display. Furthermore, as an example, when the image processing apparatus <NUM> is implemented as a server, the server may transmit the thumbnail images <NUM>, <NUM>, and <NUM> to the user device so that the user device may display the thumbnail images <NUM>, <NUM>, and <NUM> on its display.

In an example, the image processing apparatus <NUM> may display, on the display, thumbnail images obtained by changing the first image <NUM> according to styles of the candidate reference images instead of the thumbnail images <NUM>, <NUM>, and <NUM> respectively corresponding to the candidate reference images, and select, as a target reference image, a candidate reference image corresponding to a thumbnail image selected by the user.

<FIG> is a diagram for describing a method of generating a second image <NUM> based on a first image <NUM>.

Although it has been described above that each of first style data for the first image <NUM> and reference style data for a target reference image may include at least one of brightness, contrast, viewpoint, tone, white balance, sharpness, histogram, and feature map, <FIG> illustrates a case where first style data for the first image <NUM> and reference style data for a target reference image are respectively feature maps (<NUM> and <NUM>).

The image processing apparatus <NUM> obtains a first feature map <NUM> output from a feature extraction model <NUM> by inputting the first image <NUM> to the feature extraction model <NUM>. In addition, the image processing apparatus <NUM> extracts a reference feature map <NUM> corresponding to a target reference image.

The image processing apparatus <NUM> may generate a second feature map <NUM> based on the first feature map <NUM> and the reference feature map <NUM>, and input the generated second feature map <NUM> to a feature synthesis model <NUM>. The image processing apparatus <NUM> may obtain the second image <NUM> output from the feature synthesis model <NUM>.

In an embodiment, the image processing apparatus <NUM> may perform a whitening transform on the first feature map <NUM> and perform a coloring transform on a matrix related to the reference feature map <NUM> and the first feature map that has undergone the whitening transform to thereby generate the second feature map <NUM>. For example, the image processing apparatus <NUM> performs a whitening transform by taking a covariance matrix of the first feature map <NUM> and the first feature map <NUM> as an input. Style information of the first image <NUM> is removed from the first feature map that has undergone the whitening transform, and only information about an object in the first image <NUM> is included in the first feature map that has undergone the whitening transform. Furthermore, the image processing apparatus <NUM> may perform a coloring transform by taking, as an input, an inverse covariance matrix of the reference feature map <NUM> and the first feature map that has undergone the whitening transform. The second feature map <NUM> generated as a result of the coloring transform includes object information of the first image <NUM> and style information of the target reference image.

<FIG> is a diagram illustrating in detail the feature extraction model <NUM> and the feature synthesis model <NUM> shown in <FIG>.

As illustrated in <FIG>, the feature extraction model <NUM> may include a plurality of convolution layers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> for extracting a feature map of the first image <NUM>. In an embodiment, each of the plurality of convolution layers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> performs a convolution process on input data by using a predetermined number of filter kernels of a predetermined size.

In an embodiment, at least some of the plurality of convolution layers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> may include an activation layer and/or a pooling layer. An activation layer may assign non-linear features to a result output from a previous layer. The activation layer may use an activation function. The activation function may include a sigmoid function, a Tanh function, a rectified linear unit (ReLU) function, or the like, but is not limited thereto.

The feature synthesis model <NUM> may include a plurality of deconvolution layers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> which each take, as an input, feature maps that have undergone style transformation and data output from a previous layer, and then perform a convolution process.

In an embodiment, each of the plurality of deconvolution layers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> performs a convolution process on input data by using a predetermined number of filter kernels of a predetermined size. Furthermore, in an embodiment, at least some of the plurality of deconvolution layers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> may include an activation layer and/or a pooling layer.

The first deconvolution layer <NUM> among the plurality of deconvolution layers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> takes, as an input, a feature map output from the fifth convolution layer <NUM> and a feature map that has undergone style transformation after being output from the fifth convolution layer <NUM>, and then performs a convolution process.

The second deconvolution layer <NUM> takes, as an input, a feature map output from the first deconvolution layer <NUM> and a feature map that has undergone style transformation after being output from the fourth convolution layer <NUM>, and then performs a convolution process.

The third deconvolution layer <NUM> takes, as an input, a feature map output from the second deconvolution layer <NUM> and a feature map that has undergone style transformation after being output from the third convolution layer <NUM>, and then performs a convolution process.

The fourth deconvolution layer <NUM> takes, as an input, a feature map output from the third deconvolution layer <NUM> and a feature map that has undergone style transformation after being output from the second convolution layer <NUM>, and then performs a convolution process.

The fifth deconvolution layer <NUM> takes, as an input, a feature map output from the fourth deconvolution layer <NUM> and a feature map that has undergone style transformation after being output from the first convolution layer <NUM>, and then performs a convolution process.

Although <FIG> shows that the feature extraction model <NUM> includes the five convolution layers <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>, and the feature synthesis model <NUM> includes the five deconvolution layers <NUM>, <NUM>, <NUM>, <NUM> and <NUM>, this is merely an example, and the number of convolution layers that may be included in the feature extraction model <NUM> and the number of deconvolution layers that may be included in the feature synthesis model <NUM> may be changed variously. In addition, in an embodiment, the feature extraction model <NUM> and/or the feature synthesis model <NUM> may further include a fully connected layer (FCL).

Moreover, in an embodiment, the image processing apparatus <NUM> may select a target reference image having a high relevance to the first image <NUM> from among reference images included in any one of a plurality of reference image groups.

The reference image groups may be classified according to a creator of each reference image. For example, the plurality of reference image groups may include a reference image group including reference images generated by creator A, a reference image group including reference images generated by creator B, etc..

The image processing apparatus <NUM> may display information of each reference image group on the display, and select a target reference image having a high relevance to the first image <NUM> from among reference images included in a reference image group selected by the user.

For this purpose, as shown in <FIG>, information representing each reference image group (e.g., creator's identification information, etc.) may be displayed on a display <NUM>, and the user may select information representing one reference image group.

Because each image creator has his or her own style, the user may capture images according to a preferred creator's style.

As an example, when the image processing apparatus <NUM> is implemented as a user device, the image processing apparatus <NUM> may display the information of each reference image group on its own display. In addition, as an example, when the image processing apparatus <NUM> is implemented as a server, the server may transmit the information of each reference image group to the user device so that the user device may display the information on its display.

Moreover, in an embodiment, when the image processing apparatus <NUM> is implemented as a user device, assignment of visual sentiment labels to reference images, training of the recognition model <NUM>, and extraction of reference style data may be performed by a server. In other words, processes requiring reference images themselves, such as assignment of visual sentiment labels to reference images, training of the recognition model <NUM>, and extraction of reference style data, may be performed by the server instead of the user device. Visual sentiment labels assigned to reference images, reference style data for the reference images, and data related to the recognition model may be transmitted from the server to the user device. The user device may input the first image <NUM> to the recognition model <NUM>, obtain reference style data for a target reference image according to a result of comparing labels of the first image <NUM> with visual sentiment labels of reference images, and generate the second image <NUM> according to second style data generated based on first style data and reference style data.

In addition, as described above, when reference images are classified into reference image groups, the user device may further store internal parameter information of the recognition model <NUM>, which corresponds to each reference image group. The server may individually train the recognition model <NUM> for each reference image group and transmit, to the user device, internal parameter information of the recognition model <NUM>, which corresponds to each reference image group. For example, the recognition model <NUM> may be trained based on reference images included in reference image group A to have internal parameters optimized for the reference image group A, or may be trained based on reference images included in reference image group B to have internal parameters optimized for the reference image group B. The user device may control the recognition model <NUM> to operate according to internal parameters corresponding to a reference image group selected by the user. For example, the image processing apparatus <NUM> may change internal parameters of the recognition model <NUM> to internal parameters corresponding to a reference image group selected by the user and control labels of the first image <NUM> to be extracted from the recognition model <NUM> that has the changed internal parameters.

Referring to <FIG>, as a user device <NUM> is connected to a server <NUM>, the server <NUM> transmits a reference image group list to the user device <NUM>, and the user device <NUM> requests the server <NUM> for data related to a reference image group selected by the user from the reference image group list. The server <NUM> may transmit, to the user device <NUM>, visual sentiment labels <NUM> of reference images included in the reference image group selected by the user, reference style data <NUM> for the reference images, and internal parameters <NUM> of the recognition model <NUM>. In other words, as shown in <FIG>, because instead of the reference images themselves, the server <NUM> transmits, to the user device <NUM>, only the visual sentiment labels <NUM> of the reference images, the reference style data <NUM> therefor, and the internal parameters <NUM> corresponding to each reference image group, the amount of data stored in the user device <NUM> may be reduced.

Furthermore, in an embodiment, when the image processing apparatus <NUM> is implemented as a server, assignment of visual sentiment labels to reference images, training of the recognition model <NUM>, and extraction of reference style data for the reference images may be performed by the server. The server may generate the second image <NUM> by performing style transformation on the first image <NUM> received from a user device, and transmit the second image <NUM> to the user device.

In addition, in an embodiment, even when the image processing apparatus <NUM> is implemented as a user device, assignment of visual sentiment labels to reference images, training of the recognition model <NUM>, extraction of reference style data for the reference images, etc. may be performed by the user device. The user device may generate the second image <NUM> by performing style transformation on the first image <NUM>.

<FIG> is a flowchart of an image processing method according to another embodiment.

In operation S1210, the image processing apparatus <NUM> selects at least one reference image related to the first image <NUM> from among a plurality of reference images as at least one candidate reference image. Here, the image processing apparatus <NUM> may select at least one candidate reference image having a high relevance to the first image <NUM> by comparing features of the plurality of reference images (at least one of brightness, contrast, viewpoint, tone, white balance, sharpness, histogram, and feature map) with features of the first image <NUM>.

In operation S1220, the image processing apparatus <NUM> displays at least one thumbnail image corresponding to the at least one candidate reference image on the display. The image processing apparatus <NUM> may display a thumbnail image obtained by reducing a size of each candidate reference image on the display, or display a thumbnail image obtained when a style of the first image <NUM> is changed according to each candidate reference image.

As an example, when the image processing apparatus <NUM> is implemented as a user device, the image processing apparatus <NUM> may display a thumbnail image on its own display. In addition, as an example, when the image processing apparatus <NUM> is implemented as a server, the server may transmit a thumbnail image to the user device so that the user device displays the thumbnail image on its display.

In operation S1230, the image processing apparatus <NUM> selects, as a target reference image, a candidate reference image corresponding to a thumbnail image selected by the user from among at least one thumbnail image displayed on the display. The user may select one of the thumbnail images displayed on the display by using a method such as touching, etc..

In operation S1240, the image processing apparatus <NUM> may generate the second image <NUM> by changing the first image <NUM> according to features of the target reference image. For example, the image processing apparatus <NUM> may generate the second image <NUM> by changing the first image <NUM> so that features of the first image <NUM> are the same as/similar to features of the target reference image.

<FIG> is a block diagram of a configuration of an image processing apparatus <NUM> according to an embodiment.

Referring to <FIG>, the image processing apparatus <NUM> may include a memory <NUM>, a communicator <NUM>, and a processor <NUM>. The memory <NUM>, the communicator <NUM>, and the processor <NUM> may operate according to programs stored in the memory <NUM>.

The memory <NUM> may store the recognition model <NUM>. Also, the memory <NUM> may further store the feature extraction model <NUM> and the feature synthesis model <NUM>. In addition, the memory <NUM> may store data related to reference images, such as visual sentiment labels assigned to the reference images, reference style data for the reference images, and internal parameters for each reference image group.

The communicator <NUM> transmits and receives data to and from an external device over a network.

The processor <NUM> inputs the first image <NUM> to the recognition model <NUM> to obtain labels and/or probability values of the labels output from the recognition model <NUM>. The processor <NUM> obtains reference style data for a target reference image having a high degree of relevance to the first image <NUM> from among reference images, and generate second style data based on first style data for the first image <NUM> and the reference style data. In addition, the processor <NUM> generates the second image <NUM> based on the second style data.

In an embodiment, the processor <NUM> may train the recognition model <NUM> based on reference images. The processor <NUM> may input reference images and visual sentiment labels assigned to the reference images to the recognition model <NUM> so that internal parameters of the recognition model <NUM> are updated.

Hereinafter, a detailed configuration of the processor <NUM> when the processor <NUM> trains the recognition model <NUM> will be described with reference to <FIG>.

<FIG> is a block diagram of the processor <NUM> according to some embodiments.

Referring to <FIG>, the processor <NUM> according to some embodiments may include a data learner <NUM> and a data recognizer <NUM>.

The data learner <NUM> may learn a criterion for determining a situation (e.g., determining a label corresponding to an image). The data learner may learn criteria with respect to what data will be used to determine a certain situation and how to determine the situation by using data. The data learner <NUM> may obtain data to be used for training and learn a criterion for determining a situation by applying the obtained data to the recognition model <NUM>.

The data recognizer <NUM> may recognize a situation based on data. The data recognizer <NUM> may recognize a situation from data by using the trained recognition model <NUM>. The data recognizer <NUM> may obtain data according to learned preset criteria and determine a certain situation based on data by using the recognition model <NUM> that takes the obtained data as an input value. Furthermore, a resulting value output by the recognition model <NUM> that takes the obtained data as an input value may be used to update the recognition model <NUM>.

At least one of the data learner <NUM> and the data recognizer <NUM> may be fabricated in the form of at least one hardware chip and be mounted in an electronic device. For example, at least one of the data learner <NUM> and the data recognizer <NUM> may be manufactured in the form of a dedicated hardware chip for AI or as part of an existing general-purpose processor (e.g., a central processing unit (CPU) or application processor (AP)) or dedicated graphics processor (e.g., a graphics processing unit (GPU)) and may be mounted in various electronic devices as described above.

In this case, the data learner <NUM> and the data recognizer <NUM> may be mounted in one electronic device, or be respectively mounted in different electronic devices. For example, one of the data learner <NUM> and the data recognizer <NUM> may be included in the user device <NUM> while the other may be included in the server <NUM>. Furthermore, the data learner <NUM> and the data recognizer <NUM> may communicate with each other via a wire or wirelessly such that information about a model built by the data learner <NUM> may be provided to the data recognizer <NUM> and data input to the data recognizer <NUM> may be provided to the data learner <NUM> as additional training data.

Moreover, at least one of the data learner <NUM> and the data recognizer <NUM> may be implemented as a software module. When the at least one of the data learner <NUM> and the data recognizer <NUM> is implemented as a software module (or a program module including instructions), the software module may be stored in non-transitory computer readable recording media. Furthermore, in this case, the at least one software module may be provided by an operating system (OS) or application. Alternatively, some of the at least one software module may be provided by the OS while the rest thereof may be provided by the application.

<FIG> is a block diagram of the data learner <NUM> according to some embodiments.

Referring to <FIG>, according to some embodiments, the data learner <NUM> may include a data acquirer <NUM>-<NUM>, a preprocessor <NUM>-<NUM>, a training data selector <NUM>-<NUM>, a model trainer <NUM>-<NUM>, and a model evaluator <NUM>-<NUM>.

The data acquirer <NUM>-<NUM> may obtain data necessary for determining a situation. The data acquirer <NUM>-<NUM> may obtain data necessary to perform training for determining a situation.

The data acquirer <NUM>-<NUM> may obtain image data captured by a camera or received from an external device via a network. Alternatively, the data acquirer <NUM>-<NUM> may obtain preview image data recognized by a camera.

The preprocessor <NUM>-<NUM> may preprocess the obtained data such that the obtained data may be used in training for determining a situation. The preprocessor <NUM>-<NUM> may process the obtained data into a preset format so that the model trainer <NUM>-<NUM> to be described later may use the obtained data in training for determining a situation.

The training data selector <NUM>-<NUM> may select data necessary for training from among the preprocessed data. The selected data may be provided to the model trainer <NUM>-<NUM>. The training data selector <NUM>-<NUM> may select data necessary for training from among the preprocessed data according to preset criteria for determining a situation. Furthermore, the training data selector <NUM>-<NUM> may select data according to preset criteria learned by the model trainer <NUM>-<NUM> to be described later.

The model trainer <NUM>-<NUM> may learn a criterion with respect how to determine a situation based on training data. Furthermore, the model trainer <NUM>-<NUM> may learn a criterion with respect to what training data needs to be used for determining a situation.

Furthermore, the model trainer <NUM>-<NUM> may train the recognition model <NUM> used to determine a situation by using training data. In this case, the recognition model <NUM> may be a pre-built model. For example, the recognition model <NUM> may be a model pre-built by taking basic training data (e.g., sample images, etc.) as an input.

The recognition model <NUM> may be built by taking into account an application field of the recognition model <NUM>, an objective of learning, or a computer performance of a device. For example, the recognition model <NUM> may be a model based on a neural network. Models such as a DNN, a recurrent neural network (RNN), and a bidirectional recurrent DNN (BRDNN) may be used as the recognition model <NUM>, but embodiments are limited thereto.

According to various embodiments, when there are a plurality of pre-built recognition models <NUM>, the model trainer <NUM>-<NUM> may determine the recognition model <NUM> having a high correlation between input training data and basic training data as the recognition model <NUM> to be trained. In this case, the basic training data may be pre-classified according to a type of data, and the recognition model <NUM> may be pre-built for each data type. For example, the basic training data may be pre-classified based on various criteria such as an area where the training data is generated, a time at which the training data is generated, a size of the training data, a genre of the training data, a creator of the training data, a type of an object in the training data, etc..

Furthermore, the model trainer <NUM>-<NUM> may train the recognition model <NUM> by using, for example, a learning algorithm including error back-propagation or gradient descent.

Furthermore, the model trainer <NUM>-<NUM> may train the recognition model <NUM>, for example, through supervised learning using training data as an input value. Furthermore, the model trainer <NUM>-<NUM> may train the recognition model <NUM>, for example, via unsupervised learning that allows the recognition model <NUM> to discover a criterion for determining a situation by self-learning types of data necessary for determining the situation without any special guidance. Furthermore, the model trainer <NUM>-<NUM> may train the recognition model <NUM> by using, for example, reinforcement learning exploiting feedback regarding whether a result of determining a situation via training is correct.

In addition, when the recognition model <NUM> is trained, the model trainer <NUM>-<NUM> may store the trained recognition model <NUM>. In this case, the model trainer <NUM>-<NUM> may store the trained recognition model <NUM> in a memory of an electronic device including the data recognizer <NUM>. Alternatively, the model trainer <NUM>-<NUM> may store the trained recognition model <NUM> in a memory of an electronic device including the data recognizer <NUM> to be described later. Alternatively, the model trainer <NUM>-<NUM> may store the trained recognition model <NUM> in a memory of the server <NUM> connected via a wire or wirelessly to an electronic device.

In this case, the memory in which the trained recognition model <NUM> is stored may also store commands or data related to at least one other component of the electronic device. Furthermore, the memory may also store software and/or programs. For example, the programs may include kernel, middleware, application programing interface (API) and/or application program (or "application").

The model evaluator <NUM>-<NUM> may input evaluation data to the recognition model <NUM> and cause the model trainer <NUM>-<NUM> to train again the recognition model <NUM> when a recognition result obtained from the evaluation data does not satisfy a preset criterion. In this case, the evaluation data may be preset data for evaluating the recognition model <NUM>.

For example, when the number or ratio of pieces of evaluation data with respect to which recognition results are not accurate from among recognition results output from the trained recognition model <NUM> with respect to evaluation data exceeds a preset threshold, the model evaluator <NUM>-<NUM> may evaluate that the preset criterion is not satisfied. For example, when the preset criterion is defined as a ratio of <NUM>%, and when the trained recognition model <NUM> outputs wrong recognition results with respect to more than <NUM> pieces of evaluation data among a total of <NUM> pieces of evaluation data, the model evaluator <NUM>-<NUM> may evaluate the trained recognition model <NUM> as not being suitable.

Moreover, when a plurality of trained recognition models <NUM> exist, the model evaluator <NUM>-<NUM> may evaluate whether each of the trained recognition models <NUM> satisfies a preset criterion, and determine a model that has satisfied the preset criterion as the final recognition model <NUM>. In this case, when there are a plurality of models that has satisfied the preset criterion, the model evaluator <NUM>-<NUM> may determine, as the final recognition model <NUM>, one model or a certain number of models preset in order from highest to lowest evaluation scores.

Moreover, at least one of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the training data selector <NUM>-<NUM>, the model trainer <NUM>-<NUM>, and the model evaluator <NUM>-<NUM> in the data learner <NUM> may be fabricated in the form of at least one hardware chip and be mounted in an electronic device. For example, at least one of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the training data selector <NUM>-<NUM>, the model trainer <NUM>-<NUM>, and the model evaluator <NUM>-<NUM> may be manufactured in the form of a dedicated hardware chip for AI or as part of an existing general-purpose processor (e.g., a CPU or AP) or dedicated graphics processor (e.g., a GPU) and may be mounted in various electronic devices as described above.

Furthermore, the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the training data selector <NUM>-<NUM>, the model trainer <NUM>-<NUM>, and the model evaluator <NUM>-<NUM> may be mounted in one electronic device, or be respectively mounted in different electronic devices. For example, some of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the training data selector <NUM>-<NUM>, the model trainer <NUM>-<NUM>, and the model evaluator <NUM>-<NUM> may be included in the user device <NUM> while the rest thereof may be included in the server <NUM>.

In addition, at least one of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the training data selector <NUM>-<NUM>, the model trainer <NUM>-<NUM>, and the model evaluator <NUM>-<NUM> may be implemented as a software module. When the at least one of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the training data selector <NUM>-<NUM>, the model trainer <NUM>-<NUM>, and the model evaluator <NUM>-<NUM> is implemented as a software module (or a program module including instructions), the software module may be stored in non-transitory computer readable recording media. Furthermore, in this case, the at least one software module may be provided by an OS or application. Alternatively, some of the at least one software module may be provided by the OS while the rest thereof may be provided by the application.

<FIG> is a block diagram of the data recognizer <NUM> according to some embodiments.

Referring to <FIG>, according to some embodiments, the data recognizer <NUM> may include a data acquirer <NUM>-<NUM>, a preprocessor <NUM>-<NUM>, a recognition data selector <NUM>-<NUM>, a recognition result provider <NUM>-<NUM>, and a model refiner <NUM>-<NUM>.

The data acquirer <NUM>-<NUM> may obtain data necessary for determining a situation, and the preprocessor <NUM>-<NUM> may preprocess the obtained data such that the obtained data may be used for determining a situation. The preprocessor <NUM>-<NUM> may process the obtained data into a preset format such that the recognition result provider <NUM>-<NUM> to be described later may use the obtained data for determining a situation.

The recognition data selector <NUM>-<NUM> may select, from among pieces of preprocessed data, data necessary for determining a situation. The selected data may be provided to the recognition result provider <NUM>-<NUM>. The recognition data selector <NUM>-<NUM> may select some or all of the pieces of preprocessed data according to preset criteria for determining a situation. The recognition data selector <NUM>-<NUM> may select data according to preset criteria learned by the model trainer <NUM>-<NUM>.

The recognition result provider <NUM>-<NUM> may determine a situation by applying the selected data to the recognition model <NUM>. The recognition result provider <NUM>-<NUM> may provide a recognition result according to a purpose of data recognition. The recognition result provider <NUM>-<NUM> may apply the data selected by the recognition data selector <NUM>-<NUM> to the recognition model <NUM> by using the selected data as an input value. Furthermore, the recognition result may be determined by the recognition model <NUM>. For example, the recognition result provider <NUM>-<NUM> may apply the first image <NUM> selected by the recognition data selector <NUM>-<NUM> to the recognition model <NUM>. Labels corresponding to the first image <NUM> and probability values of the labels may be determined as the recognition result.

The model refiner <NUM>-<NUM> may update the recognition model <NUM> based on evaluation of the recognition result provided by the recognition result provider <NUM>-<NUM>. For example, the model refiner <NUM>-<NUM> may provide the recognition result from the recognition result provider <NUM>-<NUM> to the model trainer <NUM>-<NUM> so that the model trainer <NUM>-<NUM> may update the recognition model <NUM>.

Moreover, at least one of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the recognition data selector <NUM>-<NUM>, the recognition result provider <NUM>-<NUM>, and the model refiner <NUM>-<NUM> in the data recognizer <NUM> may be fabricated in the form of at least one hardware chip and be mounted in an electronic device. For example, at least one of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the recognition data selector <NUM>-<NUM>, the recognition result provider <NUM>-<NUM>, and the model refiner <NUM>-<NUM> may be manufactured in the form of a dedicated hardware chip for AI or as part of an existing general-purpose processor (e.g., a CPU or AP) or dedicated graphics processor (e.g., a GPU) and may be mounted in various electronic devices as described above.

Furthermore, the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the recognition data selector <NUM>-<NUM>, the recognition result provider <NUM>-<NUM>, and the model refiner <NUM>-<NUM> may be mounted in one electronic device, or be respectively mounted in different electronic devices. For example, some of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the recognition data selector <NUM>-<NUM>, the recognition result provider <NUM>-<NUM>, and the model refiner <NUM>-<NUM> may be included in the user device <NUM> while the rest thereof may be included in the server <NUM>.

In addition, at least one of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the recognition data selector <NUM>-<NUM>, the recognition result provider <NUM>-<NUM>, and the model refiner <NUM>-<NUM> may be implemented as a software module. When the at least one of the data acquirer <NUM>-<NUM>, the preprocessor <NUM>-<NUM>, the recognition data selector <NUM>-<NUM>, the recognition result provider <NUM>-<NUM>, and the model refiner <NUM>-<NUM> is implemented as a software module (or a program module including instructions), the software module may be stored in non-transitory computer readable recording media. Furthermore, in this case, the at least one software module may be provided by an OS or application. Alternatively, some of the at least one software module may be provided by the OS while the rest thereof may be provided by the application.

<FIG> is a diagram illustrating an example in which the user device <NUM> and the server <NUM> interwork with each other to learn and recognize data, according to some embodiments.

Referring to <FIG>, the server <NUM> may learn criteria for determining a situation (e.g., determining a label corresponding to an image), and the user device <NUM> may determine a situation based on a result of the learning by the server <NUM>.

In this case, a model trainer <NUM> of the server <NUM> may perform functions of the data learner <NUM> shown in <FIG>. The model trainer <NUM> of the server <NUM> may learn criteria with respect to what data will be used to determine a certain situation and how to determine the situation by using data. The model trainer <NUM> may obtain data to be used for training and learn criteria for determining a situation by applying the obtained data to the recognition model <NUM>.

Furthermore, the recognition result provider <NUM>-<NUM> of the user device <NUM> may determine a situation by applying data selected by the recognition data selector <NUM>-<NUM> to the recognition model <NUM> generated by the server <NUM>. For example, the recognition result provider <NUM>-<NUM> may transmit data selected by the recognition data selector <NUM>-<NUM> to the server <NUM> and request the server <NUM> to determine a situation by applying the data selected by the recognition data selector <NUM>-<NUM> to the recognition model <NUM>. Furthermore, the recognition result provider <NUM>-<NUM> may receive, from the server <NUM>, information about the situation determined by the server <NUM>.

Alternatively, the recognition result provider <NUM>-<NUM> of the user device <NUM> may receive the recognition model <NUM> generated by the server <NUM> from the server <NUM> and determine the situation by using the received recognition model <NUM>. In this case, the recognition result provider <NUM>-<NUM> of the user device <NUM> may determine the situation by applying the data selected by the recognition data selector <NUM>-<NUM> to the recognition model <NUM> received from the server <NUM>.

Moreover, the above-described embodiments of the present disclosure may be written as computer-executable programs, and the written programs may be stored in a medium.

The medium may be a medium for continuously storing the computer-executable programs or temporarily storing the computer-executable programs for execution or downloading. Furthermore, the medium may be any recording medium or storage medium in which a single piece or plurality of pieces of hardware are combined, and the medium is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, ROM, RAM, flash memory, and the like, and may be configured to store program instructions. Other examples of the medium may include recording media or storage media managed by application stores distributing applications or by websites, servers, etc., supplying or distributing other various types of software.

Claim 1:
A computer-implemented image processing method comprising:
obtaining a plurality of first labels (<NUM>) of a first image (<NUM>) and probability values (<NUM>) respectively corresponding to the plurality of first labels by applying the first image to a recognition model (<NUM>), wherein the first labels are visual sentiment labels;
for each of at least one candidate reference image to which at least one label common to the at least one first label among the plurality of first labels of the first image is assigned, calculating a degree of relevance to the first image by calculating an average of probability values respectively corresponding to the common at least one first label as the degree of relevance;
obtaining reference style data for a target reference image which is selected from among the at least one candidate reference image based on the calculated degree of relevance;
generating a second style data based on first style data for the first image and the obtained reference style data; and
generating a second image (<NUM>) based on the generated second style data.