Patent Description:
An image may be recognized through an image recognition model trained through sample data. However, before the image recognition model is used to recognize an image, training of the image recognition model is performed with image samples. In the related art, an image sample, after being selected, is required to be manually tagged. For ensuring an adequate recognition capability of an image recognition model, a large number of tagged image samples are required for training, while if completely manual tagging is adopted, it is apparent that the image recognition model has the problems of low training efficiency, high labor cost and the like.

<NPL> discloses a sampling by uncertainty and density technique, in which a k-Nearest-Neighbor-based density measure is adopted to determine whether an unlabeled example is an outlier. Futher, a technique of sampling by clustering is applied to build a representative initial training data set for active learning. <CIT> describes apparatuses, systems, and methods for adaptive batch mode active learning for evolving a classifier. <CIT> discloses a training set that includes training samples and corresponding assigned classification labels is obtained, and an automated classifier is trained against the training set. At least one of the training samples is selected and confirmation/re-labeling of it is requested. In response, a reply classification label is received and is used to retrain the automated classifier. <NPL>, discloses an entropy-based active learning approach.

The present disclosure provides a method and device for training an image recognition model, and a storage medium.

A first aspect of embodiments of the present disclosure provides a method for training an image recognition model according to claim <NUM>.

A second aspect of the embodiments of the present disclosure provides a device for training an image recognition model according to claim <NUM>.

A third aspect of the embodiments of the present disclosure provides a storage medium according to claim <NUM>, having computer-executable instruction stored therein, where the computer-executable instructions, when being executed by a processor, enable the processor to implement the steps of the method for training an image recognition model provided in any embodiment of the first aspect.

The technical solutions provided in the embodiments of the present disclosure may have the following beneficial effects. In the embodiments of the present disclosure, an image recognition model with an initial image recognition capability is obtained by training at first by use of a small number of tagged positive image samples and negative image samples. Then, to-be-tagged image samples in the first to-be-tagged image sample set are recognized by using the image recognition model, and the to-be-tagged image samples of which the confidence is within the preset interval are selected according to the confidence of the recognition result obtained by the image recognition model. The selected to-be-tagged image samples, i.e., images for which the present recognition capability of the image recognition model needs to be enhanced is tagged manually, and the selected to-be-tagged image samples with a high recognition confidence is not required to be tagged manually since already recognized by the image recognition model with certain confidence, so that the number of image samples required to be tagged in a model training process is reduced, and the manually tagging workload is reduced. In this way, training of the image recognition model can be achieved by tagging a small number of image samples, and the trained image recognition model is of high accuracy; on the other hand, the training speed of the image recognition model is increased, and the training cost of the image recognition model is reduced.

The implementations set forth in the following description of exemplary embodiments do not represent all implementations consistent with the embodiments of the present disclosure. Instead, they are merely examples of apparatuses and methods consistent with aspects related to the embodiments of the present disclosure as recited in the appended claims.

Terms used in the embodiments of the present disclosure are only adopted for the purpose of describing specific embodiments and not intended to limit the embodiments of the present disclosure. "A/an", "said" and "the" in a singular form in the embodiments of the present disclosure and the appended claims are also intended to include a plural form, unless other meanings are clearly denoted throughout the present disclosure. It is also to be understood that term "and/or" used in the present disclosure refers to and includes one or any or all possible combinations of multiple associated items that are listed.

It is to be understood that, although terms first, second, third and the like may be adopted to describe various information in the embodiments of the present disclosure, the information should not be limited to these terms. These terms are only adopted to distinguish the information of the same type. For example, without departing from the scope of the embodiments of the present disclosure, first information may also be called second information and, similarly, second information may also be called first information. For example, term "if" used here may be explained as "while" or "when" or "responsive to determining", which depends on the context.

As shown in <FIG>, an embodiment provides an image recognition model training method, which includes the following steps.

In S <NUM>, an image recognition model is trained by using a tagged image sample set. For example, the tagged image sample set may be a set of sample images that have been tagged by a user and may be used to train the image recognition model to identify images based on objects, people, and places in the image.

In S <NUM>, multiple to-be-tagged image samples in a to-be-tagged image sample set are recognized by using the presently trained image recognition model to obtain a confidence level of a recognition result corresponding to each of the multiple to-be-tagged image samples. For example, the image recognition model may identify multiple to-be-tagged image samples in a to-be-tagged image sample set using a confidence level.

In S <NUM>, at least one to-be-tagged image sample of which the confidence level is within a preset interval is selected to form a first to-be-tagged image sample set.

In S14, tags of the to-be-tagged image samples in the first to-be-tagged image sample set are acquired.

In S15, the tagged image sample set is updated according to the acquired tags.

In S16, the image recognition model is continued to be trained by using the updated tagged image sample set.

A positive image sample and a negative image sample are two different types of images. For example, the positive image sample is a target type of image required to be recognized, and the negative image sample may be another type of image except the target type of image.

In some embodiments of the present disclosure, a training set may include a tagged image sample set and a to-be-tagged image sample set.

At first, model training is performed by use of the tagged image sample set. In this way, after being trained based on a certain number of image samples in the tagged image sample set, the image recognition model may have an initial recognition capability.

The to-be-tagged image samples in the to-be-tagged image sample set are input into the image recognition model, and the image recognition model may output the recognition results of the to-be-tagged image samples and confidence levels for each of the recognition results. If the confidence level is high, it is indicated that the recognition result given by the image recognition model is more accurate. For example, if the image recognition model is a two-type recognition model, the recognition result output by the image recognition model may include "<NUM>" or " <NUM>". For example, "<NUM>" corresponds to the target type of image, and " <NUM>" corresponds to the other type of image except the target type. For example, when it is determined whether an image is an intensive phobia image, there are two recognition results, i.e., "YES" and "NO". In such case, confidence of the recognition result of a to-be-tagged image sample may range from <NUM> to <NUM>, if the confidence is closer to <NUM>, the recognition accuracy is higher, and if the confidence is closer to <NUM>, the recognition accuracy is lower and the recognition capability of the image recognition model is relatively low for the image sample. If the confidence of a to-be-tagged image sample is <NUM>, it means that the confidence when the image sample is determined to belong to each of the two types is <NUM> respectively. Therefore, such a to-be-tagged image sample is necessary to be tagged to continue model training so as to improve the generalization capability of the image recognition model.

In S13, the to-be-tagged image sample of which the confidence is within the preset interval may form the first to-be-tagged image sample set. The preset interval is from <NUM> to <NUM>, other intervals not claimed are from <NUM> to <NUM>, from <NUM> to <NUM>, or from <NUM> to <NUM>.

In this way, for a to-be-tagged image sample of which the confidence is <NUM> or <NUM>, the recognition capability of the image recognition model is adequate, which indicates that the present tagged image sample set has been provided with enough similar image samples and no further to-be-tagged image sample is required to be tagged.

The confidence level may be a value of a confidence function. For example, the confidence function includes, but not limited to, a softmax function. The confidence level may be a value calculated through the softmax function.

In some embodiments, if an image sample includes a positive image sample and a negative image sample, the to-be-tagged image sample of which the recognition result is a positive image sample and the confidence is within the preset interval is selected to form the first to-be-tagged image sample set in S13. The to-be-tagged image sample of which the recognition result is a negative image sample is not added into the first to-be-tagged image sample set, and thus the number of image samples required to be tagged is reduced.

The operation S14 may include that: the to-be-tagged image sample in the first to-be-tagged image sample set is displayed, and tag information for the tagged image sample is received to obtain the tag.

In condition that the tag of a to-be-tagged image sample is obtained, the to-be-tagged image sample is turned into a tagged image sample, and the tagged image sample is added into the tagged image sample set. Therefore, updating of the tagged image sample set in S15 is implemented.

Next, training of the image recognition model is continued by use of the updated tagged image sample set.

For example, model training is performed by use of the whole updated tagged image sample set. Therefore, on one hand, it is ensured that the recognition capability of the image recognition model for images for which the recognition capability is originally high may be maintained after the training is complete. On the other hand, due to introduction of new tagged image samples, the image recognition model may be endowed with a recognition capability for other images.

For another example, training may also be performed preferably by use of newly added image samples in the tagged image sample set only. After the training is complete, it is determined through a test set whether the recognition capability of the retrained image recognition model for images similar to the image samples is maintained. If the recognition capability is maintained, the old image samples are not used for training, and S12 to S16 may be continued to be executed; and if the recognition capability is not maintained, the training is continued by use of the old image samples. After the training is performed by use of the whole updated tagged image sample set, S12 to S16 are continued to be executed.

In some embodiments of the present disclosure, an image recognition model with an initial image recognition capability is trained at first by using a small number of tagged positive image samples and tagged negative image samples. Then, the to-be-tagged image sample in the first to-be-tagged image sample set is recognized by using the image recognition model, and the to-be-tagged image sample of which the confidence is within the preset interval is selected according to the confidence of the recognition result obtained by the image recognition model. The selected to-be-tagged image samples, i.e., images for which the present recognition capability of the image recognition model needs to be enhanced is tagged, and the selected to-be-tagged image samples with a high recognition confidence are not required to be tagged, so that the number of image samples required to be tagged in a model training process is reduced, and the manual tagging workload is reduced. In this way, training of the image recognition model can be achieved by tagging a small number of image samples, and the trained image recognition model is of high accuracy; on the other hand, the training speed of the image recognition model is increased, and the training cost of the image recognition model is reduced.

In the present disclosure, the image recognition model is trained for online or offline image recognition, and an obtained image recognition result obtained therefrom may be used to recommendation of images for browsing and/or intercept unwanted images. Thus, bad feelings when the user sees unwanted images are reduced, and the user experience is thereby improved.

In some embodiments, as shown in <FIG>, S14 may include the following steps.

In S141, the at least one to-be-tagged image sample in the first to-be-tagged image sample set is clustered to obtain a clustering result.

In S142, part of target to-be-tagged image samples that have cluster representativeness are selected from the first to-be-tagged image sample set according to the clustering result to form a second to-be-tagged image sample set.

In S143, tags of target to-be-tagged image samples in the second to-be-tagged image sample set are acquired.

In the present disclosure, for further reducing the number of the to-be-tagged image samples, clustering may be performed after the first to-be-tagged image sample set is obtained. For example, the to-be-tagged image samples in the first to-be-tagged image sample set may be clustered into N clusters, N being a positive integer. A clustering algorithm may adopt any clustering algorithm. For example, a k-means clustering algorithm is selected for clustering, or, an entropy clustering algorithm is selected for clustering.

Multiple clusters may be formed by clustering. A cluster including more than two to-be-tagged image samples may include multiple similar to-be-tagged image samples, and due to the similarity among the to-be-tagged image samples, part of representative to-be-tagged image samples may be selected from each cluster to form the second to-be-tagged image sample set. It is apparent that the number of the to-be-tagged image samples in the second to-be-tagged image sample set may be smaller than the number of the to-be-tagged image samples in the first to-be-tagged image sample set.

The to-be-tagged image samples having cluster representativeness may represent a corresponding cluster. Multiple images forming a cluster may be distributed differently, some of the multiple images may be located in or closest to the center of the cluster, and some may be around the cluster. The cluster representativeness of the images in the center of the cluster is more representative than those around the cluster and can be a better representative of the corresponding cluster.

Furthermore, S141 includes that: information entropy of a to-be-tagged image sample is determined according to image features of the to-be-tagged image sample in the first to-be-tagged image sample set; and
the to-be-tagged image sample of which the information entropy meets a predetermined condition is selected and clustered to obtain K clusters and a cluster center of each cluster, K being a positive integer.

During clustering, the clustering is not performed on all to-be-tagged image samples in the first to-be-tagged image sample. Instead, the information entropies are calculated, and the to-be-tagged image sample of which the information entropy meets the predetermined condition is selected for clustering. In such a manner, the to-be-tagged image samples are filtered once before clustering, and thus the to-be-tagged image samples in the second to-be-tagged image sample set are reduced.

For example, the operation that the to-be-tagged image sample of which the information entropy meets the predetermined condition is selected for clustering includes one of the following operations:.

The information entropy is determined according to the image feature of the to-be-tagged image sample in the first to-be-tagged image sample set. If the information entropy is greater than an entropy threshold value, the amount of information in the corresponding to-be-tagged image sample is larger, and it is more meaningful to adopt it for training the image recognition model.

The cluster center may be as follows: multiple to-be-tagged image samples are mapped into a predetermined space, each to-be-tagged image sample is distributed in the form of points in the predetermined space, where clusters are formed at which the points are concentrated, and the cluster center is a center coordinate of a cluster or a coordinate of the to-be-tagged image sample represented by the center point in the cluster.

The to-be-tagged image sample at a shorter distance from the cluster center is more representative than the to-be-tagged image sample at a relatively long distance from the cluster center.

Herein, the distance includes, but not limited to, a Euclidean distance.

The clustering result includes multiple clusters formed by clustering and the cluster center of each cluster. The operation S142 includes that: one or more to-be-tagged image samples closest to the corresponding cluster center are selected from each of the multiple clusters as the target to-be-tagged image samples having the cluster representativeness, to form the second to-be-tagged image sample set.

A predetermined number of representative to-be-tagged image samples are selected from a cluster to form the second to-be-tagged image sample set.

If S1 to-be-tagged image samples are clustered and the number of the representative to-be-tagged image samples in each cluster may be S2, where S2 is usually smaller than S1. In this way, the number of the to-be-tagged image samples contained in the second to-be-tagged image sample set are further reduced.

In some embodiments, the method further includes that:
after the first to-be-tagged image sample set is formed, whether the number of the to-be-tagged image sample in the first to-be-tagged image sample set is greater than a preset value is determined.

The operation S14 may include that: when the number of the to-be-tagged image sample in the first to-be-tagged image sample set is greater than the preset value, the tags of the to-be-tagged image samples in the first to-be-tagged image sample set are acquired.

For the presently formed first to-be-tagged image sample set, the number of the to-be-tagged image samples in the present first to-be-tagged image sample set is smaller than the preset value. For example, the preset value may be any predetermined value, and the preset value may be a single digit, tens digit or hundreds digit, etc. The preset value may be a predetermined percentage of the to-be-tagged image samples in the to-be-tagged image sample set. For example, the percentage is <NUM>%, and if the to-be-tagged image sample set includes <NUM>,<NUM> to-be-tagged image samples, the preset value may be <NUM>. If the number of the to-be-tagged image samples collected in the first to-be-tagged image sample set obtained in S11 to S13 is smaller than <NUM>, it is considered that the recognition capability of the present image recognition model has been high enough and training of the image recognition model may be stopped. In such case, the to-be-tagged image samples in the first to-be-tagged image sample set may not be output anymore to acquire the tags.

If the number of the to-be-tagged image samples in the first to-be-tagged image sample set is greater than the preset value, it is indicated that further training of the image recognition model is required, and then the tags of the to-be-tagged image samples in the first to-be-tagged image sample set may be acquired.

In a possible implementation not claimed, all the to-be-tagged image samples in the whole first to-be-tagged image sample set may be output to obtain the tags. In some other implementations for further reducing the number of the to-be-tagged image samples, part of representative to-be-tagged image samples in the first to-be-tagged image sample set may be selected in a clustering manner to be tagged, so as to obtain the tags. In such a manner, through acquisition of the tags of part of to-be-tagged image samples, training of the image recognition model is continued, so that on one hand, the number of the image samples required to be tagged is further reduced, and on the other hand, the accuracy of the trained image recognition model is also ensured.

In some embodiments, the method further includes that:
when the number of the to-be-tagged image sample in the first to-be-tagged image sample set is less than or equal to the preset value, training of the image recognition model is stopped.

If the number of the to-be-tagged image samples in the first to-be-tagged image sample set is less than or equal to the preset value, it is indicated that the accuracy of the image recognition model is high enough, and at this time, unnecessary training can be stopped.

In some embodiments, S16 may include that: the step that the image recognition model is trained is re-executed by use of the updated tagged image sample set. Herein, after the tagged image sample set is updated, the method returns to S11 to iteratively train the image recognition model using the updated tagged image sample set. The training process including the above steps S11 to S16 may be repeated multiple times. In some embodiments, the tagged image sample set includes a first number of positive image samples and a second number of negative image samples, the first number being smaller than the second number.

For example, for an intensive phobia image, the positive image sample is an image shown in <FIG>, and the negative image sample is an image that would not bring bad feelings, for example, as shown in <FIG>. The intensive phobia image is an image where a distribution density of identical or similar graphic elements is greater than a preset value. Under some conditions, an intensive phobia image is related to a subjective feeling of a person and refers to an image including a large number of identical or similar graphic elements. The positive image sample includes an intensive phobia image tagged as an intensive image, and the negative image sample is an image except the intensive phobia image.

The positive image sample may be a pornographic image, and the negative image sample may be any normal image except the pornographic image. For example, the pornographic image may be an image in which a specific part of a human body is naked.

The positive image sample may be a violence-related image, and the negative image sample may be any normal image except the violence-related image. For example, the violence-related image may be an image related to a bloody scene.

The image recognition model may be an image recognition model that distinguishes images based on frequencies of the times of occurrence of repeated elements. Image types include, but not limited to, intensive phobia images or normal images.

A small number of images of the target type such as intensive phobia images may be contained in a large number of normal images. In view of this, in some embodiments of the present disclosure, the number of the positive image samples is smaller than the number of the negative image samples, and thus the accuracy of recognition is relatively high for the trained image recognition model.

In some embodiments, the first to-be-tagged image sample set includes a third number of to-be-tagged image samples, the third number being larger than the second number.

Herein, the third number is larger than the second number. The third number may usually be far larger than the second number. For example, the third number may exceed the second number by one or more orders of magnitude. Therefore, manual tagging may be reduced as much as possible.

As shown in <FIG>, an embodiment provides a device for training an image recognition model, which includes:.

The training module <NUM> is further configured to continue training the image recognition model by using the updated tagged image sample set.

The training module <NUM>, the recognition module <NUM>, the selection module <NUM>, the acquisition module <NUM> and the updating module <NUM> may be program modules. The program modules, upon being executed by a processor, are enabled to obtain an image recognition model with high recognition accuracy trained by tagging as few image samples as possible.

The training module <NUM>, the recognition module <NUM>, the selection module <NUM>, the acquisition module <NUM> and the updating module <NUM> may be combined software and hardware modules. The combined software and hardware modules may include various programmable arrays. The programmable arrays include, but not limited to, a complex programmable array or a field programmable array.

The training module <NUM>, the recognition module <NUM>, the selection module <NUM>, the acquisition module <NUM> and the updating module <NUM> may be pure hardware modules. The pure hardware modules may include an Application Specific Integrated Circuit (ASIC).

The acquisition module <NUM> is specifically configured to cluster the at least one to-be-tagged image sample in the first to-be-tagged image sample set to obtain a clustering result, select part of target to-be-tagged image samples having cluster representativeness from the first to-be-tagged image sample set according to the clustering result to form a second to-be-tagged image sample set; and
acquire tags of all or part of the target to-be-tagged image samples in the second to-be-tagged image sample set.

The clustering result includes one or more clusters formed by clustering and a cluster center of each of the one or more clusters.

The acquisition module <NUM> is specifically configured to determine information entropy of the to-be-tagged image sample according to image features of the to-be-tagged image sample in the first to-be-tagged image sample set and select and cluster to-be-tagged image samples of which the information entropy meets a predetermined condition to obtain K clusters and a cluster center of each cluster, K being a positive integer.

The acquisition module <NUM> is further specifically configured to select, from each of the one or more clusters, one or more to-be-tagged image samples closest to the corresponding cluster center as the target to-be-tagged image samples having the cluster representativeness, to form the second to-be-tagged image sample set.

In some embodiments, the device further includes:
a stopping module, configured to, when the number of the to-be-tagged image sample in the first to-be-tagged image sample set is less than or equal to the preset value, stop training of the image recognition model.

In some embodiments, the training module <NUM> is further configured to re-execute the step of training the image recognition model by using the updated tagged image sample set.

In some embodiments, the tagged image sample set includes a first number of positive image samples and a second number of negative image samples, the first number being smaller than the second number; and/or, the first to-be-tagged image sample set includes a third number of to-be-tagged image samples, the third number being larger than the second number.

In some embodiments, the positive image sample includes an intensive phobia image tagged as an intensive image, and the negative image sample is an image except the intensive phobia image.

A specific example will be provided below in combination with any abovementioned embodiment.

The example relates to an active deep learning-based method for tagging and recognizing intensive phobia picture, which may include the following contents.

In an information flow recommendation system (for example, Miui browser, Tencent News and Toutiao), the recommended contents are usually presented as texts and pictures, regardless of the form. Compared with text, a picture has become an important part of a recommended information flow content by virtue of its characteristics of visualization, strong impact and easiness for rapid browsing.

Before a recommended picture content is presented to a user, a reviewing process for the picture is further required. A related reviewing manner is mainly manual reviewing, which is relatively low in efficiency and cannot deal with sharply increasing massive picture data to review and recognize pictures including bad information such as pornographic, violence-related, terrorist-related and politically sensitive information and completely eradicate propagation and recommendation of such pictures including bad information. In addition, for recognition and reviewing of pictures visually discomforting people, for example, intensive phobia pictures, there are relatively few such products on the market, and most of them do not review such pictures (resulting in poor user experiences in recommended contents), or the low-efficiency manual reviewing manner is still adopted, consuming time and labor.

The example aims to efficiently select most useful pictures that possibly belong to an intensive phobia type as few as possible for necessary manual tagging to further continuously expand a tagged sample set, so as to train a deep learning model with increasingly high classification performance to achieve recognition and reviewing of intensive phobia pictures more accurately.

A small amount of manually tagging is performed, and samples tagged as the intensive phobia type are added into the original tagged sample set, thereby increasing the number of intensive phobia picture samples.

For example, before an image recognition model is trained, the sample set includes a tagged image sample set and a to-be-tagged image sample set. All image samples in the tagged image sample set have been tagged and tags are obtained. The to-be-tagged image sample set only includes image samples, which are not tagged with tags.

At first, model training is performed by use of the tagged image sample set, the image recognition model obtained after training for a period of time are used to recognize the image samples in the to-be-tagged image sample set to obtain recognition results and confidences, and part of to-be-tagged image samples required to be manually tagged are selected according to the confidences to form a set, so that the workload of manual tagging for obtaining image sample tags may be reduced.

Furthermore, after a set including multiple to-be-tagged image samples is obtained according to the confidences, further filtering may be performed in an image clustering manner to reduce the number of the to-be-tagged image samples required to be manually tagged.

The example provides a method for training an image recognition model , which may include:
inputting data which is a tagged image sample set including a small number of tagged intensive phobia image samples (recorded as positive image samples) and a large number of tagged normal image samples (recorded as negative image samples), and a to-be-tagged image sample set including a large number of untagged images.

As shown in <FIG>, an active deep learning-based image tagging and recognition method includes the following steps.

In S51, an initial convolutional neural network deep model is trained by use of a tagged image sample set with tags. Since the number of the positive image samples with tags is relatively small, a recognition capability of the present deep model for intensive phobia pictures is relatively low.

In S52, each image sample in the to-be-tagged sample set is classified by use of the presently trained convolutional neural network deep model, and a softmax value of each image sample corresponding to a positive or negative type is output. The softmax value may be understood as a confidence that the image sample is determined to belong to each type of image sample, and if the softmax value is closer to <NUM>, the confidence is higher and the probability that the image sample belongs to this type is higher.

In S53, the image sample which belongs to the positive type and of which the output softmax value is between <NUM> and <NUM> is selected and determined as a sample that is probably an intensive phobia picture for addition into the to-be-tagged sample set. Herein, the positive type is a type to which the positive image samples belong. Herein, the to-be-tagged sample set is the abovementioned first to-be-tagged image sample set.

In S54, through adoption of an entropy clustering method, k samples with maximum entropies are selected from the initial to-be-tagged sample set and clustered, and m (m<k) samples are selected as a formal to-be-tagged sample set. Herein, the formal to-be-tagged sample set is the abovementioned second to-be-tagged image sample set.

In S55, manual tagging operations of experts are received to implement manual tagging of all the images in the present to-be-tagged sample set, which specifically includes that: the samples <NUM>% absolutely determined as normal pictures are added into the tagged image sample set; the samples <NUM>% absolutely determined as intensive phobia pictures are added into the tagged image sample set; and the pictures that cannot be absolutely determined to belong to the intensive phobia type are discarded and not put into any dataset anymore.

In S56, the deep model is retrained by use of data in the present tagged sample set. Thus, the recognition capability of the present deep model for intensive phobia pictures is further improved.

In S57, S52 and S53 are repeated; if the number of the images in the present to-be-tagged sample set is less than n, the method is stopped, and a sample set with expanded tags and an intensive phobia picture recognition model is obtained; otherwise Step <NUM> to Step <NUM> are continued.

An embodiment of the present disclosure also provides a device for training an image recognition model , which includes a processor, a memory and computer-executable instructions stored in the memory and executable by the processor, where the processor executes the computer-executable instructions to implement the method for training an image recognition model training provided in any abovementioned embodiments, for example, executing the methods shown in <FIG> and <FIG>.

<FIG> is a block diagram of an image recognition model training device <NUM>, not being part of the invention. For example, the device <NUM> may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a gaming console, a tablet, a medical device, exercise equipment, a personal digital assistant and the like.

The processing component <NUM> may include one or more processors <NUM> to execute instructions to perform all or part of the steps in the abovementioned method. Moreover, the processing component <NUM> may include one or more modules which facilitate interaction between the processing component <NUM> and the other components. For instance, the processing component <NUM> may include a multimedia module to facilitate interaction between the multimedia component <NUM> and the processing component <NUM>.

The memory <NUM> is configured to store various types of data to support the operations of the device <NUM>. Examples of such data include instructions for any applications or methods operated on the device <NUM>, contact data, phonebook data, messages, pictures, video, etc. The memory <NUM> may be implemented by any type of volatile or non-volatile memory devices, or a combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic memory, a flash memory, and a magnetic or optical disk.

The multimedia component <NUM> includes a screen providing an output interface between the device <NUM> and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes the TP, the screen may be implemented as a touch screen to receive an input signal from the user. The TP includes one or more touch sensors to sense touches, swipes and gestures on the TP. The touch sensors may sense not only a boundary of a touch or swipe action but also detect a duration and pressure associated with the touch or swipe action. The front camera and/or the rear camera may receive external multimedia data when the device <NUM> is in an operation mode, such as a photographing mode or a video mode. Each of the front camera and the rear camera may be a fixed optical lens system or have focusing and optical zooming capabilities.

The audio component <NUM> is configured to output and/or input an audio signal. For example, the audio component <NUM> includes a Microphone (MIC), and the MIC is configured to receive an external audio signal when the device <NUM> is in the operation mode, such as a call mode, a recording mode and a voice recognition mode. The received audio signal may further be stored in the memory <NUM> or sent through the communication component <NUM>. In some embodiments, the audio component <NUM> further includes a speaker configured to output the audio signal.

The communication component <NUM> is configured to facilitate wired or wireless communication between the device <NUM> and another device. The device <NUM> may access a communication-standard-based wireless network, such as a Wireless Fidelity (WiFi) network, a 2nd-Generation (<NUM>) or 3rd-Generation (<NUM>), <NUM>, <NUM> network or a combination thereof. In an exemplary embodiment, the communication component <NUM> receives a broadcast signal or broadcast associated information from an external broadcast management system through a broadcast channel. In an exemplary embodiment, the communication component <NUM> further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on a Radio Frequency Identification (RFID) technology, an Infrared Data Association (IrDA) technology, an Ultra-Wide Band (UWB) technology, a Bluetooth (BT) technology and another technology.

In an exemplary embodiment, the device <NUM> may be implemented by one or more ASICs, Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components, and is configured to execute the abovementioned method.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including an instruction, such as the memory <NUM> including an instruction, and the instruction may be executed by the processor <NUM> of the device <NUM> to implement the abovementioned method. For example, the non-transitory computer-readable storage medium may be a ROM, a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disc, an optical data storage device and the like.

An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium, which may be called a storage medium for short. Computer-executable instructions stored in the storage medium, when being executed by a processor, enable the processor to execute the method for training an image recognition model, the method including that: an image recognition model is trained by using a tagged image sample set; multiple to-be-tagged image samples in a to-be-tagged image sample set are recognized by using the presently trained image recognition model to obtain a confidence level of a recognition result corresponding to each of the multiple to-be-tagged image samples; at least one to-be-tagged image sample of which the confidence level is in a preset interval is selected as at least one to-be-tagged image sample to form a first to-be-tagged image sample set; tags of the to-be-tagged image sample in the first to-be-tagged image sample set are acquired; the tagged image sample set is updated according to the acquired tags; and the image recognition model is continued to be trained by using the updated tagged image sample set.

The operation that the tags of the to-be-tagged image sample in the first to-be-tagged image sample set are acquired includes that: the at least one to-be-tagged image sample in the first to-be-tagged image sample set is clustered to obtain a clustering result; part of target to-be-tagged image samples having cluster representativeness are selected from the first to-be-tagged image sample set according to the clustering result to form a second to-be-tagged image sample set; and tags of all or part of the target to-be-tagged image samples in the second to-be-tagged image sample set are acquired.

The operation that the at least one to-be-tagged image sample in the first to-be-tagged image sample set is clustered to obtain the clustering result includes that: information entropy of a to-be-tagged image sample is determined according to image features of the to-be-tagged image sample in the first to-be-tagged image sample set; and the to-be-tagged image samples of which the information entropy meets a predetermined condition is selected and clustered to obtain K clusters and a cluster center of each cluster, K being a positive integer.

The clustering result includes one or more clusters formed by clustering and a cluster center of each of the one or more clusters; and the operation that part of target to-be-tagged image samples having the cluster representativeness are selected from the first to-be-tagged image sample set according to the clustering result to form the second to-be-tagged image sample set includes that: one or more to-be-tagged image samples closest to the corresponding cluster center are selected from each of the one or more clusters as the target to-be-tagged image samples having the cluster representativeness, to form the second to-be-tagged image sample set.

The method further includes that: after the first to-be-tagged image sample set is formed, whether the number of the to-be-tagged image sample in the first to-be-tagged image sample set is greater than a preset value is determined; and the operation that the tags of the to-be-tagged image sample in the first to-be-tagged image sample set are acquired includes that: when the number of the to-be-tagged image samples in the first to-be-tagged image sample set are greater than the preset value, the tags of the to-be-tagged image sample in the first to-be-tagged image sample set are acquired.

The method further includes that: when the number of the to-be-tagged image samples in the first to-be-tagged image sample set are less than or equal to the preset value, training of the image recognition model is stopped. Based on the solution, the operation that the image recognition model is continued to be trained by using the updated tagged image sample set includes that: the step that the image recognition model is trained is re-executed by use of the updated tagged image sample set.

The tagged image sample set includes a first number of positive image samples and a second number of negative image samples, the first number being smaller than the second number; and/or, the first to-be-tagged image sample set includes a third number of to-be-tagged image samples, the third number being larger than the second number.

The positive image sample includes an intensive phobia image tagged as an intensive image, and the negative image sample is an image except for the intensive phobia image.

Claim 1:
A method for training an image recognition model, applied to a terminal device, the method comprising:
training (S11) an image recognition model by using a tagged image sample set;
recognizing (S12) multiple to-be-tagged image samples by using the image recognition model to obtain a confidence level of a recognition result corresponding to each of the multiple to-be-tagged image samples;
selecting (S13), in the multiple to-be-tagged image samples, at least one to-be-tagged image sample of which the confidence level is within a preset interval to form a first to-be-tagged image sample set, wherein the preset interval is from <NUM> to <NUM>;
acquiring (S14) manually tagged tags of to-be-tagged image samples in the first to-be-tagged image sample set;
updating (S15) the tagged image sample set by adding the to-be-tagged image samples with the tags to the tagged image sample set, to obtain an updated tagged image sample set; and
training (S16) the image recognition model by using the updated tagged image sample set,
wherein acquiring (S14) the manually tagged tags of the to-be-tagged image samples in the first to-be-tagged image sample set comprises:
determining information entropy of a to-be-tagged image sample according to image features of the to-be-tagged image sample in the first to-be-tagged image sample set; characterized by
clustering to-be-tagged image samples of which the information entropy meets a predetermined condition to obtain K clusters and a cluster center of at least one cluster, K being a positive integer, wherein clustering the to-be-tagged image samples of which the information entropy meets the predetermined condition comprises clustering the to-be-tagged image samples of which the information entropy is greater than an entropy threshold value or clustering a predetermined number of to-be-tagged image samples with maximum information entropies;
selecting, from the at least one cluster, one or more to-be-tagged image samples closest to a cluster center of the cluster as target to-be-tagged image samples having cluster representativeness, to form a second to-be-tagged image sample set; and
acquiring (S143) manually tagged tags of the target to-be-tagged image samples in the second to-be-tagged image sample set.