# Image Retrieval Technology of Economic Regulations Based on Semantic Segmentation

Authors: Guanyan Guo; Liangliang Sun
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

Content-based image retrieval (CBIR) is an important part of pattern recognition and artificial intelligence. It has broad application prospects in many important fields, such as digital library, medical image analysis, petroleum geological survey, and public security information retrieval. In this study, statistical modeling and discriminant learning methods are used to analyze and study some key problems in image retrieval, including image concept retrieval, image example retrieval, and relevance feedback. The main research results obtained are as follows: an image classification method based on the Gaussian mixture model (GMM) and max-min posterior pseudo-probability (MMP) discriminant learning is proposed, which is called GMM-MMP method for short; a concept retrieval method based on GMM-MMP is proposed. According to the image concept, the image is divided into two categories: the concept-related image and the concept-unrelated image. The Gaussian mixture model is used to establish the mapping from the image low-level features to the image concept, and the image is classified according to the posterior pseudo-probability classifier to realize the image concept retrieval; an example retrieval method based on GMM-MMP is proposed. According to the image similarity semantics, the image is divided into two categories: the related image and the uncorrelated image of the example image. The Gaussian mixture model is used to establish the mapping from the low-level features of the image to the image similarity semantics. The image is classified according to the posterior pseudo-probability classifier to realize the image case retrieval. Based on the above work, this study implements a content-based image retrieval system.


---

## Body

## 1. Introduction

### 1.1. Research Background and Significance

With the popularization of network technology and the rapid development of digital image acquisition technology, the number of digital images produced in many fields such as science, education, medical treatment, and industry is increasing at an alarming rate. Google, the world's largest search engine, is taken for example. In July 2018, the Google image library had about 310 million images. By March 2020, the number of images had reached 1.187 billion, an increase of nearly three times. These images contain a lot of useful information. To access and use this information effectively, we need a technology that can quickly and accurately find and access image information. Image retrieval technology came into being.In essence, traditional image retrieval is a text-based image retrieval (TBIR) method. For each image in the image library, it is summarized and labeled manually, and then, the image is retrieved according to the image name, number, annotation, additional information, link address, etc. There are two problems in traditional image retrieval methods: (1) every image needs manual annotation, so large image databases need a lot of manual annotation, which is too expensive; (2) manual annotation has strong subjectivity. Different people have different understandings of the same image, so the limited and fixed manual annotation is difficult to meet the needs of different users.A typical CBIR system is mainly composed of two modules: image database building module and image retrieval module. The image database establishment module establishes and maintains the image database and related files. Because this module consumes a lot of time, it is generally implemented offline. The image retrieval module retrieves the images in the image library according to the user’s query requirements and interacts with the user, which is generally implemented in an online manner.In view of the problems existing in traditional image retrieval methods, the National Science Foundation of the United States held a seminar on visual information management system in 2021. The participants reached a consensus on the new development direction of image database management system; that is, the most effective method for representing and indexing image information should be based on the economic regulation image itself. Economic regulation image-based image retrieval (CBIR) technology has been developed since then. The basic principle of CBIR technology is to establish an image feature vector for similarity query according to the economic regulation image of the image, that is, the color, texture, shape, spatial relationship of objects, and other information contained in the image. On the basis of trying to understand the economic regulation image, the image with a certain feature or specific economic regulation image is searched from the database. Economic regulation image-based image retrieval technology involves many research fields. At present, it has become one of the most active research hotspots in the fields of computer vision, pattern recognition, database technology, knowledge mining, information retrieval, and so on.In addition, content-based image retrieval technology has broad application prospects in many important fields, such as medical image analysis, public security information retrieval, intellectual property protection, and geological survey. In terms of medical diagnosis, doctors can find similar symptoms among multiple patients by searching the medical image database and assist doctors in diagnosis or carry out medical analysis on the condition and development of patients. In terms of public security and crime prevention, the public security police can further grasp the facial features of suspects and prevent crimes by identifying whether suspects are in the existing criminal face database. In terms of intellectual property protection, trademark registrants can better protect intellectual property and protect the interests of trademark owners by searching the existing trademark database and checking whether the applied trademark is the same or similar to the registered trademark. In geological exploration, remote sensing image retrieval can help geological workers to distinguish and identify specific targets or areas with certain characteristics in remote sensing images.


### 1.2. Research Status at Home and Abroad

In the early 1990s, economic regulation image-based image retrieval technology began to appear and quickly became a hot research field [1]. At present, economic regulation image-based image retrieval has become one of the most active research hotspots in the fields of computer vision, pattern recognition, database technology, knowledge mining, and information retrieval. Many important international journals, such as IEEE Transactions on Image Processing, Pattern Recognition, Pattern Recognition Letters, IEEE Transactions on Pattern Analysis and Machine Intelligence, and International Journal of Pattern Recognition and Artistic Intelligence, publish a large number of papers on CBIR every year. To further promote the development of economic regulation image-based image retrieval technology, many important international conferences, such as IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE International Conference on Image Processing, International Conference on Pattern Recognition, and IEEE Conference on Computer Vision and Pattern Recognition, have set up visual information retrieval topics and chapters. There are also many special international conferences on image retrieval held every year, such as International Conference on Image and Video Retrieval, ACM Sigma Workshop on Multimedia Information Retrieval, ACM Multimedia, and IEEE International Conference on Multimedia and Expo. In addition, many research institutions have conducted extensive and in-depth research on CBIR technology and have successively launched some image retrieval systems in the commercial and research fields.Early research on economic regulation image-based image retrieval mainly focused on uncompressed ordinary images from the perspective of visual features, that is, identifying economic regulation image through low-level features such as color, texture, and shape [2]. Later, to improve the effect of image retrieval, people studied from the aspects of semantic feature extraction, multidimensional indexing, relevance feedback, machine learning, and so on and put forward some new methods. With the popularity of the network, a large number of compressed images have appeared, and the object of image retrieval has gradually expanded to compressed domain images. At the same time, some images in specific fields, such as medical images and remote sensing images, are gradually included in the research scope of economic regulation image-based image retrieval [3].The key of economic regulation image-based image retrieval is how to understand image semantics. Generally, a semantic unit refers to the name or category of an object that appears in a scene. According to the complexity of image semantics, Eakins [4] divided the research work of economic regulation image-based image retrieval into the following three levels.At present, image retrieval technology is mainly in the stage of feature semantic understanding. Image retrieval based on object semantics, object spatial relationship semantics, scene semantics, behavior semantics, and emotion semantics is still very difficult.Economic regulation image-based image retrieval has broad application prospects, but there are still many key technologies that have not been solved or are not perfect.Around the problem of image retrieval, people have carried out research in image query methods, image low-level feature description methods, similarity measurement methods, test image database, evaluation criteria, and so on. Based on the research results, some image retrieval systems have emerged in the commercial and research fields. The following is a brief introduction to the existing technologies and systems of image retrieval [5].


#### 1.2.1. Common Image Query Methods

People use the following query methods in image retrieval: browsing, query by example, QBE, query by sketch, and query by keyword.


#### 1.2.2. Image Low-Level Feature Description Method

At present, the commonly used image low-level features include color features, texture features, shape features, and spatial relationships. The following briefly introduces various features commonly used in image retrieval.


#### 1.2.3. Similarity Measurement Method

The similarity measurement method is mainly used to measure the similarity between two images. It is an important part of image retrieval research and the result of the interaction of human visual system, cognition, and emotion. The quality of similarity measurement method will directly affect the effect of image retrieval, and its computational complexity will affect the user response time of image retrieval. The most intuitive similarity measurement method is to directly use the distance of feature vectors to measure the similarity of two images. The following describes several distance measurement methods commonly used in CBIR systems.(1)DI,J=∑ifiI−fiJP1/P,(2)DI,J=∑ifiI−fiJ,(3)DI,J=∑ifiI−fiJ21/2,(4)DI,J=∑iwifiI−fiJ21/2,(5)DI,J=fI−fJTC−1fI−fJ,(6)DI,J=∑iminfiI,fiJ∑ifiJ,(7)AVRR=1R∑i=1Rpi,(8)IAVRR=T2,(9)MT=RT.


### 1.3. Main Research Economic Regulation Images of This Paper

The main research economic regulation images include the following.


#### 1.3.1. Concept Retrieval Method of Image

The related images of different concepts vary greatly, and there may be great differences between the related images belonging to the same concept. To solve the problem of image concept retrieval and alleviate the semantic gap, this study studies the problem of image concept retrieval and tries to solve the problem of image concept retrieval from the perspective of image classification. The concept retrieval method based on concept annotation is discussed [6–10].


#### 1.3.2. Example Retrieval Method of Image

The core problem of image retrieval is image similarity measurement, that is, how to measure the similarity between two images. Based on the analysis of the existing similarity measurement methods, this study studies the problem of image case retrieval and tries to solve the problem of image case retrieval from the perspective of image classification. The case retrieval method based on semantic similarity measurement and visual similarity measurement is discussed.


#### 1.3.3. Relevance Feedback Method

To alleviate the semantic gap and improve the accuracy of image retrieval, relevance feedback technology is introduced into the field of economic regulation image-based image retrieval to improve the retrieval performance. Based on the analysis of the existing relevance feedback methods, this study studies the relevance feedback problem in image retrieval and tries to solve the relevance feedback problem in concept retrieval and example retrieval from the perspective of image classification. The relevance feedback method of multi-example learning is discussed [11–13].


## 1.1. Research Background and Significance

With the popularization of network technology and the rapid development of digital image acquisition technology, the number of digital images produced in many fields such as science, education, medical treatment, and industry is increasing at an alarming rate. Google, the world's largest search engine, is taken for example. In July 2018, the Google image library had about 310 million images. By March 2020, the number of images had reached 1.187 billion, an increase of nearly three times. These images contain a lot of useful information. To access and use this information effectively, we need a technology that can quickly and accurately find and access image information. Image retrieval technology came into being.In essence, traditional image retrieval is a text-based image retrieval (TBIR) method. For each image in the image library, it is summarized and labeled manually, and then, the image is retrieved according to the image name, number, annotation, additional information, link address, etc. There are two problems in traditional image retrieval methods: (1) every image needs manual annotation, so large image databases need a lot of manual annotation, which is too expensive; (2) manual annotation has strong subjectivity. Different people have different understandings of the same image, so the limited and fixed manual annotation is difficult to meet the needs of different users.A typical CBIR system is mainly composed of two modules: image database building module and image retrieval module. The image database establishment module establishes and maintains the image database and related files. Because this module consumes a lot of time, it is generally implemented offline. The image retrieval module retrieves the images in the image library according to the user’s query requirements and interacts with the user, which is generally implemented in an online manner.In view of the problems existing in traditional image retrieval methods, the National Science Foundation of the United States held a seminar on visual information management system in 2021. The participants reached a consensus on the new development direction of image database management system; that is, the most effective method for representing and indexing image information should be based on the economic regulation image itself. Economic regulation image-based image retrieval (CBIR) technology has been developed since then. The basic principle of CBIR technology is to establish an image feature vector for similarity query according to the economic regulation image of the image, that is, the color, texture, shape, spatial relationship of objects, and other information contained in the image. On the basis of trying to understand the economic regulation image, the image with a certain feature or specific economic regulation image is searched from the database. Economic regulation image-based image retrieval technology involves many research fields. At present, it has become one of the most active research hotspots in the fields of computer vision, pattern recognition, database technology, knowledge mining, information retrieval, and so on.In addition, content-based image retrieval technology has broad application prospects in many important fields, such as medical image analysis, public security information retrieval, intellectual property protection, and geological survey. In terms of medical diagnosis, doctors can find similar symptoms among multiple patients by searching the medical image database and assist doctors in diagnosis or carry out medical analysis on the condition and development of patients. In terms of public security and crime prevention, the public security police can further grasp the facial features of suspects and prevent crimes by identifying whether suspects are in the existing criminal face database. In terms of intellectual property protection, trademark registrants can better protect intellectual property and protect the interests of trademark owners by searching the existing trademark database and checking whether the applied trademark is the same or similar to the registered trademark. In geological exploration, remote sensing image retrieval can help geological workers to distinguish and identify specific targets or areas with certain characteristics in remote sensing images.


## 1.2. Research Status at Home and Abroad

In the early 1990s, economic regulation image-based image retrieval technology began to appear and quickly became a hot research field [1]. At present, economic regulation image-based image retrieval has become one of the most active research hotspots in the fields of computer vision, pattern recognition, database technology, knowledge mining, and information retrieval. Many important international journals, such as IEEE Transactions on Image Processing, Pattern Recognition, Pattern Recognition Letters, IEEE Transactions on Pattern Analysis and Machine Intelligence, and International Journal of Pattern Recognition and Artistic Intelligence, publish a large number of papers on CBIR every year. To further promote the development of economic regulation image-based image retrieval technology, many important international conferences, such as IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE International Conference on Image Processing, International Conference on Pattern Recognition, and IEEE Conference on Computer Vision and Pattern Recognition, have set up visual information retrieval topics and chapters. There are also many special international conferences on image retrieval held every year, such as International Conference on Image and Video Retrieval, ACM Sigma Workshop on Multimedia Information Retrieval, ACM Multimedia, and IEEE International Conference on Multimedia and Expo. In addition, many research institutions have conducted extensive and in-depth research on CBIR technology and have successively launched some image retrieval systems in the commercial and research fields.Early research on economic regulation image-based image retrieval mainly focused on uncompressed ordinary images from the perspective of visual features, that is, identifying economic regulation image through low-level features such as color, texture, and shape [2]. Later, to improve the effect of image retrieval, people studied from the aspects of semantic feature extraction, multidimensional indexing, relevance feedback, machine learning, and so on and put forward some new methods. With the popularity of the network, a large number of compressed images have appeared, and the object of image retrieval has gradually expanded to compressed domain images. At the same time, some images in specific fields, such as medical images and remote sensing images, are gradually included in the research scope of economic regulation image-based image retrieval [3].The key of economic regulation image-based image retrieval is how to understand image semantics. Generally, a semantic unit refers to the name or category of an object that appears in a scene. According to the complexity of image semantics, Eakins [4] divided the research work of economic regulation image-based image retrieval into the following three levels.At present, image retrieval technology is mainly in the stage of feature semantic understanding. Image retrieval based on object semantics, object spatial relationship semantics, scene semantics, behavior semantics, and emotion semantics is still very difficult.Economic regulation image-based image retrieval has broad application prospects, but there are still many key technologies that have not been solved or are not perfect.Around the problem of image retrieval, people have carried out research in image query methods, image low-level feature description methods, similarity measurement methods, test image database, evaluation criteria, and so on. Based on the research results, some image retrieval systems have emerged in the commercial and research fields. The following is a brief introduction to the existing technologies and systems of image retrieval [5].


### 1.2.1. Common Image Query Methods

People use the following query methods in image retrieval: browsing, query by example, QBE, query by sketch, and query by keyword.


### 1.2.2. Image Low-Level Feature Description Method

At present, the commonly used image low-level features include color features, texture features, shape features, and spatial relationships. The following briefly introduces various features commonly used in image retrieval.


### 1.2.3. Similarity Measurement Method

The similarity measurement method is mainly used to measure the similarity between two images. It is an important part of image retrieval research and the result of the interaction of human visual system, cognition, and emotion. The quality of similarity measurement method will directly affect the effect of image retrieval, and its computational complexity will affect the user response time of image retrieval. The most intuitive similarity measurement method is to directly use the distance of feature vectors to measure the similarity of two images. The following describes several distance measurement methods commonly used in CBIR systems.(1)DI,J=∑ifiI−fiJP1/P,(2)DI,J=∑ifiI−fiJ,(3)DI,J=∑ifiI−fiJ21/2,(4)DI,J=∑iwifiI−fiJ21/2,(5)DI,J=fI−fJTC−1fI−fJ,(6)DI,J=∑iminfiI,fiJ∑ifiJ,(7)AVRR=1R∑i=1Rpi,(8)IAVRR=T2,(9)MT=RT.


## 1.2.1. Common Image Query Methods

People use the following query methods in image retrieval: browsing, query by example, QBE, query by sketch, and query by keyword.


## 1.2.2. Image Low-Level Feature Description Method

At present, the commonly used image low-level features include color features, texture features, shape features, and spatial relationships. The following briefly introduces various features commonly used in image retrieval.


## 1.2.3. Similarity Measurement Method

The similarity measurement method is mainly used to measure the similarity between two images. It is an important part of image retrieval research and the result of the interaction of human visual system, cognition, and emotion. The quality of similarity measurement method will directly affect the effect of image retrieval, and its computational complexity will affect the user response time of image retrieval. The most intuitive similarity measurement method is to directly use the distance of feature vectors to measure the similarity of two images. The following describes several distance measurement methods commonly used in CBIR systems.(1)DI,J=∑ifiI−fiJP1/P,(2)DI,J=∑ifiI−fiJ,(3)DI,J=∑ifiI−fiJ21/2,(4)DI,J=∑iwifiI−fiJ21/2,(5)DI,J=fI−fJTC−1fI−fJ,(6)DI,J=∑iminfiI,fiJ∑ifiJ,(7)AVRR=1R∑i=1Rpi,(8)IAVRR=T2,(9)MT=RT.


## 1.3. Main Research Economic Regulation Images of This Paper

The main research economic regulation images include the following.


### 1.3.1. Concept Retrieval Method of Image

The related images of different concepts vary greatly, and there may be great differences between the related images belonging to the same concept. To solve the problem of image concept retrieval and alleviate the semantic gap, this study studies the problem of image concept retrieval and tries to solve the problem of image concept retrieval from the perspective of image classification. The concept retrieval method based on concept annotation is discussed [6–10].


### 1.3.2. Example Retrieval Method of Image

The core problem of image retrieval is image similarity measurement, that is, how to measure the similarity between two images. Based on the analysis of the existing similarity measurement methods, this study studies the problem of image case retrieval and tries to solve the problem of image case retrieval from the perspective of image classification. The case retrieval method based on semantic similarity measurement and visual similarity measurement is discussed.


### 1.3.3. Relevance Feedback Method

To alleviate the semantic gap and improve the accuracy of image retrieval, relevance feedback technology is introduced into the field of economic regulation image-based image retrieval to improve the retrieval performance. Based on the analysis of the existing relevance feedback methods, this study studies the relevance feedback problem in image retrieval and tries to solve the relevance feedback problem in concept retrieval and example retrieval from the perspective of image classification. The relevance feedback method of multi-example learning is discussed [11–13].


## 1.3.1. Concept Retrieval Method of Image

The related images of different concepts vary greatly, and there may be great differences between the related images belonging to the same concept. To solve the problem of image concept retrieval and alleviate the semantic gap, this study studies the problem of image concept retrieval and tries to solve the problem of image concept retrieval from the perspective of image classification. The concept retrieval method based on concept annotation is discussed [6–10].


## 1.3.2. Example Retrieval Method of Image

The core problem of image retrieval is image similarity measurement, that is, how to measure the similarity between two images. Based on the analysis of the existing similarity measurement methods, this study studies the problem of image case retrieval and tries to solve the problem of image case retrieval from the perspective of image classification. The case retrieval method based on semantic similarity measurement and visual similarity measurement is discussed.


## 1.3.3. Relevance Feedback Method

To alleviate the semantic gap and improve the accuracy of image retrieval, relevance feedback technology is introduced into the field of economic regulation image-based image retrieval to improve the retrieval performance. Based on the analysis of the existing relevance feedback methods, this study studies the relevance feedback problem in image retrieval and tries to solve the relevance feedback problem in concept retrieval and example retrieval from the perspective of image classification. The relevance feedback method of multi-example learning is discussed [11–13].


## 2. Gaussian Mixture Modeling and Maximum-Minimum Posteriori Pseudo-Probability Method for Image Classification

### 2.1. Introduction

In this study, concept retrieval, example retrieval, and relevance feedback in image retrieval are regarded as image classification problems; that is, according to some high-level semantics (image concept, image similarity, and user intention), the images in the image database are divided into two categories: high-level semantic-related images and uncorrelated images. On this basis, this study proposes a Gaussian mixture modeling and maximum-minimum posteriori pseudo-probability method for image classification. This method is composed of Gaussian mixture model (GMM), a posteriori pseudo-probability classifier, and max-min posteriori pseudo-probability learning (MMP). We call this method GMM-MMP for short. Under the framework of GMM-MMP method, the concept retrieval, example retrieval, and relevance feedback problems in image retrieval are processed. The specific processing methods will be introduced in the following three sections.


### 2.2. Image Gaussian Mixture Modeling

In this study, the Gaussian mixture model is used to establish the mapping between image low-level features and image high-level semantics. The extraction of image low-level features and Gaussian mixture modeling involved are introduced below.


#### 2.2.1. Feature Extraction

To avoid the negative impact of inaccurate image segmentation, which leads to low retrieval accuracy, this study directly uses the overall features of the image to describe the economic regulation image; that is, only the first two common features of the image are extracted: color features and texture features.HSV color space is considered to be closer to human perception of color. In this study, the first-, second-, and third-order color moments of HSV space are extracted to form a 9-dimensional feature vector color components, with 3 moments on each component as the color features of the image. The color moment is calculated as follows:(10)μi=1N∑j=1Npij,(11)σi=1N∑j=1Npij−μi212,(12)si=1N∑j=1Npij−μi313.The Gabor texture can minimize the uncertainty of space and frequency and can also detect edges and lines in different directions and angles in the image. A two-dimensional Gabor wavelet is defined as follows:(13)Ψk,z=k2σ2exp−k2z22σ2⋅expikz−exp−σ22,(14)kμ,v=kveiφμ,of which,(15)kv=kmaxfv,v∈0,1,…,4;φμ=πμ8,μ∈0,1,…,7.In this study, the Gabor filter coefficients of 6 directions and 4 scales are selected as the texture features of the image, that is, the first 71 coefficients of 24 coefficient matrix obtained from each image after the Gabor transformation of 6 directions and 4 scales are taken as the texture features of the image [14–16].9D color features and 71D texture features are arranged in turn to form an 80D feature vector as the overall description of the image.


#### 2.2.2. Gaussian Mixture Modeling

A Gaussian mixture model (GMM) is one of the most widely used density estimation tools at present. It uses a semi-parametric density estimation method, combines the advantages of parametric and nonparametric methods, and uses the generated model for density estimation. Although the model itself does not limit the function form, if the number of model components is selected properly and the model parameters are selected correctly, the Gaussian mixture model can approximate any continuous density and achieve any accuracy.


#### 2.2.3. Gaussian Mixture Modeling

Gaussian mixture model (GMM) is one of the most widely used density estimation tools at present. It uses a semi-parametric density estimation method, combines the advantages of parametric and nonparametric methods, and uses the generated model for density estimation. Although the model itself does not limit the function form, if the number of model components is selected properly and the model parameters are selected correctly, the Gaussian mixture model can approximate any continuous density and achieve any accuracy.(16)px|θ,α=∑j=1mαjpjx|θj,(17)∑jαj=1.The mixed model can combine several simple density functions into a more complex function.In this study, the Gaussian mixture model is used to simulate the feature distribution of image high-level semantics, and the mapping between image low-level features and image high-level semantics is established.(18)px|ω=∑k=1Kwkpkx=∑k=1KwkNx|μk,Σk,of which,(19)Nx|μk,Σk=2π−d2Σk−12exp−12x−μkTΣk−1x−μk,(20)∑k=1Kwk=1.


### 2.3. Posterior Pseudo-Probability Classifier

According to the Bayes formula,(21)Pωi|x=px|ωiPωipx.Equation (13) can be derived:(22)Pωi|xPωj|x=px|ωiPωipx|ωjPωj.According to equation (12), the correlation between the posterior probabilities of the same category under different input modes is established to obtain equation (14):(23)Pωi|xPωi|x′=px|ωipx′px′|ωipx.Formula (13) is similar to formula (14) in form but different in meaning. Equation (13) contains two classes and an input mode, indicating the relationship between posterior probabilities of different classes under the same input mode; equation (14) includes one class and two input modes, which represents the relationship between the posterior probabilities of the same class under different input modes.Equation (14) can be simplified as follows:(24)Pωi|xPωi|x′=px|ωipx′|ωi,(25)Pωi|x∞px|ωi,(26)Pωi|x≈fpx|ωi=1−exp−λpx|ωi.Equation (9) is substituted into equation (17) to obtain equation (18):(27)fX;Λ=1−exp−λ∑k=1KwkNx|μk,Σk,(28)Λ=λ,wk,μk,Σk,k=1,⋯,K.In the image retrieval problem discussed in this study, each image high-level semantics corresponds to a posterior pseudo-probability function, which we take as a class model. The class model mentioned below is the posterior pseudo-probability function of the class. After the posterior pseudo-probability value of the corresponding category in the input mode is calculated by equation (18), the category corresponding to the maximum posterior pseudo-probability value is taken as the classification result [17–19].


### 2.4. Learning of Posterior Pseudo-Probability Classifier

Before using a posteriori pseudo-probability classifier for classification, it is necessary to determine the unknown parameter set. In this study, two methods are used to learn unknown parameter sets. They are as follows: (1) expectation maximization (EM), which is a generative learning method. The EM method is used to learn the initial parameters of the posterior pseudo-probability function; (2) max-min posterior pseudo-probability (MMP) is a discriminant learning method. MMP method is used to continue to learn and optimize the model parameters on the basis of EM method, so as to achieve the best classification effect. We call the corresponding methods as GMM-EM method and GMM-MMP method, respectively.


## 2.1. Introduction

In this study, concept retrieval, example retrieval, and relevance feedback in image retrieval are regarded as image classification problems; that is, according to some high-level semantics (image concept, image similarity, and user intention), the images in the image database are divided into two categories: high-level semantic-related images and uncorrelated images. On this basis, this study proposes a Gaussian mixture modeling and maximum-minimum posteriori pseudo-probability method for image classification. This method is composed of Gaussian mixture model (GMM), a posteriori pseudo-probability classifier, and max-min posteriori pseudo-probability learning (MMP). We call this method GMM-MMP for short. Under the framework of GMM-MMP method, the concept retrieval, example retrieval, and relevance feedback problems in image retrieval are processed. The specific processing methods will be introduced in the following three sections.


## 2.2. Image Gaussian Mixture Modeling

In this study, the Gaussian mixture model is used to establish the mapping between image low-level features and image high-level semantics. The extraction of image low-level features and Gaussian mixture modeling involved are introduced below.


### 2.2.1. Feature Extraction

To avoid the negative impact of inaccurate image segmentation, which leads to low retrieval accuracy, this study directly uses the overall features of the image to describe the economic regulation image; that is, only the first two common features of the image are extracted: color features and texture features.HSV color space is considered to be closer to human perception of color. In this study, the first-, second-, and third-order color moments of HSV space are extracted to form a 9-dimensional feature vector color components, with 3 moments on each component as the color features of the image. The color moment is calculated as follows:(10)μi=1N∑j=1Npij,(11)σi=1N∑j=1Npij−μi212,(12)si=1N∑j=1Npij−μi313.The Gabor texture can minimize the uncertainty of space and frequency and can also detect edges and lines in different directions and angles in the image. A two-dimensional Gabor wavelet is defined as follows:(13)Ψk,z=k2σ2exp−k2z22σ2⋅expikz−exp−σ22,(14)kμ,v=kveiφμ,of which,(15)kv=kmaxfv,v∈0,1,…,4;φμ=πμ8,μ∈0,1,…,7.In this study, the Gabor filter coefficients of 6 directions and 4 scales are selected as the texture features of the image, that is, the first 71 coefficients of 24 coefficient matrix obtained from each image after the Gabor transformation of 6 directions and 4 scales are taken as the texture features of the image [14–16].9D color features and 71D texture features are arranged in turn to form an 80D feature vector as the overall description of the image.


### 2.2.2. Gaussian Mixture Modeling

A Gaussian mixture model (GMM) is one of the most widely used density estimation tools at present. It uses a semi-parametric density estimation method, combines the advantages of parametric and nonparametric methods, and uses the generated model for density estimation. Although the model itself does not limit the function form, if the number of model components is selected properly and the model parameters are selected correctly, the Gaussian mixture model can approximate any continuous density and achieve any accuracy.


### 2.2.3. Gaussian Mixture Modeling

Gaussian mixture model (GMM) is one of the most widely used density estimation tools at present. It uses a semi-parametric density estimation method, combines the advantages of parametric and nonparametric methods, and uses the generated model for density estimation. Although the model itself does not limit the function form, if the number of model components is selected properly and the model parameters are selected correctly, the Gaussian mixture model can approximate any continuous density and achieve any accuracy.(16)px|θ,α=∑j=1mαjpjx|θj,(17)∑jαj=1.The mixed model can combine several simple density functions into a more complex function.In this study, the Gaussian mixture model is used to simulate the feature distribution of image high-level semantics, and the mapping between image low-level features and image high-level semantics is established.(18)px|ω=∑k=1Kwkpkx=∑k=1KwkNx|μk,Σk,of which,(19)Nx|μk,Σk=2π−d2Σk−12exp−12x−μkTΣk−1x−μk,(20)∑k=1Kwk=1.


## 2.2.1. Feature Extraction

To avoid the negative impact of inaccurate image segmentation, which leads to low retrieval accuracy, this study directly uses the overall features of the image to describe the economic regulation image; that is, only the first two common features of the image are extracted: color features and texture features.HSV color space is considered to be closer to human perception of color. In this study, the first-, second-, and third-order color moments of HSV space are extracted to form a 9-dimensional feature vector color components, with 3 moments on each component as the color features of the image. The color moment is calculated as follows:(10)μi=1N∑j=1Npij,(11)σi=1N∑j=1Npij−μi212,(12)si=1N∑j=1Npij−μi313.The Gabor texture can minimize the uncertainty of space and frequency and can also detect edges and lines in different directions and angles in the image. A two-dimensional Gabor wavelet is defined as follows:(13)Ψk,z=k2σ2exp−k2z22σ2⋅expikz−exp−σ22,(14)kμ,v=kveiφμ,of which,(15)kv=kmaxfv,v∈0,1,…,4;φμ=πμ8,μ∈0,1,…,7.In this study, the Gabor filter coefficients of 6 directions and 4 scales are selected as the texture features of the image, that is, the first 71 coefficients of 24 coefficient matrix obtained from each image after the Gabor transformation of 6 directions and 4 scales are taken as the texture features of the image [14–16].9D color features and 71D texture features are arranged in turn to form an 80D feature vector as the overall description of the image.


## 2.2.2. Gaussian Mixture Modeling

A Gaussian mixture model (GMM) is one of the most widely used density estimation tools at present. It uses a semi-parametric density estimation method, combines the advantages of parametric and nonparametric methods, and uses the generated model for density estimation. Although the model itself does not limit the function form, if the number of model components is selected properly and the model parameters are selected correctly, the Gaussian mixture model can approximate any continuous density and achieve any accuracy.


## 2.2.3. Gaussian Mixture Modeling

Gaussian mixture model (GMM) is one of the most widely used density estimation tools at present. It uses a semi-parametric density estimation method, combines the advantages of parametric and nonparametric methods, and uses the generated model for density estimation. Although the model itself does not limit the function form, if the number of model components is selected properly and the model parameters are selected correctly, the Gaussian mixture model can approximate any continuous density and achieve any accuracy.(16)px|θ,α=∑j=1mαjpjx|θj,(17)∑jαj=1.The mixed model can combine several simple density functions into a more complex function.In this study, the Gaussian mixture model is used to simulate the feature distribution of image high-level semantics, and the mapping between image low-level features and image high-level semantics is established.(18)px|ω=∑k=1Kwkpkx=∑k=1KwkNx|μk,Σk,of which,(19)Nx|μk,Σk=2π−d2Σk−12exp−12x−μkTΣk−1x−μk,(20)∑k=1Kwk=1.


## 2.3. Posterior Pseudo-Probability Classifier

According to the Bayes formula,(21)Pωi|x=px|ωiPωipx.Equation (13) can be derived:(22)Pωi|xPωj|x=px|ωiPωipx|ωjPωj.According to equation (12), the correlation between the posterior probabilities of the same category under different input modes is established to obtain equation (14):(23)Pωi|xPωi|x′=px|ωipx′px′|ωipx.Formula (13) is similar to formula (14) in form but different in meaning. Equation (13) contains two classes and an input mode, indicating the relationship between posterior probabilities of different classes under the same input mode; equation (14) includes one class and two input modes, which represents the relationship between the posterior probabilities of the same class under different input modes.Equation (14) can be simplified as follows:(24)Pωi|xPωi|x′=px|ωipx′|ωi,(25)Pωi|x∞px|ωi,(26)Pωi|x≈fpx|ωi=1−exp−λpx|ωi.Equation (9) is substituted into equation (17) to obtain equation (18):(27)fX;Λ=1−exp−λ∑k=1KwkNx|μk,Σk,(28)Λ=λ,wk,μk,Σk,k=1,⋯,K.In the image retrieval problem discussed in this study, each image high-level semantics corresponds to a posterior pseudo-probability function, which we take as a class model. The class model mentioned below is the posterior pseudo-probability function of the class. After the posterior pseudo-probability value of the corresponding category in the input mode is calculated by equation (18), the category corresponding to the maximum posterior pseudo-probability value is taken as the classification result [17–19].


## 2.4. Learning of Posterior Pseudo-Probability Classifier

Before using a posteriori pseudo-probability classifier for classification, it is necessary to determine the unknown parameter set. In this study, two methods are used to learn unknown parameter sets. They are as follows: (1) expectation maximization (EM), which is a generative learning method. The EM method is used to learn the initial parameters of the posterior pseudo-probability function; (2) max-min posterior pseudo-probability (MMP) is a discriminant learning method. MMP method is used to continue to learn and optimize the model parameters on the basis of EM method, so as to achieve the best classification effect. We call the corresponding methods as GMM-EM method and GMM-MMP method, respectively.


## 3. Concept Retrieval Method of Image

### 3.1. Introduction

The process of image concept retrieval (query by concept) can be summarized as follows: according to the concepts entered by the user (in this study, it refers to image aliases), such as “bus,” “elephant,” and “tiger,” the system returns the relevant images of the concept in the image database to the user as the retrieval results [20–22].The relevant images of the concept have the following characteristics:(1)
The related images of different concepts are very different from each other.(2)
There may also be great differences between the related images of the same concept, and the magnitude of this difference is related to the representation range of the concept itself. If the representation range of image concept is relatively narrow, the main economic regulation image of the image related to the concept will be relatively single, and the difference between them is small; if the representation range of the image concept is wide, the main economic regulation image of the image related to the concept is rich, and there will be great differences between them.The above characteristics of the concept-related images can be illustrated in Figure1: for the two image concepts “horse” and “racing car” in figure (a) and figure (b), because the representation range of these two concepts is relatively narrow, the difference between the related images of the same concept is relatively small, and the features extracted from the related images of such concepts can basically be consistent. As for the image concept of “Greek island” in figure (c), since the concept of “Greek island” itself has a wide range of representation, including local customs, architectural styles, and natural scenery, the main economic regulation image of the relevant images of the concept is relatively rich and different, and the features extracted from the relevant images of such concepts will also differ greatly.Figure 1
Related images of the concept: (a) related images of “horse”; (b) relevant images of “racing car”; (c) related images of “Greek island.”
(a)(b)(c)In view of the above characteristics of concept-related images, if the traditional image retrieval method based on low-level visual feature is adopted, the retrieval accuracy will be low and the retrieval effect will be poor due to the “semantic gap.”To establish the mapping between image low-level features and image high-level semantics, many researchers use the supervised learning method or unsupervised learning method to obtain image high-level semantics, establish the association between image low-level features and image high-level semantics, and realize the mapping of image from feature space to high-level semantic space.


#### 3.1.1. Support Vector Machine Method

To improve the effect of SVM classification, Xu et al. proposed a new image classification framework. Before using SVM classification, representative features of image categories are selected and noise features are reduced to learn a more accurate image category model. The framework realizes image classification by selecting salient features, generating visual keywords to describe image categories, and finally combining SVM with the visual keyword frequency model. They conducted classification experiments on 2500 Corel images in 25 categories, and the average classification accuracy reached 67.46%. To solve the problem that classification is relatively difficult when multiple classes overlap in the feature space, Tsai et al. established the image index classification system Claire using a two-stage mapping model (TSMM). The system establishes three SVM modules for mapping color, texture, and high-level concepts. The first-order mapping uses color SVM and texture SVM to map the color features and texture features of the region into corresponding color terms and texture terms, respectively. The second-order mapping uses the high-level concept SVM to map the above two terms into high-level keywords. Through the above two-level mapping, images are associated with keywords to realize image indexing and classification. In view of the fact that the low-level features of the image cannot fully represent the economic regulation image, Lu Jing and Ma Shaoping used image classification to establish the concept index of the image and realized the mapping from the low-level visual features to the model features through the multilevel classifier based on SVM. The model features were defined as the probability combination of the multilevel classification results, and the above methods were used to realize the automatic semantic annotation of the image [23–26].


#### 3.1.2. Bayesian Classifier Method

Jin et al. combined semi-naïve Bayesian with regional clustering to learn the mapping between regional clustering and concepts and proposed an automatic image annotation method. Firstly, the region clustering of the image is realized according to the pairwise constraints learned from the training image. In the image annotation stage, a semi-naïve Bayesian model is used to calculate the posterior probability of region clusters relative to the concept to realize the automatic image annotation. Zhang and Izquierdo proposed an image high-level semantic information annotation retrieval method. Firstly, the multi-objective optimization (MOO) algorithm is used to obtain the visual representation of the target from the low-level visual features of the image, and then, the Bayesian network is used to establish the belief network model for the latent semantic annotation of the region, so as to realize the region-based image semantic annotation.


#### 3.1.3. Neural Network Method

To avoid the adverse effects caused by inaccurate image segmentation, Ma and Wang proposed the method of using a neural network to establish the mapping between image concepts and image low-level features. Firstly, the low-level features of the image are extracted and used as the input of the three-layer neural network. The output of the neural network is the concept of the image. Through the feedback from the user, the whole image retrieval system is trained online and the network is adjusted to improve its retrieval effect.


#### 3.1.4. Decision Tree Method

Sethi et al. proposed two automatic image annotation methods, which, respectively, used the decision tree method and K-means clustering to find the mapping relationship between image bottom features and high-level text descriptors.


#### 3.1.5. Clustering Method

Sheikholeslami et al. proposed the semquery algorithm, which uses different image features for clustering, then generates semantic clustering according to various low-level feature subclusters such as color and shape, and finally forms a top-down image semantic hierarchical clustering structure. In image retrieval, neuromerge, a feature merging algorithm based on a multilayer perceptron neural network model, is used to realize the mapping from visual query submitted by users to high-level semantics.


### 3.2. Concept Retrieval Method Based on GMM-MMP

#### 3.2.1. Specific Methods

The specific implementation process of the concept retrieval method based on GMM-MMP is as follows:(1)
Assuming that the feature vector of the image related to the concept obeys the Gaussian mixture distribution, the mapping from the low-level feature of the image to the image concept is established.(2)
A posterior pseudo-probability classifier is used to realize image classification. The following methods are used to learn the unknown parameters in the posterior pseudo-probability classifier: first, the EM method is used to learn the initial values of the model parameters, and then, the MMP method is used to continue to learn and optimize the model parameters based on the EM method. Here, a positive example for learning is a related image of an image concept, and a negative example is an unrelated image of an image concept.


#### 3.2.2. Concept Retrieval Experiment

Recall rate and precision rate are used to evaluate the retrieval effect of the method. In the second return method, the number of returned images (NRIs) is also used as an evaluation standard to measure the retrieval effect.(1) Concept Retrieval Experiment on 5000 Images. When 50 concepts were retrieved by GMM-MMP method, the precision ratio (P100) of the first 100 result images varied from 34% to 99%, and the average precision ratio was 58.58%. When GMM-EM method is used for concept retrieval, the precision of the first 100 result images varies from 10% to 87%, and the average precision is 37.46%. At this time, the recall rate is equal to the precision rate [27, 28].Figure2 shows the average precision change curve of the top 6, top 12, top 18, top 24, and top 30 result images when using GMM-MMP and GMM-EM methods to retrieve 50 concepts on 5000 images.Figure 2
Change curve of average precision rate during 50 concept searches on 5000 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 50 concepts retrieved by GMM-MMP method are 52.22%, 62.02%, and 122.9, respectively. When using GMM-EM method to retrieve 50 concepts, the average precision, average recall, and average number of returned images are 7.55%, 89.44%, and 2149.04, respectively. Figure3 shows the first 20 result images of “racing car” and “tiger.” In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 19 relevant images in the first 20 result images of “racing” using GMM-EM method, and the first 20 result images of “racing” using GMM-MMP method are all relevant images. In figure (b), among the top 20 result images of “tiger” using GMM-EM method, there are 19 related images, and the top 20 result images of “tiger” using GMM-MMP method are all related images.Figure 3
Top 20 result images of two concepts: (a) search results of “racing car”; (b) search results of “tiger.”
(a)(b)(2) Concept Retrieval Experiment on 2500 Images. When 50 concepts were retrieved by GMM-MMP method, the precision ratio (P50) of the first 50 result images varied from 2% to 98%, and the average precision ratio was 32.8%. When GMM-EM method is used for concept retrieval, the precision of the first 50 result images varies from 0% to 84%, and the average precision is 19.12%. At this time, the recall rate is equal to the precision rate.Figure4 shows the average precision change curve of the top 6, top 12, top 18, top 24, and top 30 result images when using GMM-MMP and GMM-EM methods to retrieve 50 concepts on 2500 images.Figure 4
Change curve of average precision rate when searching 50 concepts on 2500 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 50 concepts retrieved by GMM-MMP method are 32.50%, 34%, and 53.98, respectively. When using GMM-EM method to retrieve 50 concepts, the average precision rate, average recall rate, and average number of returned images are 6.95%, 79.2%, and 1064.8, respectively. Figure5 shows the first 20 result images of “racing car” and “tiger.” In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 8 related images in the first 20 result images of “racing” using GMM-EM method and 15 related images in the first 20 result images of “racing” using GMM-MMP method. In figure (b), there are 8 related images in the top 20 result images of “tiger” using GMM-EM method and 16 related images in the top 20 result images of “tiger” using GMM-MMP method.Figure 5
Top 20 result images of two concepts: (a) search results of “racing car”; (b) search results of “tiger.”
(a)(b)


### 3.3. Concept Retrieval Method Integrating Concept Annotation

#### 3.3.1. Specific Methods

This study presents a concept retrieval method based on concept annotation, which performs concept retrieval on the basis of image concept annotation and reduces the image set to be retrieved, thus speeding up the retrieval speed and improving the retrieval accuracy.The conceptual annotation method can be described as follows:(1)
The posterior pseudo-probability function value of the image is calculated relative to all concepts.(2)
The golden section point (0.618 points) of the image relative to the posterior pseudo-probability function value of all concepts is taken as the threshold to annotate the image. If the posterior pseudo-probability of an image relative to a concept is greater than the threshold, the image is labeled as the corresponding concept.The above concept annotation method allows one image to be annotated with multiple concepts.The concept retrieval methods based on concept annotation can be summarized as follows:(1)
The concept annotation method is used to annotate all the images to be retrieved.(2)
According to the image concept retrieved by the user, all the images labeled with this concept are taken to form a candidate image set.(3)
GMM-MMP method is used for concept retrieval on candidate image sets.


#### 3.3.2. Concept Retrieval Experiment

The concept retrieval method based on concept annotation is used to retrieve 50 concepts on 5000 Corel images to test the retrieval effect of this method. Two methods are set to return the retrieval results: (1) return the top 100 result images; (2) return all images annotated as the currently retrieved concept. When the first 100 images are returned, the average precision rate (P100) is counted. At this time, the recall rate is equal to the precision rate. When all images marked as the current retrieval concept are returned, the average precision, average recall, and average number of returned images (NRIs) are counted. The results of concept retrieval on 5000 images are compared with those of GMM-MMP method.(1) Analysis of Experimental Results.(1)
The concept retrieval method integrating concept annotation improves the concept retrieval effect of GMM-MMP method: when GMM-MMP method retrieves 50 concepts on 5000 images, the average precision rate of the first 100 images is 58.58%; the precision of the first 100 images is 59.84% when the concept retrieval method based on concept annotation is used for concept retrieval. In addition, when all the images labeled as the current retrieval concept are returned, the average recall rate of the concept retrieval method based on concept annotation is 100% and the average precision rate is close to 50%. The experimental data show that the concept retrieval method based on concept annotation performs concept retrieval on the basis of concept annotation, reduces the image set to be retrieved, and improves the concept retrieval accuracy of the original retrieval method.(2)
When all the images labeled as the current retrieval concept are returned, the average recall rate of the concept retrieval method of concept annotation fusion reaches 100%. These experimental data show that when taking the golden section point of the projection line segment of 50 posterior pseudo-probability function values as the threshold value of image concept annotation, 100 relevant images of each concept are correctly labeled.(3)
The process of concept annotation is relatively independent and can be completed in the process of image input. In addition, the retrieval is carried out on the candidate image set of the concept, which saves the retrieval time and improves the retrieval efficiency.


## 3.1. Introduction

The process of image concept retrieval (query by concept) can be summarized as follows: according to the concepts entered by the user (in this study, it refers to image aliases), such as “bus,” “elephant,” and “tiger,” the system returns the relevant images of the concept in the image database to the user as the retrieval results [20–22].The relevant images of the concept have the following characteristics:(1)
The related images of different concepts are very different from each other.(2)
There may also be great differences between the related images of the same concept, and the magnitude of this difference is related to the representation range of the concept itself. If the representation range of image concept is relatively narrow, the main economic regulation image of the image related to the concept will be relatively single, and the difference between them is small; if the representation range of the image concept is wide, the main economic regulation image of the image related to the concept is rich, and there will be great differences between them.The above characteristics of the concept-related images can be illustrated in Figure1: for the two image concepts “horse” and “racing car” in figure (a) and figure (b), because the representation range of these two concepts is relatively narrow, the difference between the related images of the same concept is relatively small, and the features extracted from the related images of such concepts can basically be consistent. As for the image concept of “Greek island” in figure (c), since the concept of “Greek island” itself has a wide range of representation, including local customs, architectural styles, and natural scenery, the main economic regulation image of the relevant images of the concept is relatively rich and different, and the features extracted from the relevant images of such concepts will also differ greatly.Figure 1
Related images of the concept: (a) related images of “horse”; (b) relevant images of “racing car”; (c) related images of “Greek island.”
(a)(b)(c)In view of the above characteristics of concept-related images, if the traditional image retrieval method based on low-level visual feature is adopted, the retrieval accuracy will be low and the retrieval effect will be poor due to the “semantic gap.”To establish the mapping between image low-level features and image high-level semantics, many researchers use the supervised learning method or unsupervised learning method to obtain image high-level semantics, establish the association between image low-level features and image high-level semantics, and realize the mapping of image from feature space to high-level semantic space.


### 3.1.1. Support Vector Machine Method

To improve the effect of SVM classification, Xu et al. proposed a new image classification framework. Before using SVM classification, representative features of image categories are selected and noise features are reduced to learn a more accurate image category model. The framework realizes image classification by selecting salient features, generating visual keywords to describe image categories, and finally combining SVM with the visual keyword frequency model. They conducted classification experiments on 2500 Corel images in 25 categories, and the average classification accuracy reached 67.46%. To solve the problem that classification is relatively difficult when multiple classes overlap in the feature space, Tsai et al. established the image index classification system Claire using a two-stage mapping model (TSMM). The system establishes three SVM modules for mapping color, texture, and high-level concepts. The first-order mapping uses color SVM and texture SVM to map the color features and texture features of the region into corresponding color terms and texture terms, respectively. The second-order mapping uses the high-level concept SVM to map the above two terms into high-level keywords. Through the above two-level mapping, images are associated with keywords to realize image indexing and classification. In view of the fact that the low-level features of the image cannot fully represent the economic regulation image, Lu Jing and Ma Shaoping used image classification to establish the concept index of the image and realized the mapping from the low-level visual features to the model features through the multilevel classifier based on SVM. The model features were defined as the probability combination of the multilevel classification results, and the above methods were used to realize the automatic semantic annotation of the image [23–26].


### 3.1.2. Bayesian Classifier Method

Jin et al. combined semi-naïve Bayesian with regional clustering to learn the mapping between regional clustering and concepts and proposed an automatic image annotation method. Firstly, the region clustering of the image is realized according to the pairwise constraints learned from the training image. In the image annotation stage, a semi-naïve Bayesian model is used to calculate the posterior probability of region clusters relative to the concept to realize the automatic image annotation. Zhang and Izquierdo proposed an image high-level semantic information annotation retrieval method. Firstly, the multi-objective optimization (MOO) algorithm is used to obtain the visual representation of the target from the low-level visual features of the image, and then, the Bayesian network is used to establish the belief network model for the latent semantic annotation of the region, so as to realize the region-based image semantic annotation.


### 3.1.3. Neural Network Method

To avoid the adverse effects caused by inaccurate image segmentation, Ma and Wang proposed the method of using a neural network to establish the mapping between image concepts and image low-level features. Firstly, the low-level features of the image are extracted and used as the input of the three-layer neural network. The output of the neural network is the concept of the image. Through the feedback from the user, the whole image retrieval system is trained online and the network is adjusted to improve its retrieval effect.


### 3.1.4. Decision Tree Method

Sethi et al. proposed two automatic image annotation methods, which, respectively, used the decision tree method and K-means clustering to find the mapping relationship between image bottom features and high-level text descriptors.


### 3.1.5. Clustering Method

Sheikholeslami et al. proposed the semquery algorithm, which uses different image features for clustering, then generates semantic clustering according to various low-level feature subclusters such as color and shape, and finally forms a top-down image semantic hierarchical clustering structure. In image retrieval, neuromerge, a feature merging algorithm based on a multilayer perceptron neural network model, is used to realize the mapping from visual query submitted by users to high-level semantics.


## 3.1.1. Support Vector Machine Method

To improve the effect of SVM classification, Xu et al. proposed a new image classification framework. Before using SVM classification, representative features of image categories are selected and noise features are reduced to learn a more accurate image category model. The framework realizes image classification by selecting salient features, generating visual keywords to describe image categories, and finally combining SVM with the visual keyword frequency model. They conducted classification experiments on 2500 Corel images in 25 categories, and the average classification accuracy reached 67.46%. To solve the problem that classification is relatively difficult when multiple classes overlap in the feature space, Tsai et al. established the image index classification system Claire using a two-stage mapping model (TSMM). The system establishes three SVM modules for mapping color, texture, and high-level concepts. The first-order mapping uses color SVM and texture SVM to map the color features and texture features of the region into corresponding color terms and texture terms, respectively. The second-order mapping uses the high-level concept SVM to map the above two terms into high-level keywords. Through the above two-level mapping, images are associated with keywords to realize image indexing and classification. In view of the fact that the low-level features of the image cannot fully represent the economic regulation image, Lu Jing and Ma Shaoping used image classification to establish the concept index of the image and realized the mapping from the low-level visual features to the model features through the multilevel classifier based on SVM. The model features were defined as the probability combination of the multilevel classification results, and the above methods were used to realize the automatic semantic annotation of the image [23–26].


## 3.1.2. Bayesian Classifier Method

Jin et al. combined semi-naïve Bayesian with regional clustering to learn the mapping between regional clustering and concepts and proposed an automatic image annotation method. Firstly, the region clustering of the image is realized according to the pairwise constraints learned from the training image. In the image annotation stage, a semi-naïve Bayesian model is used to calculate the posterior probability of region clusters relative to the concept to realize the automatic image annotation. Zhang and Izquierdo proposed an image high-level semantic information annotation retrieval method. Firstly, the multi-objective optimization (MOO) algorithm is used to obtain the visual representation of the target from the low-level visual features of the image, and then, the Bayesian network is used to establish the belief network model for the latent semantic annotation of the region, so as to realize the region-based image semantic annotation.


## 3.1.3. Neural Network Method

To avoid the adverse effects caused by inaccurate image segmentation, Ma and Wang proposed the method of using a neural network to establish the mapping between image concepts and image low-level features. Firstly, the low-level features of the image are extracted and used as the input of the three-layer neural network. The output of the neural network is the concept of the image. Through the feedback from the user, the whole image retrieval system is trained online and the network is adjusted to improve its retrieval effect.


## 3.1.4. Decision Tree Method

Sethi et al. proposed two automatic image annotation methods, which, respectively, used the decision tree method and K-means clustering to find the mapping relationship between image bottom features and high-level text descriptors.


## 3.1.5. Clustering Method

Sheikholeslami et al. proposed the semquery algorithm, which uses different image features for clustering, then generates semantic clustering according to various low-level feature subclusters such as color and shape, and finally forms a top-down image semantic hierarchical clustering structure. In image retrieval, neuromerge, a feature merging algorithm based on a multilayer perceptron neural network model, is used to realize the mapping from visual query submitted by users to high-level semantics.


## 3.2. Concept Retrieval Method Based on GMM-MMP

### 3.2.1. Specific Methods

The specific implementation process of the concept retrieval method based on GMM-MMP is as follows:(1)
Assuming that the feature vector of the image related to the concept obeys the Gaussian mixture distribution, the mapping from the low-level feature of the image to the image concept is established.(2)
A posterior pseudo-probability classifier is used to realize image classification. The following methods are used to learn the unknown parameters in the posterior pseudo-probability classifier: first, the EM method is used to learn the initial values of the model parameters, and then, the MMP method is used to continue to learn and optimize the model parameters based on the EM method. Here, a positive example for learning is a related image of an image concept, and a negative example is an unrelated image of an image concept.


### 3.2.2. Concept Retrieval Experiment

Recall rate and precision rate are used to evaluate the retrieval effect of the method. In the second return method, the number of returned images (NRIs) is also used as an evaluation standard to measure the retrieval effect.(1) Concept Retrieval Experiment on 5000 Images. When 50 concepts were retrieved by GMM-MMP method, the precision ratio (P100) of the first 100 result images varied from 34% to 99%, and the average precision ratio was 58.58%. When GMM-EM method is used for concept retrieval, the precision of the first 100 result images varies from 10% to 87%, and the average precision is 37.46%. At this time, the recall rate is equal to the precision rate [27, 28].Figure2 shows the average precision change curve of the top 6, top 12, top 18, top 24, and top 30 result images when using GMM-MMP and GMM-EM methods to retrieve 50 concepts on 5000 images.Figure 2
Change curve of average precision rate during 50 concept searches on 5000 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 50 concepts retrieved by GMM-MMP method are 52.22%, 62.02%, and 122.9, respectively. When using GMM-EM method to retrieve 50 concepts, the average precision, average recall, and average number of returned images are 7.55%, 89.44%, and 2149.04, respectively. Figure3 shows the first 20 result images of “racing car” and “tiger.” In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 19 relevant images in the first 20 result images of “racing” using GMM-EM method, and the first 20 result images of “racing” using GMM-MMP method are all relevant images. In figure (b), among the top 20 result images of “tiger” using GMM-EM method, there are 19 related images, and the top 20 result images of “tiger” using GMM-MMP method are all related images.Figure 3
Top 20 result images of two concepts: (a) search results of “racing car”; (b) search results of “tiger.”
(a)(b)(2) Concept Retrieval Experiment on 2500 Images. When 50 concepts were retrieved by GMM-MMP method, the precision ratio (P50) of the first 50 result images varied from 2% to 98%, and the average precision ratio was 32.8%. When GMM-EM method is used for concept retrieval, the precision of the first 50 result images varies from 0% to 84%, and the average precision is 19.12%. At this time, the recall rate is equal to the precision rate.Figure4 shows the average precision change curve of the top 6, top 12, top 18, top 24, and top 30 result images when using GMM-MMP and GMM-EM methods to retrieve 50 concepts on 2500 images.Figure 4
Change curve of average precision rate when searching 50 concepts on 2500 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 50 concepts retrieved by GMM-MMP method are 32.50%, 34%, and 53.98, respectively. When using GMM-EM method to retrieve 50 concepts, the average precision rate, average recall rate, and average number of returned images are 6.95%, 79.2%, and 1064.8, respectively. Figure5 shows the first 20 result images of “racing car” and “tiger.” In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 8 related images in the first 20 result images of “racing” using GMM-EM method and 15 related images in the first 20 result images of “racing” using GMM-MMP method. In figure (b), there are 8 related images in the top 20 result images of “tiger” using GMM-EM method and 16 related images in the top 20 result images of “tiger” using GMM-MMP method.Figure 5
Top 20 result images of two concepts: (a) search results of “racing car”; (b) search results of “tiger.”
(a)(b)


## 3.2.1. Specific Methods

The specific implementation process of the concept retrieval method based on GMM-MMP is as follows:(1)
Assuming that the feature vector of the image related to the concept obeys the Gaussian mixture distribution, the mapping from the low-level feature of the image to the image concept is established.(2)
A posterior pseudo-probability classifier is used to realize image classification. The following methods are used to learn the unknown parameters in the posterior pseudo-probability classifier: first, the EM method is used to learn the initial values of the model parameters, and then, the MMP method is used to continue to learn and optimize the model parameters based on the EM method. Here, a positive example for learning is a related image of an image concept, and a negative example is an unrelated image of an image concept.


## 3.2.2. Concept Retrieval Experiment

Recall rate and precision rate are used to evaluate the retrieval effect of the method. In the second return method, the number of returned images (NRIs) is also used as an evaluation standard to measure the retrieval effect.(1) Concept Retrieval Experiment on 5000 Images. When 50 concepts were retrieved by GMM-MMP method, the precision ratio (P100) of the first 100 result images varied from 34% to 99%, and the average precision ratio was 58.58%. When GMM-EM method is used for concept retrieval, the precision of the first 100 result images varies from 10% to 87%, and the average precision is 37.46%. At this time, the recall rate is equal to the precision rate [27, 28].Figure2 shows the average precision change curve of the top 6, top 12, top 18, top 24, and top 30 result images when using GMM-MMP and GMM-EM methods to retrieve 50 concepts on 5000 images.Figure 2
Change curve of average precision rate during 50 concept searches on 5000 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 50 concepts retrieved by GMM-MMP method are 52.22%, 62.02%, and 122.9, respectively. When using GMM-EM method to retrieve 50 concepts, the average precision, average recall, and average number of returned images are 7.55%, 89.44%, and 2149.04, respectively. Figure3 shows the first 20 result images of “racing car” and “tiger.” In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 19 relevant images in the first 20 result images of “racing” using GMM-EM method, and the first 20 result images of “racing” using GMM-MMP method are all relevant images. In figure (b), among the top 20 result images of “tiger” using GMM-EM method, there are 19 related images, and the top 20 result images of “tiger” using GMM-MMP method are all related images.Figure 3
Top 20 result images of two concepts: (a) search results of “racing car”; (b) search results of “tiger.”
(a)(b)(2) Concept Retrieval Experiment on 2500 Images. When 50 concepts were retrieved by GMM-MMP method, the precision ratio (P50) of the first 50 result images varied from 2% to 98%, and the average precision ratio was 32.8%. When GMM-EM method is used for concept retrieval, the precision of the first 50 result images varies from 0% to 84%, and the average precision is 19.12%. At this time, the recall rate is equal to the precision rate.Figure4 shows the average precision change curve of the top 6, top 12, top 18, top 24, and top 30 result images when using GMM-MMP and GMM-EM methods to retrieve 50 concepts on 2500 images.Figure 4
Change curve of average precision rate when searching 50 concepts on 2500 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 50 concepts retrieved by GMM-MMP method are 32.50%, 34%, and 53.98, respectively. When using GMM-EM method to retrieve 50 concepts, the average precision rate, average recall rate, and average number of returned images are 6.95%, 79.2%, and 1064.8, respectively. Figure5 shows the first 20 result images of “racing car” and “tiger.” In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 8 related images in the first 20 result images of “racing” using GMM-EM method and 15 related images in the first 20 result images of “racing” using GMM-MMP method. In figure (b), there are 8 related images in the top 20 result images of “tiger” using GMM-EM method and 16 related images in the top 20 result images of “tiger” using GMM-MMP method.Figure 5
Top 20 result images of two concepts: (a) search results of “racing car”; (b) search results of “tiger.”
(a)(b)


## 3.3. Concept Retrieval Method Integrating Concept Annotation

### 3.3.1. Specific Methods

This study presents a concept retrieval method based on concept annotation, which performs concept retrieval on the basis of image concept annotation and reduces the image set to be retrieved, thus speeding up the retrieval speed and improving the retrieval accuracy.The conceptual annotation method can be described as follows:(1)
The posterior pseudo-probability function value of the image is calculated relative to all concepts.(2)
The golden section point (0.618 points) of the image relative to the posterior pseudo-probability function value of all concepts is taken as the threshold to annotate the image. If the posterior pseudo-probability of an image relative to a concept is greater than the threshold, the image is labeled as the corresponding concept.The above concept annotation method allows one image to be annotated with multiple concepts.The concept retrieval methods based on concept annotation can be summarized as follows:(1)
The concept annotation method is used to annotate all the images to be retrieved.(2)
According to the image concept retrieved by the user, all the images labeled with this concept are taken to form a candidate image set.(3)
GMM-MMP method is used for concept retrieval on candidate image sets.


### 3.3.2. Concept Retrieval Experiment

The concept retrieval method based on concept annotation is used to retrieve 50 concepts on 5000 Corel images to test the retrieval effect of this method. Two methods are set to return the retrieval results: (1) return the top 100 result images; (2) return all images annotated as the currently retrieved concept. When the first 100 images are returned, the average precision rate (P100) is counted. At this time, the recall rate is equal to the precision rate. When all images marked as the current retrieval concept are returned, the average precision, average recall, and average number of returned images (NRIs) are counted. The results of concept retrieval on 5000 images are compared with those of GMM-MMP method.(1) Analysis of Experimental Results.(1)
The concept retrieval method integrating concept annotation improves the concept retrieval effect of GMM-MMP method: when GMM-MMP method retrieves 50 concepts on 5000 images, the average precision rate of the first 100 images is 58.58%; the precision of the first 100 images is 59.84% when the concept retrieval method based on concept annotation is used for concept retrieval. In addition, when all the images labeled as the current retrieval concept are returned, the average recall rate of the concept retrieval method based on concept annotation is 100% and the average precision rate is close to 50%. The experimental data show that the concept retrieval method based on concept annotation performs concept retrieval on the basis of concept annotation, reduces the image set to be retrieved, and improves the concept retrieval accuracy of the original retrieval method.(2)
When all the images labeled as the current retrieval concept are returned, the average recall rate of the concept retrieval method of concept annotation fusion reaches 100%. These experimental data show that when taking the golden section point of the projection line segment of 50 posterior pseudo-probability function values as the threshold value of image concept annotation, 100 relevant images of each concept are correctly labeled.(3)
The process of concept annotation is relatively independent and can be completed in the process of image input. In addition, the retrieval is carried out on the candidate image set of the concept, which saves the retrieval time and improves the retrieval efficiency.


## 3.3.1. Specific Methods

This study presents a concept retrieval method based on concept annotation, which performs concept retrieval on the basis of image concept annotation and reduces the image set to be retrieved, thus speeding up the retrieval speed and improving the retrieval accuracy.The conceptual annotation method can be described as follows:(1)
The posterior pseudo-probability function value of the image is calculated relative to all concepts.(2)
The golden section point (0.618 points) of the image relative to the posterior pseudo-probability function value of all concepts is taken as the threshold to annotate the image. If the posterior pseudo-probability of an image relative to a concept is greater than the threshold, the image is labeled as the corresponding concept.The above concept annotation method allows one image to be annotated with multiple concepts.The concept retrieval methods based on concept annotation can be summarized as follows:(1)
The concept annotation method is used to annotate all the images to be retrieved.(2)
According to the image concept retrieved by the user, all the images labeled with this concept are taken to form a candidate image set.(3)
GMM-MMP method is used for concept retrieval on candidate image sets.


## 3.3.2. Concept Retrieval Experiment

The concept retrieval method based on concept annotation is used to retrieve 50 concepts on 5000 Corel images to test the retrieval effect of this method. Two methods are set to return the retrieval results: (1) return the top 100 result images; (2) return all images annotated as the currently retrieved concept. When the first 100 images are returned, the average precision rate (P100) is counted. At this time, the recall rate is equal to the precision rate. When all images marked as the current retrieval concept are returned, the average precision, average recall, and average number of returned images (NRIs) are counted. The results of concept retrieval on 5000 images are compared with those of GMM-MMP method.(1) Analysis of Experimental Results.(1)
The concept retrieval method integrating concept annotation improves the concept retrieval effect of GMM-MMP method: when GMM-MMP method retrieves 50 concepts on 5000 images, the average precision rate of the first 100 images is 58.58%; the precision of the first 100 images is 59.84% when the concept retrieval method based on concept annotation is used for concept retrieval. In addition, when all the images labeled as the current retrieval concept are returned, the average recall rate of the concept retrieval method based on concept annotation is 100% and the average precision rate is close to 50%. The experimental data show that the concept retrieval method based on concept annotation performs concept retrieval on the basis of concept annotation, reduces the image set to be retrieved, and improves the concept retrieval accuracy of the original retrieval method.(2)
When all the images labeled as the current retrieval concept are returned, the average recall rate of the concept retrieval method of concept annotation fusion reaches 100%. These experimental data show that when taking the golden section point of the projection line segment of 50 posterior pseudo-probability function values as the threshold value of image concept annotation, 100 relevant images of each concept are correctly labeled.(3)
The process of concept annotation is relatively independent and can be completed in the process of image input. In addition, the retrieval is carried out on the candidate image set of the concept, which saves the retrieval time and improves the retrieval efficiency.


## 4. Example Retrieval Method of Image

### 4.1. Specific Methods

In this study, the GMM-MMP method for image classification proposed in Section 2 is used to transform the problem of image example retrieval into the problem of image classification according to image similarity. The images in the image database are divided into two categories: the related images of example images and the uncorrelated images, to solve the problem of image example retrieval. The specific implementation process of the example retrieval method based on GMM-MMP is as follows:(1)
Assuming that the difference in feature vectors of similar image pairs obeys the Gaussian mixture distribution with the mean value of 0, the mapping of image low-level features to image similarity semantics is established.(2)
A posterior pseudo-probability classifier is used to realize image classification. The following methods are used to learn the unknown parameters in the posterior pseudo-probability classifier: first, the EM method is used to learn the initial values of the model parameters, and then, the MMP method is used to continue to learn and optimize the model parameters based on the EM method. Here, the positive example for learning is a similar image pair, and the negative example is a dissimilar image pair.


### 4.2. Example Retrieval Experiment

5000 images in the core image library are selected to form the background image library of the image retrieval system. 5000 Corel images are composed of 50 categories, such as Africans, beaches, buildings, buses, dinosaurs, elephants, flowers, horses, and mountains; each category has 100 images [29].The specific process of image example retrieval using GMM-MMP method is as follows: first, the user inputs the example image to be retrieved, the system reads the image similarity semantic model parameters, calculates the posterior pseudo-probability function values of the image pairs composed of all the images to be retrieved and the example images in the image database, and arranges all the images to be retrieved in descending order according to the posterior pseudo-probability function values. Then, according to the user's specific requirements for the search results, the top images will be returned to the user.To test the example retrieval effect of GMM-MMP method on different image sets, the example retrieval experiments were carried out on 5000 images and 4000 images without training images, and the results were compared with the example retrieval experiment results of GMM-EM method and fuzzy SVM method proposed by Rao et al.Recall rate and precision rate are used to evaluate the retrieval effect of the method. In addition, in the experiment of automatically determining the number of returned images, the number of returned images (NRIs) is also used as an evaluation standard to measure the retrieval effect.


#### 4.2.1. Example Retrieval Experiment on 5000 Images

Five images of each category are randomly selected as example images, and the GMM-MMP method is used to retrieve 250 example images on 5000 images. Two methods are set to return the search results: (1) return the top 100 result images; (2) return all images with a posteriori pseudo-probability function value greater than 0.5. When returning the top 100 result images, the precision of the top 6, top 12, top 18, top 24, top 30, and top 100 result images will be counted. At this time, the recall is equal to the precision. When all images with a posteriori pseudo-probability function value greater than 0.5 are returned, the average recall, average precision, and average number of returned images retrieved for 250 examples are counted. The experimental results of GMM-MMP method were compared with those of GMM-EM method.When the GMM-MMP method was used to retrieve 250 example images, the precision ratio (P100) of the first 100 result images varied from 11.8% to 92%, and the average precision ratio was 31.75%. When the GMM-EM method is used to retrieve 250 example images, the precision of the first 100 images varies from 5.40% to 58.60%, and the average precision is 19%. At this time, the recall is equal to the precision.Figure6 shows the average precision change curve of the first 6, first 12, first 18, first 24, and first 30 result images when the GMM-MMP method and GMM-EM method are used to retrieve 250 example images on 5000 images.Figure 6
Change curve of average precision rate during 250 example image retrieval on 5000 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 250 example images retrieved by GMM-MMP method are 32.40%, 33.14%, and 216.65, respectively. When the GMM-EM method is used to retrieve 250 example images, the average precision, average recall, and average number of returned images are 2.31%, 98.32%, and 4709.6, respectively. Figure7 shows the first 20 result images obtained when retrieving the two example images of “horse” and “sunrise,” and the first line of image is the example image. In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 11 related images in the top 20 result images of “horse” using GMM-EM method and 17 related images in the top 20 result images of GMM-MMP method. In figure (b), there are 8 related images in the first 20 result images of “sunrise” using GMM-EM method and 13 related images in the first 20 result images of GMM-MMP method.Figure 7
First 20 result images of the two example images. The first row of images is the example image, and the rest are the search results: (a) the search results of the example image “horse”; (b) retrieval results of example image “sunrise.”
(a)(b)


## 4.1. Specific Methods

In this study, the GMM-MMP method for image classification proposed in Section 2 is used to transform the problem of image example retrieval into the problem of image classification according to image similarity. The images in the image database are divided into two categories: the related images of example images and the uncorrelated images, to solve the problem of image example retrieval. The specific implementation process of the example retrieval method based on GMM-MMP is as follows:(1)
Assuming that the difference in feature vectors of similar image pairs obeys the Gaussian mixture distribution with the mean value of 0, the mapping of image low-level features to image similarity semantics is established.(2)
A posterior pseudo-probability classifier is used to realize image classification. The following methods are used to learn the unknown parameters in the posterior pseudo-probability classifier: first, the EM method is used to learn the initial values of the model parameters, and then, the MMP method is used to continue to learn and optimize the model parameters based on the EM method. Here, the positive example for learning is a similar image pair, and the negative example is a dissimilar image pair.


## 4.2. Example Retrieval Experiment

5000 images in the core image library are selected to form the background image library of the image retrieval system. 5000 Corel images are composed of 50 categories, such as Africans, beaches, buildings, buses, dinosaurs, elephants, flowers, horses, and mountains; each category has 100 images [29].The specific process of image example retrieval using GMM-MMP method is as follows: first, the user inputs the example image to be retrieved, the system reads the image similarity semantic model parameters, calculates the posterior pseudo-probability function values of the image pairs composed of all the images to be retrieved and the example images in the image database, and arranges all the images to be retrieved in descending order according to the posterior pseudo-probability function values. Then, according to the user's specific requirements for the search results, the top images will be returned to the user.To test the example retrieval effect of GMM-MMP method on different image sets, the example retrieval experiments were carried out on 5000 images and 4000 images without training images, and the results were compared with the example retrieval experiment results of GMM-EM method and fuzzy SVM method proposed by Rao et al.Recall rate and precision rate are used to evaluate the retrieval effect of the method. In addition, in the experiment of automatically determining the number of returned images, the number of returned images (NRIs) is also used as an evaluation standard to measure the retrieval effect.


### 4.2.1. Example Retrieval Experiment on 5000 Images

Five images of each category are randomly selected as example images, and the GMM-MMP method is used to retrieve 250 example images on 5000 images. Two methods are set to return the search results: (1) return the top 100 result images; (2) return all images with a posteriori pseudo-probability function value greater than 0.5. When returning the top 100 result images, the precision of the top 6, top 12, top 18, top 24, top 30, and top 100 result images will be counted. At this time, the recall is equal to the precision. When all images with a posteriori pseudo-probability function value greater than 0.5 are returned, the average recall, average precision, and average number of returned images retrieved for 250 examples are counted. The experimental results of GMM-MMP method were compared with those of GMM-EM method.When the GMM-MMP method was used to retrieve 250 example images, the precision ratio (P100) of the first 100 result images varied from 11.8% to 92%, and the average precision ratio was 31.75%. When the GMM-EM method is used to retrieve 250 example images, the precision of the first 100 images varies from 5.40% to 58.60%, and the average precision is 19%. At this time, the recall is equal to the precision.Figure6 shows the average precision change curve of the first 6, first 12, first 18, first 24, and first 30 result images when the GMM-MMP method and GMM-EM method are used to retrieve 250 example images on 5000 images.Figure 6
Change curve of average precision rate during 250 example image retrieval on 5000 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 250 example images retrieved by GMM-MMP method are 32.40%, 33.14%, and 216.65, respectively. When the GMM-EM method is used to retrieve 250 example images, the average precision, average recall, and average number of returned images are 2.31%, 98.32%, and 4709.6, respectively. Figure7 shows the first 20 result images obtained when retrieving the two example images of “horse” and “sunrise,” and the first line of image is the example image. In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 11 related images in the top 20 result images of “horse” using GMM-EM method and 17 related images in the top 20 result images of GMM-MMP method. In figure (b), there are 8 related images in the first 20 result images of “sunrise” using GMM-EM method and 13 related images in the first 20 result images of GMM-MMP method.Figure 7
First 20 result images of the two example images. The first row of images is the example image, and the rest are the search results: (a) the search results of the example image “horse”; (b) retrieval results of example image “sunrise.”
(a)(b)


## 4.2.1. Example Retrieval Experiment on 5000 Images

Five images of each category are randomly selected as example images, and the GMM-MMP method is used to retrieve 250 example images on 5000 images. Two methods are set to return the search results: (1) return the top 100 result images; (2) return all images with a posteriori pseudo-probability function value greater than 0.5. When returning the top 100 result images, the precision of the top 6, top 12, top 18, top 24, top 30, and top 100 result images will be counted. At this time, the recall is equal to the precision. When all images with a posteriori pseudo-probability function value greater than 0.5 are returned, the average recall, average precision, and average number of returned images retrieved for 250 examples are counted. The experimental results of GMM-MMP method were compared with those of GMM-EM method.When the GMM-MMP method was used to retrieve 250 example images, the precision ratio (P100) of the first 100 result images varied from 11.8% to 92%, and the average precision ratio was 31.75%. When the GMM-EM method is used to retrieve 250 example images, the precision of the first 100 images varies from 5.40% to 58.60%, and the average precision is 19%. At this time, the recall is equal to the precision.Figure6 shows the average precision change curve of the first 6, first 12, first 18, first 24, and first 30 result images when the GMM-MMP method and GMM-EM method are used to retrieve 250 example images on 5000 images.Figure 6
Change curve of average precision rate during 250 example image retrieval on 5000 images.When all images with a posterior pseudo-probability function value greater than 0.5 are returned, the average precision, average recall, and average number of returned images of 250 example images retrieved by GMM-MMP method are 32.40%, 33.14%, and 216.65, respectively. When the GMM-EM method is used to retrieve 250 example images, the average precision, average recall, and average number of returned images are 2.31%, 98.32%, and 4709.6, respectively. Figure7 shows the first 20 result images obtained when retrieving the two example images of “horse” and “sunrise,” and the first line of image is the example image. In figure (a) and figure (b), the left figure shows the search results obtained by GMM-EM method, and the right figure shows the search results obtained by GMM-MMP method. In figure (a), there are 11 related images in the top 20 result images of “horse” using GMM-EM method and 17 related images in the top 20 result images of GMM-MMP method. In figure (b), there are 8 related images in the first 20 result images of “sunrise” using GMM-EM method and 13 related images in the first 20 result images of GMM-MMP method.Figure 7
First 20 result images of the two example images. The first row of images is the example image, and the rest are the search results: (a) the search results of the example image “horse”; (b) retrieval results of example image “sunrise.”
(a)(b)


## 5. Summary

In this study, some key technologies in image retrieval are studied and some achievements are made, but there are still some problems worthy of further study and discussion. In the next step, we will carry out further research in the following four aspects on the basis of the existing work.(1)
On the basis of the image retrieval system established in this study, the retrieval effects of different image features are further studied. On this basis, the dynamic feature selection method is used to retrieve different images with different features, which meets the retrieval requirements of different types of images and improves the accuracy and robustness of the system retrieval.(2)
Based on the GMM-MMP method for image classification proposed in this study, the optimization algorithm of MMP method is further studied to shorten the running time of the algorithm and speed up the training speed of the system.(3)
Based on the concept retrieval method proposed in this study, we further study the keyword annotation method of image region, establish the mapping relationship between image region keywords and image concepts, and improve the accuracy of image concept retrieval.Based on the stability-based region of interest multi-instance learning method proposed in this study, the mapping method from example annotation to package annotation is further studied to improve the accuracy of region-based example retrieval.



---

*Source: 6799899-2022-08-18.xml*