SDePR: Fine-Grained Leaf Image Retrieval with Structural Deep Patch Representation

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 04:24:43

Abstract: Fine-grained leaf image retrieval (FGLIR) is a new unsupervised pattern recognition task in content-based image retrieval (CBIR). It aims to distinguish varieties/cultivars of leaf images within a certain plant species and is more challenging than general leaf image retrieval task due to the inherently subtle differences across different cultivars. In this study, we for the first time investigate the possible way to mine the spatial structure and contextual information from the activation of the convolutional layers of CNN networks for FGLIR. For achieving this goal, we design a novel geometrical structure, named Triplet Patch-Pairs Composite Structure (TPCS), consisting of three symmetric patch pairs segmented from the leaf images in different orientations. We extract CNN feature map for each patch in TPCS and measure the difference between the feature maps of the patch pair for constructing local deep self-similarity descriptor. By varying the size of the TPCS, we can yield multi-scale deep self-similarity descriptors. The final aggregated local deep self-similarity descriptors, named Structural Deep Patch Representation (SDePR), not only encode the spatial structure and contextual information of leaf images in deep feature domain, but also are invariant to the geometrical transformations. The extensive experiments of applying our SDEPR method to the public challenging FGLIR tasks show that our method outperforms the state-of-the-art handcrafted visual features and deep retrieval models.

Main paper: 
 related work: 
 In this section, we briefly survey the state-of-the-art leaf image descriptors for FGLIR and deep CNN feature based image retrieval methods.The early leaf image descriptors are designed for plant species recognition and mainly extract various shape features [10, 11, 16, [26] [27] [28] to construct feature representations. The leaf texture features are also considered in this task. However, they are only taken as the complementary clues for enhancing the recognition accuracy [1, 34] . While in FGLIR, as shown in Fig. 1 , the leaf images have high intra-class similarity in their shape patterns. So, the existing methods for FGLIR mainly focuses on the use of no-shape features.Larese et al. [13] used the Self-Invariant Feature Transform (SIFT) to detect vein patterns and employed Bag of Words (BoW) model to aggregate local vein features for distinguishing soybean cultivars. Oleander has many cultivars exhibiting high inter-class similarity. Baldi et al. [3] used 18 morphometric and colorimetric parameters as leaf visual features for distinguishing 22 oleander cultivars.Recently, some methods treat leaf texture patterns as the most valuable clues for FGLIR. Wang et al. [29] proposed a novel local R-symmetry co-occurrence to de-picting discriminative local symmetry texture patterns. Chen et al. [5] designed a novel geometrical configuration, named Symmetric Binary Tree (SBT), for mining co-occurrence texture patterns. More recently, based on the fanbeam projection theory, a new texture descriptor, named Fan-Beam Binarization Difference Projec-tion (FB-BDP) [6] , was developed for various FGLIR tasks. A single leaf image pat-tern may be not powerful enough to distinguish cultivars. Wang et al. [30] for the first time constructed leaf image descriptors from the joint leaf image patterns in which the leaves from the lower, middle and upper parts of plants are combined for joint texture feature extraction.Besides the handcrafted methods, some works have also considered the use of deep learning for identifying cultivars. Tavakoli et al. [25] proposed to use convolu-tional neural networks (CNN) to classify 12 cultivars of common beans that belong to three different species. They fine-tuned the pre-trained VGG16 model [24] by replacing the last two dense layers with two new ones of 1024 and 512 neurons. However it is designed to work in the supervised setting, i.e., annotated images are required for model fine-tuning. While FGLIR is a pure unsupervised recognition task and annotated images are unavailable. Therefor this method cannot handle the challenging FGLIR.According to whether the deep feature representations encode the structural in-formation of images, the existing CNN based image retrieval methods can be classified into non-structural method and structural method. Most of the methods to the former. They generally focuses on using the activations from the fully connected layers or convolutional layers of CNN models and adopting various feature aggregat-ing strategies to build image descriptors. Wei et al. [31] proposed a method, named Selective Convolutional Descriptor Aggregation (SCDA), for fine-grained image retrieval. It takes the activations from the convolutional layers of CNN models as clues to select the useful deep descriptors for feature aggregation. Pang et al. [19] proposed to address deep convolutional feature aggregation by simulating the dynamics of heat diffusion. Yang et al. [34] made the first attempt to fuse local and global convolutional layer features in an orthogonal manner for effective single-stage image re-trieval. They proposed a novel information fusion framework, named Deep Orthogonal Local and Global (DOLG), for achieving this goal.Capturing the structural characteristics of image can enhance the discriminative power and robustness of image descriptors [8, 14] . Instead simply computing the CNN activation vector over the entire image, Gong et al. [8] proposed to combine activations of CNN extracted at multiple local image windows for getting improved recognition performance. Self-similarity is a classical structural image descriptor aim-ing to measure how similar a specific part of an image is to the entire image or its neighborhood region [14] . Lee et al. [14] revisited it in terms of convolutional and proposed a self-similarity encoder that embeds structural properties in global embeddings while learning diverse structural properties within numerous images.  
the proposed method: 
 In this section, we introduce the details of our proposed Structural Deep Patch Representation (SDePR) for fine-grained leaf image retrieval (FGLIR).Given a leaf image 𝐼 , let Ω denote the leaf contour extracted from it. We uniformly sample 𝑁 points, 𝑍 𝑖 , 𝑖 = 0, . . . 𝑁 -1, from the leaf contour Ω. Then the original leaf contour Ω is simplified as a polygon with 𝑁 edges. For each sample point 𝑍 𝑖 , the largest distance between it and the other 𝑁 -1 sample ones is defined asEQUATIONwhere ∥ • ∥ 2 denotes L2-norm. Given a length 𝜌 ∈ (0, 𝑅 𝑖 ), we use it as a radius and the contour point 𝑍 𝑖 as the center to draw a circle. Since 𝜌 < Υ 𝑖 , the circle intersects with the contour at least two points. Among all the intersection points, let 𝑍 𝑙 and 𝑍 𝑟 be separately the nearest neighboring points of the point 𝑍 𝑖 along contour in clockwise and counter-clockwise directions. We denote 𝛼 𝑖,𝜌 as the angle formed by rotating the right chord 𝑍 𝑖 𝑍 𝑟 about the point 𝑍 𝑖 to the left chord 𝑍 𝑖 𝑍 𝑙 in anticlockwise direction.Taking the left chord 𝑍 𝑖 𝑍 𝑙 as the side, we construct a rectangle 𝐵 𝑙 with the length being 𝜌 and the width being 𝜌/2 inside the leaf area. In the same way, we use the right chord 𝑍 𝑖 𝑍 𝑟 to construct another rectangle 𝐵 𝑟 inside the right area. As shown in Fig. 2 (a), the obtained two rectangle are symmetrical about the bisector of the angle 𝛼 𝑖,𝜌 . We use them to cut two rectangular patches from the leaf image 𝐼 and denote them as 𝑃 𝑟 , respectively. Where we sample each of them to 256 × 128 pixels with subtracting the mean of the pixel values over the patch. By rotating the rectangles 𝐵 𝑙 and 𝐵 𝑟 by 𝛼 𝑖,𝜌 /4, in clockwise and counter-clockwise directions, respectively and using them to cut the leaf image 𝐼 , we can achieve another two rectangular patches as shown in Fig. 2 (b ). We denote them as 𝑃 In this subsection, we use each patch pair in TPCS to measure the local self-similarity of leaf images in deep feature domain. For each pair of patches 𝑃(𝑡 ) 𝑙 , 𝑃 (𝑡 ) 𝑟in TPCS, we feed them through a pre-trained CNN network to extract two feature maps, 𝑓 𝑡 𝑙 and 𝑓 𝑡 𝑟 ∈ 𝑅 𝐻 ×𝑊 ×𝐶 , respectively, from the same convolutional layer, where 𝐻 ×𝑊 and 𝐶 are the spatial resolution and the number of the EQUATIONWe conduct channel-wise average pooling against it to yield a feature vector 𝑆 (𝑡 ) ∈ 𝑅 𝐶 . Fig. 4 presents a flow chart to illustrate the procedure of constructing the vector 𝑆 (𝑡 ) . Then using the TPCS and a pre-trained CNN network, we get three feature vectors 𝑆 (𝑡 ) , 𝑡 = 1, 2, 3. Through normalizing them with L2-norm and concatenating them, we can build a descriptorEQUATIONassociated with the contour point 𝑧 𝑖 under the parameter 𝜌, to measure the local self-similarity of the leaf image. We tern it Local Deep Self-Similarity Descriptor. The ensemble of multiple layers of CNN models can boost the final recognition performance [31] . However, the feature vectors extracted from different layers generally have different dimensionalities. The simple concatenation of them may not make them complementary with each other to contribute to the performance improvement. Considering that developing a novel feature fusion scheme is not the focus of this study, we adopt a recent feature fusion scheme, KNN-HDFF, proposed by [5] . KNN-HDFF is a simple way originally designed for fusing handcrafted features and deep features. It considers the neighbouring information of different feature spaces in the normalization of distance measures. In this study, we use it to fuse the SDePRs constructed from different layers. Limited by the length of the article, we omit the introduction of its details (refer to [5] for the details of KNN-HDFF). 
experiments: 
 In this section, to gauge the performance of the proposed method for fine-grained leaf image retrieval (FGLIR), we conduct extensive experiments on the publicly available benchmarks and compare our method with the state-of-the-art methods with standard evaluation metrics. Two standard evaluation metrics, Bulls-eye test [5, 6, 10, 16, 27] and precision-recall (PR) curves [5, 6, 26] are used to quantify the retrieval performance of all the competing algorithms. Eight stateof-the-art image descriptors, Local RsCoM [29] , SBT [5] , FB-BDP [6] , DSFH [17] , ReSW [20] , HeW-ResNet50 [19] , DOLG [34] , and SENet [14] are used as baselines. Among them, the first three descriptors are recently published handcrafted leaf image descriptors focusing on the characterization of fine-grained leaf images and report state-of-the-art performance on fine-grained leaf image recognition. While the last five descriptors are based on deep CNN features and achieve state-of-the-art performance on image retrieval tasks. DSFH [17] used the off-the-shelf features extracted from the fullyconnected layer of pr-trained VGG-16 model. ReSW [20] used the the features of convolutional layers from the pre-trained or finetuned VGG-16 models. HeW-ResNet50 [19] extracted patch-level faetures from the last convolutional layer of the pre-trained and fine-tuned ResNet50 models. While as for DOLG [34] and SENet [14] , they both took ResNet50 as backbone and consider the output of the convolutional layers for constructing their own network. Their networks require an additional dataset such as Google landmarks dataset V2 (GLDV2) [33] to train them. Their parameters all follow their original settings.All the competing methods are performed on a computer equipped with an intel i5-10500 CPU and a NVIDIA RTX 3070 GPU. Since our method focuses on developing training-free deep feature representation for FGLIR, a new challenging unsupervised pattern recognition task, we directly use the pre-trained CNN model and treat it as an generic feature extractor. VGG-16 is a popular pre-trained model which has been widely used for various pattern recognition tasks due to its great generalization ability and low computation cost. We therefore choose it as the extractor of feature maps for constructing our SDePR image-level representation. We use the publicly available VGG-16 model (imagenet-matconvnet-vgg-verydeep-16) using the open-source library MatConvNet. The existing VGG-16 based methods generally use layers in the 5th convolutional blocks such as 'pool5', 'conv5-2' and 'conv5-3' for feature extraction. Different from them, besides the the use of 'pool5', we also consider the use of 'pool4' due to its preservation of richer spatial information over the deeper layers which benefit the capturing of structural and spatial-contextual information. We use the KNN-HDFF [5] to fuse the SDePRs from the 'pool4' and 'pool5' layers of pre-trained VGG-16 for image retrieval. The parameter settings for our method are: the number of the points sampled from the contour is 𝑁 = 64 and the number of scale levels are 𝛷 = 4.Soybeans are commonly known as one of the most important economic crops in the world. SoyCultivar200 [30] is a public available soybean leaf image dataset. It has 200 cultivars with each having 30 leaves collected from different parts of soybean plants: 10 samples from the upper part, 10 samples from the middle part, and 10 samples from the lower part. All the samples from the same part of soybean plants are grouped into a subset which makes the dataset divided into three subsets, Soy-Up, Soy-Mid, and Soy-Low, respectively. Each subset accordingly consists of 200×10=2000 single leaf patterns. Besides the above three subsets focusing on the testing of single leaf image patterns, this benchmark also designs another evaluation protocol on the use of joint leaf image patterns. In this protocol, all the 6000 leaves in the dataset are divided into 2000 groups with each containing three leaves of the same cultivar separately from the upper part, middle part and lower part of different soybean plants. Each group is treated as a joint leaf pattern and all of them form a set, Soy-Joint, consisting of 200×10=2000 joint leaf patterns. For a evaluated method, its feature descriptors extracted from the joint leaf patterns are concatenated as a single vector for image retrieval. Fig. 6 shows parts of leaf images of different cultivars from the three subsets, Soy-Up, Soy-Mid, and Soy-Low. More details about the SoyCultivar200 dataset refer to [30] .The Bull-eye scores of all the competing methods are summarized in Table 1 . It can be seen that on the use of single leaf image patterns (performing on the test cases, Soy-Up, Soy-Mid and Soy-Low, respectively), the proposed methods achieves the scores of 65.59%, 64.28%, and 59.91% which are separately 4.55%, 6.09% and 3.77% higher than the other competing methods. While using joint leaf image patterns (Soy-joint), the proposed method achieves an exciting score of 92.64% which outperforms the other competing methods by 1.98%. We also report the results of our method on using 'pool4' or 'pool5' layer for soybean FGLIR. Although their scores are both lower than those of using the fusion of 'pool4' and 'pool5' layers, they are still significantly higher than those of all the other competing methods. We plot the PR curves for all the competing methods on the four test cases in Fig. 7 . It can be clearly observed that on all the test cases, the proposed method consistently achieves the best PR curves. Peanut is another economic crop in the world which is widely grown in the tropics and subtropics, contributing to both small and large commercial producers. The PeanCultiar120 dataset [6] is another public available benchmark designed for FGLIR. It has 600 leaf images from 120 peanut cultivars with each consisting of 5 samples. Different from the SoyCultivar200, this dataset is designed only for testing the performance of single leaf image pattern. An example sample for each peanut cultivar is shown in Fig. 8 .The Bull-eye scores for all the competing methods are summarized in Table 2 . As can be seen that the proposed method obtains the score of 59.93% which out-performers the other competing methods by 5.4%. We also report the scores of our method on using 'pool4' or 'pool5' layer for peanut FGLIR. Although their scores are both lower than that of using their fusion, they still achieve better scores over the all the other competing methods. In Fig. 9 , Table 1 : The Bull-eye scores (%) of all the competing methods on the three benchmarks, Soy-Up, Soy-Mid, Soy-Low of the SoyCultivar200 leaf image dataset [30] . we plot the PR curves of all the competing methods. It can be seen that the PR curve achieved by our method is obviously better than those of the other methods which consistently indicate its superior performance over the state-of-the-art methods on FGLIR. In this subsection, we use the PeanCultiar120 dataset to separately study the influences of the choice of layers in the pre-trained VGG-16 model. For the choice of layers, we first conduct a group of experiments on the individual use of 'poo1', 'poo2', 'poo3', 'poo4', and 'poo5'. The experimental results are summarized in Table 3 . It can be seen that the features extracted from the deep layers work better than those of shallow layers due to the lack of high-level semantic information. However, due to the strong ability of our proposed SDePR on modelling the structural and spatial-contextual information, the use of middle layer ('pool3') for our method achieves the score 55.43% which still outperforms all the other competing methods. Next, we conduct another group of experiments to fuse the SDePR of 'pool5' with the SDePRs of 'pool4', 'pool3', 'pool2' and 'pool1'. The results are summarized in Table 4 . It can be seen that compared with the individual use of 'pool5' layer, the fusions of 'pool5' with 'pool3' and 'pool4', respectively can both significantly improve the retrieval performance (resulting an increase of about 2%). 
conclusion: 
 Fine-grained leaf image retrieval (FGLIR) is a new challenging unsupervised pattern recognition task. Due to the very high inter-class similarity across different cultivars, FGLIR tasks require powerful leaf image descriptors to handle the subtle difference among varieties within the same plant species. In this work, we make the first attempt to investigate the possible way to encode spatial structure and contextual information into the image-level representation for , have the strong ability to capture the spatial structure and contextual information of leaf images which make it more suitable for depicting fine-grained leaf images. We conduct extensive experiments to apply our proposed SDePR method to the public challenging fine-grained leaf image retrieval tasks which greatly improve the retrieval rates of the state-of-the-art methods. 
: 
 For instance, there have been about 2500 soybean cultivars released since the scientific breeding of soybean cultivars began in the early 20th century [18] . Fig. 1 presents some typical leaves of different soybean cultivars. It can be seen that there are very high similarity of leaf image patterns across different cultivars which accordingly have caused increasing concerns [12, 25, 29, 30] about whether leaf image pattern can be effectively used for distinguishing cultivars.Unlike conventional leaf image retrieval task that focuses on distinguishing spe-cies, fine-grained leaf image retrieval (FGLIR) aims at retrieving leaf images belong-ing to the diverse cultivars of a certain plant species (e.g. soybean) and returning leaf images with the same cultivar as the query leaf image [5, 6] . Due to the great high inter-class similarity as shown in Fig. 1 , FGLIR task requires cultivar-level leaf image similarity, i.e., fine-grained image similarity in which two images are considered similar if and only if they belong to the same cultivar. Mining discriminative leaf image features is at the core of finding fine-grained image similarity. A desirable leaf image feature representation is expected to narrow the intra-class distance while increase the inter-class distance as much as possible in image feature space.In the past decades, a large body of leaf image descriptors have been devoted to plant species or cultivar recognition tasks. Most of them are based on handcrafted features which take shape, texture, vein, or edge patterns of leaf images as main clues. Although some methods [4, 21, 22] have reported high accuracies on species recognition, they are still limited to capture the subtle difference of leaf image pattern across different cultivars in FGLIR task due to the intrinsic essence of handcrafted features heavily depending on the human expertise. Inspired by the great success of deep learning in various image recognition tasks, some recent attempts [9, 15, 32] have been made to employ deep learning features to leaf image recognition. However, these methods focuses on leaf image classification, a supervised pattern recognition task, and their performance depend on a large body of labeled training samples.FGLIR is a typical unsupervised pattern recognition task. Its characteristics of fine-grained leaf images and unlabeled dataset make it more challenging than other leaf image recognition tasks. Although some recent efforts [14, 17, 19, 34] have been made to employ deep features for image retrieval, few attention has been paid on leveraging deep features for FGLIR. In this paper, we investigate the possible way to enhance the discriminative power of deep convolutional features for setting a new state-of-the-art performance on fine-grained leaf image retrieval.Image descriptors produced by deep convolutional neural networks (CNN) have emerged as state-of-the art generic descriptors for various visual recognition tasks [7, 23, 31] . Our work also relies on using CNN activations as off-the-shelf features for addressing the challenging FGLIR. Some CNN based image retrieval methods [20, 27, 38] focus on extracting feature from the fully-connected layers. However they are limited to capture the spatial structural features of images due to the loss of spatial information in the fullyconnected layers. Recently, most of methods [2, 14, 19, 31, 34, 35] , concentrate on building image descriptors from the convolutional layers. Some of them [2, 19, 31] mainly pay attention on the feature choice or aggregation of the convolutional layers and ignore the useful spatial structure feature extraction. Although some recent at-tempts [14, 34, 35] have been made to encode the spatial structure and contextual information from feature maps of the convolutional layers into the final image descriptors, they require additional labeled datasets to train the networks.In this work, we propose a novel CNN feature based leaf image feature representa-tion, named Structural Deep Patch Representation (SDePR), for fine-grained leaf image retrieval (FGLIR). The contributions of this study is highlighted as follows: (1) We for the first time investigate the possible way to construct leaf image descriptors that can encode the spatial structure and contextual information from the feature maps of the convolutional layers in a training-free manner. (2) We design a triplet patch-pair composite structure (TPCS), to measure the local structure properties and spatial correlations of leaf images. (3) We conduct extensive experiments to apply our proposed SDePR method to the public challenging fine-grained leaf image da-tasets and greatly improve the retrieval rates of the state-of-the-art methods. 
 
 Appendix:  


Decision: The paper received mixed scores. The average score is on the positive side, however there still are concerns.

Reviews: review: paper_summary: paper_summary: This article conducts research on fine-grained leaf image retrieval using unsupervised learning. By designing a triple patch-pair compositive structure (TPCS) module, a large number of patches on the leaf surface are extracted. Then, local structure properties and spatial correlations are obtained through CNN, and image retrieval is further achieved through the fusion of multi-scale and multi-layer CNN features. The effectiveness of the proposed method has been demonstrated through a large number of comparative experiments.
strength_weakness: strength_weakness: (1) This proposed method is special. Unsupervised methods have better generalization compared to supervised ones.

(2) Comparative experiments on two datasets have demonstrated the effectiveness of the features proposed in this paper.

(3) This paper is well written and easy to understand.
limitations: limitations: (1) Pretrained CNN is crucial for extracting leaf features. This paper chooses imagenet-matconvnet-vgg-verydeep-16 as the feature extractor. Could you directly initialize the weights of CNN randomly as baseline, and choose other datasets to train VGG-16 from scratch, and then use it as the feature extractor for comparison. You could choose leaf/plant datasets with similar data sizes, as well as other datasets unrelated to leaves, such as Cats/Dogs/Cars. This comparison may reflect the importance of the pretrained CNN feature extractor.

(2) Although this article is a training-free method, TPCS extracts a large number of patches, resulting in a high time complexity for extracting CNN features for each leaf. could you include the comparison results of testing time consumption in Table 1 or Table 2.

(3) N and φ should be discussed in the ablation study. 

(4) Minor errors, for example, are both Line 826 and Line 828 the results of "Proposed SDePR (Pool4)"? Spelling issues, such as "spe-ces" in line 86, "belong-ing" in line 87, "represneta- tion" in line 140, "da-tasets" in line 150, et al.

I am willing to revise my rating based on the authors’ feedback.
paper_summary: paper_summary: The paper presents a novel approach for fine-grained leaf image retrieval, focusing on the subtle differences among various cultivars within a species. It introduces a method called Structural Deep Patch Representation (SDePR), which utilizes convolutional neural network (CNN) activations to capture spatial and contextual information from leaf images. This approach leverages a Triplet Patch-Pairs Composite Structure (TPCS) to construct local deep self-similarity descriptors, allowing for effective retrieval across different scales and transformations. Extensive experiments on public datasets demonstrate that SDePR outperforms existing methods.
strength_weakness: strength_weakness: In general, the paper is well written and the motivation for this submission is easy to understand, but the novelty is limited.
limitations: limitations: 1. For Figure 4, do these two cnn's share parameters? Or are they trained separately by p_l^(t) and  p_r^(t).
2. For Figure 5, the description above mentions performing a scale rotation of the ρ, and what effect does this rotation angle have on the experimental results?
3. For ablation experiments, the description above mentions performing a scale rotation of the ρ, and what effect does this rotation angle have on the experimental results?
4.  What do the results of random sampling look like? For example, random blocks of leaves were collected as p_l^(t) and r_r^(t)
5. The method involves constructing multi-scale descriptors and employing complex structures like the Triplet Patch-Pairs Composite Structure (TPCS). The computational cost and efficiency of these processes are not discussed.
6.  Although the paper claims superior performance over state-of-the-art methods, the comparison mainly focuses on other leaf image retrieval methods. There is a lack of comparison with the latest general image retrieval techniques that use more advanced deep learning strategies
paper_summary: paper_summary: This paper proposes a Structural Deep Patch Representation (SDePR) for fine-grained leaf image retrieval (FGLIR). The proposed method investigates the possible way to mine the spatial structure and contextual information from the activation of the convolutional layers of CNN networks for FGLIR. Extensive experimental results and visual analysis show that the proposed method outperforms the state-of-the-art handcrafted visual features and deep retrieval models.
strength_weakness: strength_weakness: 1. The topic of this paper is interesting. A Triplet Patch-Pairs Composite Structure (TPCS) is introduced to extract the texture information of leaf images.
2. This paper utilizes the pre-trained CNN method to extract convolutional features from leaf image patches that represent spatial structure and contextual information.
3. Extensive experiments demonstrate the effectiveness of the proposed method in fine-grained leaf image retrieval tasks.
limitations: limitations: 1. What are the differences between the proposed Triplet Patch-Pairs Composite Structure (TPCS) in this paper and the existing FGLIR methods?
2. The authors should present figures demonstrating the differences between the proposed method and existing FGLIR methods to highlight the innovations of the proposed method.
3. This paper lacks an overall flowchart demonstrating the proposed method to ensure that readers can understand the entire process of how leaf image retrieval is implemented.
4. In the ablation experiments, the authors should present comparative results showing whether or not TPCS and multi-scale features are used.
5. What are the advantages of the proposed method compared to inputting the entire leaf image into a CNN network for feature extraction?
6. The authors should provide source code to enhance the replicability of the proposed method.
paper_summary: paper_summary: In this study, the authors for the first time investigate the possible way to mine the spatial structure and contextual information from the activation of the convolutional layers of CNN networks for FGLIR.
strength_weakness: strength_weakness: The paper for the first time investigate the possible way to mine the spatial structure and contextual information from the activation of the convolutional layers of CNN networks for FGLIR. For achieving this goal, authors design a novel geometrical structure, named Triplet Patch-Pairs Composite Structure (TPCS).
limitations: limitations: (1) There are some errors in the paper, such as “SDEPR” in the abstract, bold in Table 1, naming of the last algorithm in Table 2, and spelling errors in Table 3, which need to be carefully corrected.
(2) A leaf image is uniformly sampled with 64 points, together with multi-scale variations, yielding more local images. Therefore, how efficient is SDePR?