Writer identification and writer retrieval based on NetVLAD with Re‐ranking

IET BiometricsEarly View ORIGINAL RESEARCH PAPEROpen Access Writer identification and writer retrieval based on NetVLAD with Re-ranking Shervin Rasoulzadeh, Shervin Rasoulzadeh orcid.org/0000-0002-0019-0137 School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, IranSearch for more papers by this authorBagher BabaAli, Corresponding Author Bagher BabaAli babaali@ut.ac.ir orcid.org/0000-0001-8064-580X School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran Correspondence Bagher BabaAli, School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran. Email: babaali@ut.ac.irSearch for more papers by this author Shervin Rasoulzadeh, Shervin Rasoulzadeh orcid.org/0000-0002-0019-0137 School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, IranSearch for more papers by this authorBagher BabaAli, Corresponding Author Bagher BabaAli babaali@ut.ac.ir orcid.org/0000-0001-8064-580X School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran Correspondence Bagher BabaAli, School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran. Email: babaali@ut.ac.irSearch for more papers by this author First published: 26 April 2021 https://doi.org/10.1049/bme2.12039AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract The issue of writer identification and writer retrieval, which is considered a challenging problem in the field of document analysis and recognition is addressed here.. A novel pipeline is proposed for the problem at hand by employing a unified neural network architecture consisting of the ResNet-20 as a feature extractor and an integrated NetVLAD layer, inspired by the vector of locally aggregated descriptors (VLAD), in the head of the latter part. Having defined this architecture, the triplet semi-hard loss function is used to directly learn an embedding for individual input image patches. Subsequently, the generalised max-pooling technique is employed for the aggregation of embedded descriptors of each handwritten image. Also, a novel re-ranking strategy is introduced for the task of identification and retrieval based on the k-reciprocal nearest neighbours, and it is shown that the pipeline can benefit tremendously from this step. Experimental evaluation has been done on the three publicly available datasets: the ICDAR 2013, CVL, and KHATT datasets. Results indicate that while the performance was comparable to the state-of-the-art KHATT, the writer identification and writer retrieval pipeline achieve superior performance on the ICDAR 2013 and CVL datasets in terms of mAP. 1 INTRODUCTION Along with biometrics identifiers such as DNA, fingerprints, etc., handwriting is considered a special case of behavioural biometrics [1]. Handwriting analysis helps to extract attributes such as the writer from a handwritten document. Several factors may lead to handwriting variability, such as having diseases (Parkinson could be recalled as an example), education, time and effort spent, different pens etc. [2]. Thus, handwriting analysis is considered a complex and challenging task. To overcome the challenges and to provide an automatic handwriting analysis system, one needs to differentiate between online and offline data/systems. Online text analysis systems capture the whole procedure of writing with special devices and the input consists of temporal data such as pen-tip positions. On the other hand, offline data is usually static and is typically in the format of an image. Also, methods for handwriting analysis are categorised into two categories: text-dependent and text-independent methods. In the text-dependent methods each handwriting must contain a fixed content, while in the text-independent methods no assumptions are made on the content of the handwriting and any arbitrary text could constitute the handwriting. Here, the provision of an offline text-independent handwriting analysis system concentrated on the problem of writer recognition, specifically writer identification and writer retrieval is intended. Writer retrieval is the task of ranking document images from a large corpus of data with similar handwriting to the query sample, see Figure 1. Experts in relative fields then analyse these rankings, and thus, new documents from the same writer can be found. Historians and paleographers benefit the most from this scenario. When analysing historical documents a vast amount of data should be dealt with, where examining them individually is not possible or a very time-consuming task. However, this scenario helps to find the writer of a historical document in a shortlist without having to go through all the documents in the database. In contrast to writer retrieval, writer identification is the task of finding the writer of a query sample by assuming a set of documents where each one's writer is known in advance, see Figure 2. This scenario is often applicable in forensic sciences, for example finding the writer of a threatening letter. FIGURE 1Open in figure viewerPowerPoint Overall schema of writer retrieval FIGURE 2Open in figure viewerPowerPoint Overall schema of writer identification The methods for the two scenarios are quite similar. Both generate a feature vector describing the handwriting of each document with respect to its writer. These feature vectors are compared using a similar measure criterion such as the cosine distance, χ2-distance etc., and then the rankings are computed accordingly. For the retrieval scenario, these rankings are further analysed and a shortlist of documents written by the query's writer is returned. On the other hand, for the identification scenario, the writer of the sample with the shortest distance to the query sample is returned as the output. Our proposed pipeline uses a ResNet-20 convolutional neural network (CNN) with the NetVLAD layer, inspired by Vectors of Locally Aggregated Descriptors (VLAD), to extract local descriptors and their corresponding embeddings describing small windows in a document image. Afterwards, generalised max-pooling (GMP) aggregation is employed to achieve a single global image descriptor for each image from the computed embeddings. Dimensionality reduced global image descriptors resulting from application of the principal component analysis (PCA) are then compared, and rankings are computed. In the final stage, we make use of a re-ranking strategy based on query expansion and the k-reciprocal nearest neighbours to improve the retrieved rankings before evaluation. Here, in Section 2 some deep-learning-based related works in the field of writer identification and retrieval, as well as re-rankings are discussed. Then, in Section 3 the proposed pipeline is investigated in great detail. Section 4 introduces datasets and finally, the proposed pipeline is evaluated and compared with three different state-of-the-art datasets (ICDAR 2013, CVL, and KHATT) in Section 5. 2 RELATED WORK Nearly all common writer identification and retrieval datasets consist of writer disjoint train and test sets. Hence, end-to-end training cannot be applied. One of the first writer recognition methods using deep learning techniques was proposed by Fiel and Sablatnig [3]. They trained the 'CaffeNet' CNN on line and word segmentations. Feature vectors extracted from the penultimate layer of the CNN are compared using the χ2-distance. At the time, their results showed superior performance on the IAM [4] and ICFHR′12 datasets while showing inferior performance on the ICDAR 2013 [5] dataset. Christlein et al. [6] used activation features from a CNN as image local descriptors. Afterwards, they formed global image descriptors through the GMM supervector encoding. Their approach showed 0.21% improvement in terms of mAP on the ICDAR13 dataset. In another recent method by Christlein et al. [7] LeNet and ResNet architectures were employed to extract local descriptors followed by VLAD encoding to compute global image descriptors for document images. They experimented with both exemplar support vector machines (ESVMs) and nearest neighbours to evaluate their pipeline. To the best of our knowledge, their approach has set new standards on the ICDAR 2013 and CVL [8] datasets. In [9] Jordan et al. experimented with reciprocal relationships in two ways. First, they integrated them into the Jaccard distance and computed the final rankings based on a weighted sum of the original distance and the Jaccard distance. Second, they encoded them into new feature vectors, and hence, expanded the positive set for ESVMs. As a result, both their techniques outperformed the baseline on the ICDAR 2017 dataset [10]. Tang and Wu [11] proposed a novel approach with the convolutional neural network and joint Bayesian consisting of two stages: 1. feature extraction and 2. writer identification. They used CNNs to extract global features instead of small image patches. They used random word segmentations and generated 500 and 20 training samples per writer for training and testing, respectively. Finally, they used a Bayesian network for the computation of similarity between feature vectors. At the time, they achieved the best results when compared to the state-of-the-art ICDAR 2013 and CVL datasets. In another work by Xing and Qiao [12], two adjacent image patches were used as inputs to their proposed network, named DeepWriter, consisting of two branches sharing the convolutional layers. For the final evaluation part, two softmax layers belonging to each branch were averaged to predict the writer and achieved promising results on the IAM dataset. Despite that they worked in an end-to-end manner (i.e. 'the CNN is trained for a specific writer on a line-basis using some lines for training, one for validation and one for testing'. [7].), comparison of their work with other pipelines in the literature is impossible. Considering that, we could say that our proposed pipeline is mostly inspired by the works of Christlein et al. [6, 7] and Jordan et al. [9]. However, with our proposed pipeline consisting of the unified neural network architecture with the NetVLAD layer, and re-ranking strategy-based query expansion and the k-reciprocal nearest neighbours, we could improve on the state-of-the-art ICDAR 2013 and CVL datasets. 3 THE PROPOSED PIPELINE FOR WRITER IDENTIFICATION AND WRITER RETRIEVAL Our proposed pipeline consists of two parts: 1. A unified neural network architecture with ResNet-20 [13, 14] and the NetVLAD layer [15] and 2. A re-ranking strategy to improve the final results. The first part itself consists of three main steps (depicted in Figure 3): The ResNet-20 with the NetVLAD layer to extract the local image descriptors and their corresponding embeddings, An orderless aggregation function to the pool obtained embeddings of each image into one global image descriptor, and the normaliation and PCA [16]-based dimensionality reduction of the resulting global image descriptors. FIGURE 3Open in figure viewerPowerPoint Overview of the proposed pipeline. PCA, principal component analysis 3.1 Convolutional Neural network with the NetVLAD Layer State-of-the-art deep-learning-based methods in writer identification and writer retrieval usually employ a CNN to extract the local image descriptors, which are subsequently encoded using an encoding method. An encoding consists of two steps: (1) An embedding step, where local feature vectors are projected into a possibly high-dimensional space, and (2) An aggregation step, in which embedded local feature vectors of each input image are pooled into one global image descriptor. Christlein et al. [7] computed the local feature vectors through a ResNet-20 residual neural network and used the VLAD [17] encoding method for embedding and aggregation. Building on the success of Christlein et al. [7] we propose a unified neural network consisting of the ResNet-20 followed by the trainable NetVLAD layer [15], inspired by the VLAD at the head of the last convolutional layer of the ResNet-20 to learn the embedding of feature vectors in an end-to-end manner using a triplet loss [18]. 3.1.1 ResNet-20 and NetVLAD Details of the ResNet-20 CNN and the NetVLAD layer are described as following: ResNet-20 Convolutional Neural Network. For network inputs, 32 × 32 image patches centred at the contour of the handwriting were extracted. Same as Christlein et al. [7] we follow the architectural design of He et al. [13] on CIFAR10 dataset [19]. 6n + 2 layers are employed with n set to 3 leading to the ResNet-20 architecture. The first layer is 3 × 3 convolutions. Then a stack of 6n layers with 3 × 3 convolutions follows with every 2n layers forming a stage. At the beginning of each stage (except the first one), the feature map size is halved (downsampled) by a convolutional layer with the stride of 2, while the number of filters is doubled. Within each stage, the layers have the same number of filters. More precisely, feature maps and filters for stages are of sizes {32, 16, 8} and {16, 32, 64}, respectively. Shortcut connections are connected to the pairs of 3 × 3 layers leading to a total of 3n shortcuts. The network ends with the global average pooling layer with a size of 8 and an N-way fully connected layer. However, we discard the last fully connected layer and pass the 1 × 1 × 64 output feature vector of the global average pooling layer to the NetVLAD to further learn the VLAD embeddings of these feature vectors. (See Figure 4). FIGURE 4Open in figure viewerPowerPoint Convolutional neural network architecture composed of the ResNet-20 followed by the NetVLAD layer. Numbers in each rectangle denote kernel size, number of output filters, padding, and size of stride, respectively. The N-way fully connected is dropped and instead the 1 × 1 × 64 output vector is passed to the NetVLAD layer NetVLAD Layer. The idea behind the vectors of locally aggregated descriptors (VLAD) [17] is to compute the embeddings by means of residuals xi − cj for each local image descriptor xi. Finally, embedded local image descriptors of each image are accumulated by an orderless aggregation function. This characterises the distribution of the vectors with respect to the cluster centres. The VLAD embedding can be regarded as a simplified version of the Fisher Vectors [17]. More precisely, given N local image descriptors { x i | x i ∈ R D , i = 1 , … , N } and a dictionary of K cluster centres { c j | c j ∈ R D , j = 1 , … , K } , the VLAD embedding function is computed as follows: ϕ VLAD , k ( x i ) = α k ( x i ) ( x i − c k ) (1) α k ( x i ) = 1 , if k = argmin j = 1 , … K ‖ x i − c j ‖ 0 , else (2) Then ϕVLAD(xi) = (ϕ1(xi), …, ϕK(xi)) represents the full embedding for each local image descriptor xi. Arandjelovic et al [15] introduced a trainable generalised VLAD layer, named NetVLAD, which is pluggable into any CNN architecture. Clusters and hard-assignments ak(xi) of local image descriptors in the original VLAD are the source of discontinuities which prevent differentiability in the backpropagation procedure. The authors replaced them with a soft-assignment to make it amenable to backpropagation: α ¯ k ( x i ) = e − α ‖ x i − c k ‖ 2 ∑ j = 1 K e − α ‖ x i − c j ‖ 2 (3)where α is a parameter that controls the decay of response with the magnitude of distance. Intuitively, Equation (3) assigns the weights of local image descriptors xi proportional to their nearness to clusters cj. Moreover, factorising e − α ‖ x i ‖ 2 results in α ¯ k ( x i ) = e w k T x i + b k ∑ j = 1 K e w j T x i + b j (4)where wk = 2αck and bk = −α‖ck‖2. However, in [20] the authors propose decoupling dependencies of parameters ck, wk, and bk as it will bring greater flexibility to the model. In this manner, the NetVLAD layer consists of three independent set of learnable parameters. We crop the ResNet-20 at the last convolutional layer and view it as a D-dimensional (here D = 64) local image descriptor. As depicted in Figure 4, the NetVLAD layer can be decomposed into CNN layers connected in an acyclic graph. Equation (4) represents the soft-max activation function. So the soft-assignments of local image descriptor xi to clusters ck can be viewed as applying a 1 × 1 convolution layer with the K filters representing wk and biases as bk followed by the soft-max activation function to obtain the final soft-assignments α ¯ k ( x i ) . The final output is K × D × 1-dimensional representing the full embedding for the local image descriptor xi. The authors in [20] regard the output of the last convolutional layer with H × W × D map as a set of D-dimensional descriptors at H × W spatial locations in input image which are further embedded and pooled by the NetVLAD layer. However, by using the ResNet-20 with image patches of size 32 × 32 as feature extractors, the output of the last convolutional layer becomes 1 × 1 × 64 map which we consider as a 64-dimensional local image descriptor extracted from the input image patch. Passing this descriptor (with H = 1 and W = 1) enables the NetVLAD layer to learn the respective local image descriptor embedding. So in this manner, the NetVLAD layer functions to learn the generalised VLAD embeddings. Prior to forwarding the embeddings extracted from the NetVLAD layer to the triplet loss, they are ℓ2-normalised. Learning from Semi-Hard Triplets. A good strategy to learn the parameters of the network to reach a good encoding is through the triplet loss function. We wish to learn the VLAD embedding representation ϕVLAD(x) constrained to lie on the K × D-dimensional hypersphare, that is ‖ϕVLAD(x)‖ = 1, such that two embeddings belonging to the image(s) of the same writer are close together in the embedding space, while embeddings of images with different writers lie far away from each other. The only requirement is that given two positive embeddings of the same writer and one negative embedding, the negative should be farther away than the positive by some margin m. This requirement can be translated into a loss between triplets. The loss will be defined over triplets of embeddings: an anchor ϕa, a positive of the same writer as the anchor ϕp, and a negative of a different writer ϕn. For some distance on the embedding space d, the loss of a triplets (ϕa, ϕp, ϕn) is as follows: L = max ( d ( ϕ a , ϕ p ) − d ( ϕ a , ϕ n ) + m a r g i n , 0 ) (5)The original NetVLAD paper utilises the weakly supervised triplet ranking loss [15]. However, since the NetVLAD layer is applied here to learn embeddings in a patch-wise manner, another strategy is employed. Based on the definition of loss we tend to train on semi-hard triplets [18]: triplets where the negative is not closer to the anchor than the positive, but which still produce positive loss: d ( ϕ a , ϕ p ) < d ( ϕ a , ϕ n ) < d ( ϕ a , ϕ p ) + m a r g i n . (6) We train parameters of the proposed pipeline on a large set of semi-hard triplets and image patches triplets extracted from the respective datasets. Details and parameters of the training are given in Section 4. 3.1.2 Aggregation The aggregation step is required to obtain a single vector representing each image from its embedded local descriptors. Default aggregation method is sum-pooling. Assuming the set of N local descriptors X = { x i | i = 1 , … , N } for an image, sum-pooling constructs global descriptor ξ as follows: ξ = ψ ( ϕ ( X ) ) = ∑ x ∈ X ϕ ( x ) . (7) 'Since we sum over all descriptors, the aggregated descriptors can suffer from interference of unrelated descriptors that influence the similarity, even if they have low individual similarity' [21] as the similarity K ( X , Y ) between two images represented by sets X and Y is computed as follows: K ( X , Y ) = ∑ x ∈ X ∑ y ∈ Y ϕ ( x ) ⋅ ϕ ( y ) . (8) Hence, more frequently occurring descriptors will be more influential in the final representation and affect the final similarity between global descriptors. This phenomenon is called visual burstiness [22]. Recently, a novel approach named generalised max-pooling [23] was proposed to overcome this problem and has been successfully applied in the field of writer identification and writer retrieval in the works of Christlein et al. [7]. We employed this method in our pipeline as it has shown superior performance compared to the other two methods [24]. Generalised max-pooling balances the contribution of every embedding ϕ ( x ) ∈ R K × D where x ∈ R D is the local image descriptor, by solving a ridge regression problem. Therefore, ϕ ( x ) T ξ g m p ( X ) = C , ∀ x ∈ X , (9)where X is the set of all the local descriptors of an image, ξgmp denotes the aggregated global image descriptor and C is a constant that can be set arbitrarily since it does not influence the global image descriptors as they are subsequently normalised in the post-processing step. Equation (9) can be re-formulated for all N local image descriptors of each image as shown below: Φ T ξ g m p = 1 N , (10)where Φ and 1N denote the (K × D) × N matrix of all local image descriptor embeddings and vector of N constants set to 1, respectively. Equation (10) can be turned into a least-squares ridge regression problem [2, 7]: ξ g m p = argmin ξ ‖ Φ T ξ − 1 N ‖ 2 2 + λ ‖ ξ ‖ 2 2 (11)with λ being a regularisation parameter. In the remainder of this work, ψ denotes the aggregated global image descriptor. 3.1.3 Normalisation and dimensionality reduction While working with global image descriptors obtained in the previous step, two challenges arise: (1) visual burstiness might corrupt the visual similarity measure between the global image descriptors, that is, the cosine distance used to rank images, and (2) these global descriptors lie in a very high-dimensional space and pipeline might benefit from projecting them onto a lower-dimensional space. We address these challenges with an additional normalisation and dimensionality reduction step. Power Normalisation. A normalisation method to counter visual burstiness is named power normalisation [25] that proposes to apply function f component-wise to the global image descriptor ψ, f ( ψ ) = s i g n ( ψ i ) | ψ i | p , ∀ i , 1 ≤ i ≤ n (12)where p is a normalisation parameter and is generally set to 0.5. Power normalisation is followed by ℓ2-normalisation. Principal Component Analysis. Due to the nature of VLAD encoding, global image descriptors lie in a very high-dimensional space. PCA [16] is used to dimensionally reduce the encoding representations. However, this introduces a new parameter, dimension, to the pipeline denoting the number of components to be kept. After performing the PCA, ℓ2-normalisation along each sample is necessary. 3.2 Re-ranking Writer identification and retrieval systems are evaluated using leave-one-image-out cross validation. Each image is once used as query q and the pipeline returns a ranked list L(q) of all other images in a test set (a.k.a gallery). These ranked lists are obtained by computing the pairwise distance between the query q and each p ∈ L(q) using a similarity measure criterion, that is cosine distance. Given two vectors p and q, the cosine distance is defined as follows: d cos ( p , q ) = 1 − p q ‖ p ‖ ‖ q ‖ . (13) Our goal is to re-rank each L(q) based on the knowledge that each contains, so that more relevant samples rank top in the list and thus, boost the performance of writer identification and retrieval. 3.2.1 Nearest neighbours The k-nearest neighbours kNN(q) (top-k ranked samples of ranked list) of query q are defined as follows: k NN ( q ) = { p 1 , p 2 , … , p k } , | k NN ( q ) | = k , (14) Where |.| denotes the cardinality of the set. The k-reciprocal nearest neighbours krNN(q, k) is defined as: k rNN ( q ) = { p i | p i ∈ k NN ( q ) ∧ q ∈ k NN ( p i ) } . (15) In other words, two samples q and p are considered as k-reciprocal nearest neighbours, when both appear within the top-k ranked samples of each other. According to the previous descriptions, thek-reciprocal nearest neighbours are more related to query q than the k-nearest neighbours. 3.2.2 Query expansion and k-Reciporal nearest neighbours A common approach to boost the performance of the information retrieval systems is automatic query expansion (QE) [9]. With an initial ranked list L(q) computed, query expansion reformulates each query sample q and obtains the improved ranked list by re-quering using the newly formed query instead of q. Chum et al. [26] proposed the following query expansion approach. For query q, a new query sample can be formed by taking the average over top-n spatially verified samples F from the ranked list L(q), q avg = 1 | F | + 1 q + ∑ f i n F f (16)where fi and n denote the ith sample in F and the total number of samples in | F |, respectively. In our problem, the features do not encode any global spatial information and thus, we have no spatial verification at hand. Averaging over top-k samples in L(q) is not much reliable since the top-k samples might contain false matches. We propose the use of a more constrained strategy by taking an average over query q and its k-rNNs in the initial ranked list to minimise the risk of including false matches. Hence, the newly formed query qnew is computed as follows: q new = 1 | k rNN ( q ) | + 1 q + ∑ r ∈ k rNN ( q ) r . (17) This, however, introduces a new hyper-parameter k to the pipeline. In the following, our proposed pipeline(with re-ranking) is denoted as 'Proposed(+krNN-QEk)'. 4 EVALUATION 4.1 Datasets Our primary dataset is the ICDAR 2013. However, we compare our results against the state-of-the-art CVL and KHATT datasets as well. These datasets contain documents written in English, Greek, German, and Arabic. Successful results on these datasets also show that the proposed pipeline is language-independent. The datasets' information are summarised in Table 1. TABLE 1. Summary of the datasets used in our experiments Dataset Language Train writers Test writers Train pages/writer Test pages/writer ICDAR 2013 English + Greek 100 250 4 4 CVL English + German 27 283 7 5 KHATT Arabic 700 150 4 4 ICDAR 2013: The ICDAR 2013 dataset was introduced for the ICDAR 2013 competition on writer identification [5]. It consists of four samples per writer, two of which are written in English, while the other two are in Greek. This dataset contains two disjoint train (a.k.a experimental) and test (a.k.a benchmarking) subsets. The train set consists of 100 writers, while the test set contains 250 writers. An example document from the ICDAR 2013 dataset can be seen in Figure 5. FIGURE 5Open in figure viewerPowerPoint Example document from the ICDAR13 dataset (ID: 032_1) CVL: The CVL dataset [8] (version 1.1) consists of 27 writers contributing to seven texts (one in German and six in English) in the officially provided train set. The test set consists of 283 writers, where each writer has copied five texts (one in German and four in English). Figure 6 depicts one sample document from this dataset. FIGURE 6Open in figure viewerPowerPoint Example document from the CVL dataset (ID: 0073-4) KHATT: The KHATT dataset [27] was used as a database for the ICFHR 2014 [28] Arabic writer identification competition. It consists of 1000 writers contributing to four samples. Two of these contain similar texts, while the other two were unique to each writer. The dataset is provided by three disjoint, that is writer independent sets, with a 4: 1: 1 ratio for train, test, and validation subsets, respectively. Along with two other datasets, a sample document image of this dataset is shown in Figure 7. FIGURE 7Open in figure viewerPowerPoint Example document from the KHATT dataset (ID: AHTD3A0003-1) 4.2 Metrics Results reported in terms of the Hard Top-N and the mAP which are quite common metrics in information retrieval tasks. These metrics are defined as follows: Hard Top-N , The strictest evaluation metric is the hard top-N. A returned list L(q) for query sample q is considered as acceptable [29] if all of the top-N ranked samples in L(q) belong to the same class as sample q's class that is writ