Original: Under review as a conference paper at ICLR

Original: D ENSITY -B ASED C LUSTERING WITH K ERNEL D IFFUSION
Revised: D ENSITY -B ASED C LUSTERING WITH K ERNEL D IFFUSION

Original: Anonymous authors Paper under double-blind review
Revised: Anonymous authors Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Finding a suitable density function is essential for density-based clustering algorithms such as DBSCAN and DPC.
Revised: Finding a suitable density function is essential for density-based clustering algorithms such as DBSCAN and DPC.

Original: A naive density corresponding to the indicator function of a unit d -dimensional Euclidean ball is commonly used in these algorithms.
Revised: A naive density corresponding to the indicator function of a unit d -dimensional Euclidean ball is commonly used in these algorithms.

Original: Such  density suffers from capturing local features in complex datasets.
Revised: Such a density suffers from an inability to capture local features in complex datasets.

Original: To tackle this issue, we propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.
Revised: To tackle this issue, we propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.

Original: Furthermore, we develop a surrogate that can be efﬁciently computed in linear time and space and prove that it is asymptotically equivalent to the kernel diffusion density function.
Revised: Furthermore, we develop a surrogate that can be efficiently computed in linear time and space and prove that it is asymptotically equivalent to the kernel diffusion density function.

Original: Extensive empirical experiments on benchmark and large-scale face image datasets show that the proposed approach not only achieves a signiﬁcant improvement over classic density-based clustering algorithms but also outperforms the state-of-the-art face clustering methods by a large margin.
Revised: Extensive empirical experiments on benchmark and large-scale face image datasets show that the proposed approach not only achieves a significant improvement over classic density-based clustering algorithms but also outperforms the state-of-the-art face clustering methods by a large margin.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Density-based clustering algorithms are now widely used in a variety of applications, ranging from high energy physics (Tramacere & Vecchio, 2012;
Revised: Density-based clustering algorithms are now widely used in a variety of applications, ranging from high energy physics (Tramacere & Vecchio, 2012;

Original: Rovere et al., 2020), material sciences (Marquis et al., 2019; Reza et al., 2007), social network analysis (Shi et al., 2014; Khatoon & Banu, 2019) to molecular biology (Cao et al., 2017; Ziegler et al., 2020).
Revised: Rovere et al., 2020), material sciences (Marquis et al., 2019; Reza et al., 2007), social network analysis (Shi et al., 2014; Khatoon & Banu, 2019) to molecular biology (Cao et al., 2017; Ziegler et al., 2020).

Original: In these algorithms, data points are partitioned into clusters that are considered to be sufﬁciently or locally high-density areas with respect to an underlying probability density or a similar reference function.
Revised: In these algorithms, data points are partitioned into clusters that are considered to be sufficiently or locally high-density areas with respect to an underlying probability density or a similar reference function.

Original: We call them density functions throughout this paper.
Revised: We call them density functions throughout this paper.

Original: These techniques are attractive to practitioners, due to their nonparametric feature, which leads to ﬂexibility in discovering clusters that have arbitrary shapes, whilst classic methods such as k -means and k -medoids (Friedman et al., 2001) can only detect convex (e.g., spherical) clusters.
Revised: These techniques are attractive to practitioners, due to their nonparametric feature, which leads to flexibility in discovering clusters that have arbitrary shapes, whilst classic methods such as k -means and k -medoids (Friedman et al., 2001) can only detect convex (e.g., spherical) clusters.

Original: Seminal work in the context of density-based clustering includes DBSCAN (Ester et al., 1996) and DPC (Rodriguez & Laio, 2014), among many others (Ankerst et al., 1999;
Revised: Seminal work in the context of density-based clustering includes DBSCAN (Ester et al., 1996) and DPC (Rodriguez & Laio, 2014), among many others (Ankerst et al., 1999;

Original: Cuevas et al., 2001; Comaniciu & Meer, 2002; Hinneburg & Gabriel, 2007; Stuetzle, 2003).
Revised: Cuevas et al., 2001; Comaniciu & Meer, 2002; Hinneburg & Gabriel, 2007; Stuetzle, 2003).

Original: Most density-based clustering algorithms implicitly identify cluster centers and assign remaining points to the clusters by connecting with the higher density point nearby.
Revised: Most density-based clustering algorithms implicitly identify cluster centers and assign remaining points to the clusters by connecting with the higher density point nearby.

Original: To proceed with these methods it requires a density function, which is usually an estimate of the underlying true probability density or some variants of it.
Revised: To proceed with these methods it requires a density function, which is usually an estimate of the underlying true probability density or some variants of it.

Original: For example, a popular choice is the naive density function that is carried out by simply calculating the number of data points covered in the " -neighborhood of each x .
Revised: For example, a popular choice is the naive density function that is carried out by simply calculating the number of data points covered in the ε -neighborhood of each x .

Original: Note that such densities are not adaptive to different distribution regions.
Revised: Note that such densities are not adaptive to different distribution regions.

Original: One of the challenging scenarios is when clusters in the data have varying local features, for example, size, height, spread, and smoothness.
Revised: One of the challenging scenarios is when clusters in the data have varying local features, for example, size, height, spread, and smoothness.

Original: Therefore, the resulting density function has a tendency to ﬂatten the peaks and valleys in the data distribution, which leads to underestimation of the number of clusters (see Figure 1).
Revised: Therefore, the resulting density function has a tendency to flatten the peaks and valleys in the data distribution, which leads to underestimation of the number of clusters (see Figure 1).

Original: Many heuristics variations of DBSCAN and DPC have been proposed to magnify the local features, thus making the clustering task easier (Campello et al., 2013; Chen et al., 2018; Ertöz et al., 2003; Zhu et al., 2016).
Revised: Many heuristics variations of DBSCAN and DPC have been proposed to magnify the local features, thus making the clustering task easier (Campello et al., 2013; Chen et al., 2018; Ertöz et al., 2003; Zhu et al., 2016).

Original: Most of these methods can be viewed as performing clustering on certain transformations of the naive density function.
Revised: Most of these methods can be viewed as performing clustering on certain transformations of the naive density function.

Original: However, if the naive density function itself is quite problematic in the ﬁrst place, these methods will become less effective.
Revised: However, if the naive density function itself is quite problematic in the first place, these methods will become less effective.

Original: Moreover, even if we apply adaptive alternatives to modify the classic density functions, there are other contentious issues of the generally used linear kernel density estimator (KDE).
Revised: Moreover, even if we apply adaptive alternatives to modify the classic density functions, there are other contentious issues of the generally used linear kernel density estimator (KDE).

Original: It often suffers from severe boundary bias (Marron & Ruppert, 1994) and is acknowledged as computationally expensive.
Revised: It often suffers from severe boundary bias (Marron & Ruppert, 1994) and is acknowledged as computationally expensive.

Original: These phenomenon prevent the classic density functions being practically useful and reliable, especially for large-scale and complex clustering tasks.
Revised: These phenomenon prevent the classic density functions being practically useful and reliable, especially for large-scale and complex clustering tasks.

Original: To overcome these problems in density-based clustering algorithms, in this paper we propose a general approach to build the so-called kernel diffusion density function to replace classic density functions.
Revised: To overcome these problems in density-based clustering algorithms, in this paper we propose a general approach to build the so-called kernel diffusion density function to replace classic density functions.

Original: The key idea is to construct the density from a user-speciﬁed bivariate kernel that has desired local adaptive properties.
Revised: The key idea is to construct the density from a user-specified bivariate kernel that has desired local adaptive properties.

Original: Instead of using the naive density function and its variants, we utilize the bivariate kernel to derive a transition probability.
Revised: Instead of using the naive density function and its variants, we utilize the bivariate kernel to derive a transition probability.

Original: A diffusion process is induced by this transition probability, which admits a limiting and stationary distribution.
Revised: A diffusion process is induced by this transition probability, which admits a limiting and stationary distribution.

Original: This limiting distribution serves as a plausible density function for clustering with reduced error.
Revised: This limiting distribution serves as a plausible density function for clustering with reduced error.

Original: Under this framework, we provide examples of symmetric and asymmetric bivariate kernels to construct the kernel diffusion density function, which can tackle clustering complex and locally varying data.
Revised: Under this framework, we provide examples of symmetric and asymmetric bivariate kernels to construct the kernel diffusion density function, which can tackle clustering complex and locally varying data.

Original: We apply the resulting adapted DBSCAN and DPC algorithms to widely different empirical datasets and show signiﬁcant improvement in each of these analyses.
Revised: We apply the resulting adapted DBSCAN and DPC algorithms to widely different empirical datasets and show significant improvement in each of these analyses.

Original: The main contributions of this paper are summarized below:
Revised: The main contributions of this paper are summarized below:

Original: • We introduce new bivariate kernel functions and construct the associated kernel diffusion processes.
Revised: • We introduce new bivariate kernel functions and construct the associated kernel diffusion processes.

Original: Based on the diffusion process, we propose a kernel diffusion density function to adapt density-based clustering algorithms such as DBSACN and DPC, which attains accuracy in the presence of varying local features.
Revised: Based on the diffusion process, we propose a kernel diffusion density function to adapt density-based clustering algorithms such as DBSACN and DPC, which attains accuracy in the presence of varying local features.

Original: • We derive a computationally much more efﬁcient surrogate, and show analytically it is asymptotic equivalent to the proposed kernel diffusion density function.
Revised: • We derive a computationally much more efficient surrogate, and show analytically it is asymptotic equivalent to the proposed kernel diffusion density function.

Original: • By extensive experiments, we demonstrate the superiority of kernel diffusion density function over naive density function and its variants when applying to DBSCAN and DPC, and show it outperforms state-of-the-art GCN-based methods on face clustering tasks.
Revised: • By extensive experiments, we demonstrate the superiority of kernel diffusion density function over naive density function and its variants when applying to DBSCAN and DPC, and show it outperforms state-of-the-art GCN-based methods on face clustering tasks.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Density-Based Clustering
Revised: Density-Based Clustering

Original: There is vast literature on adapting density-based clustering algorithms to tackle large variations in different clusters in the data.
Revised: There is vast literature on adapting density-based clustering algorithms to tackle large variations in different clusters in the data.

Original: DPC itself is such a reﬁnement of DBSCAN, as it determines cluster centers not only by highest density values but also by taking into account their distances from each other, thus has a generally better performance in complex clustering tasks.
Revised: DPC itself is such a refinement of DBSCAN, as it determines cluster centers not only by highest density values but also by taking into account their distances from each other, thus has a generally better performance in complex clustering tasks.

Original: Other attempts include rescaling the data to have relative reference measures instead of KDE (Zhu 
Revised: Other attempts include rescaling the data to have relative reference measures instead of KDE (Zhu et al., 2016;

Original: Under review as a conference paper at ICLRet al., 2016; Chen et al., 2018), and using the number of shared-nearest-neighbors between two points to replace the geometric distance (Ertöz et al., 2003).
Revised: Chen et al., 2018), and using the number of shared-nearest-neighbors between two points to replace the geometric distance (Ertöz et al., 2003).

Original: Diffusion Maps The technique of diffusion maps (Coifman et al., 2005; Coifman & Lafon, 2006) gives a multi-scale organization of the data according to their underlying geometric structure.
Revised: Diffusion Maps The technique of diffusion maps (Coifman et al., 2005; Coifman & Lafon, 2006) gives a multi-scale organization of the data according to their underlying geometric structure.

Original: It uses a local similarity measure to create a diffusion process on the data which integrates local geometry at different scales along time t .
Revised: It uses a local similarity measure to create a diffusion process on the data which integrates local geometry at different scales along time t .

Original: Generally speaking, the diffusion will segment the data into several smaller clusters in small t and group data into one cluster for large t .
Revised: Generally speaking, the diffusion will segment the data into several smaller clusters in small t and group data into one cluster for large t .

Original: Applying eigenfunctions at a carefully selected time t leads to good macroscopic representations of the data, which is useful in dimension reduction and spectral clustering (Nadler et al., 2005).
Revised: Applying eigenfunctions at a carefully selected time t leads to good macroscopic representations of the data, which is useful in dimension reduction and spectral clustering (Nadler et al., 2005).

Original: Face Clustering Face clustering has been extensively studied as an important application in machine learning.
Revised: Face Clustering Face clustering has been extensively studied as an important application in machine learning.

Original: Traditional algorithms include k -means, hierarchical clustering (Sibson, 1973) and ARO (Otto et al., 2017).
Revised: Traditional algorithms include k -means, hierarchical clustering (Sibson, 1973) and ARO (Otto et al., 2017).

Original: Many recent works take advantage of supervised information and
Revised: Many recent works take advantage of supervised information and

Original: GCN models, achieving impressive improvement comparing to traditional algorithms.
Revised: GCN models, achieving impressive improvement comparing to traditional algorithms.

Original: To name a few, CDP (Zhan et al., 2018) proposes to aggregate the features extracted by different models;
Revised: To name a few, CDP (Zhan et al., 2018) proposes to aggregate the features extracted by different models;

Original: L-GCN (Wang et al., 2019) predicts the linkage in an instance pivot subgraph;
Revised: L-GCN (Wang et al., 2019) predicts the linkage in an instance pivot subgraph;

Original: LTC (Yang et al., 2019) generates a series of subgraphs as proposals and detects face clusters thereon; and GCN(V+E)
Revised: LTC (Yang et al., 2019) generates a series of subgraphs as proposals and detects face clusters thereon; and GCN(V+E)

Original: (Yang et al., 2020) learns both the conﬁdence and connectivity by GCN.
Revised: (Yang et al., 2020) learns both the confidence and connectivity by GCN.

Original: In this paper we demonstrate that the proposed density-based clustering algorithm with kernel diffusion, as a general clustering approach, even outperforms theses state-of-the-art methods that are especially designed for face clustering.
Revised: In this paper we demonstrate that the proposed density-based clustering algorithm with kernel diffusion, as a general clustering approach, even outperforms theses state-of-the-art methods that are especially designed for face clustering.

Original:  P RELIMINARIES 3.1 N OTATIONS
Revised:  P RELIMINARIES 3. N OTATIONS

Original: Let the dataset D = { x 1 , . . .
Revised: Let the dataset D = { x 1 , . . .

Original: , x n } ⇢ R d be n i.i.d samples drawn from a distribution measure F with density f on R d .
Revised: , x n } ⊂ R d be n i.i.d samples drawn from a distribution measure F with density f on R d .

Original: Let F n denote the corresponding empirical distribution measured with respect to D , i.e., F n ( A )
Revised: Let F n denote the corresponding empirical distribution measured with respect to D , i.e., F n ( A )

Original: = 1 n 
Revised: = 1 n (cid:80) ni

Original: P ni =1 1 A ( x i )  where 1 A ( · ) denotes the indicator function of set A .
Revised: =1 1 A ( x i ) , where 1 A ( · ) denotes the indicator function of set A .

Original: We write || u
Revised: We write || u

Original: || as the Euclidean norm of vector u .
Revised: || as the Euclidean norm of vector u .

Original: Let B ( x, " ) and V d denote the d -dimensional " -ball centered at x and the volume of the unit ball B (0 , 1) , respectively.
Revised: Let B ( x, ε ) and V d denote the d -dimensional ε -ball centered at x and the volume of the unit ball B (0 , 1) , respectively.

Original: Let N k ( x ) denote the set of k -nearest neighbors of point x within the dataset D .
Revised: Let N k ( x ) denote the set of k -nearest neighbors of point x within the dataset D .

Original: 3.2 D ENSITY FUNCTION
Revised: 3. D ENSITY FUNCTION

Original: Density-based algorithms perform clustering by specifying and segmenting high-value areas in a density function denoted by ⇢ .
Revised: Density-based algorithms perform clustering by specifying and segmenting high-value areas in a density function denoted by ρ .

Original: Usually, we calculate each of ⇢ ( x i ) , and then identify cluster centers with (locally) highest values.
Revised: Usually, we calculate each of ρ ( x i ) , and then identify cluster centers with (locally) highest values.

Original: Many popular algorithms such as DBSCAN and DPC employ the following naive density function:
Revised: Many popular algorithms such as DBSCAN and DPC employ the following naive density function:

Original: [Equation]
Revised: [Equation]

Original: The naive density function ⇢ naive is actually an empirical estimation of f for carefully chosen " .
Revised: The naive density function ρ naive is actually an empirical estimation of f for carefully chosen ε .

Original: It is easy to observe, for clustering purpose we only care about ⇢ naive ( x ) up to a normalising constant, which makes it simply equivalent to counting the total number of data points in the " -ball around x .
Revised: It is easy to observe, for clustering purpose we only care about ρ naive ( x ) up to a normalising constant, which makes it simply equivalent to counting the total number of data points in the ε -ball around x .

Original: In practice, the data distribution may be very complex and contains varying local features that are difﬁcult to be detected.
Revised: In practice, the data distribution may be very complex and contains varying local features that are difficult to be detected.

Original: The naive density in (1) with the same radius " for all x usually suffers from unsatisfactory empirical performance, for example, failing to identify small clusters with fewer data points.
Revised: The naive density in (1) with the same radius ε for all x usually suffers from unsatisfactory empirical performance, for example, failing to identify small clusters with fewer data points.

Original: One possible way to alleviate this problem is through a transformation into the following local contrast (LC) function (Chen et al., 2018):
Revised: One possible way to alleviate this problem is through a transformation into the following local contrast (LC) function (Chen et al., 2018):

Original: [Equation]
Revised: [Equation]

Original: In this way, ⇢ LC compares the density of each data point with its k -nearest neighbors.
Revised: In this way, ρ LC compares the density of each data point with its k -nearest neighbors.

Original: To see the beneﬁt of LC, let us consider x to be a cluster center.
Revised: To see the benefit of LC, let us consider x to be a cluster center.

Original: After local contrasting, ⇢ LC ( x ) is likely to reach the value of k regardless of the size of this cluster.
Revised: After local contrasting, ρ LC ( x ) is likely to reach the value of k regardless of the size of this cluster.

Original: Under review as a conference paper at ICLR

Original: However density functions like ⇢ LC still highly depend on the underpinning performance of ⇢ naive .
Revised: However density functions like ρ LC still highly depend on the underpinning performance of ρ naive .

Original: This restricts their applications in clustering data with challenging local features.
Revised: This restricts their applications in clustering data with challenging local features.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: In this section, we present a new type of density-based clustering algorithm, based on the notion of kernel diffusion density function.
Revised: In this section, we present a new type of density-based clustering algorithm, based on the notion of kernel diffusion density function.

Original: Towards this end, we will introduce a kernel diffusion density function, which takes account of local adaptability and is well-suited for clustering purpose.
Revised: Towards this end, we will introduce a kernel diffusion density function, which takes account of local adaptability and is well-suited for clustering purpose.

Original: We provide details on how to derive this density function from a diffusion process induced by bivariate kernels.
Revised: We provide details on how to derive this density function from a diffusion process induced by bivariate kernels.

Original: We also provide a surrogate density function that is computationally more efﬁcient.
Revised: We also provide a surrogate density function that is computationally more efficient.

Original: 4.1 D IFFUSION PROCESS AND K ERNEL D IFFUSION D ENSITY
Revised: 4. D IFFUSION PROCESS AND K ERNEL D IFFUSION D ENSITY

Original: Considering a bivariate kernel function k : D ⇥ D ! R + , such that:
Revised: Considering a bivariate kernel function k : D × D → R + , such that:

Original: • k
Revised: • k

Original: ( x, y ) is positive semi-deﬁnite, i.e., k ( x, y )  0 .
Revised: ( x, y ) is positive semi-definite, i e., k ( x, y ) ≥ 0 .

Original: • k
Revised: • k

Original: ( x, y ) is F n -integrable with respect to both x and y .
Revised: ( x, y ) is F n -integrable with respect to both x and y .

Original: We deﬁne d ( x ) = n

Original: R

Original:  D k ( x, [Equation] ) dF n ( y ) as a local measure of the volume at x and deﬁne p ( x, y
Revised: We define d ( x ) = n (cid:82) D k ( x, y ) dF n ( y ) as a local measure of the volume at x and define

Original: [Equation]
Revised: [Equation]

Original: It is easy to see that p ( x, y ) satisﬁes the conservation property n 
Revised: It is easy to see that p ( x, y ) satisfies the conservation property n (cid:82)

Original: R D p ( x, y ) dF n ( y ) = 1 .
Revised: D p ( x, y ) dF n ( y ) = 1 .

Original: As a result, p ( x, y ) can be viewed as a probability for a random walk on the dataset from point x to point y , which induces a Markov chain on D with n ⇥ n transition matrix P =
Revised: As a result, p ( x, y ) can be viewed as a probability for a random walk on the dataset from point x to point y , which induces a Markov chain on D with n × n transition matrix P =

Original: [ p ( x, y )] .
Revised: [ p ( x, y )] .

Original: This technique is standard in various applications, known as the normalized graph Laplacian construction.
Revised: This technique is standard in various applications, known as the normalized graph Laplacian construction.

Original: For example, we can view D as a graph, L = I  P as the normalized graph Laplacian, and d ( x ) as a normalization factor.
Revised: For example, we can view D as a graph, L = I − P as the normalized graph Laplacian, and d ( x ) as a normalization factor.

Original: For t  0 , the probability of transiting from x to y in t time steps is given by P t , the t -th power of P .
Revised: For t ≥ 0 , the probability of transiting from x to y in t time steps is given by P t , the t -th power of P .

Original: Running the Markov chain forward in time, we observe the dataset at different scales, which is the diffusion process X t on D .
Revised: Running the Markov chain forward in time, we observe the dataset at different scales, which is the diffusion process X t on D .

Original: Let ⇢ ( x, t ) :
Revised: Let ρ ( x, t ) :

Original: D ⇥ R + ! R + be the associated probability density, which is governed by the following second-order differential equation with initial conditions:
Revised: D × R + → R + be the associated probability density, which is governed by the following second-order differential equation with initial conditions:

Original: [Equation]
Revised: [Equation]

Original: where  0 ( x ) is a probability density at time t = 0 .
Revised: where ϕ 0 ( x ) is a probability density at time t = 0 .

Original: In practice we can use any valid choice of  0 ( x ) , e.g. the uniform density.
Revised: In practice we can use any valid choice of ϕ 0 ( x ) , e.g. the uniform density.

Original: To give an explicit example of the diffusion process, consider a sub-class of k , i.e., isotropic kernels, where k ( x, y )
Revised: To give an explicit example of the diffusion process, consider a sub-class of k , i.e., isotropic kernels, where k ( x, y )

Original: =
Revised: =

Original: K ( || x  y || 2 /h ) for some function K : R ! R + .
Revised: K ( || x − y || 2 /h ) for some function K : R → R + .

Original: Here we can dual interpret h as a scale parameter to infer local information and as a time step h =
Revised: Here we can dual interpret h as a scale parameter to infer local information and as a time step h =

Original:  t at which the random walk jumps.
Revised: ∆ t at which the random walk jumps.

Original: Then we can deﬁne the forward Chapman-Kolmogorov operator T F as
Revised: Then we can define the forward Chapman-Kolmogorov operator T F as

Original: [Equation]
Revised: [Equation]

Original: Note that T F is the data distribution at time t
Revised: Note that T F is the data distribution at time t

Original: = h , thus can be viewed as continuous analogues of the left multiplication by the transition matrix P .
Revised: = h , thus can be viewed as continuous analogues of the left multiplication by the transition matrix P .

Original: Letting h ! 0 , the random walk converges to a continuous diffusion process with probability density evolves continuously in t .
Revised: Letting h → 0 , the random walk converges to a continuous diffusion process with probability density evolves continuously in t .

Original: In this case, we can explicitly write the second-order differential equation in (4) as:
Revised: In this case, we can explicitly write the second-order differential equation in (4) as:

Original: [Equation]
Revised: [Equation]

Original: where L h = lim h [-! 
-] ( T F  I ) /h is the conventional inﬁnitesimal generator of the process.
Revised: where L h = lim h [-! 
-]→ 0 ( T F − I ) /h is the conventional infinitesimal generator of the process.

Original: Now we are ready to introduce our kernel diffusion density function.
Revised: Now we are ready to introduce our kernel diffusion density function.

Original: Deﬁnition 1.
Revised: Definition 1.

Original: (Kernel diffusion density function)
Revised: (Kernel diffusion density function)

Original: Suppose the Markov chain induced by P is ergodic, we deﬁne the kernel diffusion density function as the limiting probability density of the diffusion process X t , i.e.,
Revised: Suppose the Markov chain induced by P is ergodic, we define the kernel diffusion density function as the limiting probability density of the diffusion process X t , i.e.,

Original: [Equation]
Revised: [Equation]

Original: Under review as a conference paper at ICLR

Original: We provide some intuitions that why ⇢ KD serves as a valid density function for clustering.

Original: With increased values of t , the diffusion process X t  reveals the geometric structure (such as high-density regions) of the data distribution F at increasing scales.
Revised: Intuitively, on the one hand, with increased values of t we expect the diffusion process X t gradually reveals the geometric structure (such as high-density regions) of the data distribution F .

Original: To see this, note that the transition probability P reﬂects connectivity between data points.
Revised: To see this, note that the transition probability P reflects connectivity between data.

Original: We can interpret a cluster as a region in which the probability of staying in this region is high during a transition.
Revised: We can interpret a cluster as an underlying geometric structure in which the probability of staying in this region is high during a transition.

Original: The probability of following a path along an underlying geometric structure  increases with t , as the involved data points are dense and highly connected.
Revised: In the diffusion process, the probability of following a path along a structure usually increases with t , as the involved data points are dense and highly connected.

Original: The paths, therefore, form along with short and high probability jumps.
Revised: Therefore, the path consists of short and high probability jumps.

Original: Whilst paths that do not follow this structure form long and low probability jumps, which lowers the path’s overall probability.
Revised: Whilst paths that do not follow any structure consists of long and low probability jumps, which lowers their overall probability.

Original: Meanwhile, to prevent the diffusion process grouping of all the data into one large cluster as t ! 1 , we can use certain sophisticated forms of k ( x, y ) that focus on local adaptivity. This will help slow down the diffusion and lead the process gradually towards the correct geometry structure at the right scale.
Revised: As a result, geometry structures of F is magnified during the diffusion. On the other hand, by talking certain sophisticated forms of k ( x, y ) that take into account of local adaptivity, we also slow down the diffusion to avoid trivial geometry structures such as one big cluster for all the data points.

Original: Thus they achieve a balance between magnifying geometry structures and identifying the signal of local features.
Revised: In this way, we can eventually identify the correct geometry structure at the right scale.

Original: 4.2 L OCALLY A DAPTIVE K ERNELS
Revised: 4. L OCALLY A DAPTIVE K ERNELS

Original: To address the local adaptability in kernel diffusion density function, we propose the following two bivariate kernels.
Revised: To address the local adaptability in kernel diffusion density function, we propose the following two bivariate kernels.

Original: Both of them are very simple variations of the most commonly used classic kernels.
Revised: Both of them are very simple variations of the most commonly used classic kernels.

Original: Symmetric-Gaussian kernel:
Revised: Symmetric-Gaussian kernel:

Original: [Equation]
Revised: [Equation]

Original: Here h and ✏ are both hyper-parameters.
Revised: Here h and ϵ are both hyper-parameters.

Original: We call this kernel symmetric since k ( x, y ) =
Revised: We call this kernel symmetric since k ( x, y ) =

Original: k ( y, x ) .
Revised: k ( y, x ) .

Original: Asymmetric-Gaussian kernel:
Revised: Asymmetric-Gaussian kernel:

Original: [Equation]
Revised: [Equation]

Original: Here h and k are hyper-parameters.
Revised: Here h and k are hyper-parameters.

Original: Note that in this case k ( x, y ) is asymmetric as y 2 N k ( x ) does not imply x 2 N k
Revised: Note that in this case k ( x, y ) is asymmetric as y ∈ N k ( x ) does not imply x ∈ N k

Original: ( y ) .
Revised: ( y ) .

Original: Bivariate kernels deﬁned in (7) and (8) are just combinations of classic Gaussian kernel and " - neighbourhood or k -nearest neighbours kernels, respectively.
Revised: Bivariate kernels defined in (7) and (8) are just combinations of classic Gaussian kernel and εneighbourhood or k -nearest neighbours kernels, respectively.

Original: With these simple combinations, we truncate Gaussian kernel at local areas, and the contribution of each point y to the construction of the density function ⇢ KD ( x ) depends not only on the distance
Revised: With these simple combinations, we truncate Gaussian kernel at local areas, and the contribution of each point y to the construction of the density function ρ KD ( x ) depends not only on the distance

Original: || y  x || but also on the local geometry structure around x .
Revised: || y − x || but also on the local geometry structure around x .

Original: Hence, the new kernels are adaptive at different x , which is expected to lead to better clustering performance against local features.
Revised: Hence, the new kernels are adaptive at different x , which is expected to lead to better clustering performance against local features.

Original: We remark that the Asymmetric-Gaussian kernel takes into account a varying neighborhood around each x , thus is more adaptive comparing to the
Revised: We remark that the Asymmetric-Gaussian kernel takes into account a varying neighborhood around each x , thus is more adaptive comparing to the

Original: Symmetric-Gaussian kernel.
Revised: Symmetric-Gaussian kernel.

Original: Although here we only provide two examples of locally adaptive kernels, other options can be easily created in a similar spirit under this framework, e.g., changing the Gaussian kernels to other kernels or changing the " -neighbourhood ( k -nearest neighbours) kernels to other locally truncated functions.
Revised: Although here we only provide two examples of locally adaptive kernels, other options can be easily created in a similar spirit under this framework, e.g., changing the Gaussian kernels to other kernels or changing the ε -neighbourhood ( k -nearest neighbours) kernels to other locally truncated functions.

Original: Once k ( x, y ) is determined, we can derive the corresponding density function ⇢ KD .
Revised: Once k ( x, y ) is determined, we can derive the corresponding density function ρ KD .

Original: Next, we just need to simply apply any density clustering procedure like DPC or DBSCAN based on ⇢ KD instead of the naive density function ⇢ naive .
Revised: Next, we just need to simply apply any density clustering procedure like DPC or DBSCAN based on ρ KD instead of the naive density function ρ naive .

Original: In Section 5, we assess the empirical performance of the proposed kernel diffusion density function with the above two locally adaptive kernels.
Revised: In Section 5, we assess the empirical performance of the proposed kernel diffusion density function with the above two locally adaptive kernels.

Original: They outperform existing density-based algorithms and other state-of-the-art methods.
Revised: They outperform existing density-based algorithms and other state-of-the-art methods.

Original: 4.3 F AST K ERNEL D IFFUSION D ENSITY
Revised: 4. F AST K ERNEL D IFFUSION D ENSITY

Original: The kernel diffusion density function ⇢ KD can be calculated as the stationary distribution of a Markov chain induced by the transition matrix P .
Revised: The kernel diffusion density function ρ KD can be calculated as the stationary distribution of a Markov chain induced by the transition matrix P .

Original: Numerically, we can solve it by iteratively right multiplying P with ⇢ ( x, t ) until convergence, or applying a QR decomposition on P .
Revised: Numerically, we can solve it by iteratively right multiplying P with ρ ( x, t ) until convergence, or applying a QR decomposition on P .

Original: These methods are expensive in terms of computational cost, especially when the sample size n is large.
Revised: These methods are expensive in terms of computational cost, especially when the sample size n is large.

Original: Under review as a conference paper at ICLR

Original: To tackle this problem, we propose the following surrogate of ⇢ KD ( x ) which is computationally more efﬁcient.
Revised: To tackle this problem, we propose the following surrogate of ρ KD ( x ) which is computationally more efficient.

Original: Deﬁnition 2.
Revised: Definition 2.

Original: (Fast kernel diffusion density function) Let p ( y, x ) be the transition probability from point y to point x , as deﬁned in equation (3).
Revised: (Fast kernel diffusion density function) Let p ( y, x ) be the transition probability from point y to point x , as defined in equation (3).

Original: We deﬁne the fast kernel diffusion density function as
Revised: We define the fast kernel diffusion density function as

Original: [Equation]
Revised: [Equation]

Original: It is straightforward that ⇢ FKD can be obtained in linear time and memory space, as we only need to compute the column averages of matrix P .
Revised: It is straightforward that ρ FKD can be obtained in linear time and memory space, as we only need to compute the column averages of matrix P .

Original: Here we show that ⇢ FKD is not only computationally efﬁcient but also suitable for detecting local features.
Revised: Here we show that ρ FKD is not only computationally efficient but also suitable for detecting local features.

Original: This is illustrated through the following Theorem 1.
Revised: This is illustrated through the following Theorem 1.

Original: Consider a special case that k ( x, y )
Revised: Consider a special case that k ( x, y )

Original: = 1 B ( x, " ) ( y ) .
Revised: = 1 B ( x,ε ) ( y ) .

Original: Then it is easy to verify that
Revised: Then it is easy to verify that

Original: [Equation]
Revised: [Equation]

Original: where C d = n " d V d is a normalising constant.
Revised: where C d = nε d V d is a normalising constant.

Original: In this way, we build a connection between ⇢ FKD and the naive density function ⇢ naive in this special example.
Revised: In this way, we build a connection between ρ FKD and the naive density function ρ naive in this special example.

Original: Theorem 1.
Revised: Theorem 1.

Original: Consider the above special case that k ( x, y )
Revised: Consider the above special case that k ( x, y )

Original: = 1 B ( x, " ) ( y ) .
Revised: = 1 B ( x,ε ) ( y ) .

Original: In addition, assume the dataset D = { x 1 , ..., x n } can be split into m disjoint clusters: i.e., X = D S ...
Revised: In addition, assume the dataset D = { x 1 , ..., x n } can be split into m disjoint clusters: i.e., X = D 1 (cid:83) ...

Original: S D m and for each x 2 D , B ( x, " ) only contain data points that belong to the same cluster as x .
Revised: (cid:83) D m and for each x ∈ D , B ( x, ε ) only contain data points that belong to the same cluster as x .

Original: Denote ¯ ⇢ j = 1 | D j | 
Revised: Denote ¯ ρ j = 1 | D j | (cid:80) x

Original: P x 2 D j ⇢ FKD ( x ) as the average density in cluster j .
Revised: ∈ D j ρ FKD ( x ) as the average density in cluster j .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Theorem 1 demonstrates that the averaged ⇢ FKD in each cluster are the same regardless of cluster sizes and other local features.
Revised: Theorem 1 demonstrates that the averaged ρ FKD in each cluster are the same regardless of cluster sizes and other local features.

Original: This shows that ⇢ FKD elevates the density of small clusters, which is essential for ﬁnding the density peaks of small clusters.
Revised: This shows that ρ FKD elevates the density of small clusters, which is essential for finding the density peaks of small clusters.

Original: Previously we claim that ⇢ FKD is a surrogate of the kernel diffusion density ⇢ KD .
Revised: Previously we claim that ρ FKD is a surrogate of the kernel diffusion density ρ KD .

Original: Next, we want to reveal the relationship between these two density functions from an asymptotic viewpoint.
Revised: Next, we want to reveal the relationship between these two density functions from an asymptotic viewpoint.

Original: To proceed, we will need the following assumption.
Revised: To proceed, we will need the following assumption.

Original: Assumption 1.
Revised: Assumption 1.

Original: There exists some positive constant c < 1  such that ⇢ FKD ( x )  cuniformly holds for every x 2 D .
Revised: There exists some positive constant c < 1 that is independent of n , such that ρ FKD ( x ) ≤ c holds for every x ∈ D .

Original: This is a very mild assumption, since it always holds that ⇢ FKD ( x ) < 1 , and the average of ⇢ FKD ( x ) over the dataset is 
Revised: This is a very mild assumption, since it always holds that ρ FKD ( x ) < 1 , and the average of ρ FKD ( x ) over the dataset is (cid:82) D

Original: [Equation] which vanishes as n ! 1 .
Revised: ρ FKD ( x ) dF n ( x ) = 1 /n , which vanishes as n → ∞ .

Original: Now we are ready to present the following theorem that characterise theuniform closeness between ⇢ FKD and ⇢ KD .
Revised: Now we are ready to present the following theorem that characterise the closeness between ρ FKD and ρ KD .

Original: Theorem 2.
Revised: Theorem 2.

Original: Suppose that Assumption 1 holds and the Markov chain induced by the kernel k ( x, y ) is ergodic.
Revised: Suppose that Assumption 1 holds and the Markov chain induced by the kernel k ( x, y ) is ergodic.

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original:  Theorem 2implies that ⇢ FKD uniformly approximates ⇢ KD when n is sufﬁciently large.
Revised: As shown in the Appendix, the almost sure convergence in Theorem 2 is of a fast rate at n − 1 .

Original: So it is safe for us to use it to replace ⇢ KD in practice.
Revised: Thus it is safe for us to use it to replace ρ KD in finite sample experiments.

Original: This result is also veriﬁed by our numerical experiments in Section 5.
Revised: This result is also verified by our numerical experiments in Section 5.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we empirically evaluate the proposed kernel diffusion density functions against ⇢ naive and ⇢ LC in density-based clustering algorithms, and also compare them with other state-of-the-art methods.
Revised: In this section, we empirically evaluate the proposed kernel diffusion density functions against ρ naive and ρ LC in density-based clustering algorithms, and also compare them with other state-of-the-art [Table]

Original:  We denote ⇢ symKD and ⇢ symFKD as the kernel diffusion density functions and its fast surrogate, with symmetric-Gaussian kernel, respectively.
Revised: methods. We denote ρ symKD and ρ symFKD as the kernel diffusion density functions and its fast surrogate, with symmetric-Gaussian kernel, respectively.

Original: Similarly, we denote ⇢ asymKD and ⇢ asymFKD as the proposed two density functions with asymmetric-Gaussian kernel, respectively.
Revised: Similarly, we denote ρ asymKD and ρ asymFKD as the proposed two density functions with asymmetric-Gaussian kernel, respectively.

Original: We examine their performance on a wide range of datasets.
Revised: We examine their performance on a wide range of datasets.

Original: The clustering results are measuredin Pairwise F-score (Banerjee et al., 2005) and BCubed F-score (Amigó et al., 2009). [Table]
Revised: The clustering results are measured Pairwise F-score (Banerjee et al., 2005), BCubed F-score (Amigó et al., 2009) or NMI (Cover, 1999).

Original: The parameters " (radius of the ball, used in ⇢ naive , ⇢ LC , ⇢ symKD and ⇢ symFKD ), k (number of nearest neighbors, used in ⇢ LC , ⇢ asymKD and ⇢ asymFKD ) are tuned by searching within a suitable range in the parameter space, and h (bandwidth of Gaussian kernels, used in ⇢ symKD , ⇢ symFKD , ⇢ asymKD and ⇢ asymFKD ) is ﬁxed to be 0 .

Original: 5.1 P ERFORMANCE ON B ENCHMARK D ATASETS
Revised: 5. P ERFORMANCE ON B ENCHMARK D ATASETS

Original: We now discuss the performance on 13 benchmark datasets ( ⇠ 100 to ⇠ 5 , 000 data points) from UCI repository.
Revised: We now discuss the performance on 13 benchmark datasets ( ∼ 100 to ∼ 5 , 000 data points) from UCI repository.

Original: The metadata is summarised in the Appendix.
Revised: The metadata is summarised in the Appendix.

Original: As summarised in Table 1, both ⇢ symKD and ⇢ asymKD uniformly outperform ⇢ naive and ⇢ LC in terms of clustering accuracy.
Revised: As summarised in Table 1, both ρ symKD and ρ asymKD uniformly outperform ρ naive and ρ LC in terms of clustering accuracy in terms of F-scores.

Original:  The proposed kernel diffusion density functions with asymmetric Gaussian kernel, ⇢ asymKD , which enjoys better local adaptivity analytically, achieves the best results on most datasets and outperforms ⇢ naive and ⇢ LC by a large margin.
Revised: results based on NMI are deferred to the Appendix. The proposed kernel diffusion density functions with asymmetric Gaussian kernel, ρ asymKD , which enjoys better local adaptivity analytically, achieves the best results on most datasets and outperforms ρ naive and ρ LC by a large margin.

Original: It is worth noticing that the two fast surrogates, ⇢ symFKD and ⇢ asymFKD , achieve comparable results with their original counterparts, ⇢ symKD and ⇢ asymKD .
Revised: It is worth noticing that the two fast surrogates, ρ symFKD and ρ asymFKD , achieve comparable results with their original counterparts, ρ symKD and ρ asymKD .

Original: In the Appendix similar results are observed for the same set of density functions applied to DBSCAN.
Revised: In the Appendix similar results are observed for the same set of density functions applied to DBSCAN.

Original: 5.2 P ERFORMANCE ON FACE IMAGE D ATASETS
Revised: 5. P ERFORMANCE ON FACE IMAGE D ATASETS

Original: Clustering face images according to their latent identity becomes an important application in recent years.
Revised: Clustering face images according to their latent identity becomes an important application in recent years.

Original: It is challenging in the sense that face image datasets usually contain thousands of identities, corresponding to thousands of clusters.
Revised: It is challenging in the sense that face image datasets usually contain thousands of identities, corresponding to thousands of clusters.

Original: Meanwhile, the number of images for each identity (cluster) is quite different, corresponding to the variety of cluster sizes.
Revised: Meanwhile, the number of images for each identity (cluster) is quite different, corresponding to the variety of cluster sizes.

Original: We assess the performance of the proposed approach on two popular face image datasets: emore_200k (Zhan et al., 2018) and
Revised: We assess the performance of the proposed approach on two popular face image datasets: emore_200k (Zhan et al., 2018) and

Original: MS1M (Guo et al., 2016).
Revised: MS1M (Guo et al., 2016).

Original: [Table]
Revised: [Table]

Original: emore_200k.
Revised: emore_200k.

Original: The dataset contains 2,577 identities with 200,000 images following the protocol in Zhan et al.
Revised: The dataset contains 2,577 identities with 200,000 images following the protocol in Zhan et al.

Revised: We set ε to 0.8, k to 200, and h to 0.5 for density-based methods

Original: Results are summarized in Table 2.
Revised: Results are summarized in Table 2.

Original: The proposed diffusion density functions are applied to DPC, and compared with kmeans, HAC (Sibson, 1973), ARO (Otto et al., 2017)
Revised: The proposed diffusion density functions are applied to DPC, and compared with k -means, HAC (Sibson, 1973), ARO (Otto et al., 2017), and

Original: [Table]

Original: and CDP (Zhan et al., 2018).
Revised: CDP (Zhan et al., 2018).

Original: Again, we observe signiﬁcant improvement in the proposed density functions over ⇢ naive and ⇢ LC .
Revised: Again, we observe significant improvement in the proposed density functions over ρ naive and ρ LC .

Original: It is also worth pointing out that, density-based clustering with proposed kernel diffusion density functions also outperform the stateof-the-arts approaches such as CDP by a large margin
Revised: It is also worth pointing out that, density-based clustering with proposed kernel diffusion density functions also outperform the state-of-the-arts approaches such as CDP by a large margin.

Original: [Table]
Revised: [Table]

Original: [Table] MS1M. The dataset contains 8,573 identities with around 584,000 images following the protocols in Yang et al. 
Revised: MS1M. The dataset contains 8,573 identities with around 584,000 images following the protocols in Yang et al. We set ε to 0.8, k to 200, and h to 0.

Original:  We reported the results of clustering performance inTable 3. Precision versus Recall curves for different density functions (applied to DPC) are plotted in Figure 2.
Revised: for density-based methods. We reported the results of clustering performance in for different density functions (applied to DPC) are plotted in Figure 2.

Original: In Table 3, the proposed kernel diffusion density functions outperform ⇢ naive and ⇢ LC .
Revised: In Table 3, the proposed kernel diffusion density functions outperform ρ

Original: Note that GCN-based methods such as L-GCN (Wang et al., 2019), 
Revised: Note that GCN-based methods such as L-GCN (Wang et al., 2019), LTC

Original: LTC (Yang et al., 2019) and GCN (V+E)
Revised: (Yang et al., 2019) and GCN (V+E)

Original: (Yang et al., 2020) achieve generally better clustering performance than unsupervised methods due to their supervised nature.
Revised: (Yang et al., 2020) achieve generally better clustering performance than unsupervised methods due to their supervised nature.

Original: However, it is quite encouraging to see that the proposed kernel diffusion approaches, although are also unsupervised clustering methods, considerably outperform the GCN-based methods.
Revised: However, it is quite encouraging to see that the proposed kernel diffusion approaches, although are also unsupervised clustering methods, considerably outperform the GCN-based methods.

Original: 5.3 S ENSITIVITY A NALYSIS
Revised: 5. S ENSITIVITY A NALYSIS

Original: Next, we examine the sensitivity of the proposed kernel diffusion density functions to hyperparameters and compare it with ⇢ naive and ⇢ LC .
Revised: Next, we examine the sensitivity of the proposed kernel diffusion density functions to hyperparameters and compare it with ρ naive and ρ LC .

Original: The results are obtained via extensive experiments on emore_200k and MS1M, which are shown in Figure 3.
Revised: The results are obtained via extensive experiments on emore_200k and MS1M, which are shown in Figure 3.

Original: We can see that the clustering performance of ⇢ symKD is much more stable than ⇢ naive and ⇢ LC when we vary the value of " .
Revised: We can see that the clustering performance of ρ symKD is much more stable than ρ naive and ρ LC when we vary the value of ε .

Original: Whilst ⇢ asymKD is robust to the parameter k , and both ⇢ symKD and ⇢ asymKD are quite robust to the parameter h .
Revised: Whilst ρ asymKD is robust to the parameter k , and both ρ symKD and ρ asymKD are quite robust to the parameter h .

Original: 5.4 C OMPUTATIONAL C OST
Revised: 5. C OMPUTATIONAL C OST

Original: We carried out a series of experiments on
Revised: We carried out a series of experiments on

Original: MS1M to demonstrate the computational efﬁciency of the fast surrogate ⇢ FKD in terms of time and space.
Revised: MS1M to demonstrate the computational efficiency of the fast surrogate ρ FKD in terms of time and space.

Original: With a collection of subsampled data from MS1M at different percentile levels, we run both the kernel diffusion density ⇢ KD and the fast surrogate ⇢ FKD .
Revised: With a collection of subsampled data from MS1M at different percentile levels, we run both the kernel diffusion density ρ KD and the fast surrogate ρ FKD .

Original: As we can observe from Figure 4, the running time and memory usage of ⇢ KD increase dramatically with the sample size.
Revised: As we can observe from Figure 4, the running time and memory usage of ρ KD increase dramatically with the sample size.

Original: Whilst ⇢ FKD retains a very low level of computational cost.
Revised: Whilst ρ FKD retains a very low level of computational cost.

Original: This suggests that ⇢ FKD , which achieves an excellent computational efﬁciency, should be favored in practice.
Revised: This suggests that ρ FKD , which achieves an excellent computational efficiency, should be favored in practice.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Density-based clustering has a profound impact on machine learning and data mining.
Revised: Density-based clustering has a profound impact on machine learning and data mining.

Original: However, the underpinning naive density function suffers from detecting varying local features, causing extra errors in the clustering.
Revised: However, the underpinning naive density function suffers from detecting varying local features, causing extra errors in the clustering.

Original: We propose a new set of density functions based on the kernel diffusion process to resolve this problem, which is adaptive to density regions of varying local distributional features.
Revised: We propose a new set of density functions based on the kernel diffusion process to resolve this problem, which is adaptive to density regions of varying local distributional features.

Original: We demonstrate that DBSCAN and DPC adapted by the proposed approach have improved clustering performance comparing to their classic versions and other state-of-the-art methods. 
Revised: We demonstrate that DBSCAN and DPC adapted by the proposed approach have improved clustering performance comparing to their classic versions and other state-of-the-art methods. A A PPENDIX

Revised: In this supplementary file, we provide technical proofs of the theoretical results in Section 4.3, and present extra empirical experiments regarding our kernel diffusion approach with symmetric and asymmetric Gaussian kernels applied to DBSCAN.

Revised: All the numerical experiments are carried out on a standard work station with a Intel 64-cores CPU and two Nvidia P100 GPUs.

Revised: A. P SEUDO CODE FOR DBSCAN AND DPC. Algorithm 1 DBSCAN

Revised: x

Revised: x −

Revised: The connected compoenents of the graph are determined a clusters,a dn the remaining points are unclustered and considered as noise-points.

Revised: Algorithm 2 DPC

Revised: A. P ROOFS OF T HEORETICAL R ESULT .

Revised: Proof of Theorem 1.

Revised: Since { D 1 , · · · , D m } are disjoint, we have p ( x, y )

Original: {+= 
+}
Revised: {+= 
+}

Revised: if x and y belong to different clusters.

Revised: By the definition of matrix P , for each x ∈ D j , we have

Revised: [Equation]

Revised: [Equation]

Revised: n

Revised: [Equation]

Revised: y

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: , which implies that

Revised: [Equation]

Revised: D

Revised: [Equation]

Revised: Therefore,

Revised: [Equation]

Revised: which implies that ¯ ρ j = 1 for any j = 1 , , . .

Revised: , m .

Revised: Before proceeding to the proof of Theorem 2, we need following auxiliary lemma that relates the stationary distribution of a Markov chain to an arbitrary vector g .

Revised: Lemma A.1.

Revised: Let P be transition probability matrix of a finite inreducible discrete time Markov chain with n states, which admits a stationary distribution, denoted by vector π .

Revised: We write e = (1 , . . .

Revised: , 1) T ∈ R n as the a column vector of ones.

Revised: The following holds for any vector g such that g T e ̸

Revised: = 0 :

Revised: (1) ( I − P + eg T ) is non-singular.

Revised: (2) Let H

Revised: =

Revised: ( I − P + eg T ) − 1 , then π T =

Revised: g T H.

Revised: Proof.

Revised: Since π is the stationary distribution, we have π T e = 1 .

Revised: Applying Theorem 3.3 in (Hunter, 1982) yields that matrix

Revised: I − P + eg T ) is non-singular.

Revised: Next recall that π T P = π T , therefore we have

Revised: [Equation]

Revised: which implies π T

Revised: = g T H .

Revised: Proof of Theorem 2.

Revised: Note that for for each x ∈ D , the linear reference function ρ FKD ( x )

Revised: =

Revised: (cid:82)

Revised: D p ( y, x ) dF n ( y ) is the corresponding column average of the transition matrix P .

Revised: We write the i -th column vector of P as

Revised: [Equation]

Revised: Therefore ρ FKD ( x i ) =

Revised: e T p

Revised: i /n

Revised: Since the Markov chain induced by the kernel k ( x, y ) is ergodic, the density ρ ( x, t ) of the diffusion process X t converges to the limiting stationary distribution of the Markov chain, denoted by π .

Revised: We can write the n -vectors of g and π in the following form:

Revised: [Equation]

Revised: where g i =

Revised: e T p

Revised: i is the i -th column sums of matrix P .

Revised: As a result, we have

Revised: [Equation]

Revised: By the definition of g , we know

Revised: [Equation]

Revised: It follows from Lemma A.1 that ( I − P + eg T ) is non-singular and π T

Revised: = g T H, where H

Revised: =

Revised: ( I − P + eg T ) − 1 .

Revised: We define B =

Revised: I + eg T .

Revised: By simple algebra calculation, we can find B is non-singular with

Revised: [Equation]

Revised: As a result, it is easy to see that that g T B − 1 = g T n + 1 and

Revised: [Equation]

Revised: Use the Neumann series, we have

Revised: [Equation]

Revised: Thus

Revised: [Equation]

Revised: Since we assume for any x ∈ D , ˆ g ( x ) < c for some 0 < c < 1 .

Revised: This leads to

Revised: [Equation]

Revised: Therefore, let κ j be the j -th compoenent of g T PB − 1 , it is straightforward

Revised: [Equation]

Revised: This implies for every x ∈ D ,

Revised: [Equation]

Revised: Hence we have

Revised: [Equation]

Revised: which completes the proof.

Revised: A. A DDITIONAL N UMERICAL E XPERIMENTS

Revised: Hyperparameters The parameter ε (radius of the ball, used in ρ naive , ρ LC , ρ symKD and ρ symFKD ) is tuned by searching within the range between 0.1 and 1 with am increment of 0.1, parameter k (number of nearest neighbors, used in ρ LC , ρ asym KD and ρ asym FKD ) is tuned by searching within the range between 10% and 50% number of samples, with an increment of 10% .

Revised: Metadata of benchmark datasets.

Revised: The number of samples n , the number of clusters c , and feature dimension d for each benchmark dataset are listed in Table 4 below.

Revised: [Table]

Revised: Benckmark datasets with DBSCAN.

Revised: We provide the performance of the conventional density functions, ρ naive and ρ LC , and the proposed kernel diffusion density functions with symmetric and asymmetric Gaussian kernels, ρ ∗ KD and ρ ∗ FKD ( ∗ ∈ { sym , asym } ), applied to DBSCAN on 13 benchmark datasets.

Revised: The results are summarised in Table 6.

Revised: Similar to DPC, we see that both ρ symKD and ρ asymKD uniformly outperform ρ naive and ρ LC in terms of clustering quality.

Revised: ρ asymKD , which has better local adaptivity analytically, achieves the best results on most datasets and outperforms others by a significant margin in Breast-o, Control, Haberma and Seeds.

Revised: NMI for benchmark datasets.

Revised: Below we present in Table 6 the clustering results for benchmark datasets based on NMI metric.

Revised: [Table]

Revised: [Table]

Revised: Haberman 9.

Revised: Ionosphere 27.

Revised: Iris 51.

Revised: Libras 63.

Revised: Pageblocks 8.

Revised: Seeds 47.

Revised: Segment 63.

Revised: Wine 58.