Original: Kernel Landmarks: An Empirical Statistical Approach to Detect Covariate Shift
Revised: Kernel Landmarks: An Empirical Statistical Approach to Detect Covariate Shift

Original: Abstract
Revised: Abstract

Original: Training a predictive model with empirical risk minimization requires a distributionof the input training data that matches the testing data.
Revised: Training an effective predictive model with empirical risk minimization requires a distribution of the input training data that matches the testing data.

Original: Covariate shift can occurwhen the testing cases are not class-balanced, but the training is.
Revised: Covariate shift can occur when the testing cases are not class-balanced, but the training is.

Original: In order to detectwhen class imbalance is present in a test sample (without labels), we propose touse statistical divergence based on the Wasserstein distance and optimal transport.
Revised: In order to detect when class imbalance is present in a test sample (without labels), we propose to use statistical divergence based on the Wasserstein distance and optimal transport.

Original: Recently, slicing techniques have been proposed that provide computational andstatistical advantages in high-dimensional spaces.
Revised: Recently, slicing techniques have been proposed that provide computational and statistical advantages for the Wasserstein distance for highdimensional spaces.

Original: In this work we presented acomputationally simple approach to perform generalized slicing via kernel-based Wasserstein distance and apply itto as a two-sample test.
Revised: In this work we presented a computationally simple approach to perform generalized slicing of the kernel-based Wasserstein distance and apply it as a two-sample test.

Original: The proposed landmarkbased slicing chooses a single pointin the samples to be the sole support vector torepresent the witness function.
Revised: The proposed landmark-based slicing chooses a single point to be the sole support vector to represent the witness function.

Original: We run pseudo-real experiments using the MNISTdataset and compare our method with maximum mean discrepancy (MMD).
Revised: We run pseudo-real experiments using the MNIST dataset and compare our method with maximum mean discrepancy (MMD).

Original: Wehave shown that our proposed methods perform better than MMD on these syntheticsimulations of covariate shift.
Revised: We have shown that our proposed methods perform better than MMD on these synthetic simulations of covariate shift.

Original:  Introduction
Revised:  Introduction

Original: Optimal transport (OT) is an old problem that can be dated back to 17 th century.
Revised: Optimal transport (OT) is an old problem that can be dated back to 17 th century.

Original: Historically theobject of interest was literal dirt, but today we can apply the same approach to align two distributionsor samples of points [9].
Revised: Historically the object of interest was literal dirt, but today we can apply the same approach to align two feature distributions or data sets [Peyré and Cuturi, 2019].

Original: Generally, computing Wasserstein distance requires solving an highdimensional optimization problem.
Revised: Generally, computing Wasserstein distance requires solving a high-dimensional optimization problem.

Original: There have been many extensions and approximations of optimaltransport.
Revised: There have been many extensions and approximations of optimal transport.

Original: Perhaps the two easiest cases to compute Wasserstein distance are when the data isone-dimensional or can be described by the ﬁrst and second-moments, as in Gaussian distributions.
Revised: Perhaps the two easiest cases to compute Wasserstein distance are for one-dimensional distributions or distributions described by their first and second-moments, as in Gaussian distributions.

Original: However, data-sets usually have higher dimensions and are non-Gaussian.
Revised: However, most data sets have multivariate features and are non-Gaussian.

Original: Inspired by the Radontransform, [6] proposed to obtain slices that are one-dimensional marginal representations of thedistributions through linear projections.
Revised: Using the Radon transform, Kolouri et al. [2016] proposed to obtain slices that are one-dimensional marginal representations of the distributions through linear projections.

Original: The max-sliced Wasserstein [4] was proposed to solveprojection complexity by replacing the random projections with the one that maximized the 1D 
Revised: The max-sliced Wasserstein [Deshpande et al., 2019, Nguyen et al., 2021] replaces the random projections with a single slice or an optimized distribution over slices that maximizes the 1D Wasserstein.

Original: Wasserstein. Computing max-sliced Wasserstein still requires optimization and it could be expensive.
Revised: Alternatively, Meng et al.

Original: Alternatively, [8] proposed to ﬁnd the most meaningful projection using a sufﬁcient dimensionreduction technique [7].
Revised: [2019] proposed to find the most meaningful projection using a sufficient dimension reduction technique [Li, 1991].

Original: We propose an alternative solution calledlandmark-based kernel max-slicing to tackle this problem.
Revised: We propose an alternative solution called kernel landmarks to tackle this problem.

Original: We use a kernel-based approach, mapping the data points to a Hilbert space [12] and evaluate thediscrepancy between the distributions using each data point as a witness function and measurethe divergence between the witness function evaluations.
Revised: We use a kernelbased approach, mapping the data points to a Hilbert space [Zhang et al., 2020] and evaluate the discrepancy between the distributions using each data point as a witness function and measure the divergence between the witness function evaluations.

Original: After evaluating all data points, we pickthe point (the landmark) which identiﬁes the largest discrepancy between the distribution.
Revised: After evaluating all data points, we pick the point (the landmark) which identifies the largest discrepancy between the distribution.

Original: Ourpreliminary results show that this landmark-based max slicing is efﬁcient compared to maximummean discrepancy (MMD) [5].
Revised: Our preliminary results show that this landmark-based max slicing is nearly efficient as maximum mean discrepancy (MMD) [Gretton et al., 2012].

Original: Furthermore, the landmark-based kernel max-slicing is much simplerto compute than kernel max-slicing [2].
Revised: Furthermore, the landmark-based kernel max-slicing is much simpler to compute than kernel max-slicing [Brockmeier et al., 2021], while still being a probability distance metric.

Original:  Methodology
Revised:  Methodology

Original: We consider a feature domain X ⊆ R d .
Revised: We consider a feature domain X ⊆ R d .

Original: Let P ( X ) be the set of Borel probability measures on a givenmetric space ( X , d )  Let µ, ν ∈ P ( X ) be the probability measures and X,
Revised: Let P ( X ) be the set of Borel probability measures on the metric space ( X , d ) where d ( x, y ) is the distance metric for x, y ∈ X . Let µ, ν ∈ P ( X ) be the probability measures and X,

Original: Y ∈
Revised: Y ∈

Original: X be the randomvariables such that X ∼ µ and Y ∼ ν .
Revised: X be the random variables such that X ∼ µ and Y ∼ ν .

Original: For any p ≥ 1 ,  the Wasserstein- p distance is given as
Revised: For any p ≥ 1 , we assume the distributions µ and ν have finite p -th moments, and using the Euclidean distance, d ( x, y ) = ∥ x − y ∥ , the Wasserstein- p distance is given as

Original: [Equation]
Revised: [Equation]

Revised: Cédric [2003] shows that Eq.

Original: Villani [3] shows that square root of the objective in Eq.1 gives a metric on P ( X )  and this metric isknown as Wasserstein-2 distance [11, 9].
Revised: (1) gives a metric on P ( X ) [Villani, 2008, Peyré and Cuturi, 2019]. The max-sliced Wasserstein- p distance is given by the following saddlepoint problem [Equation] where ∥ ( ww ⊤ )( X − Y ) ∥

Original:  2.1 Divergences in the Reproducing Kernel Hilbert Space (RKHS)
Revised: = ∥⟨ X − Y, w ⟩ w ∥ = |⟨ X − Y, w ⟩| ∥ w ∥ . 2.1 Divergences in the Reproducing Kernel Hilbert Space (RKHS)

Original: Let H be the Hilbert space with the real-valued function and κ be a  positive deﬁnite kernel suchthat κ : X × X → R .
Revised: Consider a symmetric (real-valued) positive definite kernel function κ : X × X → R .

Original:  H is a reproducing kernel Hilbert space if the following conditions are met:( i ) ∀ x ∈ X , κ ( · , x ) ∈ H and( ii ) ∀ x ∈ X , f ∈ H (cid:104) f, κ ( · , x ) (cid:105) H = f ( x ) [10].
Revised: κ defines H , a reproducing kernel Hilbert space, if the following conditions are met: ∀ x ∈ X , κ ( · , x ) ∈ H and ∀ x ∈ X , f ∈ H , f ( x ) = ⟨ f, κ ( · , x ) ⟩ H [Scholkopf and Smola, 2001].

Original: Let φ : X → H bethe feature map (mapping elements x ∈ X to elements in RKHS φ ( x ) )
Revised: Let ϕ : X → H be the implicit feature map (mapping elements x ∈ X to elements in RKHS ϕ

Original: such that φ ( x ) = κ ( · , x ) andφ ( y ) 
Revised: ( x ) = κ ( · , x ) ) such that ⟨ ϕ ( x ) , ϕ ( y ) ⟩ H = κ ( x, y ) .

Original: = κ ( · , y ) , and (cid:104) φ ( x ) , φ ( y ) (cid:105) H = κ ( x, y ) .

Original: For a  family of functions  F = { ω : (cid:104) ω, ω (cid:105) H ≤ 1 } on X and ω : X → R , Maximum Mean Discrepancy (MMD) is given by
Revised: Assuming a bounded kernel E X ∼ ξ [ κ ( X, X )] ≤ ∞ , ∀ ξ ∈ P ( X ) , for bounded family of functions in the RKHS F = { ω : ⟨ ω, ω ⟩ H ≤ 1 } on X where ω : X → R , the maximum mean discrepancy (MMD) is given by

Original: [Equation]
Revised: [Equation]

Original:  m µ =
Revised: where m µ =

Original: E X ∼ µ [ φ ( X )] ∈ H and m ν 
Revised: E X ∼ µ [ ϕ ( X )] ∈ H and m ν =

Original: = E Y ∼ µ [ φ ( Y )]
Revised: E Y ∼ µ [ ϕ ( Y )]

Original: ∈ H are the ﬁrst moments of data on RKHS. MMD is the optimal transport in the RKHS between distributions described by their means. When the µ and ν are two Gaussians, the Wasserstein distance can be related to MMD and Bures distance [9] in terms of the ﬁrst and the second moments [12].
Revised: ∈ H .

Original: The kernel Gaussian W2 distanceis deﬁned as W H G ( µ, ν ) = (cid:113) (cid:107) m µ − m ν (cid:107) 2 + d H B (Σ µ , Σ ν ) 2 , where d B (Σ µ , Σ ν ) =(cid:0) Tr (Σ µ )+ Tr (Σ
Revised: Using the kernel-induced distance d ( x, y )

Original: [Equation] ) − 2 Tr (Σ 12 µ Σ ν Σ 12 µ ) (cid:1) 12 [1], Σ µ =ρ µ − m µ ⊗ m µ and Σ ν = ρ ν − m ν ⊗ m νare the covariance matrices of X and Y in the RKHS, and ρ µ = E [ φ ( X ) ⊗ φ ( X )] ∈ H andρ ν = E
Revised: =

Original: [ φ ( Y ) ⊗ φ ( Y )]
Revised: ∥ ϕ ( x ) − ϕ ( y ) ∥ H , Eq.

Original: ∈ H are the uncentered second moments. The Bures distance between the uncentered second moments in the Hilbert space [2] is also a divergence measure D H B ( µ, ν ) = d B ( ρ µ , ρ ν ) .
Revised: (1) can be extended to the kernel

Original: The kernel Wasserstein-2 distance is
Revised: Wasserstein- p distance [Zhang et al., 2020] as

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) where (cid:107) φ ( X ) − φ ( Y ) (cid:107) 2 H =κ
Revised: where ∥ ϕ ( X ) − ϕ ( Y ) ∥ p H =

Original: (  X, X ) − 2 κ ( X, Y ) 
Revised: ( κ ( X, X ) − 2 κ ( X, Y ) + κ ( Y, Y ))

Original: + κ ( Y, Y ) .
Revised: p/ 2 . For p = 2 , this simplifies such that the joint expectation moves inside and W H 2 ( µ, ν ) = inf γ ∈ Γ( µ,ν ) (cid:112) E

Original:  If one were to purse the one-dimensionalcase as the easiest way to compute Wasserstein distance, the implicit feature map φ ( · ) seems tobe an obstacle.
Revised: [ κ ( X, X )] + E [ κ ( Y, Y )] − E ( X,Y ) ∼ γ 2 κ ( X, Y ) . If one were to pursue the onedimensional case as the easiest way to compute Wasserstein distance, the implicit feature map ϕ ( · ) seems to be an obstacle to slicing compare to Eq.

Original: However, by the reproducing property, the expression in Eq.1 can be extended as(cid:104)
Revised: However, by the reproducing property of the RKHS, the sliced distance defined by ω ∈ H ,

Original: X − Y, ω (cid:105) p = 
Revised: ∥ ω ∥ H = 1 , is ∥ ( ω ⊗ ω ) ( ϕ ( X ) − ϕ ( Y ))

Original: [ ω ( X ) − ω
Revised: ∥ H = |⟨ ϕ ( X ) − ϕ

Original: ( Y )]
Revised: ( Y ) , ω ⟩|∥ ω

Original: p where ω ( X ) and ω ( Y ) are the witness functions.
Revised: ∥ H = | ω ( X ) − ω ( Y ) | , where ω ( X ) and ω ( Y ) are real-valued random variables with pushforward measures ω ♯ µ and ω ♯ ν , respectively.

Original: Consideringp = 2 , the squared witness functions can be expressed  as[-ω ( X ) 
-]
Revised: The max-sliced kernel Wassersteindistance can be expressed in terms of witness functions as[-ω ( X ) 
-]

Original: =
Revised: [Equation]

Original: (cid:104) φ ( z ) , φ ( X ) (cid:105) 2 and denoted as 66 E [ ω 2 ( X )] = (cid:104) ω⊗ ω, E [ φ ( X ) ⊗ φ ( X )] (cid:105) = (cid:107) ω (cid:107) 2 µ , and similarly E [ ω2 ( Y )]
Revised: From the second line it is clear that the optimal γ can be solved analytically by coupling the largest values of ω ( X ) with the largest values of ω ( Y ) , this can be done using their inverse cumulative distribution functions, which exist under mild conditions [Santambrogio, 2015].

Original: = (cid:107) ω (cid:107) 2 ν .
Revised: Even with analytic solutions for the optimal transport after slicing, the computation of the max-sliced Wasserstein distance still requires optimizing the parameters defining the optimal slice.

Original: The max-slicedkernel Wasserstein-2 distance can be expressed in terms of squared witness functions as
Revised: Here, we propose a restricted form of kernel max-slicing called kernel landmarks. We restrict the witness function to be an implicit mapping of a single data point in the Hilbert space, ω

Original: [Equation]
Revised: = ϕ ( z ) , z ∈

Original: For a sample, the witness function ω ( · )  = 
Revised: X , assuming normalized kernel κ ( z, z ) = 1 , z ∈ X = ⇒ ∥ ϕ ( z ) ∥ H = 1 , and compute the

Original: (cid:80) m
Revised: Wasserstein-2 distance [Equation] For characteristic kernel functions [Fukumizu et al., 2008], namely the Gaussian and Laplacian kernels, this is a probability metric as stated and proved in the appendix (Theorem 1). 2.2 Two-sample Tests Using Kernel Divergences

Original: + n i 
Revised: We now consider two finite, weighted samples { ( µ i , x i ) } m i =1 and { ( ν i , y i ) } n i =1 of size m and n with (cid:80) i µ i = 1 and (cid:80) i ν i

Original: = 
Revised: = 1 .

Original: α i κ ( · , z i ) is optimized in terms of a dual variablesα ∈ R n + m and z i = (cid:26) x i , for 1 ≤ i ≤ m y i , for m + 1 ≤ i≤ l (cid:27) , where l
Revised: The masses are represented by the vectors µ and ν . These samples can be represented by the empirical measures ˆ µ = (cid:80) i µ i δ x i and ˆ ν = (cid:80) i ν i

Original: = m + n .
Revised: δ y i .

Original: For two ﬁnite samples,the full kernel matrix is K{+parametrized as ω ( · ) = (cid:80) li =
 α i κ ( · , z i ) in terms of the dual variables α ∈ R l and { z i } li =
+} = (cid:20) K
Revised: In the sample case, the witness function ω is {+parametrized as ω ( · ) = (cid:80) li =
 α i κ ( · , z i ) in terms of the dual variables α ∈ R l and { z i } li =
+} = Z where z i = (cid:26) x i , 1 ≤ i ≤ m y

Original: [Equation]
Revised: [Equation]

Original:  K 
Revised: m , m + 1 ≤ i ≤ l , i ∈ { 1 , . . , l } and l = m + n . The kernel matrix is K = (cid:20) K XX K XY K Y X

Original: [Equation]
Revised: K Y Y (cid:21) =

Original:  K 
Revised: (cid:20) K XZ

Original: [Equation] K Y Y (cid:21)  , 
Revised: K Y Z (cid:21) ∈ R l × l where K XZ ∈ R m × l , K Y Z ∈ R n × l , and K i,j = κ

Original: where K i,j = κ ( x i , x i ) − 2 κ ( x i , y j 
Revised: ( z i , z i ) − 2 κ ( z i , z j ) + κ ( z j , z j ) for i ∈ { 1 , . . . , m + n } .

Revised: The witness function evaluations for each sample are given by the vectors [ ω ( x 1 ) , . .

Original: [Equation]
Revised: . , ω ( x m )] ⊤ = K XZ α and [ ω ( y 1 ) , . . . , ω ( y n )]

Revised: ⊤ = K Y Z α .

Original: Wasserstein distance is 
Revised: For a positive definite kernel matrix K , to ensure ω has bounded norm, the coefficient vector should be restricted to be α ∈ A = { α ∈ R l : α T K α ≤ 1 } . Additional constraints are needed if the kernel matrix is positive semi-definite. The max-sliced kernel Wasserstein-2 distance (squared) can be computed in terms of witness function evaluations as

Original: [Equation]
Revised: [Equation]

Original: where D ˆ µ = 1 m I m ,  and similarly, (cid:107) ω (cid:107) 2ˆ ν =  1  n (cid:107) K Y
Revised: where P ˆ µ, ˆ ν = { P ∈ [0 , 1] m × n : P1 n = µ and P ⊤ 1 m = ν } is the transport polytope, where 1 ⊤ n = [1 , . . , 1] is a vector of n ones, and M ◦ 2 is the element-wise squaring of the entries of M .

Revised: The optimal transport plan can be obtained analytically after sorting the values in the vectors ω Y =

Original:  Z α (cid:107) 22 with D ˆ ν = 1 n I n  .
Revised: K Y Z α ∈ R m and ω X = K Y Z α ∈ R n . Assuming Q and R are the permutations such that ω ( x Q (1) ) ≤ · · · ≤ ω ( x Q ( m ) ) and ω ( y R (1) ) ≤ · · · ≤ ω ( y R ( n ) ) .

Original: When ω is ﬁxed,the optimal transport plan can be obtained by sorting witness function evaluations ω ( X )  = 
Revised: The optimal transport plan P ⋆ is defined as [ P ⋆ ] i,j = [˜ P ] Q ( i ) ,R ( j ) where ˜ P is optimal transport plan after sorting, which is the finite differences across rows and columns of G : [˜ P ] i,j = G i +1 ,j +1 − G i +1 , j − G i,j +1 with

Original: 75 (cid:80) mj =1 (cid:104) ω, φ ( x j ) (cid:105) ={+, (cid:80) j − 1 k =
+}
Revised: G i,j = min( (cid:80) i − 1 k =1 µ Q ( k ) {+, (cid:80) j − 1 k =
+}

Original: K XZ α and ω ( Y )  = {+max ( m, n ) . For the kernel landmark Wasserstein distance (L-W2), the continuous multivariate optimization of the optimal slice is replaced by a discrete optimization over the set of possible landmarks is { z i } li =
 = Z .+}
Revised: ν R ( k ) , where the matrix G ∈ [0 , 1] m +1 × n +1 is the zero-padded joint cumulative distribution of the optimal transport plan after sorting. Overall, it is a saddle-point optimization problem, with evaluation cost O ( N log( N )) for n > m where N = {+max ( m, n ) . For the kernel landmark Wasserstein distance (L-W2), the continuous multivariate optimization of the optimal slice is replaced by a discrete optimization over the set of possible landmarks is { z i } li =
 = Z .+}

Original: [Equation]
Revised: This is equivalent to restricting α to be one-hot vector, e.g., α = [0 , . . . . , 0] ⊤ . Assuming the i -th data point is the landmark, ω ( · ) = κ ( · , z i ) ,

Original: nj =
Revised: K XZ α =

Original: [Equation]
Revised: [ κ ( x 1 , z i ) , . . . , κ ( x m , z i )] ⊤ =

Original: ω, φ ( y j ) (cid:105) = K Y Z α  . 
Revised: k Xz i , and K Y Z α = [ κ ( y 1 , z i ) , . . , κ ( y n , z i )]

Original: The third term now canbe expressed as
Revised: ⊤ = k Y z i , which are the i -th columns of K XZ and K Y Z , respectively. Using these, we introduce the landmark-based max-sliced kernel Wasserstein-2 distance

Original: [Equation]
Revised: [Equation]

Original: , where P ˆ µ
Revised: We now assume i.i.d.

Original: [Equation] ˆ ν is thetransport polytope and  P is a optimal transport plan.
Revised: samples, µ 1 = · · · = µ m = 1 m and ν 1 = · · · = ν m = 1 n . Let P i be the optimal transport matrix for landmark i ∈ { 1 , . . .

Original: Kernel max-sliced Wasserstein distance can becomputed in terms of witness function evaluations as [Equation] where (cid:107) · (cid:107) 2 is the Euclidean norm For a positive deﬁnite kernel matrix K , the solution has to bebounded by α T K α ≤ 1 in order to bound the witness function’s norm.  Landmark-based Max-Slicing
Revised: , l } .

Original: Computation of the max-sliced Wasserstein distance still involves solving optimization problems toobtain parameters for an optimal slice which can be expensive. Here, we proposed an alternativesolution called Landmark-based max-sliced Wasserstein to tackle this problem. Instead of ﬁndingthe optimal slice by using complex optimization, we simply evaluate the discrepancy betweendistributions for each data point on Hilbert space.
Revised: It can be written as row and column permutations of ˜ P the optimal transport matrix for sorted i.i.d.

Original: In the landmark case, α ∈ { e i } li =1 is the standardbasis to R l , where e i= (0 , . . .
Revised: samples: P i = Q ⊤ i ˜ PR i , Q i ∈ Π m , R i ∈ Π n , where the entries of Q i

Original: . , 0) is the l -tuple with all entries are 0, except the ith equals1.
Revised: k Xz i

Original: Now, we can introduce landmark-based max-sliced kernel Wasserstein distance in terms of Eq.6 asfollowing
Revised: =

Original: [Equation] Let K Xz i = K XZ α i and K Y
Revised: [ κ ( x Q i (1) , z i ) , . . . , κ ( x Q i ( m ) , z i )]

Original: z i
Revised: ⊤ and R i

Original: = K Y 
Revised: k Y z i =

Original: Z α i represent the projections for each landmark { z i } li =1 91 from Z .We sort the projections to max-slicing the distributions for each data point.
Revised: [ κ ( y R i (1) , z i ) , . . , κ ( y R i ( n ) , z i )] are in ascending order, κ ( x Q i (1) , z i ) ≤ · · · ≤ κ ( x Q i ( m ) , z i ) and κ ( y R i (1) , z i ) ≤ · · · ≤ κ ( y R i ( n ) , z i ) .

Original: Let
Revised: ( Q i and R

Original: P i , i ∈ [1 , l ] be the permutation which sorts the projections.
Revised: i are the permutation matrices corresponding to permutations Q i ( · ) and R i ( · ) , where Π n denotes the set of n × n permutations matrices.)

Original: The sorted projections togethercan be expressed  as˜ K XZ =
Revised: The sorted witness function evaluations for each landmark can be expressed together as

Original: [ P 1 (cid:126)K Xz 1 , P 2 (cid:126)K Xz 2 , . . , P l (cid:126)K Xz l ] =
Revised: [Equation]

Original: [ ˜ K Xz 1 , ˜ K Xz 2 , . . 
Revised: , Q l k Xz l ] = [˜ k Xz 1 , . . . ,

Original: ˜ K Xz l ] and˜ K Y Z =
Revised: ˜ k Xz l ] and ˜ K Y Z =

Original: [ P 1 (cid:126)K Y z 1 , P 2 (cid:126)K Y z 2 , . . , P l (cid:126)K Y z l ] = [ ˜ K Y z 1 ,˜ K Y z 2 , . .
Revised: [ R 1 k Y z 1 , . .

Original: ˜ K Y z l ] . For simplicity, we deﬁne Eq.7 in terms of pre-sorted witness functions as following
Revised: , R l k Y z l ] =

Original: [Equation]
Revised: [˜ k Y z 1 , . .

Original: Further, we reformulate the kernel max-sliced Bures distance is in terms of our proposedmethod. The kernel based max-sliced Bures distance in terms of squared witness functions is
Revised: ˜ k Y z l ] .

Original: [Equation]
Revised: [Equation]

Original: The empirical sample version of Buresdistance is obtained following the Eq.5. The kernel max-sliced Bures distance is D H B (ˆ µ, ˆ ν ) = 
Revised: In the case of equal sample sizes m = n and µ 1 = · · · = µ m = ν 1 = · · · = ν m = 1 m , the optimal transport matrix is a scaled product of permutation matrices P i = 1 m Q ⊤

Original: [Equation] [Table]{+i R i , since ˜ P = 
+}
Revised: {+i R i , since ˜ P = 
+}

Original: m [Table] K XZ [Table]
Revised: m I .

Original: 2 − [Table] √ [Table]
Revised: In this case, the kernel landmark Wasserstein-2 distance is simply

Original: K Y Z

Original: [Table]
Revised: [Equation]

Original: The landmark-based kernel max-sliced Bures distanceis [Equation] [Figure] [Figure]
Revised: where the second expression computes the Euclidean norm of each column of the differences.

Original:  Covariate Shift Detection
Revised:  Covariate Shift Detection

Original: [Figure]

Original: In this section, we perform simulation experiments to compare landmark-based max-sliced divergence
Revised: In this section, we perform simulation experiments to compare kernel landmark Wassersteindistance and MMD for detecting imbalanced classes, as a specific form of covariate shift detection.

Original: [Figure]
Revised: (We also compare a kernel landmark based Bures distance described in the Appendix.)

Original: and MMD for detecting imbalanced classes. Speciﬁcally, we perform a statistical power test to detectthe difference between a sample with a uniform distribution of classes and an imbalanced sampletaken from the MNIST dataset  across different levels of imbalance and sample size.
Revised: We perform a statistical power test to detect the difference between a sample with a uniform distribution of classes and a sample with the underrepresented class. We also examine the ability of the witness function to identify instances associated with the underrepresented class. Specifically, we consider the MNIST dataset split into train and test sets across different levels of imbalance and sample size.

Original: Using MNIST data-set with  ten classes l ∈ { 0 , 1 , · · · , 9 } , ﬁrst, we test the methods when ˆ µ isbalanced but ˆ ν is not. When uniformly sampled, each of ten digits appears on average 10% of thetime.
Revised: ˆ µ represents a sample of the training set with a balanced proportion of each class, and ˆ ν has less instances from one class. MNIST has ten class labels L ∈ { 0 , 1 , . .

Original: Let p ∈
Revised: Let P L be the prevalence of L -th class/digit, which is underrepresented in ˆ ν .

Original:  [0 , 1] represent the  probability that a purposefully underrepresented digit is kept whenit is drawn. The probability of the underrepresented digit is calculated by P under = 110 (1 − p )  .
Revised: P L = 110 (1 − p ) where p ∈ [0 , 1] is the Bernoulli probability that the underrepresented digit is replaced by a majority class digit when it is drawn. The prevalence of the any other digit is expected to be P L ′ ∈{ 0 , 1 , ··· , 9 }\ L = 110 (1 − p ) + 19 p .

Original: When p takes values of 0, 0.5, or 0.8 the probabilities of underrepresented digit are 110 , 120 , and 150 ,respectively.
Revised: For example, when p takes values of 0, 0.5, or 0.8, the probabilities of underrepresented digit are 110 , 120 , and 150 , respectively.

Original: We use the  divergences to test the hypothesis that the two samples come from the same distribution. 
Revised: We use the kernel-based divergences to test the hypothesis that the two samples come from the same distribution. We use the Gaussian kernel κ ( x, y ) = e

Original:  As a signiﬁcance threshold for the divergence we use the 1 − α quantile of the surrogate distributionof divergence values when instances in each sample are randomly permuted between the two samples(250 times). 
Revised: [Equation] where σ is median of the pairwise distances. As a significant threshold for the divergence we use the 1 − α quantile of the surrogate distribution of divergence values when instances in each sample are randomly permuted between the two samples (250 times). To estimate the statistical power for a given occurrence level, we use

Original: To estimate the statistical power for a given occurrence level, we use 500 Monte Carlosamples iterations.
Revised: Monte Carlo samples iterations. Fig. 1 and Fig. 3 show the statistical power for L = 0 , digit “0”, across different values of P 0 and sample size.

Original: Conclusion
Revised: We also test the precision of the witness function in detecting the specific discrepancies associated to the class imbalance. We test whether the instances in the training set with the largest magnitude [Figure] [Figure]

Original: In this paper we have investigated the max-slicing techniques on the Wasserstein distance in Hilbertspaces to detect covariate shift.
Revised: witness function evaluations are from the minority class. We report the evaluation results using precision at 10 in Fig. 2 for the specific case where digit “6” class in the test set. We also compare the proposed approach to MMD’s witness function evaluations, which are slightly less precise.

Original: Inspired by sliced Wasserstein distance we proposed a simplerapproach to solve optimal transport problems.
Revised: This approach is useful in practice, since by examining the label distribution of the top- K training set examples, the user can understand if there is any imbalances in the test set. Examples of the instances with highest witness function for both the MNIST and for CIFAR-10 are shown in the Appendix. The statistical power across different kernel bandwidths (testing the sensitivity to the median heuristic) is also shown in the Appendix. The implementation of our approach and demos can be found at https://github.com/drpointcloud/landmark .

Original:  Our approach evaluates the discrepancy betweendistribution for each data point on Hilbert space.
Revised: In this paper we have investigated max-slicing for the kernel-based Wasserstein distance to detect class-based covariate shift. Our approach evaluates the discrepancy between distributions in terms of the similarity to a landmark point.

Original: Unlike the max sliced version, our method offers asimple solution.
Revised: Unlike the generalized max-sliced Wasserstein distance, the proposed distance can be computed exactly and efficiently for the case of two samples.

Original: Statistical power tests are employed to evaluate the performance of detecting classimbalances in testing data.
Revised: Statistical power tests are employed to evaluate the performance of detecting class imbalances in testing data.

Original: We compared our approach with MMD, which is a well-known approachto ﬁnd the discrepancy.
Revised: We compared our approach with MMD, which is a well-known approach to find the discrepancy.

Original: The preliminary results shows that the proposed method detects covariateshift better than MMD. 
Revised: The preliminary results shows that the proposed method detects simple cases of covariate shift better than MMD. Acknowledgments and Disclosure of Funding

Revised: Yüksel Karahan and Austin J. Brockmeier’s effort were sponsored by the Department of the Navy,

Revised: Office of Naval Research under ONR award number N00014-21-1-2300.

Revised: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Office of Naval Research.

Revised: Bilal Riaz is supported by the

Revised: US-Pakistan Knowledge Corridor Ph.D. program at the University of Delaware.

Revised: A Appendix

Revised: The appendix contains the statement and proof of the metric property of the kernel landmark

Revised: Wasserstein-2 distance; a description of the kernel landmark Bures divergence; and additional results.

Revised: Theorem 1.

Revised: If κ is characteristic [Fukumizu et al., 2008] and normalized κ ( z, z ) = 1 ∀ z ∈ X , then W H L ∗ p ( µ, ν ) is a probability distance metric.

Revised: For µ, ν, ξ ∈ P ( X ) with finite p -th moments

Revised: [Equation]

Revised: Proof.

Revised: Non-negativity and symmetry are obvious from the proprieties of the Wasserstein distance.

Revised: If µ = ν , then for any ω ∈ H , inf γ ∈ Γ( µ,µ ) E

Revised: ( X,Y ) ∼ γ | ω

Revised: ( X ) − ω

Revised: ( Y )

Revised: | p = 0 .

Revised: We assume µ ̸ = ν and proceed to lower bound the distance and show that W H L ∗ p ( µ, ν ) > 0 .

Revised: For any ω ∈ H ,

Revised: [Equation]

Revised: where Jensen’s inequality is used based on the convexity of |·| p .

Revised: Taking the supremum over the set of landmarks yields an expression in terms of the difference of the means in the RKHS

Revised: [Equation]

Revised: When κ is characteristic

Revised: [Equation]

Revised: the mapping m ξ : ξ (cid:55)→ E X ∼ ξ

Revised: [ ϕ ( X )] is injective for ξ ∈ P ( X )

Revised: [Fukumizu et al., 2008]

Revised: [Equation]

Revised: and µ ̸

Revised: = ν = ⇒ m µ ̸

Revised: = m µ = ⇒ ∃ z, m µ ( z ) − m µ ( z )

Revised: = h ( z ) ̸

Revised: = 0 .

Revised: Together this yields W H L ∗ p (

Revised: [Equation]

Revised: The triangle inequality follows from the fact that the Wasserstein distance itself is a metric

Revised: [Equation]

Revised: A.1 Kernel Landmark Bures Distance

Revised: When the µ and ν are Gaussian distributions, the Wasserstein-2 (W2) distance can be computed analytically in terms of the first and the second moments [Peyré and Cuturi, 2019].

Revised: The kernel Gaussian

Revised: W2 distance [Zhang et al., 2020] is defined as W H G ( µ, ν ) =

Revised: (cid:112)

Revised: ∥ m µ − m ν

Revised: ∥ 2 + d B (Σ µ , Σ ν ) 2 , where d B (Σ µ , Σ ν ) =

Original: {+(cid:0) tr(Σ µ ) + tr(Σ ν ) − 2tr(Σ 12 µ Σ ν Σ 12 µ ) (cid:1) 
+}
Revised: {+(cid:0) tr(Σ µ ) + tr(Σ ν ) − 2tr(Σ 12 µ Σ ν Σ 12 µ ) (cid:1) 
+}

Revised: [Bhatia et al., 2019], Σ µ = ρ µ − m µ ⊗ m µ and Σ ν = ρ

Revised: ν − m ν ⊗ m ν are the covariance matrices of X and Y in the RKHS, and ρ µ = E

Revised: X ∼ µ [ ϕ ( X ) ⊗ ϕ ( X )]

Revised: ∈ H and ρ ν = E Y ∼ ν

Revised: [ ϕ ( Y ) ⊗ ϕ ( Y )]

Revised: ∈ H are the uncentered second moments.

Revised: The Bures distance between the uncentered second moments in the Hilbert space [Brockmeier et al., 2021] is also a divergence measure D H B ( µ, ν ) =

Revised: d B ( ρ µ , ρ ν ) .

Revised: For comparison, we consider a landmark-based version of the kernel-based max-sliced Bures distance.

Revised: The kernel-based max-sliced Bures distance is expressed in terms of the squared witness functions as D H

Revised: [Equation]

Revised: B

Revised: [Equation]

Revised: µ, ν ) =

Revised: [Equation]

Revised: ∈H

Revised: [Equation]

Revised: E

Revised: [Equation]

Revised: [ ω

Revised: [Equation]

Revised: X

Revised: )] − (cid:112) E Y ∼ ν

Revised: [ ω 2 ( Y )] (cid:12)(cid:12)(cid:12) .

Revised: For i.i.d.

Revised: samples, the kernel max-sliced Bures distance is D H ∗ B (ˆ µ, ˆ ν ) =

Revised: [Equation]

Revised: The landmark-based max-sliced kernel Bures distance (L-Bures) is

Revised: [Equation]

Revised: A.2 Additional Experiments

Revised: We performed statistical power test for various sample sizes (see Fig3) and kernel bandwidths (see

Revised: Fig. 6).

Revised: We also applied the proposed method and MMD on the purposefully imbalanced subsets of

Revised: CIFAR10 dataset (please see Fig. 4) where each instance is represented by the internal representation of the inception network [Szegedy et al., 2016]: a 2048-dimensional vector.

Revised: We used witness function evaluations to identify instances associated with the underrepresented class.

Revised: In Fig.

Revised: 7, we report the computation complexity of our method compared to MMD and the discrete Wasserstein-2 distance.

Revised: As it can be seen in Fig.

Revised: 7, the proposed method, is much easier to compute than the Wasserstein distance.

Revised: [Figure]

Revised: =

Revised: [Figure]

Revised: [Figure]

Revised: Rejection rate at =0.

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]