Original: Global Optimal K-Medoids Clustering of One Million Samples
Revised: Global Optimal K-Medoids Clustering of One Million Samples

Original: Anonymous Author(s)

Original: AffiliationAddress

Original: Abstract
Revised: Abstract

Original: We study the deterministic global optimization of the K-Medoids clustering problem.
Revised: We study the deterministic global optimization of the K-Medoids clustering problem.

Original: This work proposes a branch and bound (BB) scheme, in which a tailored
Revised: This work proposes a branch and bound (BB) scheme, in which a tailored

Original: Lagrangian relaxation method proposed in the 1970s is used to provide a lowerbound at each BB node.
Revised: Lagrangian relaxation method proposed in the 1970s is used to provide a lower bound at each BB node.

Original: The lower bounding method already guarantees themaximum gap at the root node.
Revised: The lower bounding method already guarantees the maximum gap at the root node.

Original: The closed-form solution to the lower boundcan be derived analytically without explicitly solving any optimization problems,and its computation can be easily parallelized.
Revised: A closed-form solution to the lower bound can be derived analytically without explicitly solving any optimization problems, and its computation can be easily parallelized.

Original: Moreover, with this lower bounding method, finite convergence to the global optimal solution can be guaranteedby branching only on the regions of medoids.
Revised: Moreover, with this lower bounding method, finite convergence to the global optimal solution can be guaranteed by branching only on the regions of medoids.

Original: We also present several tailoredbound tightening techniques to reduce the search space and computational costsignificantly.
Revised: We also present several tailored bound tightening techniques to reduce the search space and computational cost.

Original: Extensive computational studies on 28 machine learning datasetsdemonstrate that our algorithm can provide a provable global optimal solution withan optimality gap of 0.1% within 4 hours on datasets with up to one million samples.
Revised: Extensive computational studies on 28 machine learning datasets demonstrate that our algorithm can provide a provable global optimal solution with an optimality gap of 0.1% within 4 hours on datasets with up to one million samples.

Revised: Besides, our algorithm can obtain better or equal objective values than the heuristic method.

Original: A theoretical proof of global convergence for our algorithm is also presented.
Revised: A theoretical proof of global convergence for our algorithm is also presented.

Original:  Introduction
Revised:  Introduction

Original: In this paper, we concentrate on the K-Medoids clustering problem: given a dataset, the K-Medoidsproblem aims to select K samples from the dataset as cluster medoids that minimize the overall sumof dissimilarities from other samples to the closet medoids.
Revised: In this paper, we concentrate on the K-Medoids clustering problem: given a dataset, K-Medoids aims to select K samples from the dataset as cluster medoids that minimize the sum of dissimilarities from other samples to the closet medoids.

Original: Here, selecting from existing samples asmedoids is called the “medoids on samples” constraint.
Revised: Here, selecting from existing samples as medoids is called the “medoids on samples” constraint.

Original: The K-Medoidsproblem is similar to the K-Means problem except for the arbitrary dissimilarity measurements and the “medoids on samples”constraint.
Revised: K-Medoids is similar to the K-Means problem except for the arbitrary dissimilarity measurements and the “medoids on samples” constraint.

Original: Hence, K-Medoids generally has better interpretability than K-Means [1].
Revised: Hence, K-Medoids generally has better interpretability than K-Means [1].

Original: K-Medoidsproblems are generally NP-hard to solve exactly.
Revised: K-Medoids problems are generally NP-hard to solve exactly.

Original: So there are many heuristic methods, such as PAM[2], CLARA [3], K-means-like method [4], and Fast-PAM [5].
Revised: So there are many heuristic methods, such as PAM [2], CLARA [3], K-means-like method [4] and Fast-PAM [5].

Original: However, none of them can guaranteereaching the global minimum.
Revised: However, none of them can guarantee reaching the global minimum.

Original: To solve the K-Medoids problem deterministically, back in the 1970s, Cornuejols et al.
Revised: To solve the K-Medoids problem deterministically, back in the 1970s, Cornuejols et al.

Original: [6] proposed a
Revised: [6] proposed a

Original: Lagrangian relaxation method for obtaining a lower bound of the optimal value, which can guaranteea relative optimal gap smaller than 1 /e .
Revised: Lagrangian relaxation method for obtaining a lower bound of the optimal value, which can guarantee a relative optimal gap smaller than 1 /e .

Original: Interestingly, to the best of our knowledge, this is the onlyapplication in which the Lagrangian duality gap can be guaranteed within a certain threshold.
Revised: Interestingly, to the best of our knowledge, this is the only application in which the Lagrangian duality gap can be guaranteed within a certain threshold.

Original: Severalimprovements to this method are proposed in the literature.
Revised: Several improvements to this method are proposed in the literature.

Original: One common direction is improvingthe heuristic process of Lagrangian dual problems.
Revised: One common direction is improving the heuristic process of Lagrangian dual problems.

Original: [7] constructed a semi-Lagrangian relaxation,then accelerated the heuristic process by a cutting plane method.
Revised: [7] constructed a semi-Lagrangian relaxation, then accelerated the heuristic process by a cutting plane method.

Original: It was further improved in [8] byfixing variables in the heuristic process.
Revised: It was further improved in [8] by fixing variables in the heuristic process.

Original: Their works can deal with medium-scale datasets (e.g., 3000samples).
Revised: Their works can deal with medium-scale datasets (e.g.,samples).

Original: Another common direction is column generation [9, 10].
Revised: Another common direction is column generation [9, 10].

Original: One limitation of these Lagrangian relaxation based methods is that there is no guarantee of convergence to the global optimal solution, due to the existence of duality gap.
Revised: One limitation of these Lagrangian relaxation based methods is that there is no guarantee of convergence to the global optimal solution, due to the existence of duality gap.

Original: To address the globaloptimum guarantee, a few works have utilized the branch and bound (BB) scheme in K-Medoids 
Revised: To address the global optimum guarantee, a few works have utilized the branch and bound (BB) scheme in K-Medoids [11, 12, 13].

Original: Christofides and Beasley [11] proposed a prototype by branching on binary variablesand was capable of small-scale datasets (200 samples).
Revised: Christofides and Beasley [11] proposed a prototype by branching on binary variables and was capable of small-scale datasets (200 samples).

Original: This scheme was further improved by [12]through branching on semi-assignment constraints, which obtained solutions faster on small-scaledatasets.
Revised: This scheme was further improved by [12] through branching on semi-assignment constraints, which obtained solutions faster on small-scale datasets.

Original: [13] presented a tighter form of the K-Medoids problem and was capable of medium-scaledatasets (900 samples).
Revised: [13] presented a tighter form of the K-Medoids problem and was capable of medium-scale datasets (900 samples).

Original: Besides these specific works, several off-the-shelf solvers also containwell-implemented BB schemes, such as CPLEX [14] and Gurobi [15].
Revised: Besides these specific works, several off-the-shelf solvers also contain well-implemented BB schemes, such as CPLEX [14] and Gurobi [15].

Original: These solvers can typicallyaddress the K-Mediods problem with medium-scale datasets (e.g., 2000 samples).
Revised: These solvers can typically address the K-Mediods problem with medium-scale datasets (e.g., 2000 samples).

Original: The BB schemes used in the aforementioned works can only address medium-scale datasets becausethey need to branch on all the binary variables to guarantee convergence, which leads to poorscalability because the number of binary variables increases linearly as the number of samples.
Revised: The BB schemes used in the aforementioned works can only address medium-scale datasets because they need to branch on all the binary variables to guarantee convergence, which leads to poor scalability because the number of binary variables increases linearly as the number of samples.

Original: Therefore, we focus on the reduced-space BB scheme proposed in the stochastic programmingcommunity, which only needs to branch on the space of coupling variables (e.g., medoids).
Revised: Therefore, we focus on the reduced-space BB scheme proposed in the stochastic programming community, which only needs to branch on the space of coupling variables (e.g., medoids).

Original: This ideawas first reported for stochastic mixed-integer linear programs in [16] and then extended to stochasticmixed-integer nonlinear programs in [17].
Revised: This idea was first reported for stochastic mixed-integer linear programs in [16] and then extended to stochastic mixed-integer nonlinear programs in [17].

Original: The convergence for stochastic nonlinear programs wasproved in [18].
Revised: The convergence for stochastic nonlinear programs was proved in [18].

Original: Recently, the authors in [19] adopted this method and reported the capacity ofsolving the K-means clustering problem (a similar clustering task without the “medoids on samples”constraint) with 210,000 samples in parallel to an optimal gap of 2.5%.
Revised: Recently, the authors in [19] adopted this method and reported the capacity of solving the K-means clustering problem (a similar clustering task without the “medoids on samples” constraint) with 210,000 samples in parallel to an optimal gap of 2.5%.

Original: However, the convergenceof the reduced-space BB scheme can only be guaranteed when it is combined with suitable lowerbounding methods.
Revised: However, the convergence of the reduced-space BB scheme can only be guaranteed when it is combined with suitable lower bounding methods.

Original: In previous works, lower bounds are constructed by the Lagrangian relaxation[16] [17] [19] or removal [18] of the so-called non-anticipativity constraints (an implicit constraintthat all the samples share the same coupling variables, that is Constraint 10 in this paper).
Revised: In previous works, lower bounds are constructed by the Lagrangian relaxation [16] [17] [19] or removal [18] of the so-called non-anticipativity constraints (an implicit constraint that all the samples share the same coupling variables, that is Constraint 10 in this paper).

Original: However,these methods are unsuitable for the K-Medoids problem, since they cannot address the "medoids onsamples" constraint (a constraint that has a large dimension and connects all samples) and thus cannotensure the convergence of the reduced-space BB scheme.
Revised: However, these methods are unsuitable for the K-Medoids problem, since they cannot address the "medoids on samples" constraint (a constraint that has a large dimension and connects all samples) and thus cannot ensure the convergence of the reduced-space BB scheme.

Original: Moreover, these lower bounding methodsrequire the solution of many expensive mixed-integer nonlinear programs to global optimality ateach node.
Revised: Moreover, these lower bounding methods require the solution of many expensive mixed-integer nonlinear programs to global optimality at each node.

Original: In contrast, the work of Cornuejols et al.
Revised: In contrast, the work of Cornuejols et al.

Original: [6] generates lower bounds for the K-Medoidsproblem by the Lagrangian relaxation of the semi-assignment constraint (Constraint 2b in this paper),and the formulation directly takes the "medoids on samples" constraint into consideration.
Revised: [6] generates lower bounds for the K-Medoids problem by the Lagrangian relaxation of the semi-assignment constraint (Constraint 2b in this paper), and the formulation directly takes the "medoids on samples" constraint into consideration.

Original: Thismethod has a closed-form solution, and its computation can be easily parallelized.
Revised: This method has a closed-form solution, and its computation can be easily parallelized.

Original: Our Contributions: Firstly, we combine the method of reduced-space BB scheme and the workof Cornuejols et al.
Revised: Our Contributions: Firstly, we combine the method of reduced-space BB scheme and the work of Cornuejols et al.

Original: [6], and adapt these methods for the K-Medoids problem.
Revised: [6], and adapt these methods for the K-Medoids problem.

Original: The Lagrangianrelaxation in [6] is used to generate a tight lower bound, which already guarantees a maximumgap at the root node.
Revised: The Lagrangian relaxation in [6] is used to generate a tight lower bound, which already guarantees a maximum gap at the root node.

Original: With this lower bound, we can prove the finite convergence (detailed proofspresented in the Appendix A) by branching only on the regions of medoids (the dimension of whichis independent of the number of samples).
Revised: With this lower bound, we can prove the finite convergence (detailed proofs presented in the Appendix A) by branching only on the regions of medoids (the dimension of which is independent of the number of samples).

Original: Secondly, several tailored bound tightening techniques,including probing and feasibility based methods, are proposed to significantly reduce the search spaceand computational cost of each node.
Revised: Secondly, several tailored bound tightening techniques, including probing and feasibility based methods, are proposed to significantly reduce the search space and computational cost of each node.

Original: Lastly, we propose an upper bounding method by heuristicupdates of the candidate solutions obtained in the lower bounding procedure.
Revised: Lastly, we propose an upper bounding method by heuristic updates of the candidate solutions obtained in the lower bounding procedure.

Original: These contributionsenable our algorithm to be extremely scalable.
Revised: These contributions enable our algorithm to be extremely scalable.

Original: Capability For Million Scale Problems: we provide an open-source project in Julia and presentnumerical experiments on 28 datasets.
Revised: Capability For Million Scale Problems: we provide an open-source project in Julia and present numerical experiments on 28 datasets.

Original: Specifically, for two datasets with one million samples (whichis 400 times larger than the state-of-art BB method), our algorithm can reach a small gap (0.1%)within one hour.
Revised: Specifically, for two datasets with one million samples (which is 400 times larger than the state-of-art BB method), our algorithm can reach a small gap (0.1%) within one hour.

Original: To the best of our knowledge, no other article has ever reported results on datasetsof this scale for K-Medoids problems with a global convergence guarantee.
Revised: To the best of our knowledge, no other article has ever reported results on datasets of this scale for K-Medoids problems with a global convergence guarantee.

Original:  PreliminariesK-Medoids problem:
Revised:  Preliminaries

Original:  we denote a dataset with S samples and A attributes as X ={ x 1 , ..., x s , ..., x S } ∈ RA × S  , in which x s =
Revised: K-Medoids problem: we denote a dataset with S samples and A attributes as X = { x 1 , ..., x s , ..., x S } ∈ R S × A , in which x s =

Original: [ x s, 1 , ..., x s,A ] ∈ R
Revised: [ x s, 1 , ..., x s,A ] ∈ R

Original: A is the s th sample, and x s,ais the a th attribute of s th sample.
Revised: A is the s th sample, and x s,a is the a th attribute of s th sample.

Original: The K-Medoids problem aims to select a subset from the dataset asthe cluster medoids with the following object:
Revised: The K-Medoids problem aims to select a subset from the dataset as the cluster medoids with the following objective:

Original: [Equation]
Revised: [Equation]

Original: where µ k :=
Revised: where µ k :=

Original: [ µ k 1 , ..., µ kA ] ∈ R
Revised: [ µ k 1 , ..., µ kA ] ∈ R

Original: A is the medoid of k th cluster, µ := { µ 1 , ..., µ K } is the medoids set,
Revised: A is the medoid of k th cluster, µ := { µ 1 , ..., µ K } is the medoids set,

Original: S :
Revised: S :

Original: = { 1 , ..., S } is the index set of samples in the dataset, K := { 1 , ..., K } is the index set of clusters,
Revised: = { 1 , ..., S } is the index set of samples in the dataset, K := { 1 , ..., K } is the index set of clusters,

Original: A := { 1 , ..., A } is the index set of attributes.
Revised: A := { 1 , ..., A } is the index set of attributes.

Original: We use µ ∈ X to represent the “medoids on samples”[Equation] constraint in the K-Medoids problem.
Revised: We use µ ∈ X to represent the “medoids on samples” constraint in the K-Medoids problem.

Original: To facilitate the discussion of the BB scheme, we introduce◦ M :=
Revised: To facilitate the discussion of the BB scheme, we introduce ◦

Original:  { µ | ◦ µ ≤ µ ≤ ◦ ¯ µ } as the initial regions of medoids.
Revised: M := { µ | ◦ µ ≤ µ ≤ ◦ ¯ µ } as the initial regions of medoids.

Original: Here, ◦ µ and ◦ ¯ µ are the lower and upperbounds of medoids referred from data.
Revised: Here, ◦ µ and ◦ ¯ µ are the lower and upper bounds of medoids referred from data.

Original: Specifically, ◦ µ a = min s ∈ S x s,a and ◦ ¯ µ a = max s ∈ S x s,a are thebounds of the a th attribute.
Revised: Specifically, ◦ µ a = min s ∈ S x s,a and ◦ ¯ µ a = max s ∈ S x s,a are the bounds of the a th attribute.

Original: Note that the introduction of ◦ M does not affect the optimal solution.
Revised: Note that the introduction of ◦ M does not affect the optimal solution.

Original: Equivalently, the K-Medoids problem can also be expressed in an extensive form (EF) [20]:
Revised: Equivalently, the K-Medoids problem can also be expressed in an extensive form (EF) [20]:

Original: [Equation]
Revised: [Equation]

Original: where d s,j = || x s − x j || 22 is the distance between samples x s and x j ; y j is equal to 1 if sample x j isselected as a medoid, and otherwise 0 ; b s,j is equal to 1 if sample x s belongs to the cluster whosemedoid issample x j , and otherwise 0 .
Revised: where d s,j = || x s − x j || 22 is the distance between samples x s and x j ; y j is equal to 1 if sample x j is a medoid, and otherwise 0 ; b s,j is equal to 1 if sample x s belongs to the cluster whose medoid is x j , and otherwise 0 .

Original: Note here d s,j is computed offline, so arbitrary dissimilaritymeasurements can be used without affecting the solution process.
Revised: Note here d s,j is computed offline, so arbitrary dissimilarity measurements can be used without affecting the solution process.

Original: Constraint 2b ensures that eachsample x s must belong to one cluster.
Revised: Constraint 2b ensures that each sample x s must belong to one cluster.

Original: Constraint 2c guarantees that a total of K samples are selectedas medoids.
Revised: Constraint 2c guarantees that a total of K samples are selected as medoids.

Original: Constraint 2d expresses that sample x s can be assigned to sample x j only if x j is themedoid of a cluster.
Revised: Constraint 2d expresses that sample x s can be assigned to sample x j only if x j is the medoid of a cluster.

Original: Lagrangian Relaxation of the K-Medoids problem: by the Lagrangian relaxation of the Constraint2b, we can have a relaxed problem of the Extensive Form 2:
Revised: Lagrangian Relaxation of the K-Medoids problem: by the Lagrangian relaxation of the Constraint 2b, we can have a relaxed problem of the Extensive Form 2:

Original: [Equation]
Revised: [Equation]

Original: Constraints in the Extensive Form 2 except 2b ,
Revised: Constraints in the Extensive Form 2 except 2b ,

Original: [Equation]
Revised: [Equation]

Original: where λ s is the Lagrangian multiplier for s th sample.
Revised: where λ s is the Lagrangian multiplier for s th sample.

Original: We define the multiplier set as λ := { λ 1 , ..., λ S } .
Revised: We define the multiplier set as λ := { λ 1 , ..., λ S } .

Original: For a given λ , β LD ( ◦ M, λ ) can be solved analytically.
Revised: For a given λ , β LD ( ◦ M, λ ) can be solved analytically.

Original: Considering the objective function andconstraints of the Relaxed Problem 3, we note that the optimal values for b s,j are given by[Equation] [Equation] otherwise
Revised: Considering the objective function and constraints of the Relaxed Problem 3, we note that the optimal values for b s,j are given by

Original: [Equation]
Revised: [Equation]

Original: We define the contribution of j th sample to the objective function as:
Revised: We define the contribution of j th sample to the objective function as:

Original: [Equation]
Revised: [Equation]

Original: The Lagrangian problem can be reduced to
Revised: The Lagrangian problem can be reduced to

Original: [Equation]
Revised: [Equation]

Original: Its closed-form solution can be easily derived by selecting y j corresponding to the K smallest ρ j .
Revised: Its closed-form solution can be easily derived by selecting y j corresponding to the K smallest ρ j .

Original: Lagrangian dual: a proper selection of λ may generate a tighter lower bound.
Revised: Lagrangian dual: a proper selection of λ may generate a tighter lower bound.

Original: Hence, we need tosolve the Lagrangian dual problem to get the tightest lower bound:[Equation]
Revised: Hence, we need to solve the Lagrangian dual problem to get the tightest lower bound:

Original: [Equation]
Revised: [Equation]

Original: There are several off-the-shelf heuristic methods to update λ , such as Sub-gradient method [11],
Revised: There are several off-the-shelf heuristic methods to update λ , such as Sub-gradient method [11],

Original: Volume method [21] and Cutting-Plane method [22].
Revised: Volume method [21] and Cutting-Plane method [22].

Original: The solution to the Dual Problem 7 provides alower bound to the original problem, with β LD ( ◦ M ) ≤ z ( ◦ M ) .
Revised: The solution to the Dual Problem 7 provides a lower bound to the original problem, with β LD ( ◦ M ) ≤ z ( ◦ M ) .

Original: Particularly, for this problem, there isa unique guarantee of maximum gaps between the lower bounds and the global optimal value, whichis
Revised: Particularly, for this problem, there is a unique guarantee of maximum gaps between the lower bounds and the global optimal value, which is

Original: [Equation]
Revised: [Equation]

Original: This guarantee is theoretically proved in [6, 23].
Revised: This guarantee is theoretically proved in [6, 23].

Original:  Lagrangian Based Lower Bound
Revised:  Lagrangian Based Lower Bound

Original: The aforementioned Lagrangian method cannot close the duality gap.
Revised: The aforementioned Lagrangian method cannot close the duality gap.

Original: To obtain a global optimalsolution, we propose to combine it with the spatial branch and bound (BB) scheme, in which
Revised: To obtain a global optimal solution, we propose to combine it with the spatial branch and bound (BB) scheme, in which

Original: Lagrangian relaxation provides a lower bound for each BB node.
Revised: Lagrangian relaxation provides a lower bound for each BB node.

Original: Tailored Lagrangian Relaxation in BB nodes: Since our BB scheme branches on the regions ofthe medoids µ , at each BB node, we solve the problem with the partition set M ⊆ ◦ M .
Revised: Tailored Lagrangian Relaxation in BB nodes: Since our BB scheme branches on the regions of the medoids µ , at each BB node, we solve the problem with the partition set M ⊆ ◦ M .

Original: That is, aconstraint µ ∈ M is added at each BB node.
Revised: That is, a constraint µ ∈ M is added at each BB node.

Original: We further define M k :=
Revised: We further define M k :=

Original: { µ k | µ k ≤ µ k ≤ ¯ µ k } as themedoid region of the k th cluster, where µ k and ¯ µ k are the lower and upper bounds of the k th medoids.
Revised: { µ k | µ k ≤ µ k ≤ ¯ µ k } as the medoid region of the k th cluster, where µ k and ¯ µ k are the lower and upper bounds of the k th medoids.

Original: Since the possible region of each cluster’s medoid can be different, we need to know the belongingsof medoids to each cluster.
Revised: Since the possible region of each cluster’s medoid can be different, we need to know the assignment of medoids to each cluster.

Original: Hence, we replace y j with y kj in EF 2: y kj is equal to 1 if sample x j isselected as the medoid of the k th cluster, and otherwise 0 .
Revised: Hence, we replace y j with y kj in EF 2: y kj is equal to 1 if sample x j is selected as the medoid of the k th cluster, and otherwise 0 .

Original: The Constraint 2d is replaced by
Revised: The Constraint 2d is replaced by

Original: [Equation]
Revised: [Equation]

Original: Notice that µ k is not directly modelled in EF, we need to transform the medoid regions µ k ∈ M kto the constraints of y kj .
Revised: Notice that µ k is not directly modelled in EF, we need to transform the medoid regions µ k ∈ M k to the constraints of y kj .

Original: The regions of medoids implicitly control the selection of y kj because wehave y kj = 0 , if x j ̸∈ M k .
Revised: The regions of medoids implicitly control the selection of y kj because we have y kj = 0 , if x j ̸∈ M k .

Original: Hence, we define the feasible index set of k th cluster’s medoid as
Revised: Hence, we define the feasible index set of k th cluster’s medoid as

Original: S k + ( M ) :
Revised: S k + ( M ) :

Original: = { j ∈ S | µ k ≤ x j ≤ ¯ µ k } .
Revised: = { j ∈ S | µ k ≤ x j ≤ ¯ µ k } .

Original: In this way, we can reformulate the reduced Lagrangian
Revised: In this way, we can reformulate the reduced Lagrangian

Original: Problem 6 at the BB node with M as follow:
Revised: Problem 6 at the BB node with M as follow:

Original: [Equation]
Revised: [Equation]

Original: s.t. Constraint (2d.1), (2d.2) .

Original: [Equation]

Original: 8) with the Lagrangian dual problem defined as β LD ( M ) = max λ β LD ( M, λ ) .
Revised: with the Lagrangian dual problem defined as β LD ( M ) = max λ β LD ( M, λ ) .

Original: The optimal value of y kjin Problem 8 can be obtained by the following procedure: (1) initialize k = 

Revised: The optimal value of y kj in Problem 8 can be obtained by the following procedure: (1) initialize k =

Original: and J ∗ = ∅ (2) let j k ∗be the index of smallest ρ j ( λ ) subject to j ∈ S k +  and j / ∈ J ∗ ; (3) set y kj =[-
-]
Revised: and J ∗ = ∅ (2) let j k ∗ be the index of smallest ρ j ( λ ) subject to j ∈ S k + ( M ) and j / ∈ J ∗ ; (3) set y kj =[-
-]

Original: if j = j k ∗ , otherwise,0; (4) add j k ∗ to J ∗ ; (5) update k = k + 1 and return to (2) until k = K .
Revised: if j = j k ∗ , otherwise, 0; (4) add j k ∗ to J ∗ ; (5) update k = k + 1 and return to (2) until k = K .

Original: Parallel implementation: we give a paralleled version of the tailored Lagrangian relaxation in BBnodes, in which the computations of contributions, ρ j ( λ ) , are performed in each process.
Revised: Parallel implementation: we implement a parallel version of the tailored Lagrangian relaxation, in which the computations of contributions, ρ j ( λ ) , are performed in each process.

Revised: At the beginning of the BB procedure, given P processes, the index set S of the whole dataset is evenly divided into P subsets, S p , p = 1 , .., P .

Revised: Then, each process computes and stores the distance assigned to it, which is d s,j , s ∈ S , j ∈ S p .

Original: Firstly, wedefine the feasible selection set of all the medoids as
Revised: At each BB node with M , denote the feasible index set of all the medoids as

Original: [Equation]
Revised: [Equation]

Original: Then, given P processes, the feasible selection set S + is evenly divided into P subsets, S + p , p =1 , .., P .

Original: Finally, the p th processonly computes the  contributions of samples allocated to this process,which is ρ j ( λ ) , ∀ j ∈ S+ p  .
Revised: Then, each process computes the assigned contributions using the stored d s,j , which is ρ j ( λ ) , ∀ j ∈ S p ∩ S + ( M ) .

Original: The remaining parts of the parallel implementation (e.g., update of λ ) areidentical to the serial version.
Revised: The remaining parts of the parallel implementation (e.g., update of λ ) are identical to the serial version.

Original:  Tailored Reduced-space Branch and Bound Scheme
Revised:  Tailored Reduced-space Branch and Bound Scheme

Original: This section introduces the tailored reduced-space branch and bound algorithm for the K-Medoidsproblem, which is detailed in Algorithm 1.
Revised: This section introduces the tailored reduced-space branch and bound algorithm for the K-Medoids problem, which is detailed in Algorithm 1.

Original: Specifically, starting from the root node with ◦ M , the BBscheme recursively partitions the medoid regions.
Revised: Specifically, starting from the root node with ◦ M , the BB scheme recursively partitions the medoid regions.

Original: At each iteration, the algorithm first selects thenode with the lowest lower bound and denotes it as M .
Revised: At each iteration, the algorithm first selects the node with the lowest lower bound and denotes it as M .

Original: Then, bound tightening techniques are appliedto get a tightened medoid region ˆ M .
Revised: Then, bound tightening techniques are applied to get a tightened medoid region ˆ M .

Original: The lower and upper bounds of the tightened node are computedand updated sequentially.
Revised: The lower and upper bounds of the tightened node are computed and updated sequentially.

Original: If the gap is larger than the tolerance, we branch on the medoid variable µ k awith the maximum range to get two sub-nodes M 1 and M 2 with relint ( M 1 ) ∩ relint ( M 2 ) = ∅ ,where relint ( M ) is the relative interior of M .
Revised: If the gap is larger than the tolerance, we branch on the medoid variable µ k a with the maximum range to get two sub-nodes M 1 and M 2 with relint ( M 1 ) ∩ relint ( M 2 ) = ∅ , where relint ( M ) is the relative interior of M .

Original: On the convergence of the algorithm, we have thefollowing theorem with detailed proofs in Appendix A:
Revised: On the convergence of the algorithm, we have the following theorem with detailed proofs in Appendix A:

Original: Theorem 1.
Revised: Theorem 1.

Original: Algorithm 1 is convergent to the global optimal solution after a finite step L , withβ L =
Revised: Algorithm 1 is convergent to the global optimal solution after a finite step L , with β L =

Original: z = α L , by only branching on the space of µ .
Revised: z = α L , by only branching on the space of µ .

Original: α l .
Revised: ∈ X ∩

Original: 1: for Sample x s ∈ X ∩ M, do 2: if x s is not assigned in the parent node then 3: if dk + s,max < d k − s,min , ∀ k − ∈ K \ k +then 4:
Revised: , do 2: if x s is not assigned in the parent node then 3: if d − ,min , ∀ k − ∈ K \ k +

Original: Assign x s to k + th cluster.
Revised: Assign x s to k + th cluster.

Original: 5: end if 6: end if 7: end for
Revised: 5: end if 6: end if 7: end for

Original: 1: for Cluster k ∈ K do 2:
Revised: 1: for Cluster k ∈ K do 2:

Original: Obtain the assigned set S kA according to Alg.3.
Revised: Obtain the assigned set S kA according to Alg.3.

Original: Compute RHS =
Revised: Compute RHS =

Original: αl − β( M )− .
Revised: α − β M )

Original: 4: for Attribute
Revised: 4: for Attribute

Original: a ∈
Revised: a ∈

Original: A do 5:
Revised: A do 5:

Original: Obtain the tightened node,ˆ M ka = { µ ka | µ ka ≤ µ ka≤ ˆ¯ µ ka } , by solving the Inequality 13.
Revised: Obtain the tightened node, µ ka ≤ µ ka } , by solving the inequation 13.

Original: Updateˆ M ka ← ˆ M ka ∩ M ka 7: end for 8: end for
Revised: Update 7: end for 8: end for

Original: 4.1 Lower Bounds
Revised: 4.1 Lower Bounds

Original: Basic lower bound:
Revised: Basic lower bound:

Original: Besides the Lagrangian based lower bound, we also introduce a basic lowerbound β basic .
Revised: Besides the Lagrangian based lower bound, we also introduce a basic lower bound β basic .

Original: Although it istypically not as tight as the Lagrangian based lower bound β LD , itscomputational cost is significantly lower.
Revised: Although it is not as tight as the Lagrangian based lower bound β LD , its computational cost is significantly lower.

Original: Thus it is useful when the BB tree is deep, and the medoidregions are small.
Revised: Thus it is useful when the BB tree is deep with small medoid regions.

Original: Forthe K-MedoidsProblem 1, there is an implicit constraint that all the samplesshare the same medoid set:
Revised: For K-Medoids 1, there is an implicit constraint that all the samples share the same medoid set:

Original: [Equation]
Revised: [Equation]

Original: (10) in which µ ks is the k th medoid for the s th sample.
Revised: (10) in which µ ks is the k th medoid for the s th sample.

Original: By removing the “medoids on samples” constraintµ ∈ X and Constraint 10 in the K-Medoids Problem 1, we have the basic lower bound β basic ( M )with β basic ( M ) ≤ z ( M ) :
Revised: By removing the “medoids on samples” constraint µ ∈ X and Constraint 10, we have the basic lower bound β basic ( M ) with β basic ( M ) ≤ z ( M ) :

Original: [Equation]
Revised: [Equation]

Original: We define d ks,min ( M ) :
Revised: We define d ks,min ( M ) :

Original: = min µ k ∈ M k
Revised: = min µ k ∈ M k

Original: || x s − µ k || 22 and obviously β basic ( M )
Revised: || x s − µ k || 22 and obviously β basic ( M )

Original: = (cid:80) s
Revised: = (cid:80) s

Original: ∈ S min
Revised: ∈ S min

Original: k ∈K d
Revised: k ∈K d

Original: ks,min ( M ) .
Revised: ks,min ( M ) .

Original: Theanalytic solution to d ks,min ( M ) can be easily derived: µ ks,a = mid { µ ka , x s,a , ¯ µ ka } , a ∈ A , in whichµ ka and ¯ µ ka are the bounds of the a th attribute in k th medoid region.
Revised: The analytic solution to d ks,min ( M ) can be easily derived: µ ks,a = mid { µ ka , x s,a , ¯ µ ka } , a ∈ A , in which µ ka and ¯ µ ka are the bounds of the a th attribute in k th medoid region.

Original: Lagrangian based lower bound: as shown in Algorithm 5, by combining the basic and LD lowerbound methods, we give the final form of lower bounds: β ( M )
Revised: Lagrangian based lower bound: as shown in Algorithm 5, by combining the basic and LD lower bound methods, we give the final form of lower bounds: β ( M )

Original: = max { β basic ( M ) , β LD ( M ) } .
Revised: = max { β basic ( M ) , β LD ( M ) } .

Original: Wenote that because β basic ( M ) ≤ z ( M ) and β LD ( M ) ≤ z ( M ) , we have β ( M ) ≤ z ( M ) .
Revised: We note that because β basic ( M ) ≤ z ( M ) and β LD ( M ) ≤ z ( M ) , we have β ( M ) ≤ z ( M ) .

Original: Besides,according to the guarantee of maximum gap between the Lagrangian based lower bound and theglobal optimal value, we can also derive a guarantee of maximum gaps for β ( M ) :
Revised: Besides, according to the guarantee of maximum gap between the Lagrangian based lower bound and the global optimal value, we can also derive a guarantee of maximum gaps for β ( M ) :

Original: Proposition 2.
Revised: Proposition 2.

Original:  z ( M ) − β ( M ) z r − β
Revised: [Equation] where z [Equation]

Original: ( M ) < 1 e , where z r = (cid:80) s ∈S max j ∈S d s,j .

Original: 4.2 Upper Bounds
Revised: 4.2 Upper Bounds

Original: Upper bounds of the K-Medoids problem can be obtained by selecting a feasible set of medoids˜ µ ∈ M ∩ X .
Revised: Upper bounds of the K-Medoids problem can be obtained by selecting a feasible set of medoids ˜ µ ∈ M ∩ X .

Original: Given a feasible solution, we can obtain the assignments of samples and object valuethrough Equation 1, without solving any optimization problem explicitly.
Revised: Given a feasible solution, we can obtain the assignments of samples and object value through Equation 1, without solving any optimization problem explicitly.

Original: However, the selection of ˜ µwill critically influence the efficiency of convergence.
Revised: However, the selection of ˜ µ will critically influence the efficiency of convergence.

Original: Hence, in our implementation, we introducetwo efficient upper bounding methods: 
Revised: Hence, in our implementation, we introduce two efficient upper bounding methods: Root node: we use

Original: Root node: we use the K-Means-like method [4] as a backbone.
Revised: the K-Means- like method [4] as a backbone.

Original: Because the K-Means-like methodis sensitive to the initial seeds and apt to the local minimum [5], we run this method several timeswith randomseeds to get a candidate solution set.
Revised: Because the K-Means-like method is sensitive to the initial seeds and apt to the local minimum [5], we run this method several times with random

Original:  Then, we improve the quality of the upper boundby feeding thecandidate solution set as initial guesses to Evolutionary Centers Algorithm [24].
Revised: seeds to get a candidate solution set. Then, we improve the quality of the upper bound by feeding the

Revised: candidate

Original:  Child nodes: we utilize the candidate solution obtained in the lower bounding method to updatethe upper bounds.
Revised: solution set as initial guesses to Evolutionary Centers Algorithm [24]. Child nodes: we utilize the candidate solution obtained in the lower bounding method to update the upper bounds.

Original: Specifically, we first compute the corresponding assignments, then feed theseassignments as the initial solution to the K-Means-like method.
Revised: Specifically, we first compute the corresponding assignments, then feed these assignments as the initial solution to the K-Means-like method.

Original: It should be noted that this K-Meanslike method [4] gives a solution full-filling the "medoids on samples" constraint in K-Medoids.
Revised: It should be noted that this K-Meanslike method [4] gives a solution full-filling the "medoids on samples" constraint in K-Medoids.

Original:  Bound Tightening Techniques
Revised:  Bound Tightening Techniques

Original: Here, we introduce some bound tightening techniques tailored for the K-Medoids problem to reducethe search regions M at each BB node while guaranteeing the optimal solution is not excluded.
Revised: Here, we introduce some bound tightening techniques tailored for the K-Medoids problem to reduce the search regions M at each BB node while guaranteeing the optimal solution is not excluded.

Original: Thesebound tightening techniques are deployed before lower and upper bounds calculation.
Revised: These bound tightening techniques are deployed before lower and upper bounds calculation.

Original: 5.1 Probing
Revised: 5.1 Probing

Original: Probing is a technique to reduce bound regions by exploiting the inequalities in a mixed integerprogramming problem [25].
Revised: Probing is a technique to reduce bound regions by exploiting the inequalities in a mixed integer programming problem [25].

Original: This paper exploits the implicit inequality that the lower bound of a regioncontaining the optimal solution is smaller than the current best upper bound.
Revised: This paper exploits the implicit inequality that the lower bound of a region containing the optimal solution is smaller than the current best upper bound.

Original: As shown in Algorithm2, it tentatively restricts the bounds of µ to a subinterval and then computes the correspondinglower bound.
Revised: As shown in Algorithm 2, it tentatively restricts the bounds of µ to a subinterval and then computes the corresponding lower bound.

Original: If the lower bound is larger than the best upper bound, we can conclude that nooptimal solution exists within the subinterval, and tighten the bound on the variable accordingly.
Revised: If the lower bound is larger than the best upper bound, we can conclude that no optimal solution exists within the subinterval, and tighten the bound on the variable accordingly.

Original: Thecomputation costs are trivial since we use the basic lower bounding method.
Revised: The computation costs are trivial since we use the basic lower bounding method.

Original: 5.2 Feasibility Based Bound Tightening
Revised: 5.2 Feasibility Based Bound Tightening

Original: [Figure]

Original: This subsection introduces the Feasibility Based Bound Tightening (FBBT) technique.
Revised: This subsection introduces the Feasibility Based Bound Tightening (FBBT) technique.

Original: It is based onthe observation that the assignment of some samples to clusters can be predetermined.
Revised: It is based on the observation that the assignment of some samples to clusters can be predetermined.

Original: [Figure]
Revised: [Figure]

Original: Cluster assignment: if we have || x s − µ k
Revised: Cluster assignment: if we have || x s − µ k

Original: + ||
Revised: + ||

Original: 22 <
Revised: 22 <

Original: || x s − µ k − || 22 ,
Revised: || x s − µ k − || 22 ,

Original: ∀ k − ∈ K \ k + , then obviouslysample x s is in the k + th cluster.
Revised: ∀ k − ∈ K \ k + , then obviously sample x s is in the k + th cluster.

Original: However, the value of µ here is not known before solving theoverall problem.
Revised: However, the value of µ here is not known before solving the overall problem.

Original: Recall that we have d ks,min ( M ):= min µ k ∈ M k
Revised: Recall that we have d ks,min ( M ):= min µ k ∈ M k

Original: || x s − µ k || 22 .
Revised: || x s − µ k || 22 .

Original: Similarly, we also defined k s,max ( M ):= max µ k ∈ M k
Revised: Similarly, we also define d ks,max ( M ):= max µ k ∈ M k

Original: || x s − µ k || 2 2 as the maximum distance between x s and µ k with µ k ∈ M k .
Revised: || x s − µ k || 22 as the maximum distance between x s and µ k with µ k ∈ M k .

Original: Ata BB node with partition M , if we have d k + s,max ( M ) <
Revised: At a BB node with partition M , if we have d k + s,max ( M ) <

Original: d k − s,min ( M ) , ∀ k − ∈ K \ k + , then sample x sis guaranteed to be assigned to the k + th cluster.
Revised: d k − s,min ( M ) , ∀ k − ∈ K \ k + , then sample x s is guaranteed to be assigned to the k + th cluster.

Original: Based on this observation, we can check whetherthe cluster of some samples can be determined in a BB node as shown in Algorithm 3.
Revised: Based on this observation, we can check whether the cluster of some samples can be determined in a BB node as shown in Algorithm 3.

Original: Recall thatd ks,min ( M ) is already computed as a byproduct of the basic lower bound in Section 3.
Revised: Recall that d ks,min ( M ) is already computed as a byproduct of the basic lower bound in Section 3.

Original: Similarly, since the cluster regions for each node are rectangles, d ks,max ( M ) can also be analytically computed.
Revised: Similarly, since the cluster regions for each node are rectangles, d ks,max ( M ) can also be analytically computed.

Original: Besides, since child nodes are subsets of the parent node, once a sample is assigned to a cluster ina parent node, it is also assigned to the same cluster in its child nodes.
Revised: Besides, since child nodes are subsets of the parent node, once a sample is assigned to a cluster in a parent node, it is also assigned to the same cluster in its child nodes.

Original: Figure 1 shows a simpleillustration of this cluster assignment scheme.
Revised: Figure 1 shows a simple illustration of this cluster assignment scheme.

Original: FBBT:
Revised: FBBT:

Original: In a node with M , we denote the index sets of samples assigned to cluster k as S kA .
Revised: In a node with M , we denote the index sets of samples assigned to cluster k as S kA .

Original: FBBTinfers the regions of µ k from the following inequality:
Revised: FBBT infers the regions of µ k from the following inequation:

Original: [Equation]
Revised: [Equation]

Original: where α l is the current best upper bound, β ( M ) − represents the lower bound contributions of samplesunassigned to cluster k (either samples assigned to other clusters or samples whose assignments areundetermined).
Revised: where α l is the current best upper bound, β ( M ) − represents the lower bound contributions of samples unassigned to cluster k (either samples assigned to other clusters or samples whose assignments are undetermined).

Original: Similar to the calculation of the basic lower bounding method, we have β ( M ) − =(cid:80)
Revised: Similar to the calculation of the basic lower bounding method, we have β ( M ) − = (cid:80)

Original: [Equation]
Revised: [Equation]

Original: As shown in Algorithm 4, FBBT is applied for each cluster.
Revised: As shown in Algorithm 4, FBBT is applied for each cluster.

Original: Specifically, for the k th medoid region,1) obtain S kA according to the results of Algorithm 3; 2) compute the right-hand-side, RHS =
Revised: Specifically, for the k th medoid region, 1) obtain S kA according to the results of Algorithm 3; 2) compute the right-hand-side, RHS =

Original: α l − β ( M ) − ; 3) for each attribute a ∈ A , obtain the tightened region, ˆ M ka =
Revised: α l − β ( M ) − ; 3) for each attribute a ∈ A , obtain the tightened region, ˆ M ka =

Original: { µ ka | ˆ µ ka ≤ µ ka ≤ ˆ¯ µ ka } ,by solving the following inequality:
Revised: { µ ka | ˆ µ ka ≤ µ ka ≤ ˆ¯ µ ka } , by solving the following inequation:

Original: [Equation]
Revised: [Equation]

Original: Since 13 is a simple quadratic inequality, we can analytically solve the problem to obtain the lowerand upper bound of µ ka as ˆ µ ka and ˆ¯ µ ka .
Revised: Since 13 is a simple quadratic inequation, we can analytically solve the problem to obtain the lower and upper bound of µ ka as ˆ µ ka and ˆ¯ µ ka .

Original: Finally, update ˆ M ka by ˆ M ka ← ˆ M ka ∩ M ka .
Revised: Finally, update ˆ M ka by ˆ M ka ← ˆ M ka ∩ M ka .

Original: 5.3 Miscellanies
Revised: 5.3 Miscellaneous

Original: Symmetric breaking: we enforce a symmetric-breaking constraint on the first attribute, i.e., µ k 1 ≤µ k + , ∀ k
Revised: Symmetric-breaking: we enforce a symmetric-breaking constraint on the first attribute, i.e., µ k 1 ≤ µ k + , ∀ k

Original: = 1 , ..., K − 1 .
Revised: = 1 , ..., K − 1 .

Original: It is used to tighten the regions of M at each BB node.
Revised: It is used to tighten the regions of M at each BB node.

Original: Bound tightening according to the “medoids on samples” constraint: since the K-Medoidsproblem requires medoids to be selected from samples in the dataset, we can further tighten theregions of M by ˆ M = { µ | ˆ µ ≤ µ ≤ ˆ¯ µ } , ˆ µ ka = min s ∈ M ∩ X x s,a , and ˆ¯ µ ka = max s ∈ M ∩ X x s,a .5.4 Effects on Computation
Revised: Bound tightening according to the “medoids on samples” constraint: since the K-Medoids problem requires medoids to be selected from samples in the dataset, we can further tighten the regions of M by ˆ M = { µ | ˆ µ ≤ µ ≤ ˆ¯ µ } , ˆ µ ka = min s ∈ M ∩ X x s,a , and ˆ¯ µ ka = max s ∈ M ∩ X x s,a . 5.4 Effects on Computation

Original: With a reduced search space, these bound tightening techniques can significantly reduce the number of
Revised: With a reduced search space, these bound tightening techniques can significantly reduce the number of

Original: BB nodes to be explored.
Revised: BB nodes to be explored.

Original: Moreover, they can also accelerate the calculation of bounds.
Revised: Moreover, they can also accelerate the calculation of bounds.

Original: For example,for the Lagrangian Problem 8, we need to calculate the contributions, ρ j , of samples belonging to theoverall feasible set of medoids, ˆ S + = ˆ S 1+ ∪ ˆ S 2+ ∪ ... ∪ ˆ S K + .
Revised: For example, for the Lagrangian Problem 8, we need to calculate the contributions, ρ j , of samples belonging to the overall feasible set of medoids, ˆ S + = ˆ S 1+ ∪ ˆ S 2+ ∪ ... ∪ ˆ S K + .

Original: A reduced search space ˆ M means areduced overall feasible selection set, which further leads to fewer calculations and faster speeds.
Revised: A reduced search space ˆ M means a reduced overall feasible selection set, which further leads to fewer calculations and faster speeds.

Original:  Experiments and Discussions
Revised:  Experiments and Discussions

Original: To evaluate the performance, we use 25 datasets from the UCI Machine Learning Repository [26], onedataset called PR2392 from [27], one dataset called HEMI and one synthetic dataset from [19].
Revised: To evaluate the performance, we use 25 datasets from the UCI Machine Learning Repository [26], one dataset called PR2392 from [27], one dataset called HEMI[28] and one synthetic dataset from [19].

Original: Thedatasets from UCI Repository are in the clustering category with multivariate numerical attributes andrange from 100 to 2,458,285 instances.
Revised: The datasets from UCI Repository are in the clustering category with multivariate numerical attributes and range from 100 to 2,458,285 instances.

Original: We implement our branch and bound K-Medoids algorithmwith Lagrangian based lower bound method (BB+LD) in Julia 1.6.1.
Revised: We implement our branch and bound KMedoids algorithm with Lagrangian based lower bound method (BB+LD) in Julia 1.6.1.

Original: We also implement four othermethods to compare the performance, including the extensive form of K-Medoids problem solvedby the state-of-art global optimizer CPLEX 20.1.0 [14] (CPLEX), the heuristic method in the rootnode (Heuristic), the pure Lagrangian relaxation method without branch and bound ( LD , and the state -of-art BB method for 
Revised: We also implement four other methods to compare the performance, including the extensive form of KMedoids problem solved by the state-of-art global optimizer CPLEX 20.1.0 [14] (CPLEX), the heuristic method in the root node (Heuristic), the pure Lagrangian relaxation method without branch and bound (LD), and the state-of-art BB method for K-Means in [19] adapting to K-Medoids by adding "medoids on samples" constraint (BB+Basic).

Original: K-

Original: Means in adapting to K-Medoids by adding "medoids on samples"constraint (BB

Original: +Basic). We executed all the experiments on a high -performance computing cluster, of which each node contains 40 Intel cores at 2.
Revised: We executed all the experiments on a highperformance computing cluster, of which each node contains 40 Intel cores at 2.4 GHz and

Original: GHz and 202 GB RAM.
Revised: GB RAM.

Original: We present the results ofcluster number K = 3 in the main body with additional results K = 5 and K = 10 in Appendix 
Revised: We present the results of cluster number K = 3 in the main body with additional results K = 5 and K = 10 in Appendix C.

Original: C. The complete code  can be found in the Supplementary Material global_kmedoids.zip . After thereviewing process we will make our implementation open-source
Revised: The complete code files can be found in https: //github.com/ YankaiGroup/global_kmedoids_clustering

Original: The numerical performances are compared using three indicators, including upper bound value (UB),relative optimal gap, and the number of solved nodes.
Revised: The numerical performances are compared using three indicators, including upper bound value (UB), relative optimal gap, and the number of solved nodes.

Original: Here, UB is the best upper bound in all theiterations.
Revised: Here, UB is the best upper bound in all the iterations.

Original: The relative optimal gap is the relative difference between the best lower and upper bounds,which is calculated by Gap (%) : = α
Revised: The relative optimal gap is the relative difference between the best lower and upper bounds, which is calculated by Gap (%) :=

Original: [Equation]
Revised: [Equation]

Original: min {

Original: [Equation] [Equation] } × 100% . This relative gap represents the worst gapof the found solution from the global optimal solution.
Revised: This relative gap represents the worst gap of the found solution from the global optimal solution.

Original: Notably, this gap is a particular property ofdeterministic global optimization algorithms.
Revised: Notably, this gap is a particular property of deterministic global optimization algorithms.

Original: Heuristic methods can not quantify the worst gap fromthe global optimal solution.
Revised: Heuristic methods can not quantify the worst gap from the global optimal solution.

Original: The number of solved nodes represents the total number of BB nodes.
Revised: The number of solved nodes represents the total number of BB nodes.

Original: All the deterministic methods CPLEX, LD, BB+Basics, and BB+LD, share the same terminal criteria,including (1) Gap ≤ 0 .
Revised: All the deterministic methods CPLEX, LD, BB+Basics, and BB+LD, share the same terminal criteria, including (1) Gap ≤ 0 .

Original: 1% , (2) the solving time reaches 4 hours, (3) the number of solved nodesreaches 5 million.
Revised: 1% , (2) the solving time reaches 4 hours, (3) the number of solved nodes reaches 5 million.

Original: Both LD and BB+LD use the same sub-gradient method called the Volume method[21].
Revised: Both LD and BB+LD use the same sub-gradient method called the Volume method [21].

Original: Particularly, for BB+LD, the maximum iteration of Lagrangian multiplier updates in each nodeis 10.
Revised: Particularly, for BB+LD, the maximum iteration of Lagrangian multiplier updates in each node is 10.

Original: For LD, the maximum iteration is 10,000.
Revised: For LD, the maximum iteration is 10,000.

Original: In the heuristic method, we execute the K-Means-likemethod 20 times with a fixed seed set including 20 different seeds, then feed the solution set as initialguesses to ECA to get the best solutions.
Revised: In the heuristic method, we execute the K-Means-like method 20 times with a fixed seed set including 20 different seeds, then feed the solution set as initial guesses to ECA to get the best solutions.

Original: [Table]
Revised: [Table]

Original: 6.1 Serial Results
Revised: 6.1 Serial Results

Original: Table 1 shows the serial numerical results for the datasets with less than 100,000 samples.
Revised: Table 1 shows the serial numerical results for the datasets with less than 100,000 samples.

Original: In terms ofbest found upper bound (UB), the deterministic global optimization methods (CPLEX, BB+Basic,and BB+LD) can obtain better UB than the heuristic method in several datasets (e.g., IRIS, BM,
Revised: In terms of best found upper bound (UB), the deterministic global optimization methods (CPLEX, BB+Basic, and BB+LD) can obtain better UB than the heuristic method in several datasets (e.g., IRIS, BM,

Original: RDS).
Revised: RDS).

Original: While we try to obtain the best heuristic results by multiple initializations, there is still noguarantee for the heuristic method to reach the global optimum.
Revised: While we try to obtain the best heuristic results by multiple initializations, there is still no guarantee for the heuristic method to reach the global optimum.

Original: Moreover, the heuristic method cannot provide an optimal gap to estimate the solution quality.
Revised: Moreover, the heuristic method can not provide an optimal gap to estimate the solution quality.

Original: Notably, our method and CPLEX providethe same upper bounds and gaps for the datasets smaller than PR2392 (2392 samples).
Revised: Notably, our method and CPLEX provide the same upper bounds and gaps for the datasets smaller than PR2392 (2392 samples).

Original: As for the gaps and running times, CPLEX and LD can obtain a small gap ( ≤ 0.1%) with less timewhen datasets are smaller than HCV (572 samples).
Revised: As for the gaps and running times, CPLEX and LD can obtain a small gap ( ≤ 0.1%) with less time when datasets are smaller than HCV (572 samples).

Original: However, when datasets are larger than HCV,our method, BB+LD, can reach the small gap( ≤ 0.1%) with much less time than CPLEX and LD.
Revised: However, when datasets are larger than HCV, our method, BB+LD, can reach the small gap with much less time than CPLEX and LD.

Original: Remarkably, our method can reach the small gap in all the datasets up to 100,000 samples within4 hours.
Revised: Remarkably, our method can reach the small gap in all the datasets up to 100,000 samples within 4 hours.

Original: CPLEX fails to give a solution because of the out-of-memory error in the root node whendatasets are larger than PR2392 (2392 samples).
Revised: CPLEX fails to give a solution because of the out-of-memory error in the root node when datasets are larger than PR2392 (2392 samples).

Original: LD and BB+Basic methods may not reach the smallgap within 4 hours or maximum iteration limits when datasets are larger than BM (249 samples).
Revised: LD and BB+Basic methods may not reach the small gap within 4 hours or maximum iteration limits when datasets are larger than BM (249 samples).

Original: Recall that LD has no guarantee of reaching the global optimal solution because of the duality gap.
Revised: Recall that LD has no guarantee of reaching the global optimal solution because of the duality gap.

Original: All these results show that our method is extremely scalable for K-Medoids problems even underserial mode.
Revised: All these results show that our method is extremely scalable for K-Medoids problems even under serial mode.

Original: Notably, we followed the tuning guide in the CPLEX documentation to improve its performance.
Revised: Notably, we followed the tuning guide in the CPLEX documentation to improve its performance.

Original: For example, we initiated the root node with the Heuristic solution and specified the node file andmemory parameters.
Revised: For example, we initiated the root node with the Heuristic solution and specified the node file and memory parameters.

Original: Moreover, we tried different MIP strategies, including start, pre-solve, andheuristic strategies.
Revised: Moreover, we tried different MIP strategies, including start, pre-solve, and heuristic strategies.

Original: In Table 1, we present the best results obtained by CPLEX.
Revised: In Table 1, we present the best results obtained by CPLEX.

Original: 6.2 Parallel Results
Revised: 6.2 Parallel Results

Original: For BB+LD method, we executed parallel experiments utilizing MPIparallel interfaces in the largeand huge scale datasets with K = 3 .
Revised: For BB+LD, we executed parallel experiments utilizing MPI interfaces in the large and huge scale datasets with K = 3 .

Original: The results of large-scale datasets ranging from 10,000 to100,000 samples are presented in Appendix B. Table 2 shows the parallel numerical results for hugedatasets with 100,000 to 2,458,285 samples.
Revised: The results of large-scale datasets ranging from 10,000 to 100,000 samples are presented in Appendix B. Table 2 shows the parallel numerical results for huge datasets with 100, to 2,458,285 samples.

Original: From this table, we can see that even for the dataset with1,046,910 samples, BB+LD can still converge to the global optimal solution ( ≤ 0.10 gaps) within onehour, which is 400 times larger than the size of problem that the state-of-art global solver CPLEXcan address.
Revised: From this table, we can see that even for the dataset with 1,046,910 samples, BB+LD can still converge to the global optimal solution ( ≤ 0.10 gaps) within one hour, which is 400 times larger than the size of problem that the state-of-art global solver CPLEX can address.

Original: For USO1990 with 2,458,285 samples and 68 features, BB+LD can obtain a 6.33% gapwithin 4 hours.
Revised: For USC1990 with 2,458,285 samples and 68 features, BB+LD can obtain a 6.33% gap withinhours.

Original: According to Theorem 1, we can obtain a global ϵ -optimal solution for USO1990given more calculation time
Revised: According to Theorem 1, we can obtain a global ϵ -optimal solution for USC1990 given more calculation time

Revised: [Table]

Revised: [Table]

Revised:  Complexity Analysis 7.1 Time Complexity

Revised: Denoting the number of samples as S , attributes as A , and clusters as K

Original: [Table]
Revised: [Table]

Revised: our algorithm branches on the medoid regions to guarantee convergence, which contains A × K variables and is independent of the number of samples S .

Revised: However, the number of BB nodes to converge is hard to predict since the efficiency of bounding methods depends on datasets.

Revised: Also, the number of Lagrangian Dual (LD) iterations in each BB node is different because we set dynamic stopping criteria in case there is no update of LB in several continuous LD iterations.

Revised: The time complexity of one LD iteration is O ( S 2 ) when d s,j are precomputed.

Revised: If d s,j are not precomputed, the time complexity is O ( AS 2 ) .

Revised: Table 3 compares computing time and nodes on synthetic datasets with various dimensions and samples.

Revised: In this table, the average run-time of one LD iteration remains almost the same as the dimension changes and increases almost quadratically as the number of samples increases.

Revised: As for the BB nodes, although the number of branching variables is A × K , the actual number of explored

Revised: BB nodes does not strictly follow the growth of dimensions.

Revised: These results are consistent with the complexity analysis mentioned above.

Original: [Table]
Revised: [Table]

Revised: 7.2 Space Complexity of Distance

Revised: When distances d s,j are precomputed and stored

Revised: [Table]

Revised: the space complexity of d s,j matrix is O ( S 2 ) .

Revised: In our implementation

Revised: [Table]

Original: Conclusion and Limitation
Revised: the datasets with no more than 100,000 samples were computed on one compute node with 40 cores and 202GB RAM. For more than 100,000 samples, we executed the experiments on multiple compute nodes. Each core precomputes and stores part of the d s,j matrix, as described in Section 3.

Revised: For example, we executed the one-million dataset RETAIL-II on 150 nodes with 6, cores.

Revised: Each core only needs to store 1.25GB of d s,j matrix.

Original: Conclusion: we presented a scalable global optimization algorithm of the K-Medoids problemby applying a tailored reduced-space spatial branch and bound scheme.
Revised: For more than 1,000,000 samples, we calculated d s,j on the fly, without precomputing and storing. In this case, the time complexity of one LD iteration increases from O ( S 2 ) to O ( AS 2 ) . Hence, we expect an acceptable slowdown when the dimension of datasets is small.

Original: This algorithm includes a
Revised: For example, in Table 2, the dataset USC1990 with two-million samples and 68 features was executed without precomputing d s,j , and the result is acceptable.

Original: Lagrangian relaxation based lower bounding method and a basic lower bounding method. Boundtightening techniques are also proposed to accelerate the solution process. We demonstrate ouralgorithm’s strong scalability by extensive numerical experiments and prove the convergence bytheoretical analysis.
Revised: Table 4 compares the computing time of precomputed distance and on-the-fly computing. We can conclude that although calculating d s,j on the fly is slower than precomputing d s,j , the slowdown is acceptable.

Original: Limitation: in the future, we are interested in improving the global optimal K-Medoids algorithm bythe design and implementation of more time-efficient and memory-friendly bounding methods. Weare also interested in investing in other machine learning methods from the global optimization view. Checklist For all authors...
Revised: [Table]

Original:  (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? [Yes] (b) Did you describe the limitations of your work?
Revised:  Conclusion

Original: [Yes] See Section

Original: (c) Did you discuss any potential negative societal impacts of your work? [N/A] Our work is about a  global solver of the K-Medoids problem.
Revised: We presented a scalable global optimization algorithm of the K-Medoids problem by applying a tailored reduced-space spatial branch and bound scheme.

Original: The problem itself may havesocietal impacts.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem? [Yes] If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: This algorithm includes a Lagrangian based lower bounding method and a basic lower bounding method.

Original: [Yes] (b) Did you include complete proofs of all theoretical results? [Yes] See Appendix A
Revised: Bound tightening techniques are also proposed to accelerate the solution process.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)? [Yes] See the 
Revised: We demonstrate our algorithm’s scalability by extensive numerical experiments and prove the convergence by theoretical analysis.

Original: supplemental material (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: Besides, one interesting finding is that our algorithm can reduce the objective values for some datasets while remaining the same as the heuristic method for most datasets.

Original: [Yes] See Section 6 and the supplemental material (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)? [N/A]
Revised: We also compared other clustering evaluation metrics in Appendix D, such as Normalized Mutual Information (NMI) and Adjusted Rand Index (ARI).

Original: Our method is a deterministic method. (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)? [Yes] See Section If you are using existing assets (e
Revised: A similar conclusion with objective values holds for these metrics.

Original: g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? [Yes] We compare our method with some existing methods and we cite the creators in the main body.
Revised: However, it should be noted that only with the global optimal solution obtained in our work can we give a fair comparison of the heuristic methods and, in turn, contribute to developing a more efficient heuristic method.

Original: (b) Did you mention the license of the assets?

Original: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]We include our project codes in the supplemental material.(d)

Original: Did you discuss whether and how consent was obtained from people whose data you’reusing/curating? [N/A] We use the datasets from UCI Machine Learning Repositoryand three open-source datasets.(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content? [N/A] The meaning of datasets are described in UCIMachine Learning Repository.5.
Revised: Acknowledgement

Original: If you used crowdsourcing or conducted research with human subjects...

Original: [No] Our workdoes not use crowdsourcing or conducted research with human subjects [Table] (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable? [N/A]
Revised: The authors gratefully acknowledge the computing resources provided by SciNet (www.scinethpc.ca) and Digital Research Alliance of Canada (www.alliancecan.ca).

Original: [Table]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review [Table] Board (IRB) approvals, if applicable? [N/A](c)
Revised: Jiayang Ren acknowledges the financial support from the China Scholarship Council (Grant No. 202106320032).

Original: Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Original: [N/A] A Convergence Analysis
Revised: Appendix A Convergence Analysis

Original: As stated in Algorithm 1, the branch and bound scheme is a rooted tree, where the  level 0 root nodeis ◦ M .
Revised: As stated in Algorithm 1, the branch and bound scheme is a rooted tree, where the search space of the level 0 root node is ◦ M .

Original: We denote M l q as a sub-node at iteration l q and level q .
Revised: We denote M l q as a sub-node at iteration l q and level q .

Original: Its child node is denoted as M l q +1satisfying M l q +1 ⊂ M l q .
Revised: Its child node is denoted as M l q +1 satisfying M l q +1 ⊂ M l q .

Original: A decreasing sequence from the root node ◦ M to the node M l q is denotedas { M l q } .
Revised: A decreasing sequence from the root node ◦ M to the node M l q is denoted as { M l q } .

Original: It is obvious that the sequences { α l } and { β l } are monotonically non-increasing andnon-decreasing correspondingly.
Revised: It is obvious that the sequences { α l } and { β l } are monotonically non-increasing and non-decreasing correspondingly.

Original: In the following convergence analysis, we adapt the fundamentalresults from [19] and [28] to our work.
Revised: In the following convergence analysis, we adapt the fundamental results from [29] to our work.

Original: Differently, the BB procedureis infinite in [19], while ouralgorithm has a finite BB procedure.
Revised: Different from the BB procedure in [19], which is infinite, our algorithm has a finite BB procedure.

Original: Hence, we prove the convergence in a finite manner.
Revised: Hence, we prove the convergence in a finite manner.

Original: Definition 3.
Revised: Definition 3.

Original: (Definition IV.10 [28] ).

Original: A subdivision is called exhaustive if lim q →∞ δ ( M l q ) = 0 ,for all decreasing sub-sequences { M l q } generated by the subdivision.

Original: In our algorithm, it is obvious that if a first-stage variable µ s corresponding to the diameter of M ispartitioned, this subdivision is exhaustive without the terminal constraint | M k ∩ X | > 1 , ∀ k ∈ K .

Original: Definition 4.

Original: (Definition IV.3 [28])
Revised: (Definition IV.3 [29])

Original: A bounding operation is called finitely consistent if, at everystep, any unfathomed partition element can be further refined, and if any decreasing sequence { M l q }successively refined partition elements is finite.
Revised: A bounding operation is called finitely consistent if, at every step, any unfathomed partition element can be further refined, and if any decreasing sequence { M l q } successively refined partition elements is finite.

Original: Lemma 5.
Revised: Lemma 4.

Original: The bounding operation in Algorithm 1 is finitely consistent.
Revised: The bounding operation in Algorithm 1 is finitely consistent.

Original: Proof.
Revised: Proof.

Original: We first prove any unfathomed partition element M l q can be further refined.
Revised: We first prove any unfathomed partition element M l q can be further refined.

Original: Any unfathomed
Revised: Any unfathomed

Original: M l q satisfies ∃| M l q ∩ X | > 1 , k ∈ K and α l > β ( M l q ) + ϵ, ϵ > 0 .
Revised: M l q satisfies ∃| M l q ∩ X | > 1 , k ∈ K and α l > β ( M l q ) + ϵ, ϵ > 0 .

Original: It is obvious that there exists atleast one partition to be further refined.
Revised: It is obvious that there exists at least one partition to be further refined.

Original: We then prove any decreasing sequence { M l q } successively refined partition elements is finite.
Revised: We then prove any decreasing sequence { M l q } successively refined partition elements is finite.

Original: Assuming by contradiction that there exists a sequence { M l q } that is infinite.
Revised: Assuming by contradiction that there exists a sequence { M l q } that is infinite.

Original: Since our subdivisionis exhaustive when the sequence { M l q } is infinite, we have lim q →∞ δ
Revised: In our algorithm, since we branch on the first-stage variable µ k corresponding to the diameter of M , this subdivision is exhaustive, we have lim q →∞ δ

Original: ( M l q )
Revised: ( M l q )

Original: = 

Revised: =

Original: and { M l q } converge toone point ¯ µ .
Revised: and { M l q } converge to one point ¯ µ .

Original: If ¯ µ ∈ X , there is a ball around ¯ µ , denoted as B r (¯ µ ) =
Revised: If ¯ µ ∈ X , there is a ball around ¯ µ , denoted as B r (¯ µ ) =

Original: { µ | || µ − ¯ µ || ≤ r } , satisfying| B r (¯ µ ) ∩ X | = 1 .
Revised: { µ | || µ − ¯ µ || ≤ r } , satisfying | B r (¯ µ ) ∩ X | = 1 .

Original: There exists a q 0 that M l q ⊂ B r (¯ µ ) , ∀ q ≤ q 0 .
Revised: There exists a q 0 that M l q ⊂ B r (¯ µ ) , ∀ q ≤ q 0 .

Original: At this q 0 iteration, M l q 0 will not bebranched anymore.
Revised: At this q 0 iteration, M l q 0 will not be branched anymore.

Original: Because X is finite, we have the sequence { M l q } is finite in this case.
Revised: Because X is finite, we have the sequence { M l q } is finite in this case.

Original: If ¯ µ ̸⊂ X ,there is a ball around ¯ µ , denoted as B r (¯ µ ) =
Revised: If ¯ µ ̸⊂ X , there is a ball around ¯ µ , denoted as B r (¯ µ ) =

Original: { µ | || µ − ¯ µ || ≤ r } , satisfying | B r (¯ µ ) ∩ X | = 0 .
Revised: { µ | || µ − ¯ µ || ≤ r } , satisfying | B r (¯ µ ) ∩ X | = 0 .

Original: Thereexists a q 0 that M l q ⊂ B r (¯ µ ) , ∀ q ≤ q 0 .
Revised: There exists a q 0 that M l q ⊂ B r (¯ µ ) , ∀ q ≤ q 0 .

Original: At this q 0 iteration, M l q 0 will be deleted.
Revised: At this q 0 iteration, M l q 0 will be deleted.

Original: Consequently, inthis case, the sequence { M l q } is finite.
Revised: Consequently, in this case, the sequence { M l q } is finite.

Original: Hence, it is impossible to exists a sequence { M l q } that isinfinite.
Revised: Hence, it is impossible to exists a sequence { M l q } that is infinite.

Original: Lemma 6.

Original: Any deleted partition element M l q does not contain the global optimal solution µ ∗ ∈◦ M ∩ X .

Original: Proof.

Original: For any deleted partition element M l q , it satisfies β ( M l q ) > α l .

Original: Assuming by contradictionthat this node M l q contains the global optimal solution µ ∗ ∈ ◦ M ∩ X , we have β ( M l q ) ≤ z .

Original: It isinconsistent with the in-equation relations of β ( M l q ) >

Original: α l ≥ z .

Original: Consequently, any deleted partitionelement M l q does not contain the global optimal solution µ ∗ ∈ ◦ M ∩ X .

Original: Theorem 7.
Revised: Theorem 5.

Original: (Theorem IV.1 [28])
Revised: (Theorem IV.1 [29])

Original: In a BB procedure, suppose that the bounding operation is finitelyconsistent.
Revised: In a BB procedure, suppose that the bounding operation is finitely consistent.

Original: Then the procedure terminates after finitely many steps.
Revised: Then the procedure terminates after finitely many steps.

Original: Lemma 8.
Revised: Lemma 6.

Original: Algorithm 1 terminates after finitely many steps
Revised: Algorithm 1 terminates after finitely many steps

Original: Proof.
Revised: Proof.

Original: From Lemma 5, we have the bounding operation in Algorithm 1 is finitely consistent.
Revised: From Lemma 4, we have the bounding operation in Algorithm 1 is finitely consistent.

Original: From
Revised: From

Original: Theorem 7, Algorithm 1 terminates after finitely many steps
Revised: Theorem 5, Algorithm 1 terminates after finitely many steps

Original: Finally, we prove that the BB scheme is convergent, as shown in Theorem 1:
Revised: Finally, we prove that the BB scheme is convergent, as shown in Theorem 1:

Original: Theorem 1.
Revised: Theorem 1.

Original: Algorithm 1 is convergent to the global optimal solution after a finite step L , withβ L =
Revised: Algorithm 1 is convergent to the global optimal solution after a finite step L , with β L =

Original: z = α L , by only branching on the space of µ .
Revised: z = α L , by only branching on the space of µ .

Original: Proof.
Revised: Proof.

Original: From Lemma 8, we have Algorithm 1 terminates after finite steps.
Revised: From Lemma 6, we have Algorithm 1 terminates after finite steps.

Original: The algorithm terminateswith two situations.
Revised: The algorithm terminates with two situations.

Original: The first situations is | β l − α l | ≤ ϵ .
Revised: The first situations is | β l − α l | ≤ ϵ .

Original:  The second situations is| M kl q ∩ X | =1 , ∀ k ∈ K .
Revised: When ϵ = 0 , we have β l = z = α l . The second situations is M = ∅ .

Original: For the first situation, from Lemma 6, we have the undeleted nodeset M contains at least one nodewith the global optimal solution µ ∗ .
Revised: A node M is deleted from M and not further partitioned either because β ( M ) > α l or | M k ∩ X | = 1 , ∀ k ∈ K . The first case obviously does not contain the global optimal solution µ ∗ .

Original: Hence, we have β l ≤ z and z ≤ α l . Combining | β l − α l | ≤ ϵ ,we have β l ≤ z ≤ α l + ϵ .
Revised: Therefore, the node M ′ containing µ ∗ , is not further partitioned because | M ′ k

Original: When ϵ = 0 , we have β l = z = α l .For the second situation, from
Revised: ∩ X | = 1 , ∀ k ∈ K .

Original: Lemma 6, we have the global optimal solution µ ∗ is the only medoid set in M l q ∩ X .
Revised: After bound tightening according to the “medoids on samples” constraint, the tightened node M ′ = { µ ∗ } .

Original: It is obviousthat β l = β ( µ ∗ )=
Revised: Obviously for this node, we have β l = β ( M ′ )

Original:  z = α ( µ ∗ ) =
Revised: = z = α ( M ′ ) =

Original: α l .
Revised: α l .

Original: Consequently, we have proved Theorem 1.
Revised: Consequently, we have proved Theorem 1.

Original: B Parallel results of large scale datasets
Revised: B Parallel results of large scale datasets

Original: Table 3 shows the parallel numerical results for large scale datasets with 10,000 to 2,458,285 samples.
Revised: Table 5 shows the parallel numerical results for large scale datasets with 10,000 to 2,458,285 samples.

Original: For the large-scale datasets ranging from 10,000 to 100,000 samples, we obtain an average of 13.80xspeedup in time with 40x in cores compared to the serial results.
Revised: For the large-scale datasets ranging from 10,000 to 100,000 samples, we obtain an average of 13.80x speedup in time with 40 cores.

Original: Here, speedup ratio is defined as¯ T 1 ¯ T c , in which ¯ T c is the average calculation timeof one Lagrangian multiplier iteration with c cores.
Revised: Here, speedup ratio is defined as ¯ T 1 ¯ T c , in which ¯ T c is the overall calculation time with c cores.

Original: The average efficiency is low since the time ratio of the parallel part is relatively low in the smalldatasets.
Revised: The average efficiency is low since the time ratio of the parallel part is relatively low in the small datasets.

Original: Consequently, we can expect a higher speedup in the big datasets.
Revised: Consequently, we can expect a higher speedup in the big datasets.

Original: For example, we obtaina 25.89x Speedup for URBANGB_10 with 100,000 samples compared to 7.57x for HTRU2 with17,898 samples.
Revised: For example, we obtain a 25.89x Speedup for URBANGB_10 with 100,000 samples compared to 7.57x for HTRU2 with 17,898 samples.

Original: [Table]
Revised: [Table]

Original: C Additional results of K-Medoids problems with K = 5 and K =10
Revised: C Additional results of K-Medoids problems with K = 5 and K =

Original: In this section, we perform several additional experiments with more clusters on the datasets rangingfrom 100 to 100,000 samples.
Revised: In this section, we perform several additional experiments with more clusters on the datasets ranging from 100 to 100,000 samples.

Original: All these experiments use the same setup in Section 6.
Revised: All these experiments use the same setup in Section 6.

Original: Table 4 and 5are the results of the BB+LD method with cluster number K = 5 and K = 10 respectively.
Revised: Table 6 andare the results of the BB+LD method with cluster number K = 5 and K = 10 respectively.

Original: Whenthe cluster number is bigger, our BB+LD are more likely to obtain an upper bound smaller thanthe Heuristic UB.
Revised: When the cluster number is bigger, our BB+LD are more likely to obtain an upper bound smaller than the Heuristic UB.

Original: However, the difference in UB between BB+LD and Heuristic becomes smaller.
Revised: However, the difference in UB between BB+LD and Heuristic becomes smaller.

Original: Moreover, the search space of medoids increases as the cluster number, making it harder to obtain theglobal optimum.
Revised: Moreover, the search space of medoids increases as the cluster number, making it harder to obtain the global optimum.

Original: Nevertheless, our BB+LD method can still obtain a relative gap smaller than 0.1%within 4 hours for most datasets when K = 5 and a reasonable relative gap within 4 hours when 
Revised: Nevertheless, our BB+LD method can still obtain a relative gap smaller than 0.1% within 4 hours for most datasets when K = 5 and a reasonable relative gap within 4 hours when K = [Table]

Original: K = 10 .
Revised: [Table] [Table] D Comparison of heuristic methods for K-Medoids problems To illustrate the effectiveness of our upper bound method (Heuristic UB)

Original: [Table]
Revised: [Table]

Revised: we compares its performance with several popular heuristic methods in the literature, including Kmeans, Kmeans++, and PAM.

Revised: Here, we run all the methods several times with random seeds and select the best result.

Revised: The centers of K-Means and K-Means++ are projected to the nearest samples to fulfill the "Medoids on Samples" constraint in the KMedoids problem.

Revised: This table shows that the Heuristic UB can always obtain the same or better objective value than Kmeans, Kmeans++, and PAM.

Revised: [Table]

Revised: We also compared other clustering evaluation metrics, such as Normalized Mutual Information (NMI) and Adjusted Rand Index (ARI).

Revised: Since these clustering evaluation metrics need to compare the predicted cluster with ground truth, we can only compare the datasets with ground truth, such as

Revised: IRIS, HCV, HF, HRTU2 and UK.

Revised: Comparing Table 8 and Table 9, one interesting finding is that our algorithm can reduce the objective values for some datasets while remaining the same as the heuristic method for most datasets.

Revised: A similar conclusion with objective values also holds for ARI and NMI.

Revised: However, it should be noted that only by comparing the heuristic solution with the global optimal solution as we did in the table

Original: [Table]
Revised: [Table]

Revised: we can confidently claim that the heuristic method can do a fairly nice job in finding near-optimal solutions

Revised: [Table]

Revised: [Table]