Original: Efficient and Effective Optimal Transport-Based Biclustering
Revised: Efficient and Effective Optimal Transport-Based Biclustering

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddressemail
Revised: AffiliationAddress

Original: Abstract
Revised: Abstract

Original: Bipartite graphs can be used to model a wide variety of dyadic information such asuser-rating, document-term, and gene-disorder pairs.
Revised: Bipartite graphs can be used to model a wide variety of dyadic information such asuser-rating, document-term, and gene-disorder pairs.

Original: Biclustering is an extensionof clustering to theunderlying bipartite graph induced from this kind of data.
Revised: Biclustering is an extensionof clustering to the bipartite graphs that underlie this kind of data.

Original: Inthis paper, we leverage optimal transport (OT) which has gained  momentum inthe machine learning community to propose a novel and scalable biclusteringmodel that generalizes several classical biclustering approaches.
Revised: In this paperwe leverage optimal transport (OT), which has gained a certain momentum inthe machine learning community, in proposing a novel and scalable biclusteringmodel that generalizes several classical biclustering approaches.

Original: We performextensive experimentation to show the validity of our approach compared to other
Revised: We performextensive experimentation to show the validity of our approach in relation to other

Original: OT biclustering algorithms along both dimensions of the dyadic datasets
Revised: OT biclustering algorithms along both dimensions of the dyadic datasets

Original:  Introduction
Revised:  Introduction

Original: Let G =
Revised: Let G =

Original: ( U, V, E ) be a bipartite graph  which is a graph whose vertices can be divided into twodisjoint sets U = { 1 , 2 , . .
Revised: ( U, V, E ) be a bipartite graph , which is a graph whose vertices can be divided into twodisjoint sets U = { 1 , 2 , . .

Original: , | U |} with | U | = n , V = { 1 , 2 , . .
Revised: , | U |} with | U | = n , V = { 1 , 2 , . .

Original: , | V |} with | V | =
Revised: , | V |} with | V | =

Original: d and the set ofedges E where each edge connects a vertex of U to a vertex of V .
Revised: d and the set ofedges E where each edge connects a vertex of U to a vertex of V .

Original: The adjacency matrix for this typeof graph has the following structure
Revised: The adjacency matrix for this typeof graph has the following structure

Original: [Equation]
Revised: [Equation]

Original: where B of size n × d is called the biadjacency matrix of G , its rows and columns correspond tothe two sets of vertices; each entry represents an edge between a row and a column.
Revised: where B of size n × d is called the biadjacency matrix of G , its rows and columns corresponding tothe two sets of vertices; each entry represents an edge between a row and a column.

Original: Biclustering (or
Revised: Biclustering (or

Original: Co-clustering ) is the extention of clustering to this type of graphs.
Revised: Co-clustering ) is the extention of clustering to this type of graph.

Original: Following [15], several biclusteringmodels  attempted to solve the problem by viewing B as a two-mode matrix and searching for asimultaneous partition of its rows and columns [6].
Revised: Following [18], several biclusteringmodels have attempted to solve the problem by viewing B as a two-mode matrix and searching for asimultaneous partition of its rows and columns [8].

Original: In this way, biclustering aims to reveal subsets of
Revised: In this way, biclustering seeks to reveal subsets of

Original: U which exhibit a similar behaviour across a subset of V in matrix B .
Revised: U which exhibit a similar behaviour across a subset of V in matrix B .

Original: Biclustering has been used in several contexts, [9] used microarray data to find relations betweengenes and conditions, finding that genes with similar functions often cluster together.
Revised: Biclustering has been used in a number of different contexts. [12] used microarray data to findrelations between genes and conditions, finding that genes with similar functions often clustertogether.

Original: [14] used thisparadigm to identify drug groups with adverse effects on the US Food and Drug Administrationreporting system.
Revised: [17] applied this paradigm to data from the US Food and Drug Administration reportingsystem in order to identify groups of drugs with adverse effects.

Original: [8] used it to find market segments among tourists that are similar to each other toallow for targeted marketing, as well as many other applications [6, 30, 13].
Revised: [11] used it to find market segmentsamong tourists so as to enable more effective targeted marketing. There have been various otherapplications [8, 33, 16].

Original: Multiple solutions for the biclustering problem have been proposed in literature, [7] used aninformation-theoretic approach to solve the problem by minimizing the difference in mutual information between B and a summary matrix.
Revised: Several solutions to the biclustering problem have been proposed in the literature. [9] used aninformation-theoretic approach to solve the problem by minimizing the difference in mutual information between B and a summary matrix.

Original: [2] adapted classical modularity to bipartite networksand then used it to identify modules within them.
Revised: [2] adapted classical modularity to bipartite networksand then used it to identify modules within them.

Original: [31] proposed a biclustering paradigm based onnonnegativematrix tri-factorization of the biadjacency matrix.
Revised: [35] proposed a biclustering paradigm based onnonnegative tri-factorization of the biadjacency matrix.

Original: Recently, Optimal Transport (OT) took the machine learning community by storm and was usedin the resolution ofvarious data mining problems and biclustering was not an exception.
Revised: Recently, Optimal Transport (OT) has taken the machine learning community by storm. OT hashelped to solve a variety of data mining problems, and biclustering is no exception.

Original: First, [19] proposed two models for biclustering, a first one called CCOT which does co-clustering basedon the scaling vectors obtained from the application of the Sinkhorn-knopp algorithm on a squaresubsampled version of matrix B  and another one called CCOT-GW that uses scaling vectors obtainedfrom computing entropic Gromov-Wasserstein barycenters and which does not require subsampling.
Revised: [23] proposed two models for biclustering: a first model, CCOT, which does co-clustering based on the scalingvectors obtained by applying the Sinkhorn-Knopp algorithm on a square subsampled version ofmatrix B , and a second model, CCOT-GW, which uses scaling vectors obtained by computingentropic Gromov-Wasserstein barycenters, and which does not require subsampling.

Original: Then came [26] where  authors did biclustering by minimizing a new metric COOT, which generalizesthe Gromov-Wasserstein distance, between B and a summary matrix similar to what was done in[7].
Revised: Then came[34], where the authors did biclustering by minimizing a new metric, COOT, which generalizes the Gromov-Wasserstein distance between B and a summary matrix, similarly to what was done in [9].

Original: In particular, they proposed two metrics COOT and an entropically regularized one COOT λ .
Revised: More specifically, they proposed two new metrics: COOT, together with an entropically regularizedmetric COOT λ .

Original: These approaches, however, have some drawbacks.
Revised: However, both [23] and [34] have certain drawbacks.

Revised: First, both algorithms do nottackle the biclustering from the beginning; the co-clusters are deduced at the convergence.

Original: One of them is that both of them suffer fromhigh computational complexity and in the case of CCOT and CCOT-GW also from a large memoryconsumption.
Revised: Therebybiclustering is a consequence and not a main goal. Secondly, they suffer from high computationalcomplexity; CCOT and CCOT-GW also consume large amounts of memory.

Original: Another one is that their empirical performance on dyadic data is not satisfactory.
Revised: Finally, we will see thatthese algorithms are not suited to dyadic sparse data.

Original: In this paper,  we propose a generic framework for biclustering through optimal transport whichgeneralizes several previous biclustering approaches.
Revised: In this paper, while integrating the biclustering objective from the beginning, we propose a genericframework for biclustering through optimal transport, which generalizes some previous biclusteringapproaches.

Original: We propose two efficient methods for solvingthis problem, one that results in an almost hard biclustering and another that results in a fuzzy orsoft biclustering through entropic regularization.
Revised: We propose two efficient methods for solving this problem: one that gives an almosthard biclustering, and another that gives a fuzzy or soft biclustering through entropic regularization.

Original: These methods outperform other optimal transportbiclustering models both in term of document and term clustering on several regular and large scaledatasets while being more computationally and memory efficient.
Revised: These methods outperform other optimal transport biclustering models, in terms of both documentand term clustering, on several regular and large scale datasets, while being more computationallyand memory efficient.

Original: We emphasize once again the factthat the approach we propose is specifically tailored to datasets consisting in dyadic data which canbe represented using a bipartite graph, meaning that it should not be applied on other data types suchas images directly.
Revised: We emphasize once again that the approach we propose is specifically tailoredto datasets consisting of dyadic data that can be represented using a bipartite graph.

Original:  Methodology
Revised:  Methodology

Original: Notations In what follows, ∆ n = { p ∈ R n + | (cid:80)
Revised: Notations In what follows, ∆ n = { p ∈ R n + | (cid:80)

Original: ni =1 p i = 1 } denotes the n -dimensional standardsimplex.
Revised: ni =1 p i = 1 } denotes the n -dimensional standardsimplex.

Original: Π( w , v ) = { Z ∈ R n × k + | Z1 = w , Z ⊤ 1 = v } denotes the transportation polytope, wherew ∈
Revised: Π( w , v ) = { Z ∈ R n × k + | Z1 = w , Z ⊤ 1 = v } denotes the transportation polytope, wherew ∈

Original: ∆ n and v ∈ ∆ k are the marginals of the joint distribution Z and 1 n is a vector of ones.
Revised: ∆ n and v ∈ ∆ k are the marginals of the joint distribution Z and 1 n is a vector of ones.

Original: Wedenote matrices with uppercase boldface letters and vectors with lower case boldface letters.
Revised: Matricesare denoted with uppercase boldface letters, and vectors with lowercase boldface letters.

Original: For amatrix M , its i -th row is m and its j -th column is m ′
Revised: For a matrix M , its i -th row is m and its j -th column is m ′

Original: j We have that ∥ .
Revised: j We have that ∥ .

Original: ∥ 0 is the 0-norm which returnsthe number of nonzero elements of its argument.
Revised: ∥ 0 is the 0-norm which returns thenumber of nonzero elements of its argument.

Original: 2.1 Preliminaries
Revised: 2.1 Preliminaries

Original: We first need to introduce exact discrete OT and its entropically regularized counterpart and showhow biclustering can be posed as an integer program.
Revised: We first need to introduce exact discrete OT and its entropically regularized counterpart, and showhow biclustering can be posed as an integer program.

Original: Discrete OT as a linear program.
Revised: Discrete OT as a linear program.

Original: The goal of discrete optimal transport is to find a minimal costtransport map between a source probability distribution w and a target distribution v .
Revised: The goal of discrete optimal transport is to find a minimal costtransport map between a source probability distribution w and a target distribution v .

Original: Here we areinterested in the discrete case of the Kantorovich formulation of OT, that is
Revised: Here we areinterested in the discrete case of the Kantorovich formulation of OT, that is

Original: [Equation]
Revised: [Equation]

Original: where M ∈ R n × k is the cost matrix,  m ij quantifies the effort needed to transport a probability massfrom w i to v j .
Revised: where M ∈ R n × k is the cost matrix, and m ij quantifies the effort needed to transport a probabilitymass from w i to v j .

Original: Discrete entropy regularized OT.
Revised: Discrete entropy regularized OT.

Original: Several works [4, 3] have suggested  that the use of a regularization such asthe entropic regularization can lead to better computational and statistical efficiency.
Revised: It has been suggested in the literature [5, 4] that the use ofa regularization such as entropic regularization can lead to better computational and statisticalefficiency.

Original: [Equation]
Revised: [Equation]

Original:  where H is the entropy defined as H ( Z ) ≜ (cid:80) i,j − z ij log z ij and λ controls the strength of regularization.
Revised: (3) where H is the entropy defined as H ( Z ) ≜ (cid:80) i,j − z ij log z ij and λ controls the strength of regularization.

Original: The computational efficiency comes from the fact that the unique solution of this problemhas thefollowing structure K := − diag ( a ) exp(  M /λ ) diag ( b ) , a rescaled elementwise negativeexponential of the cost M  where a and b are scaling vectors.
Revised: The computational efficiency comes from the fact that the unique solution of this problem isof the structure K := − diag ( a ) exp( − M /λ ) diag ( b ) , a rescaled elementwise negative exponentialof the cost M , where a and b are scaling vectors.

Original: These vectors can be found efficientlyusing the Sinkhorn-Knopp algorithm.
Revised: These vectors can be found efficiently using the Sinkhorn-Knopp algorithm.

Original: Biclustering as an integer program.
Revised: Biclustering as an integer program.

Original: The Block seriation problem [21] consists in finding twopermutations matrices, one for the rows and one for the columns s.t.
Revised: The Block seriation problem [25] consists in finding twopermutation matrices, one for the rows and one for the columns s.t.

Original: dense blocks appear along the diagonal of the permuted matrix.
Revised: dense blocks appear along thediagonal of the permuted matrix.

Original: A possible definition of the block seriation problem is that givena matrix B ∈ R n × d s.t b ij gives the strength of the association between row i and column j e.g. abiadjacency matrix, it is formulated as follows
Revised: A possible definition of the block seriation problem is as follows:given a matrix B ∈ R n × d s.t b ij gives the strength of the association between row i and column j(such as in the case of a biadjacency matrix, for example), we have

Original: [Equation]
Revised: [Equation]

Original: A solution C matrix is a block diagonal matrix up to a permutation of its rows and columns.
Revised: A solution C matrix is a block diagonal matrix up to a permutation of its rows and columns.

Original: Theblock seriation problem is an integer programming problem and is consequently NP-hard.
Revised: Theblock seriation problem is an integer programming problem and is consequently NP-hard.

Original: Oneapproach for solving this problem uses a relaxed version where a rank constraint rank ( C ) ≤ k isadded for k the number of desired biclusters.
Revised: Oneapproach for solving this problem uses a relaxed version where a rank constraint rank ( C ) ≤ k isadded for k the number of desired biclusters.

Original: When integrating this constraint to 4, we can define anew equivalent problem by low-rank factorization of C  i.e. C [Equation] ⊤ , which we formulate as
Revised: Integrating this constraint into 4, we can define a newequivalent problem by low-rank factorization of C , i.e. C = ZW ⊤ , which we formulate as

Original: [Equation]
Revised: [Equation]

Original: where Γ( n, k ) = { Z ∈ { 0 , 1 } n × k | Z1 = 1 } is the set of hard partitions of dimension n × k .
Revised: where Γ( n, k ) = { Z ∈ { 0 , 1 } n × k | Z1 = 1 } is the set of hard partitions of dimension n × k .

Original: A simpleheuristic for solving this problem involves alternatingly solving for Z given W  and vice-versa usingclassical clustering algorithms before identifying biclusters through the rearranged matrix C  whichdisplays a block diagonal structure as seen in figure 1a.
Revised: A simpleheuristic for solving this problem involves alternatingly solving for Z given W , and vice-versa, usingclassical clustering algorithms, before identifying biclusters through the rearranged matrix C , whichdisplays a block diagonal structure, as shown in figure 1a.

Original: The biclusters are identified by groupingtogether the rows and columns that form a block along  diagonal.
Revised: The biclusters are identified by groupingtogether the rows and columns that form a block along the diagonal.

Original: 2.2 Biclustering using Optimal Transport
Revised: 2.2 Biclustering using Optimal Transport

Original: Here, we propose a new biclustering problem based on block seriation and optimal transport.
Revised: Here we propose a new biclustering problem based on block seriation and optimal transport.

Original: We firststart by introducing a necessary concept which we call anti-adjacency matrix 
Revised: For thispurpose we first define what we term an anti-adjacency matrix :

Original: Definition 1.
Revised: Definition 1.

Original: (Anti-adjacency matrix)
Revised: (Anti-adjacency matrix)

Original: Given a graph characterized by an adjacency matrix A , wedenote its anti-adjacency matrix A s.t.
Revised: Given a graph characterized by an adjacency matrix A , wehave a corresponding anti-adjacency matrix A s.t.

Original: a ij quantifies a discrepancy between node i and j .
Revised: a ij quantifies a discrepancy between node i and j .

Original: We consider a bipartite graph characterized by its biadjacency matrix B = ( b ij ) ∈ R n × d .
Revised: We consider a bipartite graph characterized by its biadjacency matrix B = ( b ij ) ∈ R n × d .

Original: The rowsof B are endowed with weights w ∈ ∆ n and its columns with weights v ∈ ∆ d .
Revised: The rowsof B are endowed with weights w ∈ ∆ n and its columns with weights v ∈ ∆ d .

Original: We also considera row exemplar distribution r ∈ ∆ r and a column exemplar distribution c ∈ ∆ c .
Revised: We also considera row exemplar distribution r ∈ ∆ r and a column exemplar distribution c ∈ ∆ c .

Original: Depending uponthe availability of a priori information about the data, these weight vectors can be set to uniformdistributions.
Revised: Depending onthe availability of a priori information about the data, these weight vectors can be set to uniformdistributions.

Original: Now let its anti-adjacency matrix be B
Revised: Now let its anti-adjacency matrix be B

Original: = L ( B )  where L : R n × d → R n × d leads to transform b ij ,the association between node i and node j ,  into a discrepancy measure L ( b ) ij .
Revised: = L ( B ) , where L : R n × d → R n × d means that b ij , theassociation between node i and node j , is transformed into a discrepancy measure L ( b ) ij .

Original: Z .
Revised: Z .

Original: Thus, we definethe optimal transport block seriation problem as
Revised: Thus,we define the optimal transport block seriation problem as

Original: [Equation]
Revised: [Equation]

Original: where Z is a transport map (or coupling) between between the row distribution w and the rowexemplar distribution r  and similarly for W w.r.t.
Revised: where Z is a transport map (or coupling) between between the row distribution w and the rowexemplar distribution r , and similarly for W w.r.t.

Original: the column distribution v and the column exemplardistribution c .
Revised: the column distribution v and the columnexemplar distribution c .

Original: Similar to [26] is an indefinite Bilinear Program and is related to the quadraticassignment problem (QAP)
Revised: Similar to [34] is an indefinite Bilinear Program and is related to thequadratic assignment problem (QAP)

Original: Inducing a Biclustering using BCOT We  noware going to show how to obtain a partition of therows and the columns given a solution pair ( Z , W ) .
Revised: Inducing a Biclustering using BCOT We will now show how to obtain a partition of the rowsand the columns given a solution pair ( Z , W ) .

Original: In what follows, we are interested in inducing acouple of
Revised: In what follows our aim is to identify an almost-hardclustering couple for rows and columns from the couplings Z and W .

Original: [Figure]
Revised: [Figure]

Original: Definition 2.
Revised: Definition 2.

Original: ( h -almost hard clustering)
Revised: ( h -almost hard clustering)

Original: We define an h -almost hard clustering as a clusteringwhose assignment matrix is C ∈ R n × k s.t.
Revised: We define an h -almost hard clustering as a clusteringwhose assignment matrix is C ∈ R n × k s.t.

Original: ∥
Revised: ∥

Original: C ∥ 

Revised: C ∥

Original: = n + h and for each row c of C we have that∥ c ∥ 0 > 0 .
Revised: = n + h and for each row c of C we have that∥ c ∥ 0 > 0 .

Original: When h = 0 , we obtain a standard hard clustering with one non-zero element per row.
Revised: When h = 0 , we obtain a standard hard clustering with one non-zero element per row.

Original: Proposition 1. 

Revised: Proposition 1.

Original: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .
Revised: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: Furthermore, when n = k(resp.
Revised: Furthermore, when n = k(resp.

Original: d = k ) and w = r (resp.
Revised: d = k ) and w = r (resp.

Original: v = c ), Z (resp.
Revised: v = c ), Z (resp.

Original: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.
Revised: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.

Original: W ∈ Γ( d, d ) ).
Revised: W ∈ Γ( d, d ) ).

Original: This means that the solutions are already almost a hard partition of the data since k << n, d .
Revised: This means that the solutions are already almost a hard partition of the data, since k << n, d .

Original: Toobtain a final hard clustering in the strict sense, we assign each row (resp. column) to the onecorresponding to thelargest value of each row of Z (resp. 
Revised: Toobtain a final hard clustering in the strict sense, we assign each row (resp. column) to the onecorresponding to the row of Z (resp. W ) with the largest value.

Original: W ), this should not significantly changethe structure of the solution.
Revised: This should not significantly changethe structure of the solution.

Original: To illustrate this, we look at figure 1b, it shows the block diagonalstructure generated by the product of the two coupling matrices C = ZW ⊤ . We see how it is lookssimilar to the biclustering produced by the hard block seriation 1a except for a few nonzero entriesoff the block diagonal that are hard to see immediately.
Revised: Figure 1b provides an illustration: here we see the block diagonalstructure generated by the product of the two coupling matrices C = ZW ⊤ , with its similarity inappearance to the biclustering produced by the hard block seriation 1a, apart from a few nonzeroentries off the block diagonal that are hard to see immediately.

Original: Intuition for BCOT To explain the intuition behind the proposed approach we have to look atthe way the problem is solved.
Revised: Intuition for BCOT To explain the intuition behind the proposed approach we need to look at howthe problem is solved.

Original: The optimization procedure as described in algorithm 1 consists inalternating between the computation of an optimal transportation map Z given W and vice versa.
Revised: The optimization procedure as described in algorithm 1 consists in alternatingbetween the computation of an optimal transportation map Z given W and vice versa.

Original: Ifwe look at the solving for Z given W , the problem can be rewritten as
Revised: As regardssolving for Z given W , the problem can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: This is an optimal transport problem with L ( B ) W as the cost matrix.
Revised: This is an optimal transport problem with L ( B ) W as the cost matrix.

Original: The resulting transportationmap Z can be seen as a sort ofa row cluster assignment matrix, if z ih > 0  then row i is assigned tocluster h .
Revised: The resulting transportationmap Z can be seen as a kind of row cluster assignment matrix: if z ih > 0 , then row i is assigned tocluster h .

Original: The samething holds for W  which can be seen as a column cluster assignment matrix.
Revised: The same holds for W , which can be seen as a column cluster assignment matrix.

Original: This also means that since L ( B ) is the dissimilarity between the rows and the columns, then thecost matrix L ( B ) W represents the dissimilarity between rows and row exemplars (or representativesor centroids).
Revised: Thisalso means that since L ( B ) is the dissimilarity between the rows and the columns, then the costmatrix L ( B ) W represents the dissimilarity between rows and row exemplars (or representativesor centroids).

Original: In particular, L ( B )
Revised: In particular, L ( B )

Original: i w h is the dissimilarity or cost of probability mass transportationbetween row i and row cluster exemplar h .
Revised: i w h is the dissimilarity or cost of probability mass transportationbetween row i and row cluster exemplar h .

Original: The reasoning is the same for the columns and  optimalcoupling W .
Revised: The reasoning is the same for the columns and the optimalcoupling W .

Original: Low-Rank Optimal Transport Biclustering is the main purpose of the approach we proposed, butanother interesting use case comes up.
Revised: Low-Rank Optimal Transport Biclustering is the main purpose of the approach we proposed, butthere is another interesting use case.

Original: Proposition 2.
Revised: Proposition 2.

Original: Suppose that the target row and column representatives distributions is the samei.e. r = c with no zero entries, then given a solution pair Z and W to BCOT, the matrix Q =
Revised: In the case where the target row and column representative distributions are thesame, i.e., r = c with no zero entries, then given a solution pair Z and W to BCOT, the matrix Q =

Original: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem 2and whose rank isof at most min( rank ( Z ) , rank ( W )) .
Revised: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem2 and whose rank is at most min( rank ( Z ) , rank ( W )) .

Original: Some recent works [12, 28] suggested that this kind of low-rank regularization is preferable toentropic regularizationin some aspects such as for the fact that the rank parameter is easier to select since it has simple bounds (an integer between 1 and n ) contrary to regularization strength λ in the
Revised: Some recent works [15, 31] have suggested that this kind of low-rank regularization is preferableto entropic regularization as regards certain aspects. For example, the rank parameter is easier to select, since it has simple bounds (an integer between 1 and n ).

Original:  Sinkhorn algorithm which is continuous.
Revised: This may be contrasted with theregularization strength λ in the Sinkhorn algorithm, which is continuous.

Original: 2.3 Fuzzy biclustering using regularized Optimal Transport
Revised: 2.3 Fuzzy biclustering using regularized Optimal Transport

Original: As previously mentionned, using entropic regularization can be interesting due to the many propertiesit entails like statistical and computational efficiency.
Revised: As previously mentioned, using entropic regularization may be interesting because of its varioususeful features including statistical and computational efficiency.

Original: However, another property is that the optimalcouplings Z and W are dense matrices due to the structure of the optimal solution of entropicallyregularized OT problems.
Revised: However, another feature of entropicregularization is that the optimal couplings Z and W are dense matrices as a consequence of thestructure of the optimal solution of entropically regularized OT problems.

Original: We formulate the problem as follows
Revised: We formulate the problemas follows

Original: [Equation]
Revised: [Equation]

Original: where λ Z and λ W are the regularization parameters.
Revised: where λ Z and λ W are the regularization parameters.

Original: Fuzzy Block Seriation We propose a fuzzy variant of the block seriation problem which allows usby extension to define a fuzzy variant for BCOT using entropic regularization.
Revised: Fuzzy Block Seriation We propose a fuzzy variant of the block seriation problem that allows usby extension to define a fuzzy variant for BCOT using entropic regularization.

Original: Let the fuzzy blockseriation problem be defined as
Revised: Let the fuzzy blockseriation problem be defined as

Original: [Equation]
Revised: [Equation]

Original: where Ω( Z , W ) is some regularization term introduced to make the partition matrices Z and Wdense e.g. entropic regularization or low-rank constraints and Γ s ( n, k )
Revised: where Ω( Z , W ) is some regularization term introduced to make the partition matrices Z and W dense(for example, entropic regularization or low-rank constraints), and Γ s ( n, k )

Original: = { Z ∈ R n × k + | Z1 = 1 }is the set of fuzzy partitions.
Revised: = { Z ∈ R n × k + | Z1 = 1 }is the set of fuzzy partitions.

Original: Intuitively, for a solution pair ( Z , W ) , up to a constant factor, eachentry of the block seriation matrix C =
Revised: Intuitively, for a solution pair ( Z , W ) , up to a constant factor, eachentry in the block seriation matrix C =

Original: ZW ⊤ can be seen as the probability of its correspondingrow and column belonging to the same bicluster i.e. c ij =
Revised: ZW ⊤ can be seen as the probability of its correspondingrow and column belonging to the same bicluster i.e. c ij =

Original: z i w j = [Equation] rh
Revised: z i w j = (cid:80) rh

Original: [Equation]
Revised: [Equation]

Original: z ih [Equation] jh = p ( b
Revised: 161 (cid:80)

Original: [Equation]
Revised: [Equation]

Original: , b ′ j  ) 
Revised: j ∈ h ) .

Original: = 157 (cid:80) rh =1 p b i , b ′ j ∈ h ) . It is easy to see how problem 9 isa related to problem 8 and that the couplingscorresponding to solutions to the problem give the probability of membership to the same biclustersfor the different rows and columns.
Revised: It is easy to see how problem 9 is related to problem 8 and that the couplingscorresponding to solutions of the problem give the probability that the different rows and columnsbelong to the same biclusters.

Original: Figure 1c shows biclusters produced by the solutions of BCOT λ .
Revised: Figure 1c shows biclusters produced by the solutions of BCOT λ .

Original: Similarly to BCOT there is a block diagonal structurethat is formed.
Revised: Similarly to BCOT, a block diagonal structure is formed.

Original: However, there are also severaloff-block diagonal nonzero entries that represent the probabilities of the row-columns pairs belongingto the same biclusters.
Revised: However, there are also several off-blockdiagonal nonzero entries that represent the probabilities of the row-column pairs belonging to thesame biclusters.

Original:  Connections to Existing Work
Revised:  Links to Existing Work

Original: Modularity Maximization in bipartite graphs [2] This model allows to co-cluster binary and contingency matrices by directly maximizing an adapted version of the modularity measure traditionallyused for networks.
Revised: Modularity Maximization in bipartite graphs [2] This model is able to co-cluster binary and contingency matrices by directly maximizing an adapted version of the modularity measure traditionallyused for networks.

Original: The criterion  it optimizes is
Revised: The criterion that it optimizes is

Original: [Equation]
Revised: [Equation]

Original: by setting L ( B ) = − ( B − 1 b .. B11 ⊤ B ) , this problem becomes equivalent to ours with the differencelying in the constraints on Z and W .
Revised: By setting L ( B ) = − ( B − 1 b .. B11 ⊤ B ) , this problem becomes equivalent to ours; the difference isin the constraints on Z and W .

Original: Thereby BCOT  serves as a convex relaxation to this problem.
Revised: BCOT therefore serves as a convex relaxation of this problem.

Original: Modularity-based Sparse Soft Graph Clustering [17] Here the authors proposed to fuzzy variantof the previous problem (althoughnot in a bicustering context but
Revised: Modularity-based Sparse Soft Graph Clustering [20] Here the authors proposed a fuzzy variantof the above problem (although in the context of traditional clustering rather than biclustering).

Original: rather traditionanal clustering)whose solution gives for each element of the dataset a probability to belong to a given cluster.
Revised: Solving the problem gives, for each element of the dataset, a probability of that element belonging toa given cluster.

Original: Ourproposed entropic regularization variant constitutes a sort of extension  to bipartite graphs for thisproblem.
Revised: Our proposed entropic regularization variant represents a kind of extension of thisproblem to bipartite graphs.

Original: Directional co-clustering with a conscience [27] This model relies on the block von Mises-Fishermixture model for co-clustering directional data on the unit-sphere.
Revised: Directional co-clustering with a conscience [30] This model makes use of the block von MisesFisher mixture model for co-clustering directional data on the unit-sphere.

Original: It optimizes the followingcriterion
Revised: It optimizes the followingcriterion:

Original: [Equation]
Revised: [Equation]

Original: In our formulation, if we define L ( B ) = − B and apply cluster size normalization on the optimaltransport plans ˜ Z = Z diag ( Z ⊤ 1 ) − 1 / 2 and ˜ W = W diag ( W ⊤ 1 ) − 1 / 2 after the computation of Zand W respectively in algorithm 1  we obtain a more general version of the algorithm that the authorsproposed to solve problem 11.
Revised: In our formulation, if we define L ( B ) = − B and apply cluster size normalization on the optimaltransport plans ˜ Z = Z diag ( Z ⊤ 1 ) − 1 / 2 and ˜ W = W diag ( W ⊤ 1 ) − 1 / 2 after computing Z and Wrespectively in algorithm 1 , we obtain a more general version of the algorithm proposed by theauthors for solving problem 11.

Original: Bipartite Correlation Clustering [1]
Revised: Bipartite Correlation Clustering [1]

Original: In the case where the cost function results in a completebipartite graph with ’+’ and ’-’ edges with a function
Revised: In the case where the cost function results in a completebipartite graph with ’+’ and ’-’ edges with a function

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: we get what is known as Bipartite Correlation Clustering.
Revised: we get what is known as Bipartite Correlation Clustering.

Original: The solution to this problem maximizesthe number of agreements i.e. the number of all ’+’ edges within clusters plus all ’-’ edges distributedacross clusters.
Revised: The solution to this problem maximizes thenumber of agreements, i.e. the number of all ’+’ edges within clusters plus all ’-’ edges distributedacross clusters.

Original:  Optimization and Complexity
Revised:  Optimization and Complexity

Original: [Figure]
Revised: [Figure]

Original: Optimization The block seriation problem being NPhard means that computing an exact solution is prohibitive.
Revised: Optimization Since the block seriation problem is NP-hard, computing an exact solution is prohibitive.

Original: An efficient and widely used heuristic to solvethese kind of problems in using block coordinate descent wherewe alternatingly compute rowassignmentsfor fixed column assignments  and  vice versa.
Revised: An efficient and widely used heuristic for solving thesekinds of problems involves the use of block coordinatedescent, where row assignments are computed for fixedcolumn assignments, and then vice versa, in alternation.

Original: The proposed algorithmavailable in pseudo-code 1, in eachiteration we solve two intermediate optimal transportproblems with cost matrices of dimensions n × k andd × k , since B is generally sparse and we can define Lin way thatmake L ( B ) keep a similarly sparse structure, the computation of the intermediate cost matrices
Revised: We express the proposed algorithm in pseudo-code as Alorithm 1. At each iteration we solve two intermediateoptimal transport problems with cost matrices of dimensions n × k and d × k , since B is generally sparse, and Lcan be defined such that L ( B ) retains a similarly sparsestructure.

Original:  L ( B ) W and L ( B ) ⊤ Z is quite efficient.
Revised: The computation of the intermediate cost matrices L ( B ) W and L ( B ) ⊤ Z is reasonably efficient.

Original: Furthermore,we observed that the algorithm does not need many iterations to converge as seen in figure 2, be it for BCOTor BCOT λ .
Revised: Wealso observed that the algorithm does not need manyiterations to converge, as shown in figure 2, be it for BCOT or BCOT λ .

Original: Algorithm 1: BCOT
Revised: Algorithm 1: BCOT

Original: Input :

Original: Proposition 3.
Revised: Proposition 3.

Original: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥ 

Revised: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥

Original: + tnk ( n + k ) log( n + k )
Revised: + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k )))  and when using entropicregularization, the complexity is O ( tk ∥ B ∥ 

Revised: ( d + k ) log( d + k ))) , and when using entropicregularization the complexity is O ( tk ∥ B ∥

Original: + tkn + tkd )  where t is the number of iterations.
Revised: + tkn + tkd ) , where t is the number of iterations.

Original: In table 1, we report the computational and spatial complexities of the different biclustering approaches.
Revised: In table 1, we report the computational and spatial complexities of the different biclustering approaches.

Original: Our model has the same spatial complexity as the COOT variants andis a better one than CCOT variants.
Revised: Our model has the same spatial complexity as the COOT variants and a better complexitythan CCOT variants.

Original: For the computational complexity, our model shouldbe faster in most cases, ourexperiments support this observation.
Revised: As regards the computational complexity, our model should in most cases befaster, and our experiments support this conjecture.

Original: For reproducibility, we publicly release our code2 .
Revised: For reproducibility, we publicly release our code

Original: [Table]
Revised: [Table]

Original:  Experiments
Revised:  Experiments

Original: We conduct experimentation on term-document matrices.
Revised: We ran experiments using term-document matrices.

Original: The benefit of using biclustering on thiskind of data is that the resulting biclusters contain both documents and the words that characterizewhich will help us with interpreting  clustering of the documents.
Revised: The benefit of using biclustering on this kindof data is that the resulting biclusters contain both documents and the words that characterize them,which is helpful in interpreting the clustering of the documents.

Original: 5.1 Datasets
Revised: 5.1 Datasets

Original: We evaluate BCOT on six benchmark document-term datasets, ACM, DBLP, PubMed, Wiki, Ohscaland 20 Newsgroups.
Revised: We evaluate BCOT in relation to six benchmark document-term datasets: ACM, DBLP, PubMed, Wiki, Ohscal [Table] and 20 Newsgroups.

Original: Their characteristics are shown in Table 2.
Revised: Their characteristics are shown in Table 2.

Original: ACM, DBLP, Pubmed and Wiki areattributed networks from which we use only the node-level features that correspond to term-documentmatrices.
Revised: ACM, DBLP, Pubmedand Wiki are attributed networks from which we use only the node-level features that correspond toterm-document matrices.

Original: We also selected the Ohscal collection and 20 Newsgroups as large scale document-termmatrices asthe computational efficiency benchmarks.
Revised: We also selected the Ohscal collection and 20 Newsgroups as large-scaledocument-term matrices to serve as computational efficiency benchmarks.

Original: [Table]
Revised: [Table]

Original: 5.2 Experimental setup
Revised: 5.2 Experimental setup

Original: In our experiments, we define the loss function as L ( B ) = − c B  where c is selected from { 1 , k, d, n } .
Revised: In our experiments we define the loss function as L ( B ) = − c B , where c is selected from { 1 , k, d, n } .

Original: For BCOT λ , the regularization parameter lambda is selected from { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 , 10 } .
Revised: For BCOT λ , the regularization parameter lambda is selected from { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 , 10 } .

Original: The best hyper-parameters are the ones that minimize the number of empty clusters.
Revised: The best hyper-parameters are those that minimize the number of empty clusters.

Original: In the caseof ties, we select according to the value of the Davies-Bouldin index of the partition.
Revised: In the case of ties,we select according to the value of the Davies-Bouldin index of the partition [6].

Original: We do notuse random restarts  for any algorithm including k -means.
Revised: Random restarts arenot used for any of the algorithms, including k -means.

Original: We use the implementation provided bythe authors for CCOT, CCOT λ and CCOT-GW.
Revised: We use the implementation provided by theauthors for CCOT, CCOT λ and CCOT-GW.

Original: The code for CCOT was not available so we had toimplement it based on the one for CCOT-GW.
Revised: The code for CCOT was not available, and so we had toimplement it based on the code for CCOT-GW.

Original: All the reported figures are the averages of 10 runs.
Revised: All the reported figures are the averages of 10 runs.

Original: Allthe experiments were performed on the same machine with a RAM of 12GB and Intel(R) Xeon(R) 
Revised: All the experiments were performed on the same machine with an Intel(R) Xeon(R) CPU and 12GB

Original: CPU.
Revised: RAM.

Original: For OT solvers, we relied on the POT package [11].
Revised: For OT solvers we made use of the POT package [14].

Original: 5.3 Document clustering
Revised: 5.3 Document clustering

Revised: Metrics and algorithms

Original: Metrics Here, the evaluation is straightforward, We  adopt three popular clustering metrics: clustering accuracy (ACC), normalized mutual information (NMI), adjusted rand index (ARI). Performance Results for document clustering on ACM, DBLP, PubMed and Wiki are summarizedin table 3 in terms of ACC, NMI and ARI.
Revised: Here, the evaluation is straightforward. We use three popular clusteringmetrics: Here, the evaluation is straightforward, we adopt three popular clustering metrics: clusteringaccuracy (ACC), normalized mutual information (NMI) [3], adjusted rand index (ARI) [21].

Original: On all these datasets and for all the different metrics either BCOT or BCOT λ offer the best result. Furthermore, on the wiki dataset BCOT λ gives competitiveresults when compared with state of the art attributed graph clustering methods such as [33] withouteven having access to the graph structure information of the Wiki citation network.
Revised: Inaddition to the OT biclustering algorithms we perform experiments with k -means, informationtheoretic co-clustering (ITCC) [9] and orthogonal matrix non-negative tri-factorization (ONMTF)[10] suited for document-term matrices.

Original: [Table]
Revised: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: Performance Document clustering results on ACM, DBLP, PubMed and Wiki are given in table 3for the three metrics.

Original: [Figure] [Table] [Figure] [Table]
Revised: In all cases the best result is obtained either by BCOT or by BCOT λ .

Original: [Figure] [Table] [Figure] [Table]
Revised: Moreover,on Wiki, BCOT λ gives competitive results when compared with state-of-the-art attributed graphclustering methods such as [37], despite not having access to the graph structure information in the

Original: [Figure] [Table] [Figure] [Table]
Revised: Wiki citation network.

Original: [Figure] [Table] [Figure] [Table]
Revised: Efficiency

Original: [Figure] [Table] Efficiency In Figure 3, we plot the  performance of the different methods overtheir training timerelative to that of BCOT λ on the two large scale document-term matrices, 20 Newsgroup and Ohscal.
Revised: Figure 3 plots the document clustering performance (accuracy over training time) of thedifferent methods on the two large-scale document-term matrices 20 Newsgroup and Ohscal.

Original: BCOT offers the best accuracy while BCOT λ is fastest method on both datasets.
Revised: BCOToffers the best accuracy while BCOT λ is fastest method on both datasets.

Original: We see that forboth BCOT and COOT, the entropic-regularized versions outspeed their exact counterparts and that CCOT suffers from very high computation times mostly due  to the fact that a calculation of a pairwisedistance matrices on the rows and columns is necessary.
Revised: We see that for both BCOTand COOT, the entropic-regularized versions outspeed their exact counterparts and that CCOT suffersfrom very high computation times, due mainly to the fact that this method requires pairwise distancematrices to be computed on the rows and columns.

Original: [Figure]
Revised: [Figure]

Original: .g.

Original: × 5 for BCOT means that it is approximately five times slower than

Original: BCOT λ ).

Original: We were not able to benchmark CCOT-GW since it failed to scale to these datasets.

Original: 5.4 Term Clustering
Revised: 5.4 Term Clustering

Original: Metrics Unlike document clustering, there is no ground truth partition for terms which meanswe have to look for another way to evaluate term clustering results.
Revised: Metrics Unlike document clustering, there is no ground truth partition for terms, so we need to findanother way of evaluating term clustering results.

Original: A reasonable way to performevaluation is to analyse the semantic coherence of the clusters found by the different models.
Revised: One generally acceptable technique is to analyse thesemantic coherence of the clusters obtained.

Original: Withthis in mind, we introduce a metric based onthe point mutual information (PMI).
Revised: To this end we introduce a metric based on point mutualinformation (PMI).

Original: the PMI isa frequently used information theoretic metric for quantifying the relationship between pairs ofdiscrete random variable outcomes.
Revised: PMI is a frequently used information-theoretic metric for quantifying the relationship between pairs of discrete random variable outcomes.

Original: The PMI measure was chosen because prior research [23]has shown that it is closely associated with human judgements in determining word relatedness.
Revised: The PMI measure was chosen because priorresearch [27] has shown that it is closely associated with human judgements in determining word relatedness.

Original: [Equation]
Revised: [Equation]

Original: The PMImeasure between the terms w i and w j is calculated as in (13).
Revised: The PMI between the terms w i and w j is calculated as in (13).

Original: In the context of termclustering, given the word co-occurrence matrix K = B ⊤ B ,  PMI is estimated as in (14).
Revised: In the context of term clustering, giventhe word co-occurrence matrix K = B ⊤ B , the PMI is estimated as in (14).

Original: To evaluatea partition of terms P , we propose a metric based on intra and inter PMI metrics given by
Revised: To evaluate a partition ofterms P , we propose a metric based on intra and inter PMI metrics as follows: [Table]

Original: [Equation]
Revised: [Equation]

Original: Thereby, a good clustering should reveal a large intra-cluster semantic relatedness corresponding tolarger PMI values.
Revised: In this way [Table] a good clustering should reveal a high intra-cluster semantic relatedness, correspondingto higher PMI values.

Original: From intra and inter PMIs, we propose the following coherence indexcoherence
Revised: Using the intra and inter PMIs [Table] we propose the following coherence indexcoherence

Original: [Equation]
Revised: [Equation]

Original: [Table] This index relies on the fact that a partition maximizing this criterion has semantically close termsinside the same clusters and contrasting ones across the  different groups.
Revised: Our reasoning is this: the greater the semantic proximity between terms in the same clusters, and thegreater the sematic distance between terms in different clusters, the higher the value of coherence .

Original: Results Since there is no ground truth number of term clusters [Table] wemake useof the cluster numberestimations produced by CCOT-GW for all  other models so that it is easy to compare coherencevalues between them since comparing it for different number of clusters would favormodels thatused the  larger cluster number.
Revised: Results Since there is no ground truth number of term clusters, we use the cluster number estimations produced by CCOT-GW for all the other models so that it is easy to compare coherence valuesbetween them. Comparisons based on different numbers of clusters would favor the model usingthe larger number of clusters.

Original: Table 4 shows the coherence obtained by our approach along withthose of the baselines over the different datasets.
Revised: Table 4 shows the coherences obtained across the different datasetsusing our approach, along with those of the baselines.

Original: It is clear that BCOT succeeds in capturing moresemantics than the other approaches as, on all datasets, one  of  two variants of BCOT offer the bestcoherence value.
Revised: It is clear that BCOT succeeds in capturingmore semantics than the other approaches since, whatever the dataset, one or other of the two BCOTvariants gives the highest coherence.

Original: 5.5 Statistical significance
Revised: 5.5 Statistical significance

Original: We perform a nemenyi post hoc test [22, 5] with a confidence level of 90% on theresults we obtainedin terms of document and term clustering  to see if our model outperforms other OT biclusteringapproaches in a statistically significant manner.
Revised: We did a Nemenyi post-hoc test [26, 7] with a confidence level of 90% on the document andterm clustering results that we obtained, to determine whether our model outperforms other OTbiclustering approaches in a statistically significant way.

Original: To conduct this test we generate 20 performancerankings of the OT biclustering models based on their performance for each dataset and quality metricpair for both document and term clustering.
Revised: To conduct this test we generated 20performance rankings of the OT biclustering models based on their performance for each dataset andquality metric pair for both document and term clustering.

Original: Figure 4 shows the results of the test, we see thatit hasfound two differently performing groups, one comprising BCOT and BCOT λ , which gives betterresults than the other one which comprises the remaining COOT and CCOT variants, meaning the testcould not tellapart COOT and CCOT  in a statistically significant manner with this specific number ofdatasets and metrics.
Revised: Figure 4 shows the results of the test. Wesee that two differently performing groups were identified, one comprising BCOT and BCOT λ andgiving better results than the other group comprising the remaining COOT and CCOT variants. Thisindicates that with this specific number of datasets and metrics the test was unable to tell COOT and CCOT apart in a statistically significant way.

Original: [Figure]
Revised: [Figure]

Original:  Conclusion
Revised:  Conclusion

Original: Clustering and biclustering through optimal transport is till at a nascent stage with many challengesremaining unsolved.
Revised: Clustering and biclustering through optimal transport is still at a nascent stage, with many challengesremaining unsolved.

Original: Here, we have introduced a novel problem for biclustering using optimaltransport that takes into account the sparse nature of certain types  dyadic data such document-termmatrices to make for more computationally efficient resolution.
Revised: This paper introduces a novel problem for biclustering using optimal transport that takes into account the sparse nature of certain types of dyadic data such document-term matrices,to enable more computationally efficient resolution.

Original: The problem is posed as a bilinearprogram that we solve using an efficient coordinate descent algorithm that takes into account thesparse nature of certain types of dyadic data such as document-term matrices.
Revised: The problem is posed as a bilinear programthat we solve using an efficient coordinate descent algorithm that takes into account the sparsenature of certain types of dyadic data such as document-term matrices.

Original: Experiments on anumber of document-term datasets suggest that the proposed approach does a good job of findingclusters that correspond to ground truth document classes while generating semantically coherentpartitions for the terms.
Revised: Experiments on a numberof document-term datasets suggest that the proposed approach does a good job in finding clustersthat correspond to ground truth document classes, while generating semantically coherent partitionsfor the terms.

Original: In this setting, our model outperforms recent OT biclustering methods bysignificant margin, while being more computationally efficient.
Revised: In this setting, our model outperforms recent OT biclustering methods by a significantmargin, while being more computationally efficient.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] Last paragraph of the introduction(c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] Last paragraph of the introduction (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A](d)
Revised: [N/A]

Original:  Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: [Table]

Original: were chosen)?
Revised: were chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experi-
Revised: (c) Did you report error bars (e g., with respect to the random seed after running experi-

Original: [Table]

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you citep the creators?
Revised: (a) If your work uses existing assets, did you citep the creators?

Original: [Yes]
Revised: [Yes](

Original: [Table]

Original: (b) Did you mention the license of the assets?
Revised: b) Did you mention the license of the assets?

Original: [No](c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [No](c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]
Revised: [Yes](

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re [Table] using/curating?
Revised: d ) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?

Original: [N/A] All assets used are publicly available (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] All assets used are publicly available(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?

Original: [N/A]
Revised: [N/A]5.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A]
Revised: [N/A](b)

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A](c)

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Original: [N/A
Revised: [N/A]

Original: Appendix A Proofs
Revised: Appendix A Proofs

Original: Proposition 1.
Revised: Proposition 1.

Original: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .
Revised: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: Furthermore, when n = k(resp.
Revised: Furthermore, when n = k(resp.

Original: d = k ) and w = r (resp.
Revised: d = k ) and w = r (resp.

Original: v = c ), Z (resp.
Revised: v = c ), Z (resp.

Original: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.
Revised: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.

Original: W ∈ Γ( d, d ) ).
Revised: W ∈ Γ( d, d ) ).

Original: Proof for proposition 1.
Revised: Proof of proposition 1.

Original: Problem 2 is a bounded linear program since Π( w , v ) is a polytope i.e. abounded polyhedron.
Revised: Problem 2 is a bounded linear program, since Π( w , v ) is a polytope, i.e. abounded polyhedron.

Original: The fundamental theorem of linear programming states that if the feasible setis non-empty then the solution lies in extremity the feasible region.
Revised: The fundamental theorem of linear programming states that if the feasible setis non-empty then the solution lies at the limits of the feasible region.

Original: This means that a solution Z toproblem 2 is an extreme point of Π( w , v ) .
Revised: This means that a solution Zto problem 2 is an outermost point of Π( w , v ) .

Original: We have that the extreme points of Π( w , v ) can have atmost n + d − 1 nonzero elements.
Revised: We have that the outermost points of Π( w , v ) canhave at most n + d − 1 nonzero elements.

Original: To prove this we have to show that  the bipartite graph induced bybiadjacency matrix Z , the solution to the optimal transport problem has no cycles.
Revised: To prove this we have to show that in the bipartite graphinduced by the biadjacency matrix Z , the solution to the optimal transport problem has no cycles.

Original: The maximumnumber of edges in an acyclic graph is | V | − 1 where | V | is the number of nodes in the graph.
Revised: The maximum number of edges in an acyclic graph is | V | − 1 where | V | is the number of nodesin the graph.

Original: Sincethe number of edges in the bipartite graph induced by biadjacency matrix Z is n + d − 1 , the matrix Z can not have more than n + d − 1 nonzero entries.
Revised: Since the number of edges in the bipartite graph induced by biadjacency matrix Z isn + d − 1 , the matrix Z cannot have more than n + d − 1 nonzero entries.

Original: For a detailed proof see proposition 3.3 in [25].
Revised: For a detailed proof seeproposition 3.3 in [29].

Original: We also have to show that for probability measures w and v that have no zero probability events,there is at minimum max( n, d )number of nonzero elements in Z .
Revised: We also have to show that for probability measures w and v that have no zero probability events,there are at least max( n, d ) nonzero elements in Z .

Original: This is straightforward  since wand v contain no zeros, there will always be at least one nonzero element in every row and column of Z that represents some transfer of mass between elements of w and v .
Revised: This is straightforward to show: since w and vcontain no zeros, there will always be at least one nonzero element in every row and column of Zthat represents some transfer of mass between elements of w and v .

Original: In the case of BCOT, we then have that Z has at most n + k − 1 and at least max( n, k ) =
Revised: In the case of BCOT, we then have that Z has at most n + k − 1 and at least max( n, k ) =

Original: n nonzeroentries and that W has at most d + k − 1 and at least max( d, k ) =
Revised: n nonzeroentries and that W has at most d + k − 1 and at least max( d, k ) =

Original: d elements which are bothh -almost hard clusterings with h ∈ { 0 , . .
Revised: d elements which are bothh -almost hard clusterings with h ∈ { 0 , . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: When n = k and w = r , the solution Z is a permutation matrix (up to a constant factor) andthe number of nonzero elements in it is exactly n  which means that it represents a hard partition
Revised: When n = k and w = r , the solution Z is a permutation matrix (up to a constant factor) andthe number of nonzero elements in it is exactly n , which means that it represents a hard partition

Original: Z ∈ Γ( n, n ) .
Revised: Z ∈ Γ( n, n ) .

Original: The prooffor W is the same.
Revised: The proof is the same for W .

Original: Proposition 2.
Revised: Proposition 2.

Original: Suppose that the target row and column representatives distributions is the samei.e.
Revised: Suppose that the target row and column representative distributions are the same,i.e. r = c with no zero entries.

Original: r = c with no zero entries, then given a solution pair Z and W to BCOT, the matrix Q =
Revised: Then, given a solution pair Z and W to BCOT, the matrix Q =

Original: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem 2and whose rank isof at most min( rank ( Z ) , rank ( W )) .
Revised: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem2 and whose rank is at most min( rank ( Z ) , rank ( W )) .

Original: Proof for proposition 2.
Revised: Proof of proposition 2.

Original: From linear algebra, we have that rank ( Q ) ≤min( rank ( Z ) , rank ( diag (1 / r )) , rank ( W )) .
Revised: From linear algebra, we have that rank ( Q ) ≤min( rank ( Z ) , rank ( diag (1 / r )) , rank ( W )) .

Original: Since Z and W cannot have a rank greaterthan k due to their dimension and that diag (1 / r ) is a full rank matrix due to the assumption that rhas no zero entries, we then have that rank ( Q ) ≤ min( rank ( Z ) , rank ( W )) .
Revised: Since Z and W cannot have a rank greaterthan k due to their dimension, and since diag (1 / r ) is a full rank matrix due to the assumption that rhas no zero entries, we then have that rank ( Q ) ≤ min( rank ( Z ) , rank ( W )) .

Original: For a proof that Q is indeed a valid transport map i.e. Q ∈ Π( w , v ) , we redirect the reader toproposition 2.2 in [25].
Revised: For a proof that Q is indeed a valid transport map i.e. Q ∈ Π( w , v ) , we refer the reader to proposition2.2 in [29].

Original: Proposition 3.
Revised: Proposition 3.

Original: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥ 

Revised: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥

Original: + tnk ( n + k ) log( n + k )
Revised: + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k )))  and when using entropicregularization, the complexity is O ( tk ∥ B ∥ 

Revised: ( d + k ) log( d + k ))) , and when using entropicregularization the complexity is O ( tk ∥ B ∥

Original: + tkn + tkd )  where t is the number of iterations.
Revised: + tkn + tkd ) , where t is the number of iterations.

Original: Proof for proposition 3.
Revised: Proof of proposition 3.

Original: We suppose that L ( B ) is a sparse matrix with the same number ofnonzero entries as B .
Revised: We suppose that L ( B ) is a sparse matrix with the same number of nonzeroentries as B .

Original: The complexity of computing L ( B ) W and L ( B ) W in the BCOT algorithm is
Revised: The complexity of computing L ( B ) W and L ( B ) W in the BCOT algorithm is

Original: O ( k ∥ B ∥ 0 ) .
Revised: O ( k ∥ B ∥ 0 ) .

Original: The optimal transport problem can be formulated and solved as the Earth Mover’s Distance (EMD)problem using anyalgorithm for minimum-cost flow problem  such as one of the many variantsof network simplex algorithm.
Revised: The optimal transport problem can be formulated and solved as the Earth Mover’s Distance (EMD)problem using any minimum-cost flow problem algorithm, such as one of the many variants of thenetwork simplex algorithm.

Original: Authors in [24] proposed an algorithm for the network simplexin O
Revised: The authors in [28] proposed an algorithm for the network simplexin O

Original: ( | V || E | log | V | )  where | V | is the number of nodes and | E | is the number of edges in thenetwork.
Revised: ( | V || E | log | V | ) , where | V | is the number of nodes and | E | is the number of edges in thenetwork.

Original: In our case when solving the EMD for Z and cost matrix L ( B ) W , the number of nodes is| V | = n + k and the number of edges is | E | =
Revised: In our case, when solving the EMD for Z and cost matrix L ( B ) W , the number of nodes is| V | = n + k and the number of edges is | E | =

Original: nk  which means that the complexity of the operationis
Revised: nk , which means that the complexity of the operationis

Original: O ( nk ( n + k ) log( n + k )) .
Revised: O ( nk ( n + k ) log( n + k )) .

Original: When computing the optimal transport map for W , for cost matrix
Revised: When computing the optimal transport map for W , for cost matrix

Original: L ( B ) ⊤ Z , the complexity is O
Revised: L ( B ) ⊤ Z , the complexity is O

Original: ( dk ( d + k ) log( d + k )) .
Revised: ( dk ( d + k ) log( d + k )) .

Original: The overall complexity of the BCOT algorithmis then O ( k ∥ B ∥ 0 ) + tnk ( n + k ) log( n + k )
Revised: The overall complexity of the BCOT algorithmis then O ( k ∥ B ∥ 0 ) + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k ))
Revised: ( d + k ) log( d + k ))

Original: When using entropic regularization the complexity is smaller since  the computation of the optimalmap requires only a transformation of the inputs matrix which takes O ( nk ) in the Z computation stepand O ( dk ) for W .
Revised: When using entropic regularization the complexity is smaller, since computing the optimal maprequires only a transformation of the inputs matrix, which takes O ( nk ) in the Z computation stepand O ( dk ) for W .

Original: The ensuing application of the Sinkhorn-Knopp algorithm on the transformedmatrices has a complexity O ( tnk ) and O ( tdk ) for Z and W respectively, where t is the numberof iterations necessary.
Revised: The ensuing application of the Sinkhorn-Knopp algorithm on the transformedmatrices has complexities O ( tnk ) and O ( tdk ) for Z and W respectively, where t is the numberof iterations necessary.

Original: The overall complexity of BCOT λ is then O ( k ∥ B ∥ 0 )
Revised: The overall complexity of BCOT λ is then O ( k ∥ B ∥ 0 )

Original: + tnk + tdk ) , here tincludes the number of iterations of our algorithm as well as that of Sinkhorn-Knopp.
Revised: + tnk + tdk ) , here tincludes the number of iterations of our algorithm as well as that of Sinkhorn-Knopp.