Original: Efficient and Effective Optimal Transport-Based Biclustering
Revised: Efficient and Effective Optimal Transport-Based Biclustering

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress 
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Bipartite graphs can be used to model a wide variety of dyadic information such asuser-rating, document-term, and gene-disorder pairs.
Revised: Bipartite graphs can be used to model a wide variety of dyadic information such asuser-rating, document-term, and gene-disorder pairs.

Original: Biclustering is an extensionof clustering to the  bipartite graphs that underlie this kind of data.
Revised: Biclustering is an extensionof clustering to the underlying bipartite graph induced from this kind of data.

Original: In this paperwe leverage optimal transport (OT), which has gaineda certain momentum inthe machine learning community, in proposing a novel and scalable biclusteringmodel that generalizes several classical biclustering approaches.
Revised: Inthis paper, we leverage optimal transport (OT) which has gained momentum inthe machine learning community to propose a novel and scalable biclusteringmodel that generalizes several classical biclustering approaches.

Original: We performextensive experimentation to show the validity of our approach in relation to other
Revised: We performextensive experimentation to show the validity of our approach compared to other

Original: OT biclustering algorithms along both dimensions of the dyadic datasets
Revised: OT biclustering algorithms along both dimensions of the dyadic datasets

Original:  Introduction
Revised:  Introduction

Original: Let G =
Revised: Let G =

Original: ( U, V, E ) be a bipartite graph, which is a graph whose vertices can be divided into twodisjoint sets U = { 1 , 2 , . .
Revised: ( U, V, E ) be a bipartite graph which is a graph whose vertices can be divided into twodisjoint sets U = { 1 , 2 , . .

Original: , | U |} with | U | = n , V = { 1 , 2 , . .
Revised: , | U |} with | U | = n , V = { 1 , 2 , . .

Original: , | V |} with | V | =
Revised: , | V |} with | V | =

Original: d and the set ofedges E where each edge connects a vertex of U to a vertex of V .
Revised: d and the set ofedges E where each edge connects a vertex of U to a vertex of V .

Original: The adjacency matrix for this typeof graph has the following structure
Revised: The adjacency matrix for this typeof graph has the following structure

Original: [Equation]
Revised: [Equation]

Original: where B of size n × d is called the biadjacency matrix of G , its rows and columns corresponding tothe two sets of vertices; each entry represents an edge between a row and a column.
Revised: where B of size n × d is called the biadjacency matrix of G , its rows and columns correspond tothe two sets of vertices; each entry represents an edge between a row and a column.

Original: Biclustering (or
Revised: Biclustering (or

Original: Co-clustering ) is the extention of clustering to this type of graph.
Revised: Co-clustering ) is the extention of clustering to this type of graphs.

Original: Following [18], several biclusteringmodelshave attempted to solve the problem by viewing B as a two-mode matrix and searching for asimultaneous partition of its rows and columns [8].
Revised: Following [15], several biclusteringmodels attempted to solve the problem by viewing B as a two-mode matrix and searching for asimultaneous partition of its rows and columns [6].

Original: In this way, biclustering seeks to reveal subsets of
Revised: In this way, biclustering aims to reveal subsets of

Original: U which exhibit a similar behaviour across a subset of V in matrix B .
Revised: U which exhibit a similar behaviour across a subset of V in matrix B .

Original: Biclustering has been used in a number of different contexts. [12] used microarray data to findrelations between genes and conditions, finding that genes with similar functions often clustertogether.
Revised: Biclustering has been used in several contexts, [9] used microarray data to find relations betweengenes and conditions, finding that genes with similar functions often cluster together.

Original: [17] applied this paradigm to data from the US Food and Drug Administration reportingsystem in order to identify groups of drugs with adverse effects.
Revised: [14] used thisparadigm to identify drug groups with adverse effects on the US Food and Drug Administrationreporting system.

Original: [11] used it to find market segmentsamong tourists so as to enable more effective targeted marketing. There have been various otherapplications [8, 33, 16].
Revised: [8] used it to find market segments among tourists that are similar to each other toallow for targeted marketing, as well as many other applications [6, 30, 13].

Original: Several solutions to the biclustering problem have been proposed in the literature. [9] used aninformation-theoretic approach to solve the problem by minimizing the difference in mutual information between B and a summary matrix.
Revised: Multiple solutions for the biclustering problem have been proposed in literature, [7] used aninformation-theoretic approach to solve the problem by minimizing the difference in mutual information between B and a summary matrix.

Original: [2] adapted classical modularity to bipartite networksand then used it to identify modules within them.
Revised: [2] adapted classical modularity to bipartite networksand then used it to identify modules within them.

Original: [35] proposed a biclustering paradigm based onnonnegative  tri-factorization of the biadjacency matrix.
Revised: [31] proposed a biclustering paradigm based onnonnegative matrix tri-factorization of the biadjacency matrix.

Original: Recently, Optimal Transport (OT) has taken the machine learning community by storm. OT hashelped to solve a variety of  data mining problems, and biclustering is no exception.
Revised: Recently, Optimal Transport (OT) took the machine learning community by storm and was usedin the resolution of various data mining problems and biclustering was not an exception.

Original: [23] proposed two models for biclustering: a first model, CCOT, which does co-clustering based on the scalingvectors obtained by applying the Sinkhorn-Knopp algorithm on a square subsampled version ofmatrix B, and a second model, CCOT-GW, which uses scaling vectors obtained by computingentropic Gromov-Wasserstein barycenters, and which does not require subsampling.
Revised: First, [19] proposed two models for biclustering, a first one called CCOT which does co-clustering basedon the scaling vectors obtained from the application of the Sinkhorn-knopp algorithm on a squaresubsampled version of matrix B and another one called CCOT-GW that uses scaling vectors obtainedfrom computing entropic Gromov-Wasserstein barycenters and which does not require subsampling.

Original: Then came[34], wherethe authors did biclustering by minimizing a new metric, COOT, which generalizes the Gromov-Wasserstein distance between B and a summary matrix, similarly to what was done in [9].
Revised: Then came [26] where authors did biclustering by minimizing a new metric COOT, which generalizesthe Gromov-Wasserstein distance, between B and a summary matrix similar to what was done in[7].

Original: More specifically, they proposed two new metrics: COOT, together with an entropically regularizedmetric COOT λ .
Revised: In particular, they proposed two metrics COOT and an entropically regularized one COOT λ .

Original: However, both [23] and [34] have certain drawbacks.
Revised: These approaches, however, have some drawbacks.

Original: First, both algorithms do nottackle the biclustering from the beginning; the co-clusters are deduced at the convergence.

Original: Therebybiclustering is a consequence and not a main goal. Secondly, they suffer from high computationalcomplexity; CCOT and CCOT-GW also consume large amounts of memory.
Revised: One of them is that both of them suffer fromhigh computational complexity and in the case of CCOT and CCOT-GW also from a large memoryconsumption.

Original: Finally, we will see thatthese algorithms are not suited to dyadic sparse data.
Revised: Another one is that their empirical performance on dyadic data is not satisfactory.

Original: In this paper,while integrating the biclustering objective from the beginning, we propose a genericframework for biclustering through optimal transport, which generalizes some previous biclusteringapproaches.
Revised: In this paper, we propose a generic framework for biclustering through optimal transport whichgeneralizes several previous biclustering approaches.

Original: We propose two efficient methods for solving this problem: one that gives an almosthard biclustering, and another that gives a fuzzy or soft biclustering through entropic regularization.
Revised: We propose two efficient methods for solvingthis problem, one that results in an almost hard biclustering and another that results in a fuzzy orsoft biclustering through entropic regularization.

Original: These methods outperform other optimal transport biclustering models, in terms of both documentand term clustering, on several regular and large scale datasets, while being more computationallyand memory efficient.
Revised: These methods outperform other optimal transportbiclustering models both in term of document and term clustering on several regular and large scaledatasets while being more computationally and memory efficient.

Original: We emphasize once again that the approach we propose is specifically tailoredto datasets consisting of dyadic data that can be represented using a bipartite graph.
Revised: We emphasize once again the factthat the approach we propose is specifically tailored to datasets consisting in dyadic data which canbe represented using a bipartite graph, meaning that it should not be applied on other data types suchas images directly.

Original:  Methodology
Revised:  Methodology

Original: Notations In what follows, ∆ n = { p ∈ R n + | (cid:80)
Revised: Notations In what follows, ∆ n = { p ∈ R n + | (cid:80)

Original: ni =1 p i = 1 } denotes the n -dimensional standardsimplex.
Revised: ni =1 p i = 1 } denotes the n -dimensional standardsimplex.

Original: Π( w , v ) = { Z ∈ R n × k + | Z1 = w , Z ⊤ 1 = v } denotes the transportation polytope, wherew ∈
Revised: Π( w , v ) = { Z ∈ R n × k + | Z1 = w , Z ⊤ 1 = v } denotes the transportation polytope, wherew ∈

Original: ∆ n and v ∈ ∆ k are the marginals of the joint distribution Z and 1 n is a vector of ones.
Revised: ∆ n and v ∈ ∆ k are the marginals of the joint distribution Z and 1 n is a vector of ones.

Original: Matricesare denoted with uppercase boldface letters, and vectors with lowercase boldface letters.
Revised: Wedenote matrices with uppercase boldface letters and vectors with lower case boldface letters.

Original: For a matrix M , its i -th row is m and its j -th column is m ′
Revised: For amatrix M , its i -th row is m and its j -th column is m ′

Original: j We have that ∥ .
Revised: j We have that ∥ .

Original: ∥ 0 is the 0-norm which returns thenumber of nonzero elements of its argument.
Revised: ∥ 0 is the 0-norm which returnsthe number of nonzero elements of its argument.

Original: 2.1 Preliminaries
Revised: 2.1 Preliminaries

Original: We first need to introduce exact discrete OT and its entropically regularized counterpart, and showhow biclustering can be posed as an integer program.
Revised: We first need to introduce exact discrete OT and its entropically regularized counterpart and showhow biclustering can be posed as an integer program.

Original: Discrete OT as a linear program.
Revised: Discrete OT as a linear program.

Original: The goal of discrete optimal transport is to find a minimal costtransport map between a source probability distribution w and a target distribution v .
Revised: The goal of discrete optimal transport is to find a minimal costtransport map between a source probability distribution w and a target distribution v .

Original: Here we areinterested in the discrete case of the Kantorovich formulation of OT, that is
Revised: Here we areinterested in the discrete case of the Kantorovich formulation of OT, that is

Original: [Equation]
Revised: [Equation]

Original: where M ∈ R n × k is the cost matrix,and m ij quantifies the effort needed to transport a probabilitymass from w i to v j .
Revised: where M ∈ R n × k is the cost matrix, m ij quantifies the effort needed to transport a probability massfrom w i to v j .

Original: Discrete entropy regularized OT.
Revised: Discrete entropy regularized OT.

Original: It has been suggestedin the literature [5, 4] that the use ofa regularization such as  entropic regularization can lead to better computational and statisticalefficiency.
Revised: Several works [4, 3] have suggested that the use of a regularization such as the entropic regularization can lead to better computational and statistical efficiency.

Original: [Equation]
Revised: [Equation]

Original: (3) where H is the entropy defined as H ( Z ) ≜ (cid:80) i,j − z ij log z ij and λ controls the strength of regularization.
Revised: where H is the entropy defined as H ( Z ) ≜ (cid:80) i,j − z ij log z ij and λ controls the strength of regularization.

Original: The computational efficiency comes from the fact that the unique solution of this problem isof the  structure K := − diag ( a ) exp(− M /λ ) diag ( b ) , a rescaled elementwise negative exponentialof the cost M, where a and b are scaling vectors.
Revised: The computational efficiency comes from the fact that the unique solution of this problemhas the following structure K := − diag ( a ) exp( M /λ ) diag ( b ) , a rescaled elementwise negativeexponential of the cost M where a and b are scaling vectors.

Original: These vectors can be found efficiently using the Sinkhorn-Knopp algorithm.
Revised: These vectors can be found efficientlyusing the Sinkhorn-Knopp algorithm.

Original: Biclustering as an integer program.
Revised: Biclustering as an integer program.

Original: The Block seriation problem [25] consists in finding twopermutation matrices, one for the rows and one for the columns s.t.
Revised: The Block seriation problem [21] consists in finding twopermutations matrices, one for the rows and one for the columns s.t.

Original: dense blocks appear along thediagonal of the permuted matrix.
Revised: dense blocks appear along the diagonal of the permuted matrix.

Original: A possible definition of the block seriation problem is as follows:given a matrix B ∈ R n × d s.t b ij gives the strength of the association between row i and column j(such as in the case of a biadjacency matrix, for example), we have
Revised: A possible definition of the block seriation problem is that givena matrix B ∈ R n × d s.t b ij gives the strength of the association between row i and column j e.g. abiadjacency matrix, it is formulated as follows

Original: [Equation]
Revised: [Equation]

Original: A solution C matrix is a block diagonal matrix up to a permutation of its rows and columns.
Revised: A solution C matrix is a block diagonal matrix up to a permutation of its rows and columns.

Original: Theblock seriation problem is an integer programming problem and is consequently NP-hard.
Revised: Theblock seriation problem is an integer programming problem and is consequently NP-hard.

Original: Oneapproach for solving this problem uses a relaxed version where a rank constraint rank ( C ) ≤ k isadded for k the number of desired biclusters.
Revised: Oneapproach for solving this problem uses a relaxed version where a rank constraint rank ( C ) ≤ k isadded for k the number of desired biclusters.

Original: Integrating this constraint into 4, we can define a newequivalent problem by low-rank factorization of C, i.e. C = ZW ⊤ , which we formulate as
Revised: When integrating this constraint to 4, we can define anew equivalent problem by low-rank factorization of C i.e. C [Equation] ⊤ , which we formulate as

Original: [Equation]
Revised: [Equation]

Original: where Γ( n, k ) = { Z ∈ { 0 , 1 } n × k | Z1 = 1 } is the set of hard partitions of dimension n × k .
Revised: where Γ( n, k ) = { Z ∈ { 0 , 1 } n × k | Z1 = 1 } is the set of hard partitions of dimension n × k .

Original: A simpleheuristic for solving this problem involves alternatingly solving for Z given W, and vice-versa, usingclassical clustering algorithms, before identifying biclusters through the rearranged matrix C, whichdisplays a block diagonal structure, as shown in figure 1a.
Revised: A simpleheuristic for solving this problem involves alternatingly solving for Z given W and vice-versa usingclassical clustering algorithms before identifying biclusters through the rearranged matrix C whichdisplays a block diagonal structure as seen in figure 1a.

Original: The biclusters are identified by groupingtogether the rows and columns that form a block alongthe diagonal.
Revised: The biclusters are identified by groupingtogether the rows and columns that form a block along diagonal.

Original: 2.2 Biclustering using Optimal Transport
Revised: 2.2 Biclustering using Optimal Transport

Original: Here we propose a new biclustering problem based on block seriation and optimal transport.
Revised: Here, we propose a new biclustering problem based on block seriation and optimal transport.

Original: For thispurpose we first define what we term an anti-adjacency matrix:
Revised: We firststart by introducing a necessary concept which we call anti-adjacency matrix

Original: Definition 1.
Revised: Definition 1.

Original: (Anti-adjacency matrix)
Revised: (Anti-adjacency matrix)

Original: Given a graph characterized by an adjacency matrix A , wehave a corresponding anti-adjacency matrix A s.t.
Revised: Given a graph characterized by an adjacency matrix A , wedenote its anti-adjacency matrix A s.t.

Original: a ij quantifies a discrepancy between node i and j .
Revised: a ij quantifies a discrepancy between node i and j .

Original: We consider a bipartite graph characterized by its biadjacency matrix B = ( b ij ) ∈ R n × d .
Revised: We consider a bipartite graph characterized by its biadjacency matrix B = ( b ij ) ∈ R n × d .

Original: The rowsof B are endowed with weights w ∈ ∆ n and its columns with weights v ∈ ∆ d .
Revised: The rowsof B are endowed with weights w ∈ ∆ n and its columns with weights v ∈ ∆ d .

Original: We also considera row exemplar distribution r ∈ ∆ r and a column exemplar distribution c ∈ ∆ c .
Revised: We also considera row exemplar distribution r ∈ ∆ r and a column exemplar distribution c ∈ ∆ c .

Original: Depending onthe availability of a priori information about the data, these weight vectors can be set to uniformdistributions.
Revised: Depending uponthe availability of a priori information about the data, these weight vectors can be set to uniformdistributions.

Original: Now let its anti-adjacency matrix be B
Revised: Now let its anti-adjacency matrix be B

Original: = L ( B ), where L : R n × d → R n × d means that b ij , theassociation between node i and node j ,is transformed into a discrepancy measure L ( b ) ij .
Revised: = L ( B ) where L : R n × d → R n × d leads to transform b ij ,the association between node i and node j , into a discrepancy measure L ( b ) ij .

Original: Z .
Revised: Z .

Original: Thus,we define the optimal transport block seriation problem as
Revised: Thus, we definethe optimal transport block seriation problem as

Original: [Equation]
Revised: [Equation]

Original: where Z is a transport map (or coupling) between between the row distribution w and the rowexemplar distribution r, and similarly for W w.r.t.
Revised: where Z is a transport map (or coupling) between between the row distribution w and the rowexemplar distribution r and similarly for W w.r.t.

Original: the column distribution v and the columnexemplar distribution c .
Revised: the column distribution v and the column exemplardistribution c .

Original: Similar to [34] is an indefinite Bilinear Program and is related to thequadratic assignment problem (QAP)
Revised: Similar to [26] is an indefinite Bilinear Program and is related to the quadraticassignment problem (QAP)

Original: Inducing a Biclustering using BCOT Wewill now  show how to obtain a partition of the rowsand the columns given a solution pair ( Z , W ) .
Revised: Inducing a Biclustering using BCOT We now are going to show how to obtain a partition of therows and the columns given a solution pair ( Z , W ) .

Original: In what follows our aim is to identify an almost-hardclustering couple for rows and columns from the couplings Z and W .
Revised: In what follows, we are interested in inducing acouple of

Original: [Figure]
Revised: [Figure]

Original: Definition 2.
Revised: Definition 2.

Original: ( h -almost hard clustering)
Revised: ( h -almost hard clustering)

Original: We define an h -almost hard clustering as a clusteringwhose assignment matrix is C ∈ R n × k s.t.
Revised: We define an h -almost hard clustering as a clusteringwhose assignment matrix is C ∈ R n × k s.t.

Original: ∥
Revised: ∥

Original: C ∥ 

Revised: C ∥

Original: = n + h and for each row c of C we have that∥ c ∥ 0 > 0 .
Revised: = n + h and for each row c of C we have that∥ c ∥ 0 > 0 .

Original: When h = 0 , we obtain a standard hard clustering with one non-zero element per row.
Revised: When h = 0 , we obtain a standard hard clustering with one non-zero element per row.

Original: Proposition 1. 

Revised: Proposition 1.

Original: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .
Revised: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: Furthermore, when n = k(resp.
Revised: Furthermore, when n = k(resp.

Original: d = k ) and w = r (resp.
Revised: d = k ) and w = r (resp.

Original: v = c ), Z (resp.
Revised: v = c ), Z (resp.

Original: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.
Revised: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.

Original: W ∈ Γ( d, d ) ).
Revised: W ∈ Γ( d, d ) ).

Original: This means that the solutions are already almost a hard partition of the data, since k << n, d .
Revised: This means that the solutions are already almost a hard partition of the data since k << n, d .

Original: Toobtain a final hard clustering in the strict sense, we assign each row (resp. column) to the onecorresponding to the  row of Z (resp.W ) with the largest value.
Revised: Toobtain a final hard clustering in the strict sense, we assign each row (resp. column) to the onecorresponding to the largest value of each row of Z (resp.

Original: This should not significantly changethe structure of the solution.
Revised: W ), this should not significantly changethe structure of the solution.

Original: Figure 1b provides an illustration: here we see the block diagonalstructure generated by the product of the two coupling matrices C = ZW ⊤ , with its similarity inappearance to the biclustering produced by the hard block seriation 1a, apart from a few nonzeroentries off the block diagonal that are hard to see immediately.
Revised: To illustrate this, we look at figure 1b, it shows the block diagonalstructure generated by the product of the two coupling matrices C = ZW ⊤ . We see how it is lookssimilar to the biclustering produced by the hard block seriation 1a except for a few nonzero entriesoff the block diagonal that are hard to see immediately.

Original: Intuition for BCOT To explain the intuition behind the proposed approach we need to look at howthe problem is solved.
Revised: Intuition for BCOT To explain the intuition behind the proposed approach we have to look atthe way the problem is solved.

Original: The optimization procedure as described in algorithm 1 consists in alternatingbetween the computation of an optimal transportation map Z given W and vice versa.
Revised: The optimization procedure as described in algorithm 1 consists inalternating between the computation of an optimal transportation map Z given W and vice versa.

Original: As regardssolving for Z given W , the problem can be rewritten as
Revised: Ifwe look at the solving for Z given W , the problem can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: This is an optimal transport problem with L ( B ) W as the cost matrix.
Revised: This is an optimal transport problem with L ( B ) W as the cost matrix.

Original: The resulting transportationmap Z can be seen as a kind of  row cluster assignment matrix: if z ih > 0, then row i is assigned tocluster h .
Revised: The resulting transportationmap Z can be seen as a sort of a row cluster assignment matrix, if z ih > 0 then row i is assigned tocluster h .

Original: The same  holds for W, which can be seen as a column cluster assignment matrix.
Revised: The same thing holds for W which can be seen as a column cluster assignment matrix.

Original: Thisalso means that since L ( B ) is the dissimilarity between the rows and the columns, then the costmatrix L ( B ) W represents the dissimilarity between rows and row exemplars (or representativesor centroids).
Revised: This also means that since L ( B ) is the dissimilarity between the rows and the columns, then thecost matrix L ( B ) W represents the dissimilarity between rows and row exemplars (or representativesor centroids).

Original: In particular, L ( B )
Revised: In particular, L ( B )

Original: i w h is the dissimilarity or cost of probability mass transportationbetween row i and row cluster exemplar h .
Revised: i w h is the dissimilarity or cost of probability mass transportationbetween row i and row cluster exemplar h .

Original: The reasoning is the same for the columns andthe optimalcoupling W .
Revised: The reasoning is the same for the columns and optimalcoupling W .

Original: Low-Rank Optimal Transport Biclustering is the main purpose of the approach we proposed, butthere is another interesting use case.
Revised: Low-Rank Optimal Transport Biclustering is the main purpose of the approach we proposed, butanother interesting use case comes up.

Original: Proposition 2.
Revised: Proposition 2.

Original: In the case where the target row and column representative distributions are thesame, i.e., r = c with no zero entries, then given a solution pair Z and W to BCOT, the matrix Q =
Revised: Suppose that the target row and column representatives distributions is the samei.e. r = c with no zero entries, then given a solution pair Z and W to BCOT, the matrix Q =

Original: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem2 and whose rank is  at most min( rank ( Z ) , rank ( W )) .
Revised: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem 2and whose rank is of at most min( rank ( Z ) , rank ( W )) .

Original: Some recent works [15, 31] have suggested that this kind of low-rank regularization is preferableto entropic regularization  as regards certain aspects. For example, the rank parameter is easier to select, since it has simple bounds (an integer between 1 and n ).
Revised: Some recent works [12, 28] suggested that this kind of low-rank regularization is preferable toentropic regularization in some aspects such as for the fact that the rank parameter is easier to select since it has simple bounds (an integer between 1 and n ) contrary to regularization strength λ in the

Original: This may be contrasted with theregularization strength λ in the Sinkhorn algorithm, which is continuous.
Revised: Sinkhorn algorithm which is continuous.

Original: 2.3 Fuzzy biclustering using regularized Optimal Transport
Revised: 2.3 Fuzzy biclustering using regularized Optimal Transport

Original: As previously mentioned, using entropic regularization may be interesting because of its varioususeful features including statistical and computational efficiency.
Revised: As previously mentionned, using entropic regularization can be interesting due to the many propertiesit entails like statistical and computational efficiency.

Original: However, another feature of entropicregularization is that the optimal couplings Z and W are dense matrices as a consequence of thestructure of the optimal solution of entropically regularized OT problems.
Revised: However, another property is that the optimalcouplings Z and W are dense matrices due to the structure of the optimal solution of entropicallyregularized OT problems.

Original: We formulate the problemas follows
Revised: We formulate the problem as follows

Original: [Equation]
Revised: [Equation]

Original: where λ Z and λ W are the regularization parameters.
Revised: where λ Z and λ W are the regularization parameters.

Original: Fuzzy Block Seriation We propose a fuzzy variant of the block seriation problem that allows usby extension to define a fuzzy variant for BCOT using entropic regularization.
Revised: Fuzzy Block Seriation We propose a fuzzy variant of the block seriation problem which allows usby extension to define a fuzzy variant for BCOT using entropic regularization.

Original: Let the fuzzy blockseriation problem be defined as
Revised: Let the fuzzy blockseriation problem be defined as

Original: [Equation]
Revised: [Equation]

Original: where Ω( Z , W ) is some regularization term introduced to make the partition matrices Z and W dense(for example, entropic regularization or low-rank constraints), and Γ s ( n, k )
Revised: where Ω( Z , W ) is some regularization term introduced to make the partition matrices Z and Wdense e.g. entropic regularization or low-rank constraints and Γ s ( n, k )

Original: = { Z ∈ R n × k + | Z1 = 1 }is the set of fuzzy partitions.
Revised: = { Z ∈ R n × k + | Z1 = 1 }is the set of fuzzy partitions.

Original: Intuitively, for a solution pair ( Z , W ) , up to a constant factor, eachentry in the block seriation matrix C =
Revised: Intuitively, for a solution pair ( Z , W ) , up to a constant factor, eachentry of the block seriation matrix C =

Original: ZW ⊤ can be seen as the probability of its correspondingrow and column belonging to the same bicluster i.e. c ij =
Revised: ZW ⊤ can be seen as the probability of its correspondingrow and column belonging to the same bicluster i.e. c ij =

Original: z i w j = (cid:80) rh
Revised: z i w j = [Equation] rh

Original: [Equation]
Revised: [Equation]

Original: 161 (cid:80)
Revised: z ih [Equation] jh = p ( b

Original: [Equation]
Revised: [Equation]

Original:  j∈ h ).
Revised: , b ′ j )

Original:  It is easy to see how problem 9 is  related to problem 8 and that the couplingscorresponding to solutions of the problem give the probability that the different rows and columnsbelong to the same biclusters.
Revised: = 157 (cid:80) rh =1 p b i , b ′ j ∈ h ) . It is easy to see how problem 9 is a related to problem 8 and that the couplingscorresponding to solutions to the problem give the probability of membership to the same biclustersfor the different rows and columns.

Original: Figure 1c shows biclusters produced by the solutions of BCOT λ .
Revised: Figure 1c shows biclusters produced by the solutions of BCOT λ .

Original: Similarly to BCOT, a block diagonal structure  is formed.
Revised: Similarly to BCOT there is a block diagonal structure that is formed.

Original: However, there are also several off-blockdiagonal nonzero entries that represent the probabilities of the row-column pairs belonging to thesame biclusters.
Revised: However, there are also severaloff-block diagonal nonzero entries that represent the probabilities of the row-columns pairs belongingto the same biclusters.

Original:  Links to Existing Work
Revised:  Connections to Existing Work

Original: Modularity Maximization in bipartite graphs [2] This model is able to co-cluster binary and contingency matrices by directly maximizing an adapted version of the modularity measure traditionallyused for networks.
Revised: Modularity Maximization in bipartite graphs [2] This model allows to co-cluster binary and contingency matrices by directly maximizing an adapted version of the modularity measure traditionallyused for networks.

Original: The criterionthat it optimizes is
Revised: The criterion it optimizes is

Original: [Equation]
Revised: [Equation]

Original: By setting L ( B ) = − ( B − 1 b .. B11 ⊤ B ) , this problem becomes equivalent to ours; the difference isin the constraints on Z and W .
Revised: by setting L ( B ) = − ( B − 1 b .. B11 ⊤ B ) , this problem becomes equivalent to ours with the differencelying in the constraints on Z and W .

Original:  BCOTtherefore serves as a convex relaxation of this problem.
Revised: Thereby BCOT serves as a convex relaxation to this problem.

Original: Modularity-based Sparse Soft Graph Clustering [20] Here the authors proposed a fuzzy variantof the above problem (although  in the context of traditional clustering rather than biclustering).
Revised: Modularity-based Sparse Soft Graph Clustering [17] Here the authors proposed to fuzzy variantof the previous problem (although not in a bicustering context but

Original: Solving the problem gives, for each element of the dataset, a probability of that element belonging toa given cluster.
Revised: rather traditionanal clustering)whose solution gives for each element of the dataset a probability to belong to a given cluster.

Original: Our proposed entropic regularization variant represents a kind of extensionof thisproblem to bipartite graphs.
Revised: Ourproposed entropic regularization variant constitutes a sort of extension to bipartite graphs for thisproblem.

Original: Directional co-clustering with a conscience [30] This model makes use of the block von MisesFisher mixture model for co-clustering directional data on the unit-sphere.
Revised: Directional co-clustering with a conscience [27] This model relies on the block von Mises-Fishermixture model for co-clustering directional data on the unit-sphere.

Original: It optimizes the followingcriterion:
Revised: It optimizes the followingcriterion

Original: [Equation]
Revised: [Equation]

Original: In our formulation, if we define L ( B ) = − B and apply cluster size normalization on the optimaltransport plans ˜ Z = Z diag ( Z ⊤ 1 ) − 1 / 2 and ˜ W = W diag ( W ⊤ 1 ) − 1 / 2 after computing Z and Wrespectively in algorithm 1, we obtain a more general version of the algorithm proposed by theauthors for solving problem 11.
Revised: In our formulation, if we define L ( B ) = − B and apply cluster size normalization on the optimaltransport plans ˜ Z = Z diag ( Z ⊤ 1 ) − 1 / 2 and ˜ W = W diag ( W ⊤ 1 ) − 1 / 2 after the computation of Zand W respectively in algorithm 1 we obtain a more general version of the algorithm that the authorsproposed to solve problem 11.

Original: Bipartite Correlation Clustering [1]
Revised: Bipartite Correlation Clustering [1]

Original: In the case where the cost function results in a completebipartite graph with ’+’ and ’-’ edges with a function
Revised: In the case where the cost function results in a completebipartite graph with ’+’ and ’-’ edges with a function

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: we get what is known as Bipartite Correlation Clustering.
Revised: we get what is known as Bipartite Correlation Clustering.

Original: The solution to this problem maximizes thenumber of agreements, i.e. the number of all ’+’ edges within clusters plus all ’-’ edges distributedacross clusters.
Revised: The solution to this problem maximizesthe number of agreements i.e. the number of all ’+’ edges within clusters plus all ’-’ edges distributedacross clusters.

Original:  Optimization and Complexity
Revised:  Optimization and Complexity

Original: [Figure]
Revised: [Figure]

Original: Optimization Since the block seriation problem is NP-hard, computing an exact solution is prohibitive.
Revised: Optimization The block seriation problem being NPhard means that computing an exact solution is prohibitive.

Original: An efficient and widely used heuristic for solving thesekinds of problems involves the use of block coordinatedescent, where  row  assignmentsare computed for fixedcolumn assignments, andthen vice versa, in alternation.
Revised: An efficient and widely used heuristic to solvethese kind of problems in using block coordinate descent where we alternatingly compute row assignmentsfor fixed column assignments and vice versa.

Original: We express the proposed algorithm  in pseudo-code as Alorithm 1. At each iteration we solve two intermediateoptimal transport problems with cost matrices of dimensions n × k and d × k , since B is generally sparse, and Lcan be defined such that  L ( B ) retains a similarly sparsestructure.
Revised: The proposed algorithm available in pseudo-code 1, in eachiteration we solve two intermediate optimal transportproblems with cost matrices of dimensions n × k andd × k , since B is generally sparse and we can define Lin way that make L ( B ) keep a similarly sparse structure, the computation of the intermediate cost matrices

Original: The computation of the intermediate cost matrices L ( B ) W and L ( B ) ⊤ Z is reasonably efficient.
Revised: L ( B ) W and L ( B ) ⊤ Z is quite efficient.

Original: Wealso observed that the algorithm does not need manyiterations to converge, as shown in figure 2, be it for BCOT or BCOT λ .
Revised: Furthermore,we observed that the algorithm does not need many iterations to converge as seen in figure 2, be it for BCOTor BCOT λ .

Original: Algorithm 1: BCOT
Revised: Algorithm 1: BCOT

Revised: Input :

Original: Proposition 3.
Revised: Proposition 3.

Original: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥ 

Revised: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥

Original: + tnk ( n + k ) log( n + k )
Revised: + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k ))), and when using entropicregularization the complexity is O ( tk ∥ B ∥ 

Revised: ( d + k ) log( d + k ))) and when using entropicregularization, the complexity is O ( tk ∥ B ∥

Original: + tkn + tkd ), where t is the number of iterations.
Revised: + tkn + tkd ) where t is the number of iterations.

Original: In table 1, we report the computational and spatial complexities of the different biclustering approaches.
Revised: In table 1, we report the computational and spatial complexities of the different biclustering approaches.

Original: Our model has the same spatial complexity as the COOT variants and  a better complexitythan CCOT variants.
Revised: Our model has the same spatial complexity as the COOT variants and is a better one than CCOT variants.

Original: As regards the computational complexity, our model should  in most cases befaster, and our experiments support this conjecture.
Revised: For the computational complexity, our model should be faster in most cases, ourexperiments support this observation.

Original: For reproducibility, we publicly release our code 
Revised: For reproducibility, we publicly release our code 2 .

Original: [Table]
Revised: [Table]

Original:  Experiments
Revised:  Experiments

Original: We ran experiments using term-document matrices.
Revised: We conduct experimentation on term-document matrices.

Original: The benefit of using biclustering on this kindof data is that the resulting biclusters contain both documents and the words that characterize them,which is helpful in interpretingthe clustering of the documents.
Revised: The benefit of using biclustering on thiskind of data is that the resulting biclusters contain both documents and the words that characterizewhich will help us with interpreting clustering of the documents.

Original: 5.1 Datasets
Revised: 5.1 Datasets

Original: We evaluate BCOT in relation to six benchmark document-term datasets: ACM, DBLP, PubMed, Wiki, Ohscal [Table] and 20 Newsgroups.
Revised: We evaluate BCOT on six benchmark document-term datasets, ACM, DBLP, PubMed, Wiki, Ohscaland 20 Newsgroups.

Original: Their characteristics are shown in Table 2.
Revised: Their characteristics are shown in Table 2.

Original: ACM, DBLP, Pubmedand Wiki are attributed networks from which we use only the node-level features that correspond toterm-document matrices.
Revised: ACM, DBLP, Pubmed and Wiki areattributed networks from which we use only the node-level features that correspond to term-documentmatrices.

Original: We also selected the Ohscal collection and 20 Newsgroups as large-scaledocument-term matrices to serve as  computational efficiency benchmarks.
Revised: We also selected the Ohscal collection and 20 Newsgroups as large scale document-termmatrices as the computational efficiency benchmarks.

Original: [Table]
Revised: [Table]

Original: 5.2 Experimental setup
Revised: 5.2 Experimental setup

Original: In our experiments we define the loss function as L ( B ) = − c B, where c is selected from { 1 , k, d, n } .
Revised: In our experiments, we define the loss function as L ( B ) = − c B where c is selected from { 1 , k, d, n } .

Original: For BCOT λ , the regularization parameter lambda is selected from { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 , 10 } .
Revised: For BCOT λ , the regularization parameter lambda is selected from { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 , 10 } .

Original: The best hyper-parameters are those that minimize the number of empty clusters.
Revised: The best hyper-parameters are the ones that minimize the number of empty clusters.

Original: In the case of ties,we select according to the value of the Davies-Bouldin index of the partition [6].
Revised: In the caseof ties, we select according to the value of the Davies-Bouldin index of the partition.

Original: Random restartsarenot used for any of the algorithms, including k -means.
Revised: We do notuse random restarts for any algorithm including k -means.

Original: We use the implementation provided by theauthors for CCOT, CCOT λ and CCOT-GW.
Revised: We use the implementation provided bythe authors for CCOT, CCOT λ and CCOT-GW.

Original: The code for CCOT was not available, and so we had toimplement it based on the code for CCOT-GW.
Revised: The code for CCOT was not available so we had toimplement it based on the one for CCOT-GW.

Original: All the reported figures are the averages of 10 runs.
Revised: All the reported figures are the averages of 10 runs.

Original: All the experiments were performed on the same machine with an Intel(R) Xeon(R)CPU and 12GB
Revised: Allthe experiments were performed on the same machine with a RAM of 12GB and Intel(R) Xeon(R)

Original: RAM.
Revised: CPU.

Original: For OT solvers we made use of the POT package [14].
Revised: For OT solvers, we relied on the POT package [11].

Original: 5.3 Document clustering
Revised: 5.3 Document clustering

Original: Metricsand algorithms Here, the evaluation is straightforward. We use three popular clusteringmetrics: Here, the evaluation is straightforward, we adopt three popular clustering metrics: clusteringaccuracy (ACC), normalized mutual information (NMI) [3], adjusted rand index (ARI) [21].
Revised: Metrics Here, the evaluation is straightforward, We adopt three popular clustering metrics: clustering accuracy (ACC), normalized mutual information (NMI), adjusted rand index (ARI).

Original: Inaddition to the OT biclustering algorithms we perform experiments with k -means, informationtheoretic co-clustering (ITCC) [9] and orthogonal matrix non-negative tri-factorization (ONMTF)[10] suited for document-term matrices.

Original: [Table] Performance Document clusteringresults on ACM, DBLP, PubMed and Wiki are given in table 3for the three metrics.
Revised: Performance Results for document clustering on ACM, DBLP, PubMed and Wiki are summarizedin table 3 in terms of ACC, NMI and ARI.

Original: In allcases the best result is obtained eitherby BCOT orby BCOT λ .
Revised: On all these datasets and for all the different metrics either BCOT or BCOT λ offer the best result.

Original: Moreover,on Wiki, BCOT λ gives competitive results when compared with state-of-the-art attributed graphclustering methods such as [37], despite not having access to the graph structure information in the 
Revised: Furthermore, on the wiki dataset BCOT λ gives competitiveresults when compared with state of the art attributed graph clustering methods such as [33] withouteven having access to the graph structure information of the Wiki citation network. [Table] [Figure] [Table]

Original: Wiki citation network.
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Original: Efficiency
Revised: [Table] [Figure] [Table] [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original:  Figure 3 plots thedocument clustering performance (accuracy over  training time) of thedifferent methods on the two large-scale document-term matrices 20 Newsgroup and Ohscal.
Revised: [Table] [Figure] [Table] Efficiency In Figure 3, we plot the performance of the different methods over their training timerelative to that of BCOT λ on the two large scale document-term matrices, 20 Newsgroup and Ohscal.

Original: BCOToffers the best accuracy while BCOT λ is fastest method on both datasets.
Revised: BCOT offers the best accuracy while BCOT λ is fastest method on both datasets.

Original: We see that for both BCOTand COOT, the entropic-regularized versions outspeed their exact counterparts and that CCOT suffersfrom very high computation times, duemainly to the fact that this method requires pairwise distancematrices to be computed on the rows and columns.
Revised: We see that forboth BCOT and COOT, the entropic-regularized versions outspeed their exact counterparts and that CCOT suffers from very high computation times mostly due to the fact that a calculation of a pairwisedistance matrices on the rows and columns is necessary.

Original: [Figure]
Revised: [Figure]

Revised: .g.

Revised: × 5 for BCOT means that it is approximately five times slower than

Revised: BCOT λ ).

Revised: We were not able to benchmark CCOT-GW since it failed to scale to these datasets.

Original: 5.4 Term Clustering
Revised: 5.4 Term Clustering

Original: Metrics Unlike document clustering, there is no ground truth partition for terms, so we need to findanother way of evaluating term clustering results.
Revised: Metrics Unlike document clustering, there is no ground truth partition for terms which meanswe have to look for another way to evaluate term clustering results.

Original: One generally acceptable technique is to analyse thesemantic coherence of the clusters obtained.
Revised: A reasonable way to performevaluation is to analyse the semantic coherence of the clusters found by the different models.

Original: To this end we introduce a metric based on  point mutualinformation (PMI).
Revised: Withthis in mind, we introduce a metric based on the point mutual information (PMI).

Original:  PMI is a frequently used information-theoretic metric for quantifying the relationship between pairs of discrete random variable outcomes.
Revised: the PMI isa frequently used information theoretic metric for quantifying the relationship between pairs ofdiscrete random variable outcomes.

Original: The PMI measure was chosen because priorresearch [27] has shown that it is closely associated with human judgements in determining word relatedness.
Revised: The PMI measure was chosen because prior research [23]has shown that it is closely associated with human judgements in determining word relatedness.

Original: [Equation]
Revised: [Equation]

Original: The PMI  between the terms w i and w j is calculated as in (13).
Revised: The PMI measure between the terms w i and w j is calculated as in (13).

Original: In the context of term clustering, giventhe word co-occurrence matrix K = B ⊤ B ,the PMI is estimated as in (14).
Revised: In the context of termclustering, given the word co-occurrence matrix K = B ⊤ B , PMI is estimated as in (14).

Original: To evaluate a partition ofterms P , we propose a metric based on intra and inter PMI metrics as follows: [Table]
Revised: To evaluatea partition of terms P , we propose a metric based on intra and inter PMI metrics given by

Original: [Equation]
Revised: [Equation]

Original: In this way [Table] a good clustering should reveal a high intra-cluster semantic relatedness, correspondingto higher PMI values.
Revised: Thereby, a good clustering should reveal a large intra-cluster semantic relatedness corresponding tolarger PMI values.

Original: Using the intra and inter PMIs [Table] we propose the following coherence indexcoherence
Revised: From intra and inter PMIs, we propose the following coherence indexcoherence

Original: [Equation]
Revised: [Equation]

Original: Our reasoning is this: the greater the semantic proximity between terms in the same clusters, and thegreater thesematic distance between terms in different clusters, the higher the value of coherence .
Revised: [Table] This index relies on the fact that a partition maximizing this criterion has semantically close termsinside the same clusters and contrasting ones across the different groups.

Original: Results Since there is no ground truth number of term clusters, we  use  the cluster number estimations produced by CCOT-GW for allthe other models so that it is easy to compare coherence valuesbetween them. Comparisons based on different numbers of clusters would favor  themodel usingthe larger number of clusters.
Revised: Results Since there is no ground truth number of term clusters [Table] we make use of the cluster numberestimations produced by CCOT-GW for all other models so that it is easy to compare coherencevalues between them since comparing it for different number of clusters would favor models thatused the larger cluster number.

Original: Table 4 shows the coherences obtained across the different datasetsusing our approach, along with those of the baselines.
Revised: Table 4 shows the coherence obtained by our approach along withthose of the baselines over the different datasets.

Original: It is clear that BCOT succeeds in capturingmore semantics than the other approaches since, whatever the dataset, oneor other ofthe two BCOTvariants gives the highest coherence.
Revised: It is clear that BCOT succeeds in capturing moresemantics than the other approaches as, on all datasets, one of two variants of BCOT offer the bestcoherence value.

Original: 5.5 Statistical significance
Revised: 5.5 Statistical significance

Original: We did a Nemenyi post-hoc test [26, 7] with a confidence level of 90% on thedocument andterm clustering resultsthat we obtained, to determine whether our model outperforms other OTbiclustering approaches in a statistically significant way.
Revised: We perform a nemenyi post hoc test [22, 5] with a confidence level of 90% on the results we obtainedin terms of document and term clustering to see if our model outperforms other OT biclusteringapproaches in a statistically significant manner.

Original: To conduct this test we generated 20performance rankings of the OT biclustering models based on their performance for each dataset andquality metric pair for both document and term clustering.
Revised: To conduct this test we generate 20 performancerankings of the OT biclustering models based on their performance for each dataset and quality metricpair for both document and term clustering.

Original: Figure 4 shows the results of the test. Wesee that  two differently performing groups were identified, one comprising BCOT and BCOT λ andgiving better results than the other group comprising the remaining COOT and CCOT variants. Thisindicates that with this specific number of datasets and metrics the test was unable to tell  COOT and CCOTapart in a statistically significant way.
Revised: Figure 4 shows the results of the test, we see that it hasfound two differently performing groups, one comprising BCOT and BCOT λ , which gives betterresults than the other one which comprises the remaining COOT and CCOT variants, meaning the testcould not tell apart COOT and CCOT in a statistically significant manner with this specific number ofdatasets and metrics.

Original: [Figure]
Revised: [Figure]

Original:  Conclusion
Revised:  Conclusion

Original: Clustering and biclustering through optimal transport is still at a nascent stage, with many challengesremaining unsolved.
Revised: Clustering and biclustering through optimal transport is till at a nascent stage with many challengesremaining unsolved.

Original: This paper introduces a novel problem for biclustering using optimal transport that takes into account the sparse nature of certain typesof dyadic data such document-term matrices,to enable more computationally efficient resolution.
Revised: Here, we have introduced a novel problem for biclustering using optimaltransport that takes into account the sparse nature of certain types dyadic data such document-termmatrices to make for more computationally efficient resolution.

Original: The problem is posed as a bilinear programthat we solve using an efficient coordinate descent algorithm that takes into account the sparsenature of certain types of dyadic data such as document-term matrices.
Revised: The problem is posed as a bilinearprogram that we solve using an efficient coordinate descent algorithm that takes into account thesparse nature of certain types of dyadic data such as document-term matrices.

Original: Experiments on a numberof document-term datasets suggest that the proposed approach does a good job in finding clustersthat correspond to ground truth document classes, while generating semantically coherent partitionsfor the terms.
Revised: Experiments on anumber of document-term datasets suggest that the proposed approach does a good job of findingclusters that correspond to ground truth document classes while generating semantically coherentpartitions for the terms.

Original: In this setting, our model outperforms recent OT biclustering methods by a significantmargin, while being more computationally efficient.
Revised: In this setting, our model outperforms recent OT biclustering methods bysignificant margin, while being more computationally efficient.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] Last paragraph of the introduction (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] Last paragraph of the introduction(c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A](d)

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Revised: [Table]

Original: were chosen)?
Revised: were chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e g., with respect to the random seed after running experi-
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experi-

Revised: [Table]

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you citep the creators?
Revised: (a) If your work uses existing assets, did you citep the creators?

Original: [Yes](
Revised: [Yes]

Revised: [Table]

Original: b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [No](c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [No](c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes](
Revised: [Yes]

Original: d ) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re [Table] using/curating?

Original: [N/A] All assets used are publicly available(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
Revised: [N/A] All assets used are publicly available (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A]5.
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A](b)
Revised: [N/A]

Original:  Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A](c)
Revised: [N/A]

Original:  Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A

Original:  Appendix A Proofs
Revised:  Appendix A Proofs

Original: Proposition 1.
Revised: Proposition 1.

Original: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .
Revised: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: Furthermore, when n = k(resp.
Revised: Furthermore, when n = k(resp.

Original: d = k ) and w = r (resp.
Revised: d = k ) and w = r (resp.

Original: v = c ), Z (resp.
Revised: v = c ), Z (resp.

Original: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.
Revised: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.

Original: W ∈ Γ( d, d ) ).
Revised: W ∈ Γ( d, d ) ).

Original: Proof of proposition 1.
Revised: Proof for proposition 1.

Original: Problem 2 is a bounded linear program, since Π( w , v ) is a polytope, i.e. abounded polyhedron.
Revised: Problem 2 is a bounded linear program since Π( w , v ) is a polytope i.e. abounded polyhedron.

Original: The fundamental theorem of linear programming states that if the feasible setis non-empty then the solution lies at the limits of the feasible region.
Revised: The fundamental theorem of linear programming states that if the feasible setis non-empty then the solution lies in extremity the feasible region.

Original: This means that a solution Zto problem 2 is an outermost point of Π( w , v ) .
Revised: This means that a solution Z toproblem 2 is an extreme point of Π( w , v ) .

Original: We have that the outermost points of Π( w , v ) canhave at most n + d − 1 nonzero elements.
Revised: We have that the extreme points of Π( w , v ) can have atmost n + d − 1 nonzero elements.

Original: To prove this we have to show thatin the bipartite graphinduced by the biadjacency matrix Z , the solution to the optimal transport problem has no cycles.
Revised: To prove this we have to show that the bipartite graph induced bybiadjacency matrix Z , the solution to the optimal transport problem has no cycles.

Original: The maximum number of edges in an acyclic graph is | V | − 1 where | V | is the number of nodesin the graph.
Revised: The maximumnumber of edges in an acyclic graph is | V | − 1 where | V | is the number of nodes in the graph.

Original: Since the number of edges in the bipartite graph induced by biadjacency matrix Z isn + d − 1 , the matrix Z cannot have more than n + d − 1 nonzero entries.
Revised: Sincethe number of edges in the bipartite graph induced by biadjacency matrix Z is n + d − 1 , the matrix Z can not have more than n + d − 1 nonzero entries.

Original: For a detailed proof seeproposition 3.3 in [29].
Revised: For a detailed proof see proposition 3.3 in [25].

Original: We also have to show that for probability measures w and v that have no zero probability events,there are at least max( n, d )  nonzero elements in Z .
Revised: We also have to show that for probability measures w and v that have no zero probability events,there is at minimum max( n, d ) number of nonzero elements in Z .

Original: This is straightforwardto show: since w and vcontain no zeros, there will always be at least one nonzero element in every row and column of Zthat represents some transfer of mass between elements of w and v .
Revised: This is straightforward since wand v contain no zeros, there will always be at least one nonzero element in every row and column of Z that represents some transfer of mass between elements of w and v .

Original: In the case of BCOT, we then have that Z has at most n + k − 1 and at least max( n, k ) =
Revised: In the case of BCOT, we then have that Z has at most n + k − 1 and at least max( n, k ) =

Original: n nonzeroentries and that W has at most d + k − 1 and at least max( d, k ) =
Revised: n nonzeroentries and that W has at most d + k − 1 and at least max( d, k ) =

Original: d elements which are bothh -almost hard clusterings with h ∈ { 0 , . .
Revised: d elements which are bothh -almost hard clusterings with h ∈ { 0 , . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: When n = k and w = r , the solution Z is a permutation matrix (up to a constant factor) andthe number of nonzero elements in it is exactly n, which means that it represents a hard partition
Revised: When n = k and w = r , the solution Z is a permutation matrix (up to a constant factor) andthe number of nonzero elements in it is exactly n which means that it represents a hard partition

Original: Z ∈ Γ( n, n ) .
Revised: Z ∈ Γ( n, n ) .

Original: The proofis the same for W .
Revised: The proof for W is the same.

Original: Proposition 2.
Revised: Proposition 2.

Original: Suppose that the target row and column representative distributions are the same,i.e. r = c with no zero entries.
Revised: Suppose that the target row and column representatives distributions is the samei.e.

Original: Then, given a solution pair Z and W to BCOT, the matrix Q =
Revised: r = c with no zero entries, then given a solution pair Z and W to BCOT, the matrix Q =

Original: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem2 and whose rank is  at most min( rank ( Z ) , rank ( W )) .
Revised: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem 2and whose rank is of at most min( rank ( Z ) , rank ( W )) .

Original: Proof of proposition 2.
Revised: Proof for proposition 2.

Original: From linear algebra, we have that rank ( Q ) ≤min( rank ( Z ) , rank ( diag (1 / r )) , rank ( W )) .
Revised: From linear algebra, we have that rank ( Q ) ≤min( rank ( Z ) , rank ( diag (1 / r )) , rank ( W )) .

Original: Since Z and W cannot have a rank greaterthan k due to their dimension, and since diag (1 / r ) is a full rank matrix due to the assumption that rhas no zero entries, we then have that rank ( Q ) ≤ min( rank ( Z ) , rank ( W )) .
Revised: Since Z and W cannot have a rank greaterthan k due to their dimension and that diag (1 / r ) is a full rank matrix due to the assumption that rhas no zero entries, we then have that rank ( Q ) ≤ min( rank ( Z ) , rank ( W )) .

Original: For a proof that Q is indeed a valid transport map i.e. Q ∈ Π( w , v ) , we refer the reader to proposition2.2 in [29].
Revised: For a proof that Q is indeed a valid transport map i.e. Q ∈ Π( w , v ) , we redirect the reader toproposition 2.2 in [25].

Original: Proposition 3.
Revised: Proposition 3.

Original: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥ 

Revised: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥

Original: + tnk ( n + k ) log( n + k )
Revised: + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k ))), and when using entropicregularization the complexity is O ( tk ∥ B ∥ 

Revised: ( d + k ) log( d + k ))) and when using entropicregularization, the complexity is O ( tk ∥ B ∥

Original: + tkn + tkd ), where t is the number of iterations.
Revised: + tkn + tkd ) where t is the number of iterations.

Original: Proof of proposition 3.
Revised: Proof for proposition 3.

Original: We suppose that L ( B ) is a sparse matrix with the same number of nonzeroentries as B .
Revised: We suppose that L ( B ) is a sparse matrix with the same number ofnonzero entries as B .

Original: The complexity of computing L ( B ) W and L ( B ) W in the BCOT algorithm is
Revised: The complexity of computing L ( B ) W and L ( B ) W in the BCOT algorithm is

Original: O ( k ∥ B ∥ 0 ) .
Revised: O ( k ∥ B ∥ 0 ) .

Original: The optimal transport problem can be formulated and solved as the Earth Mover’s Distance (EMD)problem using any  minimum-cost flow problemalgorithm, such as one of the many variants of thenetwork simplex algorithm.
Revised: The optimal transport problem can be formulated and solved as the Earth Mover’s Distance (EMD)problem using any algorithm for minimum-cost flow problem such as one of the many variantsof network simplex algorithm.

Original: The authors in [28] proposed an algorithm for the network simplexin O
Revised: Authors in [24] proposed an algorithm for the network simplexin O

Original: ( | V || E | log | V | ), where | V | is the number of nodes and | E | is the number of edges in thenetwork.
Revised: ( | V || E | log | V | ) where | V | is the number of nodes and | E | is the number of edges in thenetwork.

Original: In our case, when solving the EMD for Z and cost matrix L ( B ) W , the number of nodes is| V | = n + k and the number of edges is | E | =
Revised: In our case when solving the EMD for Z and cost matrix L ( B ) W , the number of nodes is| V | = n + k and the number of edges is | E | =

Original: nk, which means that the complexity of the operationis
Revised: nk which means that the complexity of the operationis

Original: O ( nk ( n + k ) log( n + k )) .
Revised: O ( nk ( n + k ) log( n + k )) .

Original: When computing the optimal transport map for W , for cost matrix
Revised: When computing the optimal transport map for W , for cost matrix

Original: L ( B ) ⊤ Z , the complexity is O
Revised: L ( B ) ⊤ Z , the complexity is O

Original: ( dk ( d + k ) log( d + k )) .
Revised: ( dk ( d + k ) log( d + k )) .

Original: The overall complexity of the BCOT algorithmis then O ( k ∥ B ∥ 0 ) + tnk ( n + k ) log( n + k )
Revised: The overall complexity of the BCOT algorithmis then O ( k ∥ B ∥ 0 ) + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k ))
Revised: ( d + k ) log( d + k ))

Original: When using entropic regularization the complexity is smaller, sincecomputing the optimal maprequires only a transformation of the inputs matrix, which takes O ( nk ) in the Z computation stepand O ( dk ) for W .
Revised: When using entropic regularization the complexity is smaller since the computation of the optimalmap requires only a transformation of the inputs matrix which takes O ( nk ) in the Z computation stepand O ( dk ) for W .

Original: The ensuing application of the Sinkhorn-Knopp algorithm on the transformedmatrices has complexities O ( tnk ) and O ( tdk ) for Z and W respectively, where t is the numberof iterations necessary.
Revised: The ensuing application of the Sinkhorn-Knopp algorithm on the transformedmatrices has a complexity O ( tnk ) and O ( tdk ) for Z and W respectively, where t is the numberof iterations necessary.

Original: The overall complexity of BCOT λ is then O ( k ∥ B ∥ 0 )
Revised: The overall complexity of BCOT λ is then O ( k ∥ B ∥ 0 )

Original: + tnk + tdk ) , here tincludes the number of iterations of our algorithm as well as that of Sinkhorn-Knopp.
Revised: + tnk + tdk ) , here tincludes the number of iterations of our algorithm as well as that of Sinkhorn-Knopp.