Original: Efficient and Effective Optimal Transport-Based Biclustering
Revised: Efficient and Effective Optimal Transport-Based Biclustering

Original: Anonymous Author(s)

Original: AffiliationAddress

Original: Abstract
Revised: Abstract

Original: Bipartite graphs can be used to model a wide variety of dyadic information such asuser-rating, document-term, and gene-disorder pairs.
Revised: Bipartite graphs can be used to model a wide variety of dyadic information such as user-rating, document-term, and gene-disorder pairs.

Original: Biclustering is an extensionof clustering to the  bipartite graphs that underlie this kind of data.
Revised: Biclustering is an extension of clustering to the underlying bipartite graph induced from this kind of data.

Original: In this paperwe leverage optimal transport (OT), which has gaineda certain momentum inthe machine learning community, in proposing a novel and scalable biclusteringmodel that generalizes several classical biclustering approaches.
Revised: In this paper, we leverage optimal transport (OT) which has gained momentum in the machine learning community to propose a novel and scalable biclustering model that generalizes several classical biclustering approaches.

Original: We performextensive experimentation to show the validity of our approach in relation to other
Revised: We perform extensive experimentation to show the validity of our approach compared to other

Original: OT biclustering algorithms along both dimensions of the dyadic datasets
Revised: OT biclustering algorithms along both dimensions of the dyadic datasets.

Original:  Introduction
Revised:  Introduction

Original: Let G =
Revised: Let G =

Original: ( U, V, E ) be a bipartite graph , which is a graph whose vertices can be divided into twodisjoint sets U = { 1 , 2 , . .
Revised: ( U, V, E ) be a bipartite graph , which is a graph whose vertices can be divided into two disjoint sets U = { 1 , 2 , . .

Original: , | U |} with | U | = n , V = { 1 , 2 , . .
Revised: , | U |} with | U | = n , V = { 1 , 2 , . .

Original: , | V |} with | V | =
Revised: , | V |} with | V | =

Original: d and the set ofedges E where each edge connects a vertex of U to a vertex of V .
Revised: d and the set of edges E where each edge connects a vertex of U to a vertex of V .

Original: The adjacency matrix for this typeof graph has the following structure
Revised: The adjacency matrix for this type of graph has the following structure

Original: [Equation]
Revised: [Equation]

Original: where B of size n × d is called the biadjacency matrix of G, its rows and columns corresponding tothe two sets of vertices; each entry represents an edge between a row and a column.
Revised: where B of size n × d is called the biadjacency matrix of G its rows and columns corresponding to the two sets of vertices; each entry represents an edge between a row and a column.

Original: Biclustering (or
Revised: Biclustering (or

Original: Co-clustering ) is the extention of clustering to this type of graph.
Revised: Co-clustering ) is the extention of clustering to this type of graph.

Original: Following [18], several biclusteringmodels have attempted to solve the problem by viewing B as a two-mode matrix and searching for asimultaneous partition of its rows and columns [8].
Revised: Following [23] several biclustering models have attempted to solve the problem by viewing B as a two-mode matrix and searching for a simultaneous partition of its rows and columns [9].

Original: In this way, biclustering seeks to reveal subsets of
Revised: In this way, biclustering seeks to reveal subsets of

Original: U which exhibit a similar behaviour across a subset of V in matrix B .
Revised: U which exhibit a similar behaviour across a subset of V in matrix B .

Original: Biclustering has been used in a number of different contexts.
Revised: Biclustering has been used in a number of different contexts.

Original: [12] used microarray data to findrelations between genes and conditions, finding that genes with similar functions often clustertogether.
Revised: [12] used microarray data to find relations between genes and conditions, finding that genes with similar functions often cluster together.

Original: [17] applied this paradigm to data from the US Food and Drug Administration reportingsystem in order to identify groups of drugs with adverse effects.
Revised: [22] applied this paradigm to data from the US Food and Drug Administration reporting system in order to identify groups of drugs with adverse effects.

Original: [11] used it to find market segmentsamong tourists so as to enable more effective targeted marketing.
Revised: [11] used it to find market segments among tourists so as to enable more effective targeted marketing.

Original: There have been various otherapplications [8, 33, 16].
Revised: There have been various other applications [9, 38, 21].

Original: Several solutions to the biclustering problem have been proposed in the literature.
Revised: Several solutions to the biclustering problem have been proposed in the literature (see [19]).

Original: [9] used aninformation-theoretic approach to solve the problem by minimizing the difference in mutual information between B and a summary matrix.
Revised: [10] used an information-theoretic approach to solve the problem by minimizing the difference in mutual information between B and a summary matrix; they implicitly assume that the data points are generated from a Poisson latent block model [20].

Original: [2] adapted classical modularity to bipartite networksand then used it to identify modules within them.
Revised: [3] adapted classical modularity to bipartite networks and then used it to identify modules within them.

Original: [35] proposed a biclustering paradigm based onnonnegative tri-factorization of the biadjacency matrix.
Revised: [40] proposed a biclustering paradigm based on nonnegative matrix tri-factorization of the biadjacency matrix.

Original: Recently, Optimal Transport (OT) has taken the machine learning community by storm.
Revised: Recently, Optimal Transport (OT) has taken the machine learning community by storm.

Original: OT hashelped to solve a variety of data mining problems, and biclustering is no exception.
Revised: OT has helped to solve a variety of data mining problems, and biclustering is no exception.

Original: [23] proposed two models for biclustering: a first model, CCOT, which does co-clustering based on the scalingvectors obtained by applying the Sinkhorn-Knopp algorithm on a square subsampled version ofmatrix B , and a second model, CCOT-GW, which uses scaling vectors obtained by computingentropic Gromov-Wasserstein barycenters, and which does not require subsampling.
Revised: [28] proposed two models for biclustering: a first model, CCOT, which does co-clustering based on the scaling vectors obtained by applying the Sinkhorn-Knopp algorithm on a square subsampled version of matrix B , and a second model, CCOT-GW, which uses scaling vectors obtained by computing entropic Gromov-Wasserstein barycenters, and which does not require subsampling.

Original: Then came[34], where the authors did biclustering by minimizing a new metric, COOT, which generalizes the
Revised: Then came [39], where the authors did biclustering by minimizing a new metric, COOT, which generalizes the

Original: Gromov-Wasserstein distance between B and a summary matrix, similarly to what was done in [9].
Revised: Gromov-Wasserstein distance between B and a summary matrix, similarly to what was done in [10].

Original: More specifically, they proposed two new metrics: COOT, together with an entropically regularizedmetric COOT λ .
Revised: More specifically, they proposed two new metrics: COOT, together with an entropically regularized metric COOT λ .

Original: However, both [23] and [34] have certain drawbacks.
Revised: However, both [28] and [39] have certain drawbacks.

Original: First, both algorithms do nottackle the biclustering from the beginning; the co-clusters are deduced at the convergence.
Revised: First, both algorithms do not tackle the biclustering from the beginning; the co-clusters are deduced at the convergence.

Original: Therebybiclustering is a consequence and not a main goal.
Revised: Thereby biclustering is a consequence and not a main goal.

Original: Secondly, they suffer from high computationalcomplexity; CCOT and CCOT-GW also consume large amounts of memory.
Revised: Secondly, they suffer from high computational complexity; CCOT and CCOT-GW also consume large amounts of memory.

Original: Finally, we will see thatthese algorithms are not suited to dyadic sparse data.
Revised: Finally, we will see that these algorithms are not suited to dyadic sparse data.

Original: In this paper, while integrating the biclustering objective from the beginning, we propose a genericframework for biclustering through optimal transport, which generalizes some previous biclusteringapproaches.
Revised: In this paper, while integrating the biclustering objective from the beginning, we propose a generic framework for biclustering through optimal transport, which generalizes some previous biclustering approaches.

Original: We propose two efficient methods for solving this problem: one that gives an almosthard biclustering, and another that gives a fuzzy or soft biclustering through entropic regularization.
Revised: We propose two efficient methods for solving this problem: one that gives an almost hard biclustering, and another that gives a fuzzy or soft biclustering through entropic regularization.

Original: These methods outperform other optimal transport biclustering models, in terms of both documentand term clustering, on several regular and large scale datasets, while being more computationallyand memory efficient.
Revised: These methods outperform other optimal transport biclustering models, in terms of both document and term clustering, on several regular and large scale datasets, while being more computationally and memory efficient.

Original: We emphasize once again that the approach we propose is specifically tailoredto datasets consisting of dyadic data that can be represented using a bipartite graph.
Revised: We emphasize once again that the approach we propose is specifically tailored to datasets consisting of dyadic data.

Original:  Methodology
Revised:  Methodology

Original: Notations In what follows, ∆ n = { p ∈ R n + | (cid:80) ni =1 p i = 1 } denotes the n -dimensional standardsimplex.
Revised: Notations. In what follows, ∆ n = { p ∈ R n + | (cid:80) ni =1 p i = 1 } denotes the n -dimensional standard simplex.

Original: Π( w , v ) = { Z ∈ R n × k + | Z1 = w , Z ⊤ 1 = v } denotes the transportation polytope, wherew ∈ ∆ n and v ∈ ∆ k are the marginals of the joint distribution Z and 1 n is a vector of ones.
Revised: Π( w , v ) = { Z ∈ R n × k + | Z1 = w , Z ⊤ 1 = v } denotes the transportation polytope, where w ∈ ∆ n and v ∈ ∆ k are the marginals of the joint distribution Z and 1 n is a vector of ones.

Original: Matricesare denoted with uppercase boldface letters, and vectors with lowercase boldface letters.
Revised: Matrices are denoted with uppercase boldface letters, and vectors with lowercase boldface letters.

Original: For a matrix
Revised: For a matrix

Original: M , its i -th row is m  and its j -th column is m ′
Revised: M , its i -th row is m i and its j -th column is m ′

Original: j We have that ∥ .
Revised: j We have that ∥ .

Original: ∥ 0 is the 0-norm which returns thenumber of nonzero elements of its argument.
Revised: ∥ 0 is the 0-norm which returns the number of nonzero elements of its argument.

Original: 2.1 Preliminaries
Revised: 2.1 Preliminaries

Original: We first need to introduce exact discrete OT and its entropically regularized counterpart, and showhow biclustering can be posed as an integer program.
Revised: We first need to introduce exact discrete OT and its entropically regularized counterpart, and show how biclustering can be posed as an integer program.

Original: Discrete OT as a linear program.
Revised: Discrete OT as a linear program.

Original: The goal of discrete optimal transport is to find a minimal costtransport map between a source probability distribution w and a target distribution v .
Revised: The goal of discrete optimal transport is to find a minimal cost transport map between a source probability distribution w and a target distribution v .

Original: Here we areinterested in the discrete case of the Kantorovich formulation of OT, that is
Revised: Here we are interested in the discrete case of the Kantorovich formulation of OT, that is

Original: [Equation]
Revised: [Equation]

Original: where M ∈ R n × k is the cost matrix, and m ij quantifies the effort needed to transport a probabilitymass from w i to v j .
Revised: where M ∈ R n × k is the cost matrix, and m ij quantifies the effort needed to transport a probability mass from w i to v j .

Original: Discrete entropy regularized OT.
Revised: Discrete entropy regularized OT.

Original: It has been suggested in the literature [5, 4] that the use ofa regularization such as entropic regularization can lead to better computational and statisticalefficiency.
Revised: It has been suggested in the literature [6, 5] that the use of a regularization such as entropic regularization can lead to better computational and statistical efficiency.

Original: [Equation]
Revised: [Equation]

Original: (3) where H is the entropy defined as H ( Z ) ≜  (cid:80) i,j− z ij log z ij and λ controls the strength of regularization.
Revised: where H is the entropy defined as H ( Z ) ≜ − (cid:80) i,j z ij log z ij and λ controls the strength of regularization.

Original: The computational efficiency comes from the fact that the unique solution of this problem isof the structure K :=− diag ( a ) exp( − M /λ ) diag ( b ) , a rescaled elementwise negative exponentialof the cost M , where a and b are scaling vectors.
Revised: The computational efficiency comes from the fact that the unique solution of this problem is of the structure Z := diag ( a ) exp( − M /λ ) diag ( b ) , a rescaled elementwise negative exponential of the cost M , where a and b are scaling vectors.

Original: These vectors can be found efficiently using the
Revised: These vectors can be found efficiently using the

Original: Sinkhorn-Knopp algorithm.
Revised: Sinkhorn-Knopp algorithm.

Original: Biclustering as an integer program.
Revised: Biclustering as an integer program.

Original: The Block seriation problem [25] consists in finding twopermutation matrices, one for the rows and one for the columns s.t.
Revised: The Block seriation problem [30] consists in finding two permutation matrices, one for the rows and one for the columns s.t.

Original: dense blocks appear along thediagonal of the permuted matrix.
Revised: dense blocks appear along the diagonal of the permuted matrix.

Original: A possible definition of the block seriation problem is as follows:given a matrix B ∈ R n × d s.t b ij gives the strength of the association between row i and column j(such as in the case of a biadjacency matrix, for example), we have
Revised: A possible definition of the block seriation problem is as follows: given a matrix B ∈ R n × d s.t b ij gives the strength of the association between row i and column j (such as in the case of a biadjacency matrix, for example), we have

Original: [Equation]
Revised: [Equation]

Original: A solution Cmatrix is a block diagonal matrix up to a permutation of its rows and columns.
Revised: A solution C is a block diagonal matrix up to a permutation of its rows and columns.

Original: Theblock seriation problem is an integer programming problem and isconsequently NP-hard.
Revised: The block seriation problem is an integer programming problem that is NP-hard.

Original: Oneapproach for solving this problem uses a relaxed version where a rank constraint rank ( C ) ≤ k isadded for k the number of desired biclusters.
Revised: One approach for solving this problem uses a simplified version where a rank constraint rank ( C ) ≤ k is added for k the number of desired biclusters.

Original: Integrating this constraint into 4, we can define a newequivalent problem by low-rank factorization of C , i.e. C = ZW ⊤ , which we formulate as
Revised: Integrating this constraint into (4), we can define a new equivalent problem by low-rank factorization of C , i.e. C = ZW ⊤ , which we formulate as

Original: [Equation]
Revised: [Equation]

Original: where Γ( n, k ) = { Z ∈ { 0 , 1 } n × k | Z1 = 1 } is the set of hard partitions of dimension n × k .
Revised: where Γ( n, k ) = { Z ∈ { 0 , 1 } n × k | Z1 = 1 } is the set of hard partitions of dimension n × k .

Original: A simpleheuristic for solving this problem involves alternatingly solving for Z given W , and vice-versa, usingclassical clustering algorithms, before identifying biclusters through the rearranged matrix C , whichdisplays a block diagonal structure, as shown in figure 1a.
Revised: A simple heuristic for solving this problem involves alternatingly solving for Z given W , and vice-versa, using classical clustering algorithms, before identifying biclusters through the rearranged matrix C , which displays a block diagonal structure, as shown in figure 1a.

Original: The biclusters are identified by groupingtogether the rows and columns that form a block along the diagonal.
Revised: The biclusters are identified by grouping together the rows and columns that form a block along the diagonal.

Original: 2.2 Biclustering using Optimal Transport
Revised: 2.2 Biclustering using Optimal Transport

Original: Here we propose a new biclustering problem based on block seriation and optimal transport.
Revised: Here we propose a new biclustering problem based on block seriation and optimal transport.

Original: For thispurpose we first define what we term an anti-adjacency matrix :
Revised: For this purpose we first define what we term an anti-adjacency matrix .

Revised: Note that a similar concept has been discussed in [41].

Original: Definition 1. (Anti-adjacency matrix)
Revised: Definition 1 (Anti-adjacency matrix)

Original: Given a graph characterized by an adjacency matrix A , wehave a corresponding anti-adjacency matrix A s.t.
Revised: Given a graph characterized by an adjacency matrix A , we have a corresponding anti-adjacency matrix A s.t.

Original: a ij quantifies a discrepancy between node i and j .
Revised: a ij quantifies the discrepancy between node i and j .

Original: We consider a bipartite graph characterized by its biadjacency matrix B = ( b ij ) ∈ R n × d .
Revised: We consider a bipartite graph characterized by its biadjacency matrix B = ( b ij ) ∈ R n × d .

Original: The rowsof B are endowed with weights w ∈ ∆ n and its columns with weights v ∈ ∆ d .
Revised: The rows of B are endowed with weights w ∈ ∆ n and its columns with weights v ∈ ∆ d .

Original: We also considera row exemplar distribution r ∈ ∆ r and a column exemplar distribution c ∈ ∆ c .
Revised: We also consider a row exemplar distribution r ∈ ∆ r and a column exemplar distribution c ∈ ∆ c .

Original: Depending onthe availability of a priori information about the data, these weight vectors can be set to uniformdistributions.
Revised: Depending on the availability of a priori information about the data, these weight vectors can be set to uniform distributions.

Original: Now let its anti-adjacency matrix be B
Revised: Now let its anti-adjacency matrix be B

Original: =
Revised: =

Original: L ( B ) , where L : R n ×
Revised: L ( B ) , where L : R n ×

Original: d → R n × d means that b ij , theassociation between node i and node j , is transformed into a discrepancy measure L ( b ) ij .Z .
Revised: d → R n × d means that b ij , the association between node i and node j , is transformed into a discrepancy measure L ( B ) ij .

Original: Thus,we define the optimal transport block seriation problem as 
Revised: Thus, we define the optimal transport block seriation problem as the following bilinear program:

Original: [Equation]
Revised: [Equation]

Original:  where Z is a transport map (or coupling) between between the row distribution w and the rowexemplar distribution r , and similarly for W w.r.t.
Revised: [Figure] where Z is a transport map (or coupling) between between the row distribution w and the row exemplar distribution r , and similarly for W w.r.t.

Original: the column distribution v and the columnexemplar distribution c .
Revised: the column distribution v and the column exemplar distribution c .

Original: Similar to [34] is an indefinite Bilinear Program and is related to thequadratic assignment problem (QAP)

Original: Inducing a Biclustering using BCOT We will now show how to obtain a partition of the rowsand the columns given a solution pair ( Z , W ) .
Revised: Inducing a biclustering via BCOT. We will now show how to obtain a partition of the rows and the columns given a solution pair ( Z , W ) .

Original: In what follows our aim is to identify an almost-hardclustering couple for rows and columns from the couplings Z and W .[Figure]
Revised: In what follows our aim is to identify an almost-hard clustering couple for rows and columns from the couplings Z and W .

Original: Definition 2. ( h -almost hard clustering)
Revised: Definition 2 ( h -almost hard clustering)

Original: We define an h -almost hard clustering as a clusteringwhose assignment matrix is C ∈ R n × k s.t.
Revised: We define an h -almost hard clustering as a clustering whose assignment matrix is C ∈ R n × k s.t.

Original: ∥
Revised: ∥

Original: C ∥ 

Revised: C ∥

Original: = n + h and for each row c of C we have that∥ c ∥ 0 > 0 .
Revised: = n + h and for each row c of C we have that ∥ c ∥ 0 > 0 .

Original: When h = 0 , we obtain a standard hard clustering with one non-zero element per row.
Revised: When h = 0 , we obtain a standard hard clustering with one non-zero element per row.

Original: Proposition [-1. 
-] For w , v , r and c containing no zeros, the resulting optimal  coupling matrices Zand W  arealways an h -almost hard clustering with h ∈ { 0 , . . .
Revised: Proposition [-1. 
-]1 1 For w , v , r and c containing no zeros, there exists an optimal pair of coupling matrices Z and W that are h -almost hard clusterings with h ∈ { 0 , . . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: Furthermore, when n = k(resp.
Revised: Furthermore, when n = k (resp.

Original: d = k ) and w = r (resp.
Revised: d = k ) and w = r (resp.

Original: v = c ),  Z (resp.
Revised: v = c ), this Z (resp.

Original: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.
Revised: W ) becomes a hard clustering, i.e., Z ∈ Γ( n, n ) (resp.

Original: W ∈ Γ( d, d ) ).
Revised: W ∈ Γ( d, d ) ).

Original: This means that the solutions are already almost a hard partition of the data, since k << n, d .
Revised: This means that the solutions are already almost a hard partition of the data, since k << n, d .

Original: Toobtain a final hard clustering in the strict sense, we assign each row (resp. column) to the onecorresponding to the row of Z (resp.
Revised: To obtain a final hard clustering in the strict sense, we assign each row (resp. column) to the one corresponding to the row of Z (resp.

Original: W ) with the largest value.
Revised: W ) with the largest value.

Original: This should not significantly changethe structure of the solution.
Revised: This should not significantly change the structure of the solution.

Original: Figure 1b provides an illustration: here we see the block diagonalstructure generated by the product of the two coupling matrices C = ZW ⊤ , with its similarity inappearance to the biclustering produced by the hard block seriation 1a, apart from a few nonzeroentries off the block diagonal that are hard to see immediately.
Revised: Figure 1b provides an illustration: here we see the block diagonal structure generated by the product of the two coupling matrices C = ZW ⊤ , with its similarity in appearance to the biclustering produced by the hard block seriation 1a, apart from a few nonzero entries off the block diagonal that are hard to see immediately.

Original: Intuition for BCOT To explain the intuition behind the proposed approach we need to look at howthe problem is solved.
Revised: Intuition for BCOT. To explain the intuition behind the proposed approach we need to look at how the problem is solved.

Original: The optimization procedure as described in algorithm 1 consists in alternatingbetween the computation of an optimal transportation map Z given W and vice versa.
Revised: The optimization procedure as described in algorithm 1 consists in alternating between the computation of an optimal transportation map Z given W and vice versa.

Original: As regardssolving for Z given W , the problem can be rewritten as
Revised: As regards solving for Z given W , the problem can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: This is an optimal transport problem with L ( B ) W as the cost matrix.
Revised: This is an optimal transport problem with L ( B ) W as the cost matrix.

Original: The resulting transportationmap Z can be seen as a kind of row cluster assignment matrix: if z ih > 0 , then row i is assigned tocluster h .
Revised: The resulting transportation map Z can be seen as a kind of row cluster assignment matrix: if z ih > 0 , then row i is assigned to cluster h .

Original: The same holds for W , which can be seen as a column cluster assignment matrix.
Revised: The same holds for W , which can be seen as a column cluster assignment matrix.

Original: Thisalso means that since L ( B ) is the dissimilarity between the rows and the columns, then the costmatrix L ( B ) W represents the dissimilarity between rows and row exemplars (or representativesor centroids).
Revised: This also means that since L ( B ) is the dissimilarity between the rows and the columns, then the cost matrix L ( B ) W represents the dissimilarity between rows and row exemplars (or representatives or centroids).

Original: In particular, L ( B )
Revised: In particular, L ( B )

Original: i w h is the dissimilarity or cost of probability mass transportationbetween row i and row cluster exemplar h .
Revised: i w h is the dissimilarity or cost of probability mass transportation between row i and row cluster exemplar h .

Original: The reasoning is the same for the columns and the optimalcoupling W .
Revised: The reasoning is the same for the columns and the optimal coupling W .

Original: Low-Rank Optimal Transport Biclustering is the main purpose of the approach we proposed, butthere is another interesting use case.
Revised: Low-rank optimal transport. Biclustering is the main purpose of the approach we proposed, but there is another interesting use case.

Original: Proposition 2. In the case where the target row and column representative distributions are thesame, i.e., r = c with no zero entries, then given a solution pair Z and W to BCOT, the matrix Q = Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem2 and whose rank is at most min( rank ( Z ) , rank ( W )) .
Revised: Proposition 2 For equal target row and column representative distributions, i.e., r = c , and containing no zero entries, then given a solution pair Z and W to BCOT, the matrix Q = Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem (2) and whose rank is at most min( rank ( Z ) , rank ( W )) .

Original: Some recent works [15, 31] have suggested that this kind of low-rank regularization is preferableto entropic regularization as regards certain aspects.
Revised: Some recent works [17, 36] have suggested that this kind of low-rank regularization is preferable to entropic regularization as regards certain aspects.

Original: For example, the rank parameter is easier to select, since it has simple bounds (an integer between 1 and n ).
Revised: For example, the rank parameter is easier to select, since it has simple bounds (an integer between 1 and n ).

Original: This may be contrasted with theregularization strength λ in the Sinkhorn algorithm, which is continuous.
Revised: This may be contrasted with the regularization strength λ in the Sinkhorn algorithm, which is continuous.

Original: 2.3 Fuzzy biclustering using regularized Optimal Transport
Revised: 2.3 Fuzzy Biclustering via Regularized Optimal Transport

Original: As previously mentioned, using entropic regularization may be interesting because of its varioususeful features including statistical and computational efficiency.
Revised: As previously mentioned, using entropic regularization may be interesting because of its various useful features including statistical and computational efficiency.

Original: However, another feature of entropicregularization is that the optimal couplings Z and W are dense matrices as a consequence of thestructure of the optimal solution of entropically regularized OT problems.
Revised: However, another feature of entropic regularization is that the optimal couplings Z and W are dense matrices as a consequence of the structure of the optimal solution of entropically regularized OT problems.

Original: We formulate the problemas follows
Revised: We formulate the problem as follows

Original: [Equation]
Revised: [Equation]

Original: where λ Z and λ W are the regularization parameters.
Revised: where λ Z and λ W are the regularization parameters.

Original: Fuzzy Block Seriation We propose a fuzzy variant of the block seriation problem that allows usby extension to define a fuzzy variant for BCOT using entropic regularization.
Revised: Fuzzy block seriation. We propose a fuzzy variant of the block seriation problem that allows us by extension to define a fuzzy variant for BCOT using entropic regularization.

Original: Let the fuzzy blockseriation problem be defined as
Revised: Let the fuzzy block seriation problem be defined as

Original: [Equation]
Revised: [Equation]

Original: where Ω( Z , W ) is some regularization term introduced to make the partition matrices Z and W dense(for example, entropic regularization or low-rank constraints), and Γ s ( n, k )
Revised: where Ω( Z , W ) is some regularization term introduced to make the partition matrices Z and W dense (for example, entropic regularization or low-rank constraints), and Γ s ( n, k )

Original: = { Z ∈ R n × k + | Z1 = 1 }is the set of fuzzy partitions.
Revised: = { Z ∈ R n × k + | Z1 = 1 } is the set of fuzzy partitions.

Original: Intuitively, for a solution pair ( Z , W ) , up to a constant factor, eachentry in the block seriation matrix C =
Revised: Intuitively, for a solution pair ( Z , W ) , up to a constant factor, each entry in the block seriation matrix C =

Original: ZW ⊤ can be seen as the probability of its correspondingrow and column belonging to the same bicluster i.e. c ij =
Revised: ZW ⊤ can be seen as the probability of its corresponding row and column belonging to the same bicluster i.e. c ij =

Original: z i w j = (cid:80) rh 
Revised: z i w j = (cid:80) rh =1 z ih w jh

Original: [Equation]
Revised: = p ( b i , b ′ j )

Original: 161 (cid:80) [Equation] j ∈ h ) .
Revised: = (cid:80) rh =1 p ( b i , b ′ j ∈ h ) .

Original: It is easy to see how problem 9 is related to problem 8 and that the couplingscorresponding to solutions of the problem give the probability that the different rows and columnsbelong to the same biclusters.
Revised: It is easy to see how problem (9) is related to problem (8) and that the couplings corresponding to solutions of the problem give the probability that the different rows and columns belong to the same biclusters.

Original: Figure 1c shows biclusters produced by the solutions of BCOT λ .
Revised: Figure 1c shows biclusters produced by the solutions of BCOT λ .

Original: Similarly to BCOT, a block diagonal structure is formed.
Revised: Similarly to BCOT, a block diagonal structure is formed.

Original: However, there are also several off-blockdiagonal nonzero entries that represent the probabilities of the row-column pairs belonging to thesame biclusters.
Revised: However, there are also several off-block diagonal nonzero entries that represent the probabilities of the row-column pairs belonging to the same biclusters.

Original:  Links to Existing Work
Revised:  Links to Existing Work

Original: Modularity Maximization in bipartite graphs [2] This model is able to co-cluster binary and contingency matrices by directly maximizing an adapted version of the modularity measure traditionallyused for networks.
Revised: Modularity Maximization in Bipartite Graphs [3] This model is able to co-cluster binary and contingency matrices by directly maximizing an adapted version of the modularity measure traditionally used for networks.

Original: The criterion that it optimizes is
Revised: The criterion that it optimizes is

Original: [Equation]
Revised: [Equation]

Original: By setting L ( B ) = − ( B − 1 b .. B11 ⊤ B ) , this problem becomes equivalent to ours; the difference isin the constraints on Z and W .
Revised: By setting L ( B ) = − ( B − 1 b .. B11 ⊤ B ) , this problem becomes equivalent to ours; the difference is in the constraints on Z and W .

Original: BCOT therefore serves as a convex relaxation of this problem.

Original: Modularity-based Sparse Soft Graph Clustering [20] Here the authors proposed a fuzzy variantof the above problem (although in the context of traditional clustering rather than biclustering).
Revised: 3.2 Modularity-Based Sparse Soft Graph Clustering [25]. Here the authors proposed a fuzzy variant of the above problem (although in the context of traditional clustering rather than biclustering).

Original: Solving the problem gives, for each element of the dataset, a probability of that element belonging toa given cluster.
Revised: Solving the problem gives, for each element of the dataset, a probability of that element belonging to a given cluster.

Original: Our proposed entropic regularization variant represents a kind of extension of thisproblem to bipartite graphs.
Revised: Our proposed entropic regularization variant represents a kind of extension of this problem to bipartite graphs.

Original:  Directional co-clustering with a conscience [30] This model makes use of the block von MisesFisher mixture model for co-clustering directional data on the unit-sphere.
Revised: 3.3 Directional Co-clustering with a Conscience [35, 1]. This model makes use of the block von Mises-Fisher mixture model for co-clustering directional data on the unit-sphere.

Original: It optimizes the followingcriterion:
Revised: It optimizes the following criterion:

Original: [Equation]
Revised: [Equation]

Original: In our formulation, if we define L ( B ) = − B and apply cluster size normalization on the optimaltransport plans ˜ Z = Z diag ( Z ⊤ 1 ) − 1 / 2 and ˜ W = W diag ( W ⊤ 1 ) − 1 / 2 after computing Z and Wrespectively in algorithm 1 , we obtain a more general version of the algorithm proposed by theauthors for solving problem 11.
Revised: In our formulation, if we define L ( B ) = − B and apply cluster size normalization on the optimal transport plans ˜ Z = Z diag ( Z ⊤ 1 ) − 1 / 2 and ˜ W = W diag ( W ⊤ 1 ) − 1 / 2 after computing Z and W respectively in algorithm 1 , we obtain a more general version of the algorithm proposed by the authors for solving problem (11).

Original:  Bipartite Correlation Clustering [1]
Revised: 3.4 Bipartite Correlation Clustering [2].

Original: In the case where the cost function results in a completebipartite graph with ’+’ and ’-’ edges with a function
Revised: In the case where the cost function results in a complete bipartite graph with ’+’ and ’-’ edges with a function

Original: [Equation]
Revised: [Equation]

Original: we get what is known as Bipartite Correlation Clustering.
Revised: we get what is known as Bipartite Correlation Clustering.

Original: The solution to this problem maximizes thenumber of agreements, i.e. the number of all ’+’ edges within clusters plus all ’-’ edges distributedacross clusters.
Revised: The solution to this problem maximizes the number of agreements, i.e. the number of all ’+’ edges within clusters plus all ’-’ edges distributed across clusters.

Original:  Optimization and Complexity
Revised:  Optimization and Complexity

Original: [Figure]
Revised: [Figure]

Original: Optimization Since the block seriation problem is NP-hard, computing an exact solution is prohibitive.
Revised: Optimization. Since the block seriation problem is NP-hard, computing an exact solution is prohibitive.

Original: An efficient and widely used heuristic for solving thesekinds of problems involves the use of block coordinatedescent, where row assignments are computed for fixedcolumn assignments, and then vice versa, in alternation.
Revised: An efficient and widely used heuristic for solving these kinds of problems involves the use of block coordinate descent, where row assignments are computed for fixed column assignments, and then vice versa, in alternation.

Original: We express the proposed algorithm in pseudo-code as Alorithm 1.
Revised: We express the proposed algorithm in pseudo-code as algorithm 1.

Original: At each iteration we solve two intermediateoptimal transport problems with cost matrices of dimensions n × k and d × k , since B is generally sparse, and Lcan be defined such that L ( B ) retains a similarly sparsestructure.
Revised: At each iteration we solve two intermediate optimal transport problems with cost matrices of dimensions n × k and d × k , since B is generally sparse, and L can be defined such that L ( B ) retains a similarly sparse structure.

Original: The computation of the intermediate cost matrices L ( B ) W and L ( B ) ⊤ Z is reasonably efficient.
Revised: The computation of the intermediate cost matrices L ( B ) W and L ( B ) ⊤ Z is reasonably efficient.

Original: Wealso observed that the algorithm does not need manyiterations to converge, as shown in figure 2, be it for 
Revised: We also observed that the algorithm does not need many iterations to converge, as shown in figure 2, be it for BCOT or

Original: BCOTor BCOT λ .
Revised: BCOT λ .

Original: Algorithm 1: BCOT Proposition 3. The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥ 

Revised: Proposition 3 The computational complexity of the BCOT algorithm 1 when using an exact OT solver is O ( tk ∥ B ∥

Original: + tnk ( n + k ) log( n + k )
Revised: + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k ))) , and when using entropicregularization the complexity is O ( tk ∥ B ∥ 

Revised: ( d + k ) log( d + k ))) , and when using entropic regularization the complexity is O ( tk ∥ B ∥

Original: + tkn + tkd ) , where t is the number of iterations.
Revised: + tkn + tkd ) , where t is the number of iterations.

Original: In table 1, we report the computational and spatial complexities of the different biclustering approaches.
Revised: In table 1, we report the computational and spatial complexities of the different biclustering approaches.

Original: Our model has the same spatial complexity as the COOT variants and a better complexitythan CCOT variants.
Revised: Our model has the same spatial complexity as the COOT variants and a better complexity than CCOT variants.

Original: As regards the computational complexity, our model should in most cases befaster, and our experiments support this conjecture.
Revised: As regards the computational complexity, our model should in most cases be faster with sparse data, and our experiments support this conjecture.

Original: For reproducibility, we publicly release our code 
Revised: For reproducibility, we publicly release our code 2 .

Revised: W while not converged do

Revised: Z ← arg OT ( L ( B ) W , w , r ) ;

Revised: W ← arg OT (cid:0) L ( B )

Revised: ⊤ Z v , c

Original:  [Table]
Revised: ; end Generate [Table]

Original:  Experiments
Revised:  Experiments

Original: We ran experiments using term-document matrices.
Revised: We ran experiments using term-document matrices.

Original: The benefit of using biclustering on this kindof data is that the resulting biclusters contain both documents and the words that characterize them,which is helpful in interpreting the clustering of the documents.
Revised: The benefit of using biclustering on this kind of data is that the resulting biclusters contain both documents and the words that characterize them, which is helpful in interpreting the clustering of the documents.

Revised: Additional experiments over synthetic and gene expression data are available in the appendix.

Original: 5.1 Datasets
Revised: 5.1 Datasets

Original: We evaluate BCOT in relation to six benchmark document-term datasets: ACM, DBLP, PubMed,
Revised: We evaluate BCOT in relation to six benchmark document-term datasets: ACM, DBLP, PubMed,

Original: Wiki, Ohscal [Table] and 20 Newsgroups.
Revised: Wiki, Ohscal, and 20 Newsgroups.

Original: Their characteristics are shown in Table 2.
Revised: Their characteristics are shown in Table 2.

Original: ACM, DBLP, Pubmedand Wiki are attributed networks from which we use only the node-level features that correspond toterm-document matrices.
Revised: ACM, DBLP, Pubmed and Wiki are attributed networks from which we use only the node-level features that correspond to term-document matrices.

Original: We also selected the Ohscal collection and 20 Newsgroups as large-scaledocument-term matrices to serve as computational efficiency benchmarks.
Revised: We also selected the Ohscal collection and 20 Newsgroups as large-scale document-term matrices to serve as computational efficiency benchmarks.

Original: [Table]
Revised: [Table]

Original: 5.2 Experimental setup
Revised: 5.2 Experimental Setup

Original: In our experiments we define the loss function as L ( B ) = − c B , where c is selected from { 1 , k, d, n } .
Revised: In our experiments we define the loss function as L ( B ) = − c B , where c is selected from { 1 , k, d, n } .

Original: For BCOT λ , the regularization parameter lambda is selected from { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 , 10 } .
Revised: For BCOT λ , the regularization parameter lambda is selected from { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 , 10 } .

Original: The best hyper-parameters are those that minimize the number of empty clusters.
Revised: The best hyper-parameters are those that minimize the number of empty clusters.

Original: In the case of ties,we select according to the value of the Davies-Bouldin index of the partition [6].
Revised: In the case of ties, we select according to the value of the Davies-Bouldin index of the partition [7].

Original: Random restarts arenot used for any of the algorithms, including k -means.
Revised: Random restarts are not used for any of the algorithms, including k -means.

Original: We use the implementation provided by theauthors for CCOT, CCOT λ and CCOT-GW.
Revised: We use the implementation provided by the authors for CCOT, CCOT λ and CCOT-GW.

Original: The code for CCOT was not available, and so we had toimplement it based on the code for CCOT-GW.
Revised: The code for CCOT was not available, and so we had to implement it based on the code for CCOT-GW.

Original: All the reported figures are the averages of 10 runs.
Revised: All the reported figures are the averages of 10 runs.

Original: All the experiments were performed on the same machine with an Intel(R) Xeon(R) CPU and 12GB
Revised: All the experiments were performed on the same machine with an Intel(R) Xeon(R) CPU and 12GB

Original: RAM.
Revised: RAM.

Original: For OT solvers we made use of the POT package [14].
Revised: For OT solvers we made use of the POT package [16].

Original: 5.3 Document clustering
Revised: 5.3 Document Clustering

Original: Metrics and algorithms
Revised: Metrics.

Original: Here, the evaluation isstraightforward. We use three popular clusteringmetrics: Here, the evaluation is straightforward, we adopt three popular clustering metrics: clusteringaccuracy (ACC), normalized mutual information (NMI) [3], adjusted rand index (ARI) [21].
Revised: Here, the evaluation is straightforward, we adopt three popular clustering metrics: clustering accuracy (CA), normalized mutual information (NMI) [4], adjusted rand index (ARI) [26].

Original: Inaddition to the OT biclustering algorithms we perform experiments with k -means, informationtheoretic co-clustering (ITCC) [9] and orthogonal matrix non-negative tri-factorization (ONMTF)[10] suited for document-term matrices.

Original: [Table]
Revised: [Table]

Original: Performance Document clustering results on ACM, DBLP, PubMed and Wiki are given in table 3for the three metrics.
Revised: Performance. Document clustering results on ACM, DBLP, PubMed and Wiki are given in tablefor the three metrics.

Original: In all cases the best result is obtained either by BCOT or by BCOT λ .
Revised: In all cases the best result is obtained either by BCOT or by BCOT λ .

Original: Moreover,on Wiki, BCOT λ gives competitive results when compared with state-of-the-art attributed graphclustering methods such as [37], despite not having access to the graph structure information in the Wiki citation network.
Revised: Moreover, on Wiki, BCOT λ gives competitive results when compared with state-of-the-art attributed graph clustering methods presented in [15], despite not having access to the graph structure information in the Wiki citation network.

Original: Efficiency
Revised: Efficiency.

Original: Figure 3 plots the document clustering performance (accuracy over training time) of thedifferent methods on the two large-scale document-term matrices 20 Newsgroup and Ohscal.
Revised: Figure 3 plots the document clustering performance (accuracy against training time) of the different methods on the two large-scale document-term matrices 20 Newsgroup and Ohscal.

Original: BCOToffers the best accuracy while BCOT λ is fastest method on both datasets.
Revised: BCOT offers the best accuracy while BCOT λ is fastest method on both datasets.

Original: We see that for both BCOTand COOT, the entropic-regularized versions outspeed their exact counterparts and that CCOT suffersfrom very high computation times, due mainly to the fact that this method requires pairwise distancematrices to be computed on the rows and columns.
Revised: We see that for both BCOT and COOT, the entropic-regularized versions outspeed their exact counterparts and that CCOT suffers from very high computation times, due mainly to the fact that this method requires pairwise distance matrices to be computed on the rows and columns.

Original: [Figure]
Revised: [Figure]

Original: 5.4 Term Clustering
Revised: 5.4 Term Clustering

Original: Metrics Unlike document clustering, there is no ground truth partition for terms, so we need to findanother way of evaluating term clustering results.
Revised: Metrics. Unlike document clustering, there is no ground truth partition for terms, so we need to find another way of evaluating term clustering results.

Original: One generally acceptable technique is to analyse thesemantic coherence of the clusters obtained.
Revised: One generally acceptable technique is to analyse the semantic coherence of the clusters obtained.

Original: To this end we introduce a metric based on point mutualinformation (PMI).
Revised: To this end we introduce a metric based on point mutual information (PMI).

Original: PMI is a frequently used information-theoretic metric for quantifying the relationship between pairs of discrete random variable outcomes.
Revised: PMI is a frequently used information-theoretic metric for quantifying the relationship between pairs of discrete random variable outcomes.

Original: The PMI measure was chosen because priorresearch [27] has shown that it is closely associated with human judgements in determining word relatedness.
Revised: The PMI measure was chosen because prior research [32] has shown that it is closely associated with human judgements in determining word relatedness.

Revised: The PMI between the terms w i and w j is calculated as

Original: [Equation]
Revised: [Equation]

Original: The PMI between the terms w i and w j is calculated as in (13).

Original: In the context of term clustering, giventhe word co-occurrence matrix K = B ⊤ B , the PMI is estimated as in(14).
Revised: In the context of term clustering, given the word co-occurrence matrix K = B ⊤ B , the PMI is estimated as in

Revised: [Equation]

Original: To evaluate a partition ofterms P , we propose a metric based on intra and inter PMI metrics as follows:
Revised: To evaluate a partition of terms P , we propose a metric based on intra and inter PMI metrics as follows:

Original: [Table]

Original: [Equation]
Revised: [Equation]

Original: In this way [Table] a good clustering should reveal a high intra-cluster semantic relatedness, correspondingto higher PMI values.
Revised: In this way, a good clustering should reveal a high intra-cluster semantic relatedness, corresponding to higher PMI values.

Original: Using the intra and inter PMIs [Table] we propose the following coherence indexcoherence
Revised: Using the intra and inter PMIs, we propose the following coherence index

Original: [Equation]
Revised: [Equation]

Original: Our reasoning is this: the greater the semantic proximity between terms in the same clusters, and thegreater the sematic distance between terms in different clusters, the higher the value of coherence .
Revised: Our reasoning is this: the greater the semantic proximity between terms in the same clusters, and the greater the sematic distance between terms in different clusters, the higher the value of coherence .

Original: Results Since there is no ground truth number of term clusters, we use the cluster number estimations produced by CCOT-GW for all the other models so that it is easy to compare coherence valuesbetween them.
Revised: Results. Since there is no ground truth number of term clusters, we use the cluster number estimations produced by CCOT-GW for all the other models so that it is easy to compare coherence values between them.

Original: Comparisons based on different numbers of clusters would favor the model usingthe larger number of clusters.
Revised: Comparisons based on different numbers of clusters would favor the model using the larger number of clusters.

Original: Table 4 shows the coherences obtained across the different datasetsusing our approach, along with those of the baselines.
Revised: Table 4 shows the coherences obtained across the different datasets using our approach, along with those of the baselines.

Original: It is clear that BCOT succeeds in capturingmore semantics than the other approaches since, whatever the dataset, one or other of the two BCOTvariants gives the highest coherence.
Revised: It is clear that BCOT succeeds in capturing more semantics than the other approaches since, whatever the dataset, one or other of the two BCOT variants gives the highest coherence [Table] [Table]

Original: 5.5 Statistical significance
Revised: 5.5 Statistical Significance

Original: We did a Nemenyi post-hoc test [26, 7] with a confidence level of 90% on the document andterm clustering results that we obtained, to determine whether our model outperforms other OTbiclustering approaches in a statistically significant way.
Revised: We performed a Nemenyi post-hoc test [31, 8] with a confidence level of 90% on the document and term clustering results that we obtained, to determine whether our model outperforms other OT biclustering approaches in a statistically significant way.

Original: To conduct this test we generated 20performance rankings of the OT biclustering models based on their performance for each dataset andquality metric pair for both document and term clustering.
Revised: To conduct this test we generatedperformance rankings of the OT biclustering models based on their performance for each dataset and quality metric pair for both document and term clustering.

Original: Figure 4 shows the results of the test.
Revised: Figure 4 shows the results of the test.

Original: Wesee that two differently performing groups were identified, one comprising BCOT and BCOT λ andgiving better results than the other group comprising the remaining COOT and CCOT variants.
Revised: We see that two differently performing groups were identified, one comprising BCOT and BCOT λ and giving better results than the other group comprising the remaining COOT and CCOT variants.

Original: Thisindicates that with this specific number of datasets and metrics the test was unable to tell COOT and
Revised: This indicates that with this specific number of datasets and metrics the test was unable to tell COOT and

Original: CCOT apart in a statistically significant way.
Revised: CCOT apart in a statistically significant way.

Original: [Figure]
Revised: [Figure]

Original:  Conclusion
Revised:  Conclusion

Original: Clustering and biclustering through optimal transport is still at a nascent stage, with many challengesremaining unsolved.
Revised: Clustering and biclustering through optimal transport is still at a nascent stage, with many challenges remaining unsolved.

Original: This paper introduces a novel problem for biclustering using optimal transport that takes into account the sparse nature of certain types of dyadic data such document-term matrices,to enable more computationally efficient resolution.
Revised: This paper introduces a novel problem for biclustering using optimal transport that takes into account the sparse nature of certain types of dyadic data such document-term matrices, to enable more computationally efficient resolution.

Original: The problem is posed as a bilinear programthat we solve using an efficient  coordinate descent algorithm that takes into account the sparsenature of certain types of dyadic data such as document-term matrices.
Revised: The problem is posed as a bilinear program that we solve using an efficient block coordinate descent algorithm to find a vertex solution.

Original: Experiments on a numberof document-term datasets suggest that the proposed approach does a good job in finding clustersthat correspond to ground truth document classes, while generating semantically coherent partitionsfor the terms.
Revised: Experiments on a number of document-term datasets suggest that the proposed approach does a good job in finding clusters that correspond to ground truth document classes, while generating semantically coherent partitions for the terms.

Original: In this setting, our model outperforms recent OT biclustering methods by a significantmargin, while being more computationally efficient.Checklist
Revised: In this setting, our model outperforms recent OT biclustering methods by a significant margin, while being more computationally efficient.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? [Yes]
Revised: Acknowledgments :

Original: (b) Did you describe the limitations of your work?

Original: [Yes] Last paragraph of the introduction

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
Revised: This work has been funded by the Association Nationale de la Recherche et de la

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [Yes]

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,

Original: how they

Original: were chosen)?

Original: [Yes]

Original: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-

Original: ments multiple times)? [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: Technologie (ANRT), Informatique Caisse des Dépôts et Consignations (ICDC), and Idex-Spectrans of Université Paris Cité.

Original: [Yes]

Original: If you are using existing

Original: assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a)

Original: If your work uses existing assets, did you citep the creators?

Original: [Yes](

Original: b)

Original: Did you mention the license of the assets?

Original: [No](c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes](

Original: d

Original: ) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?

Original: [N/A] All assets used are publicly available(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?

Original: [N/A]5.

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A](b)

Original: Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A](c)

Original: Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Original: [N/A]

Original: Appendix A Proofs
Revised: Appendix A Proofs

Original: Proposition 1.
Revised: Proposition 1.

Original: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Zand W are always an h -almost hard clustering with h ∈ { 0 , . . .
Revised: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Z and W are always an h -almost hard clustering with h ∈ { 0 , . . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: Furthermore, when n = k(resp.
Revised: Furthermore, when n = k (resp.

Original: d = k ) and w = r (resp.
Revised: d = k ) and w = r (resp.

Original: v = c ), Z (resp.
Revised: v = c ), Z (resp.

Original: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.
Revised: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.

Original: W ∈ Γ( d, d ) ).
Revised: W ∈ Γ( d, d ) ).

Original: Proof of proposition 1.
Revised: Proof for proposition 1.

Original: Problem 2 is a bounded linear program, since Π( w , v ) is a polytope, i.e. abounded polyhedron.
Revised: Problem (2) is a bounded linear program since Π( w , v ) is a polytope i.e. a bounded polyhedron.

Original: The fundamental theorem of linear programming states that if the feasible setis non-empty then the solution lies at the limits of the feasible region.
Revised: The fundamental theorem of linear programming states that if the feasible set is non-empty then the solution lies in the extremity of the feasible region.

Original: This means that a solution Zto problem 2 is an outermost point of Π( w , v ) .
Revised: This means that a solution Z to problem (2) is an extreme point of Π( w , v ) .

Original: We have that the outermost points of Π( w , v ) canhave at most n + d − 1 nonzero elements.
Revised: We have that the extreme points of Π( w , v ) can have at most n + d − 1 nonzero elements.

Original: To prove this we have to show thatin the bipartite graphinduced bythe biadjacency matrix Z , the solution to the optimal transport problem has no cycles.
Revised: To prove this we have to show that the bipartite graph induced by biadjacency matrix Z , the solution to the optimal transport problem has no cycles.

Original: The maximum number of edges in an acyclic graph is | V | − 1 where | V | is the number of nodesin the graph.
Revised: The maximum number of edges in an acyclic graph is | V | − 1 where | V | is the number of nodes in the graph.

Original: Since the number of edges in the bipartite graph induced by biadjacency matrix Z isn + d − 1 , the matrix Z cannot have more than n + d − 1 nonzero entries.
Revised: Since the number of edges in the bipartite graph induced by biadjacency matrix Z is n + d − 1 , the matrix Z can not have more than n + d − 1 nonzero entries.

Original: For a detailed proof seeproposition 3.3 in [29].
Revised: For a detailed proof see proposition 3.3 in [34].

Original: We also have to show that for probability measures w and v that have no zero probability events,there are at least max( n, d )  nonzero elements in Z .
Revised: We also have to show that for probability measures w and v that have no zero probability events, there is at minimum max( n, d ) number of nonzero elements in Z .

Original: This is straightforwardto show: since w and vcontain no zeros, there will always be at least one nonzero element in every row and column of Zthat represents some transfer of mass between elements of w and v .
Revised: This is straightforward since w and v contain no zeros, there will always be at least one nonzero element in every row and column of Z that represents some transfer of mass between elements of w and v .

Revised: BCOT is a bilinear program that has a finite global solution which means that there exists at least one optimal solution pair ( Z , W ) such that Z is an extreme point of Π( w , r ) and W is an extreme point of Π( v , c ) (theorem 1 in [27]).

Original: In the case of BCOT, we then have that Z has at most n + k − 1 and at least max( n, k ) =
Revised: We then have that, For BCOT, Z has at most n + k − 1 and at least max( n, k ) =

Original: n nonzeroentries and that W has at most d + k − 1 and at least max( d, k ) =
Revised: n nonzero entries and that W has at most d + k − 1 and at least max( d, k ) =

Original: d elements which are bothh -almost hard clusterings with h ∈ { 0 , . .
Revised: d elements which are both h -almost hard clusterings with h ∈ { 0 , . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: When n = k and w = r , the solution Z is a permutation matrix (up to a constant factor) andthe number of nonzero elements in it is exactly n, which means that it represents a hard partition
Revised: When n = k and w = r , the solution Z is a permutation matrix (up to a constant factor) and the number of nonzero elements in it is exactly n which means that it represents a hard partition

Original: Z ∈ Γ( n, n ) .
Revised: Z ∈ Γ( n, n ) .

Original: The proof is the same for W .
Revised: The proof is the same for W .

Original: Proposition 2.
Revised: Proposition 2.

Original: Suppose that the target row and column representative distributions are the same,i.e. r = c with no zero entries.
Revised: Suppose that the target row and column representative distributions are the same, i.e. r = c with no zero entries.

Original: Then, given a solution pair Z and W to BCOT, the matrix
Revised: Then, given a solution pair Z and W to BCOT, the matrix

Original: Q =
Revised: Q =

Original: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem2 and whose rank is at most min( rank ( Z ) , rank ( W )) .
Revised: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport map that is a solution to problem (2) and whose rank is at most min( rank ( Z ) , rank ( W )) .

Original: Proof of proposition 2.
Revised: Proof of proposition 2.

Original: From linear algebra, we have that rank ( Q ) ≤min( rank ( Z ) , rank ( diag (1 / r )) , rank ( W )) .
Revised: From linear algebra, we have that rank ( Q ) ≤ min( rank ( Z ) , rank ( diag (1 / r )) , rank ( W )) .

Original: Since Z and W cannot have a rank greaterthan k due to their dimension, and since diag (1 / r ) is a full rank matrix due to the assumption that rhas no zero entries, we then have that rank ( Q ) ≤ min( rank ( Z ) , rank ( W )) .
Revised: Since Z and W cannot have a rank greater than k due to their dimension, and since diag (1 / r ) is a full rank matrix due to the assumption that r has no zero entries, we then have that rank ( Q ) ≤ min( rank ( Z ) , rank ( W )) .

Original: For a proof that Q is indeed a valid transport map i.e. Q ∈ Π( w , v ) , we refer the reader to proposition2.2 in [29].
Revised: For a proof that Q is indeed a valid transport map i.e. Q ∈ Π( w , v ) , we refer the reader to proposition 2.2 in [34].

Original: Proposition 3.
Revised: Proposition 3.

Original: The computational complexity of the BCOT algorithm 1 when using an exact OTsolver is O ( tk ∥ B ∥ 

Revised: The computational complexity of the BCOT algorithm 1 when using an exact OT solver is O ( tk ∥ B ∥

Original: + tnk ( n + k ) log( n + k )
Revised: + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k ))) , and when using entropicregularization the complexity is O ( tk ∥ B ∥ 

Revised: ( d + k ) log( d + k ))) , and when using entropic regularization the complexity is O ( tk ∥ B ∥

Original: + tkn + tkd ) , where t is the number of iterations.
Revised: + tkn + tkd ) , where t is the number of iterations.

Original: Proof of proposition 3.
Revised: Proof of proposition 3.

Original: We suppose that L ( B ) is a sparse matrix with the same number of nonzeroentries as B .
Revised: We suppose that L ( B ) is a sparse matrix with the same number of nonzero entries as B .

Original: The complexity of computing L ( B ) W and L ( B ) W in the BCOT algorithm is
Revised: The complexity of computing L ( B ) W and L ( B ) W in the BCOT algorithm is

Original: O ( k ∥ B ∥ 0 ) .
Revised: O ( k ∥ B ∥ 0 ) .

Original: The optimal transport problem can be formulated and solved as the Earth Mover’s Distance (EMD)problem using any minimum-cost flow problem algorithm, such as one of the many variants of thenetwork simplex algorithm.
Revised: The optimal transport problem can be formulated and solved as the Earth Mover’s Distance (EMD) problem using any minimum-cost flow problem algorithm, such as one of the many variants of the network simplex algorithm.

Original: The authors in [28] proposed an algorithm for the network simplexin O ( | V || E | log | V | ) , where | V | is the number of nodes and | E | is the number of edges in thenetwork.
Revised: The authors in [33] proposed an algorithm for the network simplex in O ( | V || E | log | V | ) , where | V | is the number of nodes and | E | is the number of edges in the network.

Original: In our case, when solving the EMD for Z and cost matrix L ( B ) W , the number of nodes is| V | = n + k and the number of edges is | E | =
Revised: In our case, when solving the EMD for Z and cost matrix L ( B ) W , the number of nodes is | V | = n + k and the number of edges is | E | =

Original: nk , which means that the complexity of the operationis O ( nk ( n + k ) log( n + k )) .
Revised: nk , which means that the complexity of the operation is O ( nk ( n + k ) log( n + k )) .

Original: When computing the optimal transport map for W , for cost matrix
Revised: When computing the optimal transport map for W , for cost matrix

Original: L ( B ) ⊤ Z , the complexity is O
Revised: L ( B ) ⊤ Z , the complexity is O

Original: ( dk ( d + k ) log( d + k )) .
Revised: ( dk ( d + k ) log( d + k )) .

Original: The overall complexity of the BCOT algorithmis then O ( k ∥ B ∥ 0 ) + tnk ( n + k ) log( n + k )
Revised: The overall complexity of the BCOT algorithm is then O ( k ∥ B ∥ 0 ) + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k ))
Revised: ( d + k ) log( d + k ))

Original: When using entropic regularization the complexity is smaller, since computing the optimal maprequires only a transformation of the inputs matrix, which takes O ( nk ) in the Z computation stepand O ( dk ) for W .
Revised: When using entropic regularization the complexity is smaller, since computing the optimal map requires only a transformation of the inputs matrix, which takes O ( nk ) in the Z computation step and O ( dk ) for W .

Original: The ensuing application of the Sinkhorn-Knopp algorithm on the transformedmatrices has complexities O ( tnk ) and O ( tdk ) for Z and W respectively, where t is the numberof iterations necessary.
Revised: The ensuing application of the Sinkhorn-Knopp algorithm on the transformed matrices has complexities O ( tnk ) and O ( tdk ) for Z and W respectively, where t is the number of iterations necessary.

Original: The overall complexity of BCOT λ is then O ( k ∥ B ∥ 0 )
Revised: The overall complexity of BCOT λ is then O ( k ∥ B ∥ 0 )

Original: + tnk + tdk ) , here tincludes the number of iterations of our algorithm as well as that of Sinkhorn-Knopp.
Revised: + tnk + tdk ) , here t includes the number of iterations of our algorithm as well as that of Sinkhorn-Knopp.

Revised: Appendix B Additional Experiments B.1 Experiments on Synthetic Data

Revised: Datasets.

Revised: As datasets with labels along both rows and columns are unavailable, we use synthetic data as in [28, 39].

Revised: Their structure is shown in figure 5, while their characteristics are reported in table 5.

Revised: [Figure]

Revised: [Figure]

Revised: Biclusters

Revised: Sizes

Revised: Sparse

Revised: Structure

Revised: [Figure]

Revised: Metrics.

Revised: From row π r and column π c clusters, we use the Co-Clustering Accuracy (CCA) proposed by [18] to compare two pairs of partitions.

Revised: CCA is defined from Clustering Accuracy (CA) associated to π r and π c in comparison with the true row and column clusters; it is given by

Revised: [Equation]

Revised: Results.

Revised: We report the biclustering performance on the synthetic datasets in table 6.

Revised: At least one of our models finds the perfect partition in all cases.

Revised: These tests additionally allow us to show the utility of the the row cluster distribution r and column cluster distribution c .

Revised: The use of these ground truth distributions resulted in an increase of 19.5 and 4.2 points for BCOT on C and D, and an increase of 0.3 and decrease of 0.8 for BCOT λ on C and D.

Revised: [Table]

Revised: B.2 Experiments on Gene Expression Data

Revised: Datasets.

Revised: The gene-expression matrices used are the Cumida Breast Cancer and Leukemia datasets.

Revised: Their characteristics are shown in Table 7.

Revised: [Table]

Revised: Metrics.

Revised: The metrics are the same as for document clustering.

Revised: Performance In table 8, we report results on the two micro-array datasets, BCOT λ has the best performance on both of them.

Revised: [Table]