Original: Efficient and Effective Optimal Transport-Based Biclustering
Revised: Efficient and Effective Optimal Transport-Based Biclustering

Original: Abstract
Revised: Abstract

Original: Bipartite graphs can be used to model a wide variety of dyadic information such as user-rating, document-term, and gene-disorder pairs.
Revised: Bipartite graphs can be used to model a wide variety of dyadic information such as user-rating, document-term, and gene-disorder pairs.

Original: Biclustering is an extension of clustering to the underlying bipartite graph induced from this kind of data.
Revised: Biclustering is an extension of clustering to the underlying bipartite graph induced from this kind of data.

Original: In this paper, we leverage optimal transport (OT) which has gained momentum in the machine learning community to propose a novel and scalable biclustering model that generalizes several classical biclustering approaches.
Revised: In this paper, we leverage optimal transport (OT) which has gained momentum in the machine learning community to propose a novel and scalable biclustering model that generalizes several classical biclustering approaches.

Original: We perform extensive experimentation to show the validity of our approach compared to other
Revised: We perform extensive experimentation to show the validity of our approach compared to other

Original: OT biclustering algorithms along both dimensions of the dyadic datasets.
Revised: OT biclustering algorithms along both dimensions of the dyadic datasets.

Original:  Introduction
Revised:  Introduction

Original: Let G =
Revised: Let G =

Original: ( U, V, E ) be a bipartite graph , which is a graph whose vertices can be divided into two disjoint sets U = { 1 , 2 , . .
Revised: ( U, V, E ) be a bipartite graph , which is a graph whose vertices can be divided into two disjoint sets U = { 1 , 2 , . .

Original: , | U |} with | U | = n , V = { 1 , 2 , . .
Revised: , | U |} with | U | = n , V = { 1 , 2 , . .

Original: , | V |} with | V | =
Revised: , | V |} with | V | =

Original: d and the set of edges E where each edge connects a vertex of U to a vertex of V .
Revised: d and the set of edges E where each edge connects a vertex of U to a vertex of V .

Original: The adjacency matrix for this type of graph has the following structure
Revised: The adjacency matrix for this type of graph has the following structure

Original: [Equation]
Revised: [Equation]

Original: where B of size n × d is called the biadjacency matrix of G its rows and columns corresponding to the two sets of vertices; each entry represents an edge between a row and a column.
Revised: where B of size n × d is called the biadjacency matrix of G its rows and columns corresponding to the two sets of vertices; each entry represents an edge between a row and a column.

Original: Biclustering (or
Revised: Biclustering (or

Original: Co-clustering ) is the extention of clustering to this type of graph.
Revised: Co-clustering ) is the extention of clustering to this type of graph.

Original: Following [23] several biclustering models have attempted to solve the problem by viewing B as a two-mode matrix and searching for a simultaneous partition of its rows and columns [9].
Revised: Following [21] several biclustering models have attempted to solve the problem by viewing B as a two-mode matrix and searching for a simultaneous partition of its rows and columns [9].

Original: In this way, biclustering seeks to reveal subsets of
Revised: In this way, biclustering seeks to reveal subsets of

Original: U which exhibit a similar behaviour across a subset of V in matrix B .
Revised: U which exhibit a similar behaviour across a subset of V in matrix B .

Original: Biclustering has been used in a number of different contexts.
Revised: Biclustering has been used in a number of different contexts.

Original: [12] used microarray data to find relations between genes and conditions, finding that genes with similar functions often cluster together.
Revised: [12] used microarray data to find relations between genes and conditions, finding that genes with similar functions often cluster together.

Original: [22] applied this paradigm to data from the US Food and Drug Administration reporting system in order to identify groups of drugs with adverse effects.
Revised: [20] applied this paradigm to data from the US Food and Drug Administration reporting system in order to identify groups of drugs with adverse effects.

Original: [11] used it to find market segments among tourists so as to enable more effective targeted marketing.
Revised: [11] used it to find market segments among tourists so as to enable more effective targeted marketing.

Original: There have been various other applications [9, 38, 21].
Revised: There have been various other applications [9, 33, 19].

Original: Several solutions to the biclustering problem have been proposed in the literature (see [19]).
Revised: Several solutions to the biclustering problem have been proposed in the literature (see [17]).

Original: [10] used an information-theoretic approach to solve the problem by minimizing the difference in mutual information between B and a summary matrix; they implicitly assume that the data points are generated from a Poisson latent block model [20].
Revised: [10] used an information-theoretic approach to solve the problem by minimizing the difference in mutual information between B and a summary matrix; they implicitly assume that the data points are generated from a Poisson latent block model [18].

Original: [3] adapted classical modularity to bipartite networks and then used it to identify modules within them.
Revised: [3] adapted classical modularity to bipartite networks and then used it to identify modules within them.

Original: [40] proposed a biclustering paradigm based on nonnegative matrix tri-factorization of the biadjacency matrix.
Revised: [35] proposed a biclustering paradigm based on nonnegative matrix tri-factorization of the biadjacency matrix.

Original: Recently, Optimal Transport (OT) has taken the machine learning community by storm.
Revised: Recently, Optimal Transport (OT) has taken the machine learning community by storm.

Original: OT has helped to solve a variety of data mining problems, and biclustering is no exception.
Revised: OT has helped to solve a variety of data mining problems, and biclustering is no exception.

Original: [28] proposed two models for biclustering: a first model, CCOT, which does co-clustering based on the scaling vectors obtained by applying the Sinkhorn-Knopp algorithm on a square subsampled version of matrix B , and a second model, CCOT-GW, which uses scaling vectors obtained by computing entropic Gromov-Wasserstein barycenters, and which does not require subsampling.
Revised: [25] proposed two models for biclustering: a first model, CCOT, which does co-clustering based on the scaling vectors obtained by applying the Sinkhorn-Knopp algorithm on a square subsampled version of matrix B , and a second model, CCOT-GW, which uses scaling vectors obtained by computing entropic Gromov-Wasserstein barycenters, and which does not require subsampling.

Original: Then came [39], where the authors did biclustering by minimizing a new metric, COOT, which generalizes the
Revised: Then came [34], where the authors did biclustering by minimizing a new metric, COOT, which generalizes the

Original: Gromov-Wasserstein distance between B and a summary matrix, similarly to what was done in [10].
Revised: Gromov-Wasserstein distance between B and a summary matrix, similarly to what was done in [10].

Original: More specifically, they proposed two new metrics: COOT, together with an entropically regularized metric COOT λ .
Revised: More specifically, they proposed two new metrics: COOT, together with an entropically regularized metric COOT λ .

Original: However, both [28] and [39] have certain drawbacks.
Revised: However, both [25] and [34] have certain drawbacks.

Original: First, both algorithms do not tackle the biclustering from the beginning; the co-clusters are deduced at the convergence.
Revised: First, both algorithms do not tackle the biclustering from the beginning; the co-clusters are deduced at the convergence.

Original: Thereby biclustering is a consequence and not a main goal.
Revised: Thereby biclustering is a consequence and not a main goal.

Original: Secondly, they suffer from high computational complexity; CCOT and CCOT-GW also consume large amounts of memory.
Revised: Secondly, they suffer from high computational complexity; CCOT and CCOT-GW also consume large amounts of memory.

Original: Finally, we will see that these algorithms are not suited to dyadic sparse data.
Revised: Finally, we will see that these algorithms are not suited to dyadic sparse data.

Original: In this paper, while integrating the biclustering objective from the beginning, we propose a generic framework for biclustering through optimal transport, which generalizes some previous biclustering approaches.
Revised: In this paper, while integrating the biclustering objective from the beginning, we propose a generic framework for biclustering through optimal transport, which generalizes some previous biclustering approaches.

Original: We propose two efficient methods for solving this problem: one that gives an almost hard biclustering, and another that gives a fuzzy or soft biclustering through entropic regularization.
Revised: We propose two efficient methods for solving this problem: one that gives an almost hard biclustering, and another that gives a fuzzy or soft biclustering through entropic regularization.

Original: These methods outperform other optimal transport biclustering models, in terms of both document and term clustering, on several regular and large scale datasets, while being more computationally and memory efficient.
Revised: These methods outperform other optimal transport biclustering models, in terms of both document and term clustering, on several regular and large scale datasets, while being more computationally and memory efficient.

Original: We emphasize once again that the approach we propose is specifically tailored to datasets consisting of dyadic data.
Revised: We emphasize once again that the approach we propose is specifically tailored to datasets consisting of dyadic data.

Original:  Methodology
Revised:  Methodology

Original: Notations.
Revised: Notations.

Original: In what follows, ∆ n = { p ∈ R n + | (cid:80) ni =1 p i = 1 } denotes the n -dimensional standard simplex.
Revised: In what follows, ∆ n = { p ∈ R n + | (cid:80) ni =1 p i = 1 } denotes the n -dimensional standard simplex.

Original: Π( w , v ) = { Z ∈ R n × k + | Z1 = w , Z ⊤ 1 = v } denotes the transportation polytope, where w ∈ ∆ n and v ∈ ∆ k are the marginals of the joint distribution Z and 1 n is a vector of ones.
Revised: Π( w , v ) = { Z ∈ R n × k + | Z1 = w , Z ⊤ 1 = v } denotes the transportation polytope, where w ∈ ∆ n and v ∈ ∆ k are the marginals of the joint distribution Z and 1 n is a vector of ones.

Original: Matrices are denoted with uppercase boldface letters, and vectors with lowercase boldface letters.
Revised: Matrices are denoted with uppercase boldface letters, and vectors with lowercase boldface letters.

Original: For a matrix
Revised: For a matrix

Original: M , its i -th row is m i and its j -th column is m ′
Revised: M , its i -th row is m i and its j -th column is m ′

Original: j We have that ∥ .
Revised: j We have that ∥ .

Original: ∥ 0 is the 0-norm which returns the number of nonzero elements of its argument.
Revised: ∥ 0 is the 0-norm which returns the number of nonzero elements of its argument.

Original: 2.1 Preliminaries
Revised: 2.1 Preliminaries

Original: We first need to introduce exact discrete OT and its entropically regularized counterpart, and show how biclustering can be posed as an integer program.
Revised: We first need to introduce exact discrete OT and its entropically regularized counterpart, and show how biclustering can be posed as an integer program.

Original: Discrete OT as a linear program.
Revised: Discrete OT as a linear program.

Original: The goal of discrete optimal transport is to find a minimal cost transport plan between a source probability distribution w and a target distribution v .
Revised: The goal of discrete optimal transport is to find a minimal cost transport plan between a source probability distribution w and a target distribution v .

Original: Here we are interested in the discrete case of the Kantorovich formulation of OT, that is
Revised: Here we are interested in the discrete case of the Kantorovich formulation of OT, that is

Original: [Equation]
Revised: [Equation]

Original: where M ∈ R n × k is the cost matrix, and m ij quantifies the effort needed to transport a probability mass from w i to v j .
Revised: where M ∈ R n × k is the cost matrix, and m ij quantifies the effort needed to transport a probability mass from w i to v j .

Original: Discrete entropy regularized OT.
Revised: Discrete entropy regularized OT.

Original: It has been suggested in the literature [6, 5] that the use of a regularization such as entropic regularization can lead to better computational and statistical efficiency.
Revised: It has been suggested in the literature [6, 5] that the use of a regularization such as entropic regularization can lead to better computational and statistical efficiency.

Original: [Equation]
Revised: [Equation]

Original: where H is the entropy defined as H ( Z ) ≜ − (cid:80) i,j z ij log z ij and λ controls the strength of regularization.
Revised: where H is the entropy defined as H ( Z ) ≜ − (cid:80) i,j z ij log z ij and λ controls the strength of regularization.

Original: The computational efficiency comes from the fact that the unique solution of this problem is of the structure Z :=
Revised: The computational efficiency comes from the fact that the unique solution of this problem is of the structure Z :=

Original: diag ( a ) exp( − M /λ ) diag ( b ) , a rescaled elementwise negative exponential of the cost M , where a and b are scaling vectors.
Revised: diag ( a ) exp( − M /λ ) diag ( b ) , a rescaled elementwise negative exponential of the cost M , where a and b are scaling vectors.

Original: These vectors can be found efficiently using the
Revised: These vectors can be found efficiently using the

Original: Sinkhorn-Knopp algorithm.
Revised: Sinkhorn-Knopp algorithm.

Original: Biclustering as an integer program.
Revised: Biclustering as an integer program.

Original: The Block seriation problem [30] consists in finding two permutation matrices, one for the rows and one for the columns s.t.
Revised: The Block seriation problem [27] consists in finding two permutation matrices, one for the rows and one for the columns s.t.

Original: dense blocks appear along the diagonal of the permuted matrix.
Revised: dense blocks appear along the diagonal of the permuted matrix.

Original: A possible definition of the block seriation problem is as follows: given a matrix B ∈ R n × d s.t b ij gives the strength of the association between row i and column j (such as in the case of a biadjacency matrix, for example), we have
Revised: A possible definition of the block seriation problem is as follows: given a matrix B ∈ R n × d s.t b ij gives the strength of the association between row i and column j (such as in the case of a biadjacency matrix, for example), we have

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: A solution C is a block diagonal matrix up to a permutation of its rows and columns.
Revised: A solution C is a block diagonal matrix up to a permutation of its rows and columns.

Original: The block seriation problem is an integer programming problem that is NP-hard.
Revised: The block seriation problem is an integer programming problem that is NP-hard.

Original: One approach for solving this problem uses a simplified version where a rank constraint rank ( C ) ≤ k is added for k the number of desired biclusters.
Revised: One approach for solving this problem uses a simplified version where a rank constraint rank ( C ) ≤ k is added for k the number of desired biclusters.

Original: Integrating this constraint into (4), we can define a new problem by low-rank factorization of C , i.e. C = ZW ⊤ , which we formulate as
Revised: Integrating this constraint into (4), we can define a new problem by low-rank factorization of C , i.e. C = ZW ⊤ , which we formulate as

Original: [Equation]
Revised: [Equation]

Original: where Γ( n, k ) = { Z ∈ { 0 , 1 } n × k | Z1 = 1 } is the set of hard partitions of dimension n × k .
Revised: where Γ( n, k ) = { Z ∈ { 0 , 1 } n × k | Z1 = 1 } is the set of hard partitions of dimension n × k .

Original: A simple heuristic for solving this problem involves alternatingly solving for Z given W , and vice-versa, using classical clustering algorithms, before identifying biclusters through the rearranged matrix C , which displays a block diagonal structure, as shown in figure 1a.
Revised: A simple heuristic for solving this problem involves alternatingly solving for Z given W , and vice-versa, using classical clustering algorithms, before identifying biclusters through the rearranged matrix C , which displays a block diagonal structure, as shown in figure 1a.

Original: The biclusters are identified by grouping together the rows and columns that form a block along the diagonal.
Revised: The biclusters are identified by grouping together the rows and columns that form a block along the diagonal.

Original: 2.2 Biclustering using Optimal Transport
Revised: 2.2 Biclustering using Optimal Transport

Original: Here we propose a new biclustering problem based on block seriation and optimal transport.
Revised: Here we propose a new biclustering problem based on block seriation and optimal transport.

Original: For this purpose we first define what we term an anti-adjacency matrix .
Revised: For this purpose we first define what we term an anti-adjacency matrix .

Original: Note that a similar concept has been discussed in [41].
Revised: Note that a similar concept has been discussed in [36].

Original: Definition 1 (Anti-adjacency matrix)
Revised: Definition 1 (Anti-adjacency matrix)

Original: Given a graph characterized by an adjacency matrix A , we have a corresponding anti-adjacency matrix A s.t.
Revised: Given a graph characterized by an adjacency matrix A , we have a corresponding anti-adjacency matrix A s.t.

Original: a ij quantifies the discrepancy between nodes i and j .
Revised: a ij quantifies the discrepancy between nodes i and j .

Original: We consider a bipartite graph characterized by its biadjacency matrix B = ( b ij ) ∈ R n × d .
Revised: We consider a bipartite graph characterized by its biadjacency matrix B = ( b ij ) ∈ R n × d .

Original: The rows of B are endowed with weights w ∈ ∆ n and its columns with weights v ∈ ∆ d .
Revised: The rows of B are endowed with weights w ∈ ∆ n and its columns with weights v ∈ ∆ d .

Original: We also consider a row exemplar distribution r ∈ ∆ r and a column exemplar distribution c ∈ ∆ c .
Revised: We also consider a row exemplar distribution r ∈ ∆ r and a column exemplar distribution c ∈ ∆ c .

Original: Depending on the availability of a priori information about the data, these weight vectors can be set to uniform distributions.
Revised: Depending on the availability of a priori information about the data, these weight vectors can be set to uniform distributions.

Original: Now let its anti-biadjacency matrix be B
Revised: Now let its anti-biadjacency matrix be B

Original: =
Revised: =

Original: L ( B ) , where L : R n ×
Revised: L ( B ) , where L : R n ×

Original: d → R n × d means that b ij , the association between node i and node j , is transformed into a discrepancy measure L ( B ) ij .
Revised: d → R n × d means that b ij , the association between node i and node j , is transformed into a discrepancy measure L ( B ) ij .

Original: Thus, we define the optimal transport block seriation problem as the following bilinear program
Revised: Thus, we define the optimal transport block seriation problem as the following bilinear program

Original: [Equation]
Revised: [Equation]

Original: where Z is a transport plan (or coupling) between between the row distribution w and the row exemplar distribution r , and similarly for W w.r.t.
Revised: where Z is a transport plan (or coupling) between between the row distribution w and the row exemplar distribution r , and similarly for W w.r.t.

Original: the column distribution v and the column exemplar distribution c .
Revised: the column distribution v and the column exemplar distribution c .

Original: [Figure]
Revised: [Figure]

Original: Inducing a biclustering via BCOT.
Revised: Inducing a biclustering via BCOT.

Original: We will now show how to obtain a partition of the rows and the columns given a solution pair ( Z , W ) .
Revised: We will now show how to obtain a partition of the rows and the columns given a solution pair ( Z , W ) .

Original: In what follows our aim is to identify an almost-hard clustering couple for rows and columns from the couplings Z and W .
Revised: In what follows our aim is to identify an almost-hard clustering couple for rows and columns from the couplings Z and W .

Original: Definition 2 ( h -almost hard clustering)
Revised: Definition 2 ( h -almost hard clustering)

Original: We define an h -almost hard clustering as a clustering whose assignment matrix is C ∈ R n × k s.t.
Revised: We define an h -almost hard clustering as a clustering whose assignment matrix is C ∈ R n × k s.t.

Original: ∥
Revised: ∥

Original: C ∥ 

Revised: C ∥

Original: = n + h and for each row c of C we have that ∥ c ∥ 0 > 0 .
Revised: = n + h and for each row c of C we have that ∥ c ∥ 0 > 0 .

Original: When h = 0 , we obtain a standard hard clustering with one non-zero element per row.
Revised: When h = 0 , we obtain a standard hard clustering with one non-zero element per row.

Original: Proposition 1 1 For w , v , r and c containing no zeros, there exists an optimal pair of coupling matrices Z and W that are h -almost hard clusterings with h ∈ { 0 , . . .
Revised: Proposition 1 1 For w , v , r and c containing no zeros, there exists an optimal pair of coupling matrices Z and W that are h -almost hard clusterings with h ∈ { 0 , . . .

Original: , k − 1 } .
Revised: , k − 1 } .

Original: Furthermore, when n = k (resp.
Revised: Furthermore, when n = k (resp.

Original: d = k ) and w = r (resp.
Revised: d = k ) and w = r (resp.

Original: v = c ), this Z (resp.
Revised: v = c ), this Z (resp.

Original: W ) becomes a hard clustering, i.e.,
Revised: W ) becomes a hard clustering, i.e.,

Original: Z ∈ Γ( n, n ) (resp.
Revised: Z ∈ Γ( n, n ) (resp.

Original: W ∈ Γ( d, d ) ).
Revised: W ∈ Γ( d, d ) ).

Original: This means that the solutions are already almost a hard partition of the data, since k << n, d .
Revised: This means that the solutions are already almost a hard partition of the data, since k << n, d .

Original: To obtain a final hard clustering in the strict sense, we assign each row (resp. column) to the one corresponding to the row of Z (resp.
Revised: To obtain a final hard clustering in the strict sense, we assign each row (resp. column) to the one corresponding to the row of Z (resp.

Original: W ) with the largest value.
Revised: W ) with the largest value.

Original: This should not significantly change the structure of the solution.
Revised: This should not significantly change the structure of the solution.

Original: Figure 1b provides an illustration: here we see the block diagonal structure generated by the product of the two coupling matrices C = ZW ⊤ , with its similarity in appearance to the biclustering produced by the hard block seriation 1a, apart from a few nonzero entries off the block diagonal that are hard to see immediately.
Revised: Figure 1b provides an illustration: here we see the block diagonal structure generated by the product of the two coupling matrices C = ZW ⊤ , with its similarity in appearance to the biclustering produced by the hard block seriation 1a, apart from a few nonzero entries off the block diagonal that are hard to see immediately.

Original: Intuition for BCOT.
Revised: Intuition for BCOT.

Original: To explain the intuition behind the proposed approach we need to look at how the problem is solved.
Revised: To explain the intuition behind the proposed approach we need to look at how the problem is solved.

Original: The optimization procedure as described in algorithm 1 consists in alternating between the computation of an optimal transport plan
Revised: The optimization procedure as described in algorithm 1 consists in alternating between the computation of an optimal transport plan

Original: Z given W and vice versa.
Revised: Z given W and vice versa.

Original: As regards solving for Z given W , the problem can be rewritten as
Revised: As regards solving for Z given W , the problem can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: This is an optimal transport problem with L ( B ) W as the cost matrix.
Revised: This is an optimal transport problem with L ( B ) W as the cost matrix.

Original: The resulting transport plan Z can be seen as a kind of row cluster assignment matrix: if z ih > 0 , then row i is assigned to cluster h .
Revised: The resulting transport plan Z can be seen as a kind of row cluster assignment matrix: if z ih > 0 , then row i is assigned to cluster h .

Original: The same holds for W , which can be seen as a column cluster assignment matrix.
Revised: The same holds for W , which can be seen as a column cluster assignment matrix.

Original: This also means that since L ( B ) is the dissimilarity between the rows and the columns, then the cost matrix L ( B ) W represents the dissimilarity between rows and row exemplars (or representatives or centroids).
Revised: This also means that since L ( B ) is the dissimilarity between the rows and the columns, then the cost matrix L ( B ) W represents the dissimilarity between rows and row exemplars (or representatives or centroids).

Original: In particular, L ( B )
Revised: In particular, L ( B )

Original: i w h is the dissimilarity or cost of probability mass transportation between row i and row cluster exemplar h .
Revised: i w h is the dissimilarity or cost of probability mass transportation between row i and row cluster exemplar h .

Original: The reasoning is the same for the columns and the optimal coupling W .
Revised: The reasoning is the same for the columns and the optimal coupling W .

Original: Low-rank optimal transport.
Revised: Low-rank optimal transport.

Original: Biclustering is the main purpose of the approach we proposed, but there is another interesting use case.
Revised: Biclustering is the main purpose of the approach we proposed, but there is another interesting use case.

Original: Proposition 2 For equal target row and column representative distributions, i.e., r = c , and containing no zero entries, then given a solution pair Z and W to BCOT, the matrix Q = Z diag (1 / r ) W ⊤ is an approximation of the optimal transport plan that is a solution to problem (2) and whose rank is at most min( rank ( Z ) , rank ( W )) .
Revised: Proposition 2 For equal target row and column representative distributions, i.e., r = c , and containing no zero entries, then given a solution pair Z and W to BCOT, the matrix Q = Z diag (1 / r ) W ⊤ is an approximation of the optimal transport plan that is a solution to problem (2) and whose rank is at most min( rank ( Z ) , rank ( W )) .

Original: Some recent works [17, 36] have suggested that this kind of low-rank regularization is preferable to entropic regularization as regards certain aspects.
Revised: Some recent works [16, 31] have suggested that this kind of low-rank regularization is preferable to entropic regularization as regards certain aspects.

Original: For example, the rank parameter is easier to select, since it has simple bounds (an integer between 1 and n ).
Revised: For example, the rank parameter is easier to select, since it has simple bounds (an integer between 1 and n ).

Original: This may be contrasted with the regularization strength λ in the Sinkhorn algorithm, which is continuous.
Revised: This may be contrasted with the regularization strength λ in the Sinkhorn algorithm, which is continuous.

Original: 2.3 Fuzzy Biclustering via Regularized Optimal Transport
Revised: 2.3 Fuzzy Biclustering via Regularized Optimal Transport

Original: As previously mentioned, using entropic regularization may be interesting because of its various useful features including statistical and computational efficiency.
Revised: As previously mentioned, using entropic regularization may be interesting because of its various useful features including statistical and computational efficiency.

Original: However, another feature of entropic regularization is that the optimal couplings Z and W are dense matrices as a consequence of the structure of the optimal solution of entropically regularized OT problems.
Revised: However, another feature of entropic regularization is that the optimal couplings Z and W are dense matrices as a consequence of the structure of the optimal solution of entropically regularized OT problems.

Original: We formulate the problem as follows
Revised: We formulate the problem as follows

Original: [Equation]
Revised: [Equation]

Original: where λ Z and λ W are the regularization parameters.
Revised: where λ Z and λ W are the regularization parameters.

Original: Fuzzy block seriation.
Revised: Fuzzy block seriation.

Original: We propose a fuzzy variant of the block seriation problem that allows us by extension to define a fuzzy variant for BCOT using entropic regularization.
Revised: We propose a fuzzy variant of the block seriation problem that allows us by extension to define a fuzzy variant for BCOT using entropic regularization.

Original: Let the fuzzy block seriation problem be defined as
Revised: Let the fuzzy block seriation problem be defined as

Original: [Equation]
Revised: [Equation]

Original: where Ω( Z , W ) is some regularization term introduced to make the partition matrices Z and W dense (for example, entropic regularization or low-rank constraints), and Γ s ( n, k )
Revised: where Ω( Z , W ) is some regularization term introduced to make the partition matrices Z and W dense (for example, entropic regularization or low-rank constraints), and Γ s ( n, k )

Original: = { Z ∈ R n × k + | Z1 = 1 } is the set of fuzzy partitions.
Revised: = { Z ∈ R n × k + | Z1 = 1 } is the set of fuzzy partitions.

Original: Intuitively, for a solution pair ( Z , W ) , up to a constant factor, each entry in the block seriation matrix C =
Revised: Intuitively, for a solution pair ( Z , W ) , up to a constant factor, each entry in the block seriation matrix C =

Original: ZW ⊤ can be seen as the probability of its corresponding row and column belonging to the same bicluster i.e. c ij =
Revised: ZW ⊤ can be seen as the probability of its corresponding row and column belonging to the same bicluster i.e. c ij =

Original: z
Revised: z

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: It is easy to see how problem (9) is related to problem (8) and that the couplings corresponding to solutions of the problem give the probability that the different rows and columns belong to the same biclusters.
Revised: It is easy to see how problem (9) is related to problem (8) and that the couplings corresponding to solutions of the problem give the probability that the different rows and columns belong to the same biclusters.

Original: Figure 1c shows biclusters produced by the solutions of BCOT λ .
Revised: Figure 1c shows biclusters produced by the solutions of BCOT λ .

Original: Similarly to BCOT, a block diagonal structure is formed.
Revised: Similarly to BCOT, a block diagonal structure is formed.

Original: However, there are also several off-block diagonal nonzero entries that represent the probabilities of the row-column pairs belonging to the same biclusters.
Revised: However, there are also several off-block diagonal nonzero entries that represent the probabilities of the row-column pairs belonging to the same biclusters.

Original:  Links to Existing Work
Revised:  Links to Existing Work

Original: 3.1 Modularity Maximization in Bipartite Graphs [3].
Revised: 3.1 Modularity Maximization in Bipartite Graphs [3].

Original: This model is able to co-cluster binary and contingency matrices by directly maximizing an adapted version of the modularity measure traditionally used for networks.
Revised: This model is able to co-cluster binary and contingency matrices by directly maximizing an adapted version of the modularity measure traditionally used for networks.

Original: The criterion that it optimizes is
Revised: The criterion that it optimizes is

Original: [Equation]
Revised: [Equation]

Original: By setting L ( B ) = − ( B − 1 b .. B11 ⊤ B ) , this problem becomes equivalent to ours; the difference is in the constraints on Z and W .
Revised: By setting L ( B ) = − ( B − 1 b .. B11 ⊤ B ) , this problem becomes equivalent to ours; the difference is in the constraints on Z and W .

Original: 3.2 Modularity-Based Sparse Soft Graph Clustering [25].
Revised: Modularity-Based Sparse Soft Graph Clustering [

Original: Here the authors proposed a fuzzy variant of the above problem (although in the context of traditional clustering rather than biclustering).
Revised: Here the authors proposed a fuzzy variant of the above problem (although in the context of traditional clustering rather than biclustering).

Original: Solving the problem gives, for each element of the dataset, a probability of that element belonging to a given cluster.
Revised: Solving the problem gives, for each element of the dataset, a probability of that element belonging to a given cluster.

Original: Our proposed entropic regularization variant represents a kind of extension of this problem to bipartite graphs.
Revised: Our proposed entropic regularization variant represents a kind of extension of this problem to bipartite graphs.

Original: 3.3 Directional Co-clustering with a Conscience [35, 1].
Revised: 3.3 Directional Co-clustering with a Conscience [30, 1].

Original: This model makes use of the block von Mises-Fisher mixture model for co-clustering directional data on the unit-sphere.
Revised: This model makes use of the block von Mises-Fisher mixture model for co-clustering directional data on the unit-sphere.

Original: It optimizes the following criterion:
Revised: It optimizes the following criterion:

Original: [Equation]
Revised: [Equation]

Original: In our formulation, if we define L ( B ) = − B and apply cluster size normalization on the optimal transport plans ˜ Z
Revised: In our formulation, if we define L ( B ) = − B and apply cluster size normalization on the optimal transport plans ˜ Z

Original: = Z diag ( Z ⊤ 1 ) − 1 / 2 and ˜ W = W diag ( W ⊤ 1 ) − 1 / 2 after computing Z and W respectively in algorithm 1 , we obtain a more general version of the algorithm proposed by the authors for solving problem (11).
Revised: = Z diag ( Z ⊤ 1 ) − 1 / 2 and ˜ W = W diag ( W ⊤ 1 ) − 1 / 2 after computing Z and W respectively in algorithm 1 , we obtain a more general version of the algorithm proposed by the authors for solving problem (11).

Original: 3.4 Bipartite Correlation Clustering [2]
Revised: 3.4 Bipartite Correlation Clustering [2]

Original: In the case where the cost function results in a complete bipartite graph with ’+’ and ’-’ edges with a function
Revised: In the case where the cost function results in a complete bipartite graph with ’+’ and ’-’ edges with a function

Original: [Equation]
Revised: [Equation]

Original: we get what is known as Bipartite Correlation Clustering.
Revised: we get what is known as Bipartite Correlation Clustering.

Original: The solution to this problem maximizes the number of agreements, i.e. the number of all ’+’ edges within clusters plus all ’-’ edges distributed across clusters.
Revised: The solution to this problem maximizes the number of agreements, i.e. the number of all ’+’ edges within clusters plus all ’-’ edges distributed across clusters.

Original:  Optimization and Complexity
Revised:  Optimization and Complexity

Original: [Figure]
Revised: [Figure]

Original: Optimization.
Revised: Optimization.

Original: Since the block seriation problem is NP-hard, computing an exact solution is prohibitive.
Revised: Since the block seriation problem is NP-hard, computing an exact solution is prohibitive.

Original: An efficient and widely used heuristic for solving these kinds of problems involves the use of block coordinate descent, where row assignments are computed for fixed column assignments, and then vice versa, in alternation.
Revised: An efficient and widely used heuristic for solving these kinds of problems involves the use of block coordinate descent, where row assignments are computed for fixed column assignments, and then vice versa, in alternation.

Original: We express the proposed algorithm in pseudo-code as algorithm 1.
Revised: We express the proposed algorithm in pseudo-code as algorithm 1.

Original: At each iteration we solve two intermediate optimal transport problems with cost matrices of dimensions n × k and d × k , since B is generally sparse, and L can be defined such that
Revised: At each iteration we solve two intermediate optimal transport problems with cost matrices of dimensions n × k and d × k , since B is generally sparse, and L can be defined such that

Original: L ( B ) retains a similarly sparse structure.
Revised: L ( B ) retains a similarly sparse structure.

Original: The computation of the intermediate cost matrices L ( B ) W and L ( B ) ⊤ Z is reasonably efficient.
Revised: The computation of the intermediate cost matrices L ( B ) W and L ( B ) ⊤ Z is reasonably efficient.

Original: We also observed that the algorithm does not need many iterations to converge, as shown in figure 2, be it for BCOT or
Revised: We also observed that the algorithm does not need many iterations to converge, as shown in figure 2, be it for BCOT or

Original: BCOT λ .
Revised: BCOT λ .

Original: Algorithm 1: BCOT converged end
Revised: Algorithm 1: BCOT converged end

Original: Proposition 3 The computational complexity of the BCOT algorithm 1 when using an exact OT solver is O ( tk ∥ B ∥ 

Revised: Proposition 3 The computational complexity of the BCOT algorithm 1 when using an exact OT solver is O ( tk ∥ B ∥

Original: + tnk ( n + k ) log( n + k )
Revised: + tnk ( n + k ) log( n + k )

Original: + tdk
Revised: + tdk

Original: ( d + k ) log( d + k ))) , and when using entropic regularization the complexity is O ( tk ∥ B ∥ 

Revised: ( d + k ) log( d + k ))) , and when using entropic regularization the complexity is O ( tk ∥ B ∥

Original: + tkn + tkd ) , where t is the number of iterations.
Revised: + tkn + tkd ) , where t is the number of iterations.

Original: In table 1, we report the computational and spatial complexities of the different biclustering approaches.
Revised: In table 1, we report the computational and spatial complexities of the different biclustering approaches.

Original: Our model has the same spatial complexity as the COOT variants and a better complexity
Revised: Our model has the same spatial complexity as the COOT variants and a better complexity

Original: [Table]
Revised: [Table]

Original: than CCOT variants.
Revised: than CCOT variants.

Original: As regards the computational complexity, our model should in most cases be faster with sparse data, and our experiments support this conjecture.
Revised: As regards the computational complexity, our model should in most cases be faster with sparse data, and our experiments support this conjecture.

Original: For reproducibility, we publicly release our code 2 .
Revised: For reproducibility, we publicly release our code 2 .

Original:  Experiments
Revised:  Experiments

Original: We ran experiments using term-document matrices.
Revised: We ran experiments using term-document matrices.

Original: The benefit of using biclustering on this kind of data is that the resulting biclusters contain both documents and the words that characterize them, which is helpful in interpreting the clustering of the documents.
Revised: The benefit of using biclustering on this kind of data is that the resulting biclusters contain both documents and the words that characterize them, which is helpful in interpreting the clustering of the documents.

Original: Additional experiments over synthetic and gene expression data are available in the appendix.
Revised: Additional experiments over synthetic and gene expression data are available in the appendix.

Original: 5.1 Datasets
Revised: 5.1 Datasets

Original: We evaluate BCOT in relation to six benchmark document-term datasets: ACM, DBLP, PubMed,
Revised: We evaluate BCOT in relation to six benchmark document-term datasets: ACM, DBLP, PubMed,

Original: Wiki, Ohscal
Revised: Wiki, Ohscal

Original: [Table]
Revised: [Table]

Original: and 20 Newsgroups.
Revised: and 20 Newsgroups.

Original: Their characteristics are shown in Table 2.
Revised: Their characteristics are shown in Table 2.

Original: ACM, DBLP, Pubmed and Wiki are attributed networks from which we use only the node-level features that correspond to term-document matrices.
Revised: ACM, DBLP, Pubmed and Wiki are attributed networks from which we use only the node-level features that correspond to term-document matrices.

Original: We also selected the Ohscal collection and 20 Newsgroups as large-scale document-term matrices to serve as computational efficiency benchmarks.
Revised: We also selected the Ohscal collection and 20 Newsgroups as large-scale document-term matrices to serve as computational efficiency benchmarks.

Original: the datasets.
Revised: the datasets.

Original: Dataset
Revised: Dataset

Original: [Table]
Revised: [Table]

Original: Documents

Original: [Table]

Original: ACM [13]
Revised: ACM [13]

Original: [Table]
Revised: [Table]

Original: DBLP

Original: [Table][Table]
Revised: [Table]

Original: PubMed [37]
Revised: PubMed [32]

Original: [Table]
Revised: [Table]

Original: Wiki
Revised: Wiki

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Ohscal [24]
Revised: Ohscal [22]

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: 20 Newsgroups [29]
Revised: 20 Newsgroups [26]

Original: [Table]
Revised: [Table]

Original: 5.2 Experimental Setup
Revised: 5.2 Experimental Setup

Original: In our experiments we define the loss function as L ( B ) = − c B , where c is selected from { 1 , k, d, n } .
Revised: In our experiments we define the loss function as L ( B ) = − c B , where c is selected from { 1 , k, d, n } .

Original: For BCOT λ , the regularization parameter lambda is selected from { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 , 10 } .
Revised: For BCOT λ , the regularization parameter lambda is selected from { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 , 10 } .

Original: The best hyper-parameters are those that minimize the number of empty clusters.
Revised: The best hyper-parameters are those that minimize the number of empty clusters.

Original: In the case of ties, we select according to the value of the Davies-Bouldin index of the partition [7].
Revised: In the case of ties, we select according to the value of the Davies-Bouldin index of the partition [7].

Original: Random restarts are not used for any of the algorithms, including k -means.
Revised: Random restarts are not used for any of the algorithms, including k -means.

Original: We use the implementation provided by the authors for CCOT, CCOT λ and CCOT-GW.
Revised: We use the implementation provided by the authors for CCOT, CCOT λ and CCOT-GW.

Original: The code for CCOT was not available, and so we had to implement it based on the code for CCOT-GW.
Revised: The code for CCOT was not available, and so we had to implement it based on the code for CCOT-GW.

Original: All the reported figures are the averages of 10 runs.
Revised: All the reported figures are the averages of 10 runs.

Original: All the experiments were performed on the same machine with an Intel(R) Xeon(R) CPU and 12GB
Revised: All the experiments were performed on the same machine with an Intel(R) Xeon(R) CPU and 12GB

Original: RAM.
Revised: RAM.

Original: For OT solvers we made use of the POT package [16].
Revised: For OT solvers we made use of the POT package [15].

Original: 5.3 Document Clustering
Revised: 5.3 Document Clustering

Original: Metrics.
Revised: Metrics.

Original: Here, the evaluation is straightforward, we adopt three popular clustering metrics: clustering accuracy (CA), normalized mutual information (NMI) [4], adjusted rand index (ARI)[26]
Revised: Here, the evaluation is straightforward, we adopt three popular clustering metrics: clustering accuracy (CA), normalized mutual information (NMI) [4], adjusted rand index (ARI)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Performance.
Revised: Performance.

Original: Document clustering results on ACM, DBLP, PubMed and Wiki are given in tablefor the three metrics.
Revised: Document clustering results on ACM, DBLP, PubMed and Wiki are given in tablefor the three metrics.

Original: In all cases the best result is obtained either by BCOT or by BCOT λ .
Revised: In all cases the best result is obtained either by BCOT or by BCOT λ .

Original: Moreover, on Wiki, BCOT λ gives competitive results when compared with state-of-the-art attributed graph clustering methods presented in [15], despite not having access to the graph structure information in the Wiki citation network.
Revised: Moreover, on Wiki, BCOT λ gives competitive results when compared with state-of-the-art attributed graph clustering methods presented in [14], despite not having access to the graph structure information in the Wiki citation network.

Original: Efficiency.
Revised: Efficiency.

Original: Figure 3 plots the document clustering performance (accuracy against training time) of the different methods on the two large-scale document-term matrices 20 Newsgroup and Ohscal.
Revised: Figure 3 plots the document clustering performance (accuracy against training time) of the different methods on the two large-scale document-term matrices 20 Newsgroup and Ohscal.

Original: BCOT offers the best accuracy while BCOT λ is fastest method on both datasets.
Revised: BCOT offers the best accuracy while BCOT λ is fastest method on both datasets.

Original: We see that for both
Revised: We see that for both

Original: BCOT and COOT, the entropic-regularized versions outspeed their exact counterparts and that CCOT suffers from very high computation times, due mainly to the fact that this method requires pairwise distance matrices to be computed on the rows and columns.
Revised: BCOT and COOT, the entropic-regularized versions outspeed their exact counterparts and that CCOT suffers from very high computation times, due mainly to the fact that this method requires pairwise distance matrices to be computed on the rows and columns.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: 5.4 Term Clustering
Revised: 5.4 Term Clustering

Original: Metrics.
Revised: Metrics.

Original: Unlike document clustering, there is no ground truth partition for terms, so we need to find another way of evaluating term clustering results.
Revised: Unlike document clustering, there is no ground truth partition for terms, so we need to find another way of evaluating term clustering results.

Original: One generally acceptable technique is to analyse the semantic coherence of the clusters obtained.
Revised: One generally acceptable technique is to analyse the semantic coherence of the clusters obtained.

Original: To this end we introduce a metric based on point mutual information (PMI).
Revised: To this end we introduce a metric based on point mutual information (PMI).

Original: PMI is a frequently used information-theoretic metric for quantifying the relationship between pairs of discrete random variable outcomes.
Revised: PMI is a frequently used information-theoretic metric for quantifying the relationship between pairs of discrete random variable outcomes.

Original: The PMI measure was chosen because prior research [32] has shown that it is closely associated with human judgements in determining word relatedness.
Revised: The PMI measure was chosen because prior research [29] has shown that it is closely associated with human judgements in determining word relatedness.

Original: The PMI between the terms w i and w j is calculated as
Revised: The PMI between the terms w i and w j is calculated as

Original: [Equation]
Revised: [Equation]

Original: In the context of term clustering, given the word co-occurrence matrix K = B ⊤ B , the PMI is estimated as in
Revised: In the context of term clustering, given the word co-occurrence matrix K = B ⊤ B , the PMI is estimated as in

Original: [Equation]
Revised: [Equation]

Original: To evaluate a partition of terms P , we propose a metric based on intra and inter PMI metrics as follows:
Revised: To evaluate a partition of terms P , we propose a metric based on intra and inter PMI metrics as follows:

Original: [Equation]
Revised: [Equation]

Original: In this way, a good clustering should reveal a high intra-cluster semantic relatedness, corresponding to higher PMI values.
Revised: In this way, a good clustering should reveal a high intra-cluster semantic relatedness, corresponding to higher PMI values.

Original: Using the intra and inter PMIs, we propose the following coherence index
Revised: Using the intra and inter PMIs, we propose the following coherence index

Original: [Equation]
Revised: [Equation]

Original: Our reasoning is this: the greater the semantic proximity between terms in the same clusters, and the greater the sematic distance between terms in different clusters, the higher the value of coherence .
Revised: Our reasoning is this: the greater the semantic proximity between terms in the same clusters, and the greater the sematic distance between terms in different clusters, the higher the value of coherence .

Original: Results.
Revised: Results.

Original: Since there is no ground truth number of term clusters, we use the cluster number estimations produced by CCOT-GW for all the other models so that it is easy to compare coherence values between them.
Revised: Since there is no ground truth number of term clusters, we use the cluster number estimations produced by CCOT-GW for all the other models so that it is easy to compare coherence values between them.

Original: Comparisons based on different numbers of clusters would favor the model using the larger number of clusters.
Revised: Comparisons based on different numbers of clusters would favor the model using the larger number of clusters.

Original: Table 4 shows the coherences obtained across the different datasets using our approach, along with those of the baselines.
Revised: Table 4 shows the coherences obtained across the different datasets using our approach, along with those of the baselines.

Original: It is clear that BCOT succeeds in capturing more semantics than the other approaches since, whatever the dataset, one or other of the two BCOT variants gives the highest coherence.
Revised: It is clear that BCOT succeeds in capturing more semantics than the other approaches since, whatever the dataset, one or other of the two BCOT variants gives the highest coherence.

Original: [Table]
Revised: [Table]

Original: 5.5 Statistical Significance
Revised: 5.5 Statistical Significance

Original: We performed a Nemenyi post-hoc test [31, 8] with a confidence level of 90% on the document and term clustering results that we obtained, to determine whether our model outperforms other
Revised: We performed a Nemenyi post-hoc test [28, 8] with a confidence level of 90% on the document and term clustering results that we obtained, to determine whether our model outperforms other

Original: OT biclustering approaches in a statistically significant way.
Revised: OT biclustering approaches in a statistically significant way.

Original: To conduct this test we generatedperformance rankings of the OT biclustering models based on their performance for each dataset and quality metric pair for both document and term clustering.
Revised: To conduct this test we generatedperformance rankings of the OT biclustering models based on their performance for each dataset and quality metric pair for both document and term clustering.

Original: Figure 4 shows the results of the test.
Revised: Figure 4 shows the results of the test.

Original: We see that two differently performing groups were identified, one comprising BCOT and BCOT λ and giving better results than the other group comprising the remaining COOT and CCOT variants.
Revised: We see that two differently performing groups were identified, one comprising BCOT and BCOT λ and giving better results than the other group comprising the remaining COOT and CCOT variants.

Original: This indicates that with this specific number of datasets and metrics the test was unable to tell COOT and
Revised: This indicates that with this specific number of datasets and metrics the test was unable to tell COOT and

Original: CCOT apart in a statistically significant way.
Revised: CCOT apart in a statistically significant way.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original:  Conclusion
Revised:  Conclusion

Original: Clustering and biclustering through optimal transport is still at a nascent stage, with many challenges remaining unsolved.
Revised: Clustering and biclustering through optimal transport is still at a nascent stage, with many challenges remaining unsolved.

Original: This paper introduces a novel problem for biclustering using optimal transport that takes into account the sparse nature of certain types of dyadic data such document-term matrices, to enable more computationally efficient resolution.
Revised: This paper introduces a novel problem for biclustering using optimal transport that takes into account the sparse nature of certain types of dyadic data such document-term matrices, to enable more computationally efficient resolution.

Original: The problem is posed as a bilinear program that we solve using an efficient block coordinate descent algorithm to find a vertex solution.
Revised: The problem is posed as a bilinear program that we solve using an efficient block coordinate descent algorithm to find a vertex solution.

Original: Experiments on a number of document-term datasets suggest that the proposed approach does a good job in finding clusters that correspond to ground truth document classes, while generating semantically coherent partitions for the terms.
Revised: Experiments on a number of document-term datasets suggest that the proposed approach does a good job in finding clusters that correspond to ground truth document classes, while generating semantically coherent partitions for the terms.

Original: In this setting, our model outperforms recent OT biclustering methods by a significant margin, while being more computationally efficient
Revised: In this setting, our model outperforms recent OT biclustering methods by a significant margin, while being more computationally efficient

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: This work has been funded by Informatique Caisse des Dépôts et Consignations (ICDC), Association
Revised: This work has been funded by Informatique Caisse des Dépôts et Consignations (ICDC), Association

Original: Nationale de la Recherche et de la Technologie (ANRT) and Idex-Spectrans of Université Paris Cité.Appendix A Proofs Proposition 1.
Revised: Nationale de la Recherche et de la Technologie (ANRT) and Idex-Spectrans of Université Paris Cité.

Original: For w , v , r and c containing no zeros, the resulting optimal coupling matrices Z and W are always an h -almost hard clustering with h ∈ { 0 , . . .

Original: , k − 1 } .

Original: Furthermore, when n = k (resp.

Original: d = k ) and w = r (resp.

Original: v = c ), Z (resp.

Original: W ) represents a hard clustering Z ∈ Γ( n, n ) (resp.

Original: W ∈ Γ( d, d ) ).

Original: Proof for proposition 1.

Original: Problem (2) is a bounded linear program since Π( w , v ) is a polytope i.e. a bounded polyhedron.

Original: The fundamental theorem of linear programming states that if the feasible set is non-empty then the solution lies in the extremity of the feasible region.

Original: This means that a solution Z to problem (2) is an extreme point of Π( w , v ) .

Original: We have that the extreme points of Π( w , v ) can have at most n + d − 1 nonzero elements.

Original: To prove this we have to show that the bipartite graph induced by biadjacency matrix Z , the solution to the optimal transport problem has no cycles.

Original: The maximum number of edges in an acyclic graph is | V | − 1 where | V | is the number of nodes in the graph.

Original: Since the number of edges in the bipartite graph induced by biadjacency matrix Z is n + d − 1 , the matrix

Original: Z can not have more than n + d − 1 nonzero entries.

Original: For a detailed proof see proposition 3.3 in [34].

Original: We also have to show that for probability measures w and v that have no zero probability events, there is at minimum max( n, d ) number of nonzero elements in Z .

Original: This is straightforward since w and v contain no zeros, there will always be at least one nonzero element in every row and column of

Original: Z that represents some transfer of mass between elements of w and v .

Original: BCOT is a bilinear program that has a finite global solution which means that there exists at least one optimal solution pair ( Z , W ) such that Z is an extreme point of Π( w , r ) and W is an extreme point of Π( v , c ) (theorem 1 in [27]).

Original: We then have that, For BCOT, Z has at most n + k − 1 and at least max( n, k ) =

Original: n nonzero entries and that W has at most d + k − 1 and at least max( d, k )

Original: =

Original: d elements which are both h -almost hard clusterings with h ∈ { 0 , . .

Original: , k − 1 } .

Original: When n = k and w = r , the solution Z is a permutation matrix (up to a constant factor) and the number of nonzero elements in it is exactly n which means that it represents a hard partition

Original: Z ∈ Γ( n, n ) .

Original: The proof is the same for W .

Original: Proposition 2.

Original: Suppose that the target row and column representative distributions are the same, i.e. r = c with no zero entries.

Original: Then, given a solution pair Z and W to BCOT, the matrix

Original: Q =

Original: Z diag (1 / r ) W ⊤ is an approximation of the optimal transport plan that is a solution to problem (2) and whose rank is at most min( rank ( Z ) , rank ( W )) .

Original: Proof of proposition 2.

Original: From linear algebra, we have that rank ( Q ) ≤ min( rank ( Z ) , rank ( diag (1 / r )) , rank ( W )) .

Original: Since Z and W cannot have a rank greater than k due to their dimension, and since diag (1 / r ) is a full rank matrix due to the assumption that r has no zero entries, we then have that rank ( Q ) ≤ min( rank ( Z ) , rank ( W )) .

Original: For a proof that Q is indeed a valid transport plan i.e. Q ∈ Π( w , v ) , we refer the reader to proposition 2.2 in [34].

Original: Proposition 3.

Original: [-The computational complexity of the BCOT algorithm 1 when using an exact OT solver is O ( tk ∥ B ∥ 
-]
Revised: [-The computational complexity of the BCOT algorithm 1 when using an exact OT solver is O ( tk ∥ B ∥ 
-]

Original: + tnk ( n + k ) log( n + k )

Original: + tdk

Original: [-( d + k ) log( d + k ))) , and when using entropic regularization the complexity is O ( tk ∥ B ∥ 
-]
Revised: [-( d + k ) log( d + k ))) , and when using entropic regularization the complexity is O ( tk ∥ B ∥ 
-]

Original: + tkn + tkd ) , where t is the number of iterations.

Original: Proof of proposition 3.

Original: We suppose that L ( B ) is a sparse matrix with the same number of nonzero entries as B .

Original: The complexity of computing L ( B ) W and L ( B ) W in the BCOT algorithm is

Original: O ( k ∥ B ∥ 0 ) .

Original: The optimal transport problem can be formulated and solved as the Earth Mover’s Distance (EMD) problem using any minimum-cost flow problem algorithm, such as one of the many variants of the network simplex algorithm.

Original: The authors in [33] proposed an algorithm for the network simplex in O ( | V || E | log | V | ) , where | V | is the number of nodes and | E | is the number of edges in the network.

Original: In our case, when solving the EMD for Z and cost matrix L ( B ) W , the number of nodes is | V | = n + k and the number of edges is | E | =

Original: nk , which means that the complexity of the operation is O ( nk ( n + k ) log( n + k )) .

Original: When computing the optimal transport plan for W , for cost matrix

Original: L ( B ) ⊤ Z , the complexity is O

Original: ( dk ( d + k ) log( d + k )) .

Original: The overall complexity of the BCOT algorithm is then O ( k ∥ B ∥ 0 ) + tnk ( n + k ) log( n + k )

Original: + tdk

Original: ( d + k ) log( d + k ))

Original: When using entropic regularization the complexity is smaller, since computing the optimal transport plan requires only a transformation of the inputs matrix, which takes O ( nk ) in the Z computation step and O ( dk ) for W .

Original: The ensuing application of the Sinkhorn-Knopp algorithm on the transformed matrices has complexities O ( tnk ) and O ( tdk ) for Z and W respectively, where t is the number of iterations necessary.

Original: The overall complexity of BCOT λ is then O ( k ∥ B ∥ 0 )

Original: + tnk + tdk ) , here t includes the number of iterations of our algorithm as well as that of Sinkhorn-Knopp.

Original: Appendix B Additional Experiments B.1 Experiments on Synthetic Data

Original: Datasets.

Original: As datasets with labels along both rows and columns are unavailable, we use synthetic data as in [28, 39].

Original: Their structure is shown in figure 5, while their characteristics are reported in table 5.

Original: [Figure]

Original: [Figure]

Original: Biclusters

Original: Sizes

Original: Sparse

Original: [Figure]

Original: Metrics.

Original: From row π r and column π c clusters, we use the Co-Clustering Accuracy (CCA) proposed by [18] to compare two pairs of partitions.

Original: CCA is defined from Clustering Accuracy (CA) associated to π r and π c in comparison with the true row and column clusters; it is given by

Original: [Equation]

Original: Results.

Original: We report the biclustering performance on the synthetic datasets in table 6.

Original: At least one of our models finds the perfect partition in all cases.

Original: These tests additionally allow us to show the utility of the the row cluster distribution r and column cluster distribution c .

Original: The use of these ground truth distributions resulted in an increase of 19.5 and 4.2 points for BCOT on C and D, and an increase of 0.3 and decrease of 0.8 for BCOT λ on C and D.

Original: [Table]

Original: B.2 Experiments on Gene Expression Data

Original: Datasets.

Original: The gene-expression matrices used are the Cumida Breast Cancer and Leukemia datasets.

Original: Their characteristics are shown in Table 7.

Original: [Table]

Original: Metrics.

Original: The metrics are the same as for document clustering.

Original: Performance In table 8, we report results on the two micro-array datasets, BCOT λ has the best performance on both of them.

Original: [Table]