Original: C HROM F ORMER : A transformer-based model for 3D genome structure prediction
Revised: C HROM F ORMER : A transformer-based model for 3D genome structure prediction

Original: Abstract
Revised: Abstract

Original: Recent research has shown that the three-dimensional (3D) genome structure is strongly linked to cell function.
Revised: Recent research has shown that the three-dimensional (3D) genome structure is strongly linked to cell function.

Original: Modeling the 3D genome structure can not only elucidate vital biological processes, but also reveal structural disruptions linked to disease.
Revised: Modeling the 3D genome structure can not only elucidate vital biological processes, but also reveal structural disruptions linked to disease.

Original: In the absence of experimental techniques able to determine 3D chromatin structure, this task is achieved computationally by exploiting chromatin interaction frequencies as measured by high-throughput chromosome conformation capture (Hi-C) data.
Revised: In the absence of experimental techniques able to determine 3D chromatin structure, this task is achieved computationally by exploiting chromatin interaction frequencies as measured by high-throughput chromosome conformation capture (Hi-C) data.

Original: However, existing methods are unsupervised, and limited by underlying assumptions.
Revised: However, existing methods are unsupervised, and limited by underlying assumptions.

Original: In this work, we present a novel framework for 3D chromatin structure prediction from Hi-C data.
Revised: In this work, we present a novel framework for 3D chromatin structure prediction from Hi-C data.

Original: The framework consists of, a novel synthetic data generation module that simulates realistic structures and corresponding Hi-C matrices, and C HROM F ORMER , a transformer-based model to predict 3D chromatin structures from standalone Hi-C data, while providing local structural-level confidence estimates.
Revised: The framework consists of, a novel synthetic data generation module that simulates realistic structures and corresponding Hi-C matrices, and C HROM F ORMER , a transformer-based model to predict 3D chromatin structures from standalone Hi-C data, while providing local structural-level confidence estimates.

Original: Our solution outperforms existing methods when tested on unseen synthetic data, and achieves comparable results on experimental data for a full eukaryotic genome.
Revised: Our solution outperforms existing methods when tested on unseen synthetic data, and achieves comparable results on experimental data for a full eukaryotic genome.

Original: The code, data, and models can be accessed at https://github.com/AI4SCR/ChromFormer .
Revised: The code, data, and models can be accessed at https://github.com/AI4SCR/ChromFormer .

Original:  Introduction
Revised:  Introduction

Original: A plethora of recent studies have illustrated that the 3D genome organization affects if, when, and how genetic information is expressed [2] to ensure controlled execution of essential cellular processes [20, 19].
Revised: A plethora of recent studies have illustrated that the 3D genome organization affects if, when, and how genetic information is expressed [2] to ensure controlled execution of essential cellular processes [20, 19].

Original: As genome misfolding is increasingly linked to different diseases [5, 13], modeling the 3D structure of chromatin can not only provide mechanistic insights on vital cell processes, but also identify structural disease biomarkers [1].
Revised: As genome misfolding is increasingly linked to different diseases [5, 13], modeling the 3D structure of chromatin can not only provide mechanistic insights on vital cell processes, but also identify structural disease biomarkers [1].

Original: However, there exists no method to experimentally determine 3D genome structure.
Revised: However, there exists no method to experimentally determine 3D genome structure.

Original: Chromatin organization is studied implicitly using high-throughput chromosome conformation capture (Hi-C) experiments [12] that produce a contact map containing interaction frequencies among tiny DNA fragments (loci) across the genome.
Revised: Chromatin organization is studied implicitly using high-throughput chromosome conformation capture (Hi-C) experiments [12] that produce a contact map containing interaction frequencies among tiny DNA fragments (loci) across the genome.

Original: In absence of experimental methods, a plethora of computational approaches that infer 3D chromatin structure from Hi-C contact maps have emerged [17].
Revised: In absence of experimental methods, a plethora of computational approaches that infer 3D chromatin structure from Hi-C contact maps have emerged [17].

Original: Most methods map interaction frequencies to 3D Euclidean distances via a parametric transfer function and use them as constraints to solve an optimization problem [18].
Revised: Most methods map interaction frequencies to 3D Euclidean distances via a parametric transfer function and use them as constraints to solve an optimization problem [18].

Original: However, the transfer function differs between organisms and resolutions [26].
Revised: However, the transfer function differs between organisms and resolutions [26].

Original: Recently, data-driven methods employing manifold learning to learn a 3D representation of
Revised: Recently, data-driven methods employing manifold learning to learn a 3D representation of

Original: Hi-C data without a transfer function assumption have emerged, such as GEM [27]) and REACH-3D 36th Conference
Revised: Hi-C data without a transfer function assumption have emerged, such as GEM [27]) and REACH-3D 36th Conference

Original: Still, in the absence of ground truth, the learning capacity of these unsupervised methods is limited, as their architecture is not designed to generate encodings resembling 3D structures.
Revised: Still, in the absence of ground truth, the learning capacity of these unsupervised methods is limited, as their architecture is not designed to generate encodings resembling 3D structures.

Original: In this work we propose a novel framework for predicting 3D chromatin structure, inspired by TECH-3D, a state-of-the-art transfer learning method [6].
Revised: In this work we propose a novel framework for predicting 3D chromatin structure, inspired by TECH-3D, a state-of-the-art transfer learning method [6].

Original: In contrast to TECH-3D, we propose a synthetic data generation module that simulates biologically-informed 3D structures and corresponding Hi-C matrices using optimal transport (OT), and C HROM F ORMER , a transformer-based model that predicts 3D structures from Hi-C matrices, while also estimating the confidence of the prediction.
Revised: In contrast to TECH-3D, we propose a synthetic data generation module that simulates biologically-informed 3D structures and corresponding Hi-C matrices using optimal transport (OT), and C HROM F ORMER , a transformer-based model that predicts 3D structures from Hi-C matrices, while also estimating the confidence of the prediction.

Original:  Methods
Revised:  Methods

Original: Our proposed framework, presented in Figure 1, consists of a synthetic data generation component that simulates paired chromatin structures and Hi-C matrices, and C HROM F ORMER , a transformerbased model that is trained on the synthetic data and learns to map Hi-C matrices to 3D chromatin structures.
Revised: Our proposed framework, presented in Figure 1, consists of a synthetic data generation component that simulates paired chromatin structures and Hi-C matrices, and C HROM F ORMER , a transformerbased model that is trained on the synthetic data and learns to map Hi-C matrices to 3D chromatin structures.

Original: C HROM F ORMER also outputs loci-level confidence values using a calibration network that corrects the predicted confidence logits to estimated confidence values.
Revised: C HROM F ORMER also outputs loci-level confidence values using a calibration network that corrects the predicted confidence logits to estimated confidence values.

Original: Synthetic Data Generation : To address the lack of ground truth data, in our framework we first propose a Uniform Cluster Walk to generate pairs of synthetic 3D chromatin structures and Hi-C matrices.
Revised: Synthetic Data Generation : To address the lack of ground truth data, in our framework we first propose a Uniform Cluster Walk to generate pairs of synthetic 3D chromatin structures and Hi-C matrices.

Original: To resemble real 3D chromatin structures, the synthetic structures need to satisfy the following constraints: i) smoothness and compactness, ii) equidistance of consecutive loci, and iii) presence of clustered regions resembling topologically associated domains (TADs) [16].
Revised: To resemble real 3D chromatin structures, the synthetic structures need to satisfy the following constraints: i) smoothness and compactness, ii) equidistance of consecutive loci, and iii) presence of clustered regions resembling topologically associated domains (TADs) [16].

Original: To this end, given a number of n loci, we generate a structure S ∈ R n × 3 , while employing rejection sampling at different levels to ensure biological plausibility.
Revised: To this end, given a number of n loci, we generate a structure S ∈ R n × 3 , while employing rejection sampling at different levels to ensure biological plausibility.

Original: More formally, for a given locus { p i , ⃗q i }, we first sample r ∼ U ( − 1 , 1)
Revised: More formally, for a given locus { p i , ⃗q i }, we first sample r ∼ U ( − 1 , 1)

Original: where || r || 2 ≤ 1 to ensure isotropy.
Revised: where || r || 2 ≤ 1 to ensure isotropy.

Original: ⃗q i +1 is then chosen as (1 − δ ) ⃗q
Revised: ⃗q i +1 is then chosen as (1 − δ ) ⃗q

Original: i + δ⃗r/ || r || 2 , where δ is a smoothness parameter, followed by L 2 normalization.
Revised: i + δ⃗r/ || r || 2 , where δ is a smoothness parameter, followed by L 2 normalization.

Original: The next locus p i +1 is derived as p i + ⃗q
Revised: The next locus p i +1 is derived as p i + ⃗q

Original: i +1 .
Revised: i +1 .

Original: To ensure S is compact, p i +1 is subject to rejection sampling according to a Gaussian
Revised: To ensure S is compact, p i +1 is subject to rejection sampling according to a Gaussian

Original: N ( p 0 , σ ) , where p 0 is the starting locus and σ controls the degree of compactness.
Revised: N ( p 0 , σ ) , where p 0 is the starting locus and σ controls the degree of compactness.

Original: p i +1 is rejected if,
Revised: p i +1 is rejected if,

Original: [Equation]
Revised: [Table]

Original: where, ρ is a sampled probability.
Revised: where, ρ is a sampled probability.

Original: Further, we create a TAD of length l << n , starting at p i , by using rejection sampling with a Gaussian N ( p i , ν ) and Equation 1.
Revised: Further, we create a TAD of length l << n , starting at p i , by using rejection sampling with a Gaussian N ( p i , ν ) and Equation 1.

Original: As TADs are more compact, we set ν < σ .
Revised: As TADs are more compact, we set ν < σ .

Original: We enforce a distance of at least k loci after a TAD ending before being eligible to form another TAD.
Revised: We enforce a distance of at least k loci after a TAD ending before being eligible to form another TAD.

Original: More details on the structure generation are given in the Appendix A.1.
Revised: More details on the structure generation are given in the Appendix A.1.

Original: S is then centered and normalized by the maximum L 2 norm of loci in S. The corresponding distance matrix
Revised: S is then centered and normalized by the maximum L 2 norm of loci in S. The corresponding distance matrix

Original: D and Hi-C matrix H are generated using pairwise Euclidean distances between all loci in S, and a power law D − 1 /α , respectively, where α is a tunable parameter.
Revised: D and Hi-C matrix H are generated using pairwise Euclidean distances between all loci in S, and a power law D − 1 /α , respectively, where α is a tunable parameter.

Original: We normalize H using iterative correction and eigenvector decomposition (ICE) normalization [14] and employ optimal transport (OT) [25] to match the synthetic to real Hi-C matrix.
Revised: We normalize H using iterative correction and eigenvector decomposition (ICE) normalization [14] and employ optimal transport (OT) [25] to match the synthetic to real Hi-C matrix.

Original: First, the real Hi-C is min-max normalized, and then OT is applied to transform the frequency distribution of the values in the synthetic Hi-C matrix to the frequency distribution of the values in the real Hi-C matrices (more details in the Appendix A.2).
Revised: First, the real Hi-C is min-max normalized, and then OT is applied to transform the frequency distribution of the values in the synthetic Hi-C matrix to the frequency distribution of the values in the real Hi-C matrices (more details in the Appendix A.2).

Original: Notably, the usage of OT mitigates the distribution gap without any training and without requiring the real chromatin structures.
Revised: Notably, the usage of OT mitigates the distribution gap without any training and without requiring the real chromatin structures.

Original: C HROM
Revised: C HROM

Original: F
Revised: F

Original: ORMER :
Revised: ORMER :

Original: The
Revised: The

Original: synthetic data generator outputs sets of three matrices, S ∈ R n × 3 , D
Revised: synthetic data generator outputs sets of three matrices, S ∈ R n × 3 , D

Original: ∈
Revised: ∈

Original: R
Revised: R

Original: n ×
Revised: n ×

Original: n ,
Revised: n ,

Original: and H ∈ R n ×
Revised: and H ∈ R n ×

Original: n
Revised: n

Original: . C HROM F
Revised: . C HROM F

Original: ORMER
Revised: ORMER

Original: operates on H
Revised: operates on H

Original: to
Revised: to

Original: predict
Revised: predict

Original: ˜ S
Revised: ˜ S

Original: ˜
Revised: ˜

Original: D and confidence
Revised: D and confidence

Original: ˜
Revised: ˜

Original: C
Revised: C

Original: ∈ n ×. C HROM F ORMER builds on the hypothesis that the relative spatial distribution influences the formulation of a 3D chromatin structure, and we use a Transformer [24] to exploit this hypothesis.
Revised: ∈ n ×. C HROM F ORMER builds on the hypothesis that the relative spatial distribution influences the formulation of a 3D chromatin structure, and we use a Transformer [24] to exploit this hypothesis.

Original: First, C HROM F ORMER employs E proj : R n × n → R n × d to project H onto lower dimensional embeddings E ∈ R n × d , d < n .
Revised: First, C HROM F ORMER employs E proj : R n × n → R n × d to project H onto lower dimensional embeddings E ∈ R n × d , d < n .

Original: E is summed with the positional embeddings from E pos : Z n → R n × d to include the loci spatial information.
Revised: E is summed with the positional embeddings from E pos : Z n → R n × d to include the loci spatial information.

Original: E is processed by a Multi-head Transformer, consisting of two transformer encoders, which utilizes self-attention to learn the inter-loci relations and contextualizes
Revised: E is processed by a Multi-head Transformer, consisting of two transformer encoders, which utilizes self-attention to learn the inter-loci relations and contextualizes

Original: E. Subsequently, E is processed by E str : R n × d → R n × 3 , consisting of a linear decoder, to predict 3D loci coordinates.
Revised: E. Subsequently, E is processed by E str : R n × d → R n × 3 , consisting of a linear decoder, to predict 3D loci coordinates.

Original: These coordinates are centered and divided by the maximum loci L 2 norm to produce ˜ S. ˜ D is derived from ˜ S by computing pairwise Euclidean distances.
Revised: These coordinates are centered and divided by the maximum loci L 2 norm to produce ˜ S. ˜ D is derived from ˜ S by computing pairwise Euclidean distances.

Original: A major challenge in the addressed task is the lack of ground truth 3D chromatin structures, which inhibits assessment of model predictions.
Revised: A major challenge in the addressed task is the lack of ground truth 3D chromatin structures, which inhibits assessment of model predictions.

Original: As a proxy, we approximate the prediction uncertainty by producing loci-level confidence scores.
Revised: As a proxy, we approximate the prediction uncertainty by producing loci-level confidence scores.

Original: Formally, we linearly project E to ˜ C, which represents the non-normalised predictions/logits for belonging to 100 confidence interval bins.
Revised: Formally, we linearly project E to ˜ C, which represents the non-normalised predictions/logits for belonging to 100 confidence interval bins.

Original: ˜ C is used for confidence score optimization and model calibration as described below.
Revised: ˜ C is used for confidence score optimization and model calibration as described below.

Original: Loss objectives :
Revised: Loss objectives :

Original: We optimize L = L d + λ k L k + λ c L c , consisting of three loss terms, to tune
Revised: We optimize L = L d + λ k L k + λ c L c , consisting of three loss terms, to tune

Original: C HROM F ORMER .
Revised: C HROM F ORMER .

Original: L d is the mean squared error (MSE) between D and ˜ D to ensure the distance preservation.
Revised: L d is the mean squared error (MSE) between D and ˜ D to ensure the distance preservation.

Original: The predicted structure ˜ S can be oriented and scaled differently with respect to S while being similar in structure.
Revised: The predicted structure ˜ S can be oriented and scaled differently with respect to S while being similar in structure.

Original: To mitigate this, we use the Kabsch algorithm [10] to align ˜ S to S, resulting in ˜ S aligned .
Revised: To mitigate this, we use the Kabsch algorithm [10] to align ˜ S to S, resulting in ˜ S aligned .

Original: L k computes the MSE between S and ˜ S aligned .
Revised: L k computes the MSE between S and ˜ S aligned .

Original: L c aims to ensure the correct prediction loci confidence, computed using a cross entropy between Softmax( ˜ C) and estimated confidence scores C. C is derived using a modified AlphaFold’s [9] Local Distance Difference Test [15].
Revised: L c aims to ensure the correct prediction loci confidence, computed using a cross entropy between Softmax( ˜ C) and estimated confidence scores C. C is derived using a modified AlphaFold’s [9] Local Distance Difference Test [15].

Original: First, an inverse relative error ReLU (cid:0) 1 − | D − ˜ D
Revised: First, an inverse relative error ReLU (cid:0) 1 − | D − ˜ D

Original: | / D (cid:1) is computed to assign low confidence to large deviations between D and ˜ D .
Revised: | / D (cid:1) is computed to assign low confidence to large deviations between D and ˜ D .

Original: The error is row-normalized excluding self-interacting diagonal to produce the scores.
Revised: The error is row-normalized excluding self-interacting diagonal to produce the scores.

Original: C is defined as the one hot encoding of ⌊ scores × 100 ⌋ .
Revised: C is defined as the one hot encoding of ⌊ scores × 100 ⌋ .

Original: All implementation details are given in the Appendix A.3.
Revised: All implementation details are given in the Appendix A.3.

Original: Calibration network :
Revised: Calibration network :

Original: In the absence of real ground truth structures, it is crucial to define loci-level confidence scores.
Revised: In the absence of real ground truth structures, it is crucial to define loci-level confidence scores.

Original: L c is weighted low during model training to give more emphasize to accurate structure prediction that induces sub-optimal confidence ˜ C .
Revised: L c is weighted low during model training to give more emphasize to accurate structure prediction that induces sub-optimal confidence ˜ C .

Original: We design a calibration network to correct for this and improve confidence estimates on unseen real test data.
Revised: We design a calibration network to correct for this and improve confidence estimates on unseen real test data.

Original: Additionally, the network can also correct for overconfident predictions in ˜ C .
Revised: Additionally, the network can also correct for overconfident predictions in ˜ C .

Original: To train the calibration network, we use C and ˜ C from the synthetic validation set v , which are then split into C v tr , C v val , ˜ C v tr , and ˜ C v val for network training and validation.
Revised: To train the calibration network, we use C and ˜ C from the synthetic validation set v , which are then split into C v tr , C v val , ˜ C v tr , and ˜ C v val for network training and validation.

Original: We train three networks supporting three calibration techniques, i .
Revised: We train three networks supporting three calibration techniques, i .

Original: e ., temperature scaling [7], isotonic calibration [8], and beta calibration [11].
Revised: e ., temperature scaling [7], isotonic calibration [8], and beta calibration [11].

Original: The networks transform ˜ C v tr to calibrated [˜ C v tr ] to match C v tr .
Revised: The networks transform ˜ C v tr to calibrated [˜ C v tr ] to match C v tr .

Original: The performance of the networks is quantified by,
Revised: The performance of the networks is quantified by,

Original: MSE ( calibrated [˜ C v val ] , C v val ) − MSE
Revised: MSE ( calibrated [˜ C v val ] , C v val ) − MSE

Original: ˜ C v val , C v val ) .
Revised: ˜ C v val , C v val ) .

Original: Upon training, these networks are applied on ˜
Revised: Upon training, these networks are applied on ˜

Original: C, predicted by C HROM F ORMER , for the real test data.
Revised: C, predicted by C HROM F ORMER , for the real test data.

Original:  Results and Discussion
Revised:  Results and Discussion

Original: Public Datasets : We exploit the following publicly available datasets: the T RUSSART dataset [23], a set of 100 simulated structures that contain loops, TADs and long-range interactions paired with a single Hi-C matrix (202 loci at 5 kilobase resolution), and the T ANIZAWA dataset [22], a real experimental single Hi-C measurement across the full fission yeast genome that contains 1258 loci at 5 kilobase resolution together with 18 ground truth pair-wise 3D Euclidean distances as measured by
Revised: Public Datasets : We exploit the following publicly available datasets: the T RUSSART dataset [23], a set of 100 simulated structures that contain loops, TADs and long-range interactions paired with a single Hi-C matrix (202 loci at 5 kilobase resolution), and the T ANIZAWA dataset [22], a real experimental single Hi-C measurement across the full fission yeast genome that contains 1258 loci at 5 kilobase resolution together with 18 ground truth pair-wise 3D Euclidean distances as measured by

Original: Fluorescence In Situ Hybridization (FISH).
Revised: Fluorescence In Situ Hybridization (FISH).

Original: Synthetic dataset : Using the Uniform Cluster Walk algorithm, we generated 1000 and 500 structures with 202 and 1258 loci to match the T RUSSART and T ANIZAWA datasets, respectively (see Appendix
Revised: Synthetic dataset : Using the Uniform Cluster Walk algorithm, we generated 1000 and 500 structures with 202 and 1258 loci to match the T RUSSART and T ANIZAWA datasets, respectively (see Appendix

Original: Figure 4).
Revised: Figure 4).

Original: We follow a 4:1 train-validation split for training C HROM F ORMER .
Revised: We follow a 4:1 train-validation split for training C HROM F ORMER .

Original: The validation set is further split into a 9:1 ratio for the calibration.
Revised: The validation set is further split into a 9:1 ratio for the calibration.

Original: Evaluation metrics : For the T RUSSART data, the predicted ˜ S by C HROM F ORMER is aligned with the { S i } i =1:100 ground truth structures using the Kabsch algorithm and the mean of { MSE ( S i , ˜ S ) } i =1: is used to quantify the model performance.
Revised: Evaluation metrics : For the T RUSSART data, the predicted ˜ S by C HROM F ORMER is aligned with the { S i } i =1:100 ground truth structures using the Kabsch algorithm and the mean of { MSE ( S i , ˜ S ) } i =1: is used to quantify the model performance.

Original: For the T ANIZAWA data, we match the experimentally determined 18 ground truth distances to the equivalent entries in the predicted ˜ D, and compute the Pearson correlation between the two sets.
Revised: For the T ANIZAWA data, we match the experimentally determined 18 ground truth distances to the equivalent entries in the predicted ˜ D, and compute the Pearson correlation between the two sets.

Original: To quantify the calibration networks, we compute
Revised: To quantify the calibration networks, we compute

Original: MSE ( C , Softmax (˜ C )) on the Trussart dataset.
Revised: MSE ( C , Softmax (˜ C )) on the Trussart dataset.

Original: ReLU (cid:0) 1 − | D − ˜ D
Revised: ReLU (cid:0) 1 − | D − ˜ D

Original: | / D (cid:1) is row-normalized to get C, where D is derived from the mean of the Trussart structures.
Revised: | / D (cid:1) is row-normalized to get C, where D is derived from the mean of the Trussart structures.

Original: Qualitative and quantitative analysis : Figure 2 displays a synthetic structure S and its corresponding
Revised: Qualitative and quantitative analysis : Figure 2 displays a synthetic structure S and its corresponding

Original: Hi-C matrix H. We clearly observe that S is smooth with equidistant consecutive points and that it includes chromatin loops and TAD-like domains, that are also visible in H.
Revised: Hi-C matrix H. We clearly observe that S is smooth with equidistant consecutive points and that it includes chromatin loops and TAD-like domains, that are also visible in H.

Original: The Hi-C matrices in
Revised: The Hi-C matrices in

Original: Figure 2(b) and (e) highlight the distribution gap between the synthetic and the real Trussart data, which is progressively reduced following ICE (c) and OT (d), leading to a final Hi-C matrix that is closer to the real data distribution while retaining the initial Hi-C information.
Revised: Figure 2(b) and (e) highlight the distribution gap between the synthetic and the real Trussart data, which is progressively reduced following ICE (c) and OT (d), leading to a final Hi-C matrix that is closer to the real data distribution while retaining the initial Hi-C information.

Original: Appendix Figure 5 presents more sample examples.
Revised: Appendix Figure 5 presents more sample examples.

Original: Figure 3 presents the ground-truth Trussart structure (mean of 100 simulations) and the structure as predicted by C HROM F ORMER using the Trussart Hi-C.
Revised: Figure 3 presents the ground-truth Trussart structure (mean of 100 simulations) and the structure as predicted by C HROM F ORMER using the Trussart Hi-C.

Original: We observe that the predicted structure greatly resembles the real one as it preserves all loops and TADs.
Revised: We observe that the predicted structure greatly resembles the real one as it preserves all loops and TADs.

Original: Predicted structures by the competing methods are presented in Appendix Figure 7.
Revised: Predicted structures by the competing methods are presented in Appendix Figure 7.

Original: Methods
Revised: Methods

Original: [Table]
Revised: [Table]

Original: Quantitative results on the test datasets are presented in Table 1.
Revised: Quantitative results on the test datasets are presented in Table 1.

Original: C HROM F ORMER achieves the lowest Kabsch distance on the Trussart data, outperforming competing methods by a large margin.
Revised: C HROM F ORMER achieves the lowest Kabsch distance on the Trussart data, outperforming competing methods by a large margin.

Original: On the Tanizawa data, C HROM F ORMER outperforms the state-of-the-art method TECH-3D [6], with comparable performance to MiniMDS [18] and GEM [27].
Revised: On the Tanizawa data, C HROM F ORMER outperforms the state-of-the-art method TECH-3D [6], with comparable performance to MiniMDS [18] and GEM [27].

Original: Still, qualitative results (Appendix
Revised: Still, qualitative results (Appendix

Original: Figure 8) demonstrate that C HROM F ORMER reconstructs smooth and loopy structures with a clear distinction between chromosomes, while miniMDS outputs a structure which is a compact set of points without loops and biological feasibility.
Revised: Figure 8) demonstrate that C HROM F ORMER reconstructs smooth and loopy structures with a clear distinction between chromosomes, while miniMDS outputs a structure which is a compact set of points without loops and biological feasibility.

Original: Among the calibration networks, isotonic calibration results in the best score, followed by beta calibration.
Revised: Among the calibration networks, isotonic calibration results in the best score, followed by beta calibration.

Original: The average ground truth confidence, and the average predicted confidence before and after isotonic calibration are 88.90%, 87.40%, and 88.24%, respectively.
Revised: The average ground truth confidence, and the average predicted confidence before and after isotonic calibration are 88.90%, 87.40%, and 88.24%, respectively.

Original: The average uncalibrated confidence is already good, and the calibration step brings it closer to the upper bound (Appendix Table 2 and Figure 9).
Revised: The average uncalibrated confidence is already good, and the calibration step brings it closer to the upper bound (Appendix Table 2 and Figure 9).

Original: On the MSE metric, isotonic calibration produces a score of 8.64 compared to 10.39 before calibration.
Revised: On the MSE metric, isotonic calibration produces a score of 8.64 compared to 10.39 before calibration.

Original:  Conclusion and Future work
Revised:  Conclusion and Future work

Original: In this work we present a novel approach for reconstructing 3D genome structures that overcomes the lack of ground-truth data using a biologically-informed synthetic data generation and a transformer- based architecture.
Revised: In this work we present a novel approach for reconstructing 3D genome structures that overcomes the lack of ground-truth data using a biologically-informed synthetic data generation and a transformer- based architecture.

Original: Our model can accurately reconstruct publicly available synthetic structures, outperforming state-of-the-art methods and shows comparable performance to state-of-the-art algorithms on experimental microscopy measurements from real genomes.
Revised: Our model can accurately reconstruct publicly available synthetic structures, outperforming state-of-the-art methods and shows comparable performance to state-of-the-art algorithms on experimental microscopy measurements from real genomes.

Original: A limitation of our framework is data generation, a computationally intensive process that needs to be tuned to tested datasets.
Revised: A limitation of our framework is data generation, a computationally intensive process that needs to be tuned to tested datasets.

Original: We are currently tuning our model hyperparameters and extending its validation to other datasets, especially single-cell Hi-C data [21]
Revised: We are currently tuning our model hyperparameters and extending its validation to other datasets, especially single-cell Hi-C data [21]

Original: A Appendix A.1 Uniform Cluster Walk algorithm
Revised: A.1 Uniform Cluster Walk algorithm

Original: 1: Inputs:
Revised: 1: Inputs:

Original: • n : number of loci in a synthetic structure • σ : degree of compactness of the structure • ν : degree of compactness of a TAD • β : probability of creating a TAD • δ : smoothness coefficient between consecutive loci • l : number of loci in a TAD •  : number of aging loci before being eligible to start a new TAD
Revised: • n : number of loci in a synthetic structure • σ : degree of compactness of the structure • ν : degree of compactness of a TAD • β : probability of creating a TAD • δ : smoothness coefficient between consecutive loci • l : number of loci in a TAD • k : number of aging loci before being eligible to start a new TAD

Original: 2: 3: Initialize:
Revised: 2: 3: Initialize:

Original: ⃗q
Revised: ⃗q

Original: 4: 5: function GET_NEXT_LOCUS( p in , ⃗q in , p center , v ) ▷ Get the next locus co-ordinates 6: ρ ∼ [0 , 1] 7: while
Revised: 4: 5: function GET_NEXT_LOCUS( p in , ⃗q in , p center , v ) ▷ Get the next locus co-ordinates 6: ρ ∼ [0 , 1] 7: while

Original: [Table]
Revised: [Table]

Original: [Equation] [Table] do ▷ Gaussian rejection criterion 8: r ∼ U ( − 1 , 1) 9: while ∥ r ∥ 2 ≥ 1 do 10: r ∼ U ( − 1 , 1) 11: end while 12: ⃗q out ← (1 − δ ) ⃗q in + δ ⃗r ∥
Revised: ρ do ▷ Gaussian rejection criterion 8: r ∼ U ( − 1 , 1) 9: while ∥ r ∥ 2 ≥ 1 do 10: r ∼ U ( − 1 , 1) 11: end while 12: ⃗q out ← (1 − δ ) ⃗q in + δ ⃗r ∥

Original: r ∥▷ Smoothness constraint 13: p out ← p in + ⃗q out 14: end while 15: return p out , q out 16: end function 17: 18: i tadBegin ←19: i tadEnd ←20: for i
Revised: r ∥▷ Smoothness constraint 13: p out ← p in + ⃗q out 14: end while 15: return p out , q out 16: end function 17: 18: i tadBegin ←19: i tadEnd ←20: for i

Original: = 1 to n do▷ Loop to derive structure defining loci 21: if i < i tadEnd then 22: continue; 23: end if 24: t ∼ [0 , 1] 25: if t > β and i < n − l
Revised: = 1 to n do 21: if i < i tadEnd then 22: continue; 23: end if 24: t ∼ [0 , 1] 25: if t > β and i < n − l

Original: and i > i tadBegin then ▷ Begin a TAD 26: i tadBegin ← i + l + k 27:
Revised: and i > i tadBegin then ▷ Begin a TAD 26: i tadBegin ← i + l + k 27:

Original: i tadEnd ← i + l 28: p center ← p i −29: for j = 0 to l − 1 do 30: p i + j , ⃗q i + j ← GET_NEXT_LOCUS
Revised: i tadEnd ← i + l 28: p center ← p i −29: for j = 0 to l − 1 do 30: p i + j , ⃗q i + j ← GET_NEXT_LOCUS

Original: ( p i + j − 1 , ⃗q i + j − 1 , p center , ν ) 31: end for 32: else 33: p i , ⃗q i ← GET_NEXT_LOCUS
Revised: ( p i + j − 1 , ⃗q i + j − 1 , p center , ν ) 31: end for 32: else 33: p i , ⃗q i ← GET_NEXT_LOCUS

Original: ( p i − 1 , ⃗q i − 1 , p 0 , σ ) 34: end if 35: end for
Revised: ( p i − 1 , ⃗q i − 1 , p 0 , σ ) 34: end if 35: end for

Original: A.2 Optimal transport
Revised: A.2 Optimal transport

Original: We apply OT as an unsupervised domain adaptation method to minimize the differences in values between the synthetic and the real Hi-C matrices by matching the frequency distribution of the synthetic and real values.
Revised: We apply OT as an unsupervised domain adaptation method to minimize the differences in values between the synthetic and the real Hi-C matrices by matching the frequency distribution of the synthetic and real values.

Original: Figure 5 shows the histograms of the synthetic Hi-C values before and after applying OT.
Revised: Figure 5 shows the histograms of the synthetic Hi-C values before and after applying OT.

Original: In particular, given a set of synthetic and real Hi-C matrices, we first compute the histograms of the frequencies of the values in the two sets by using 30 uniformly-spaced bins.
Revised: In particular, given a set of synthetic and real Hi-C matrices, we first compute the histograms of the frequencies of the values in the two sets by using 30 uniformly-spaced bins.

Original: Then, the histograms are normalized to have a total mass of 1, so that they can be considered discrete probability distributions.
Revised: Then, the histograms are normalized to have a total mass of 1, so that they can be considered discrete probability distributions.

Original: Given the two normalized histograms α, β , their supports A, B with equal size ( | A | = | B | = 30 ), and squared euclidean distance costs c : A × B → R ; c ( a, b ) =
Revised: Given the two normalized histograms α, β , their supports A, B with equal size ( | A | = | B | = 30 ), and squared euclidean distance costs c : A × B → R ; c ( a, b ) =

Original: ( a − b ) 2 , the OT problem is defined as follows:
Revised: ( a − b ) 2 , the OT problem is defined as follows:

Original: [Equation]
Revised: [Equation]

Original: The solution of the above linear program is the optimal transport plan Γ that maps each bin a ∈ A of the source histogram α to one or more bins b ∈ B of the target histogram β .
Revised: The solution of the above linear program is the optimal transport plan Γ that maps each bin a ∈ A of the source histogram α to one or more bins b ∈ B of the target histogram β .

Original: The value of each Γ( a, b ) denotes the amount of transported mass from α ( a ) to β ( b ) .
Revised: The value of each Γ( a, b ) denotes the amount of transported mass from α ( a ) to β ( b ) .

Original: Finally, the source histogram is transported to the target histogram by using the optimal transport plan Γ .
Revised: Finally, the source histogram is transported to the target histogram by using the optimal transport plan Γ .

Original: With the transported histogram, the values of the synthetic Hi-C matrices are transformed to match the frequency distribution of the values of the real Hi-C matrices.
Revised: With the transported histogram, the values of the synthetic Hi-C matrices are transformed to match the frequency distribution of the values of the real Hi-C matrices.

Original: For the implementation, we used the OT solver from Python Optimal Transport [4] library.
Revised: For the implementation, we used the OT solver from Python Optimal Transport [4] library.

Original: Notably, OT transformation is independent of the specific structure of the target Hi-C, and only relies on the frequency distribution of values in the target
Revised: Notably, OT transformation is independent of the specific structure of the target Hi-C, and only relies on the frequency distribution of values in the target

Original: Hi-C matrices.
Revised: Hi-C matrices.

Original: Figure 2 shows the effect of the OT transformation with source
Revised: Figure 2 shows the effect of the OT transformation with source

Original: A.3 Implementation details
Revised: A.3 Implementation details

Original: On a local CPU, it takes 30 minutes to generate the Trussart synthetic data, and 6 hours to generate the Fission Yeast synthetic data.
Revised: On a local CPU, it takes 30 minutes to generate the Trussart synthetic data, and 6 hours to generate the Fission Yeast synthetic data.

Original: To create the T RUSSART and T ANIZAWA synthetic structures, the selected parameters were:
Revised: To create the T RUSSART and T ANIZAWA synthetic structures, the selected parameters were:

Original: Parameters
Revised: Parameters

Original: T RUSSART
Revised: T RUSSART

Original: T ANIZAWA # of loci n  smoothness δ 0.
Revised: T ANIZAWA # of loci n  smoothness δ 0.

Original: degree of compactness of a structure σ  probability of creating a TAD 0.
Revised: degree of compactness of a structure σ  probability of creating a TAD 0.

Original: degree of compactness of a TAD σ 1.
Revised: degree of compactness of a TAD σ 1.

Original: # of loci per TAD l  # of aging loci k  power law α
Revised: # of loci per TAD l  # of aging loci k  power law α

Original: C HROM F ORMER model and training parameters were the same for both the datasets:
Revised: C HROM F ORMER model and training parameters were the same for both the datasets:

Original: • embedding dimension d :• E proj : number of layers = 1, hidden dimensions =
Revised: • embedding dimension d :• E proj : number of layers = 1, hidden dimensions =

Original: • # transformer blocks: 2 (dimension of the feedforward networks 100 and 48, respectively) • # heads in a transformer block:• E str : number of layers = 1, hidden dimensions =
Revised: • # transformer blocks: 2 (dimension of the feedforward networks 100 and 48, respectively) • # heads in a transformer block:• E str : number of layers = 1, hidden dimensions =

Original: • E conf : number of layers = 1, hidden dimensions =
Revised: • E conf : number of layers = 1, hidden dimensions =

Original: • E cal : number of layers = 1, hidden dimensions =
Revised: • E cal : number of layers = 1, hidden dimensions =

Original: Training:
Revised: Training:

Original: #epochs: 100, batch size: 10, optimized: Adamw with lr: 0.0005,
Revised: #epochs: 100, batch size: 10, optimized: Adamw with lr: 0.0005,

Original: : 1e-
Revised: : 1e-

Original: Loss weights: λ k = 0 .
Revised: Loss weights: λ k = 0 .

Original: 1 and λ c = 0 .
Revised: 1 and λ c = 0 .

Original: A.4 Quantitative and qualitative results
Revised: A.4 Quantitative and qualitative results

Original: Methods average(Softmax( ˜ C))
Revised: Methods average(Softmax( ˜ C))

Original: [in %][ ↑ ] MSE(C, 100 × Softmax( ˜ C))
Revised: [in %][ ↑ ] MSE(C, 100 × Softmax( ˜ C))

Original: [ ↓ ]
Revised: [ ↓ ]

Original: Ground truth 88.
Revised: Ground truth 88.

Original: Before calibration 87.
Revised: Before calibration 87.

Original: Temperature calibration [
Revised: Temperature calibration [

Original: [Table]
Revised: [Table]