Title: Multimodal 3D Genome Pre-training
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Genomics

Multimodal Foundation Model

Hi-C data

Chromatin Accessibility
Abstract: Deep learning techniques have driven significant progress in various analytical tasks within 3D genomics in computational biology. However, a holistic understanding of 3D genomics knowledge remains underexplored. Here, we propose ***MIX-HIC***, the first multimodal foundation model of 3D genome that integrates both 3D genome structure and epigenomic tracks, which obtains unified and comprehensive semantics. For accurate heterogeneous semantic fusion, we design the cross-modal interaction and mapping blocks for robust unified representation, yielding the accurate aggregation of 3D genome knowledge. Besides, we introduce the first large-scale dataset comprising over ***1 million*** pairwise samples of Hi-C contact maps and epigenomic tracks for high-quality pre-training, enabling the exploration of functional implications in 3D genomics. Extensive experiments show that MIX-HIC significantly surpasses existing state-of-the-art methods in diverse downstream tasks. This work provides a valuable resource for advancing 3D genomics research.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a multi-modal foundation model that takes as input Hi-C data and epigenomic data. The model itself includes some interesting techniques to capture cross-modal information while preserving both shared and distinct information from the two modalities.  After pre-training, the model is fine-tuned for several downstream tasks, where it outperforms state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nOne fairly significant omission is any reference to or comparison to the existing Hi-C foundation model, HiCFoundation (Wang et al. Biorxiv 2024).  The model proposed here is applicable to more tasks, since it takes in both Hi-C data and epigenomics, whereas HiCFoundation operates only on Hi-C data.  But for loop calling, both models could be compared head-to-head.  And certainly in the related work section, this existing Hi-C foundation model should be discussed.\n\nOne nice feature is that you have done train/test splitting at the level of both chromosomes and cell lines.  On the other hand, I think you should only report the cross-cell line + cross-chromosome results.  I.e., the experiments in Section 5.1 all should have been done in the test set cell lines and the test set chromosomes.  Later sections (e.g., results in Table 5) should also use this setting.  The cross-chromosomal prediction task is not really practically interesting.\n\nYou have also done a good job of identifying state-of-the-art methods to compare against (with the obvious exception of the other Hi-C foundation model mentioned above).\n\nI also like the model design, and in particular the idea of maintaining cell type-specific and cell type-agnostic latent representations of each of the inputs.\n\nThe sentence starting in line 398 does not make sense to me: \"Notably, the Hi-C contact map prediction task inherently predicts contact maps, thus lacking some experiments with Hi-C input.\"  Similarly, this sentence is hard to parse: \"Further constraint for orthogonal constaint [sic] is provided in Appendix C.4.\"  What is a constraint for a constraint?\n\nQUESTIONS:\nThe data set is described as if it's very large, but it's really not.  There are only six cell lines being used here, even though there are many more available paired datasets in ENCODE and 4DN.  Why was such a small number of cell lines used?\n\nThe text largely discussed the model inputs in terms of \"epigenomic tracks.\"  Only with some effort did I figure out that the inputs are DNAse-seq and ATAC-seq.  This seems like a weird choice, because these two are largely redundant with one another.  Why not use a histone modification instead?  If you stick with DNAse and ATAC-seq, then you should just refer to this as \"chromatin accessibility\" rather than the more general term \"epigenomic.\"\n\nThe paper claims (line 229) that MIX-HIC can infer a missing modality based on one modality.  I would like to see empirical evidence that this inference is accurate, both from Hi-C to chromatin accessibility and vice versa.\n\nI was pretty surprised by the shape of the learning curves in Figure 4.  I would expect that if you eliminate 99.99% of the training data the performance should drop precipitously for most methods.  I would also expect the gap between a pre-trained model and a non-pre-trained model to get much larger when the training set is tiny.  Why don't we see these trends?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-11"}

comments: SUMMARY:
This work proposes to pretrain models for better 3D genomics representation - jointly training models to integrates genome structure information (HiC) and epigenomic tracks. They curate a pretraining data of size 1 million (after cleaning). After training, the model performed the best across tasks such as 3D chromatin organization prediction, loop detection, and subsequence expression prediction.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper is well-written and the idea presented is very clear.
2. Bringing forward a dataset of this magnitude is important for this AI4Science domain.
3. The evaluation is pretty comprehensive and the baselines are strong enough to robustly support the superiority of the proposed model.

QUESTIONS:
1. Details and citation in Section 3. As the reviewer is not familiar with proprocessing the Hi-C contact maps and the epigenomic tracks, can the authors add citation regarding preprocessing? Also, here are a few questions
- Is log-transformation and 50 x 50 matrices standard? Would a 5kb resolution be too rough?
- Why filtering out 10% non-zero Hi-C interactions? How many data is filtered out during this process?

2. Error analysis:
- What are the main obstacles that stops the MiX-HIC-Bimodal to achieve even higher performance? Have the authors consider performing some error analysis or stratified analysis to dive further into the how and why the proposed model is better than others?

3. Open source
- Do the authors plan to open-source pre-training data?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-06-25

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a deep neural network (DNN) architecture called **MIX-HIC** for DNA analysis. The model takes as input Hi-C data alongside several genomic assays—including ATAC-seq, DNase-seq, CAGE-seq expression data, and CTCF ChIA-PET. It is pretrained with a representation-learning objective and then fine-tuned for downstream tasks. The contributions are as follows:\n\n- **A model for learning joint representations** of Hi-C and genomic assay tracks.  \n- **A large-scale paired dataset** comprising Hi-C maps and common genomic assay tracks.  \n- **Demonstrated applications** of the model to genomic track prediction.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper is well-motivated; Hi-C is an important data type that should be integrated into DNA-sequence modelling.  \n- The model architecture, training-loss design, and ablation study are comprehensive.  \n- The experiments include both conventional and deep-learning baselines to validate performance.  \n\n**Weaknesses**\n\n- The absence of a DNA-sequence modality restricts the model’s use cases, especially because many available tasks use DNA sequence as the sole input.  \n- The terms “3D genome” and “Hi-C” are not interchangeable. While Hi-C provides a good indicator of the genome’s 3D conformation, it does not fully capture it. Previous work predicts 3D genome structure from both Hi-C and DNA sequence. The DNA sequence itself determines physical properties—e.g., bond rigidity between adjacent nucleotides and the ease with which DNA bends—that may be important.  \n- See the *Questions* section for detailed inquiries about the experiments.\n\nQUESTIONS:\n- **What is the total parameter count for the MIX-HIC model?**  \n  It would be helpful if the authors showed how model size affects the pre-training loss. According to the ablation study in Figure 8, performance appears insensitive to model size, which is counter-intuitive. If that observation is accurate, why does model size have so little impact on downstream performance?\n\n- **Why was MSE loss used for CAGE-track prediction?**  \n  The common practice is to employ the Poisson negative log-likelihood loss, as in [1] and [2].\n\n- **Could you report Enformer’s zero-shot performance on the CAGE-track prediction task?**  \n  Including Enformer [1] as a baseline would clarify how MIX-HIC compares with sequence-based models.\n\n\n[1] Avsec, Ž., Agarwal, V., Visentin, D., Ledsam, J.R., Grabska-Barwinska, A., Taylor, K.R., Assael, Y., Jumper, J., Kohli, P. and Kelley, D.R., 2021. Effective gene expression prediction from sequence by integrating long-range interactions. Nature methods, 18(10), pp.1196-1203.]\n\n[2] Kelley, D. R. Cross-species regulatory sequence activity prediction. PLoS Comput. Biol. 16, e1008050 (2020).\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MIX-HIC, a multimodal pretraining framework designed to learn rich representations of 3D genome structure and epigenomic signals. The method integrates Hi-C data (3D genome conformation) and epigenomic tracks (e.g., DNase, ChIP-seq signals) across hundreds of cell types to train a transformer-based model.\n\nSTRENGTHS AND WEAKNESSES:\n>Strengths\n1. Innovative Multimodal Integration: Combines epigenomic tracks and Hi-C contact maps, which are typically modeled separately.\n\n2. Strong Pretraining Design: The contrastive learning objective aligns the 1D (track) and 2D (Hi-C) modalities, improving cross-view consistency.\n\n3. Empirical Performance: Demonstrates state-of-the-art results on multiple downstream tasks:\n\n>Weaknesses\n1. Lack of Interpretability: The model is powerful but opaque — biological interpretability of learned embeddings is not deeply explored.\n\n2. Limited Ablation on Modalities: The contribution of each modality (Hi-C vs. epigenomic) is not independently quantified in every downstream task.\n\n3. Sparse Evaluation on Rare Cell Types: Performance in rare or noisy datasets (e.g., low-coverage Hi-C) is not discussed.\n\nQUESTIONS:\nWe consider the prediction task: Use epigenomic tracks to predict the Hi-C contact map. How many benefits can you get from using contrastive learning? I.e., if you directly train a model (the same architecture as shown in the paper) using epigenomic tracks as inputs to predict the Hi-C contact map, what is the performance compared with your model?\n\nLIMITATIONS:\nThe author should provide more biological background in the paper, as this is a conference primarily focused on machine learning. The reviews are hard to understand the biological part.\n\nSparse Evaluation on Rare Cell Types: Performance in rare or noisy datasets (e.g., low-coverage Hi-C) is not discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}