Kati8/ICLR_2023_2025 huggingface dataset:
authors: Philip Kenneweg
authors: Raghuram Dandinasivara Rangaram
authors: Xiao Luo
authors: Alexander Schoenhuth
reviews: 
Comments: The authors develop diffusion models to generate realistic human genotypes. Their claimed contribution is the first models to produce full-length human genotypes. They develop CNN-, CNN+MLP- and Transformer- based models optimized for reconstruction loss. They evaluate on ALS and 1000 Genomes data. They evaluate the ability of the models to augment classifier training and also test the privacy loss associated with generating genotypes with the models.
Strengths: The paper is relatively clear although the language and grammar could use polishing. The claimed contribution is clear and the authors discuss prior work clearly. For the most part the paper is easy to follow. The problem of synthetic data generation is a significant one and the development of synthetic human genomes is surely of interest to some in the research community, although I do not work in this area.
Weaknesses: Although the authors discuss prior work, they do not seem to compare to modeling approaches used in it. It is not clear if this is because of technical limitations or an oversight. The motivation for generating synthetic genotypes to improve ML models is clear, but the authors do not spend enough time discussing the possible negative ethical implications of generating human genotypes. I appreciated the evaluation of privacy loss but the implications were not fleshed out. 

Specific comments:

- All of the figure texts are too small to read.

- change " to `` for front quotes

- " that do not only" -> "that not only"
Recommendation: 5
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 2
reviews: 
Comments: The authors present a novel approach to generating full-length synthetic human genotypes with diffusion models. They empirically show that the synthetic genotype closely resembles the actual data. They show that the performance of disease and population classifiers was maintained when synthetically generated data was used for training those classifiers. The study also demonstrates that augmenting the training data with synthetically generated data improves the performance of the classifiers. This approach can significantly improve data availability and access and preserve data privacy.
Strengths: 1. The paper is well-written and presents a technically sound and well-structured approach.
2. Using diffusion models to generate full-length genotypes is a novel approach. The rationale and motivation for this approach are clearly stated. 
 2. The paper presents a significant contribution to addressing the challenges with data access restrictions and privacy in genome data. 
 3. The paper presents empirical evidence for comparable performance between the models trained with synthetic and real data.
Weaknesses: 1. Authors should discuss potential limitations of their current evaluation approach in more detail.
2. Could authors discuss any limitations in generalizing their approach to other genomic datasets?
Recommendation: 8
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This article presents a method for generating full synthetic human genomes using diffusion. Four diffusion architectures are compared, and the quality of the generated genomes is analyzed. The article demonstrates that this method can generate useful synthetic data for geneticists.
Strengths: This article is very exciting. It is incredibly well-written with a clear and enjoyable prose. It tackles an important issue, human genome analysis, with state of the art methods, diffusion models. It presents an interesting comparative study for deep learning in a data regime not often considered, especially in the diffusion literature, but of great import and scientific interest. The insights from the architecture comparison study are useful for a machine learning readership beyond the biomedical application; an understanding of diffusion methods applied to different data types, and how generated data can be measured and made useful, is relevant to ICLR. The demonstration of generating full human genomes is both an impressive showcase of the capabilities of diffusion and an important step in the study of the human genome.
Weaknesses: It would have been helpful to contextualize the results in comparison to other methods, like the work of Szatkownik et al, or HyenaDNA. It is unclear to me how feasible the application of these methods to the full human genome is, but this could have been discussed in the results. However, it should be possible to generate smaller sequences with the proposed diffusion method in order to compare to previous works. The experimental results, as they are, demonstrate the viability of diffusion models on the full human genome (and provide a useful architecture comparison). However, they do not give an indication of the relative quality of the generated data over the data generated by other works.

Other than that, I identified no major weaknesses. Rather, here are some minor quibbles.

+ the plots are not vectors and have small fonts, making them blurry in standard sized readers. pdf plots with larger fonts would help with clarity.
+ line 451: "We draw two conclusions from this:" this is an incomplete sentence, or the next paragraph shouldn't be presented as a new paragraph.
+ eq 2 could be defined inline
+ lines 453 and lines 483 begin paragraphs that say roughly the same thing. Those two paragraphs could be merged or each trimmed.
+ there's an unneeded comma in line 492
Recommendation: 10
Reviewer confidence: 4
Soundness/correctness: 4
Clarity: 4