From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Looking forward to your further comments!
comment: Dear reviewer REBB,

We appreciate the reviewer for spending time reviewing our paper and supporting our work. If you have any questions, please let us know, and we are willing to address your concerns.

Thanks,

Authors

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Looking forward to your further comments!
comment: Dear reviewer sowP,

 We would appreciate it if you could check our responses and re-evaluate our paper. If you have any questions, please tell us, and we are willing to address your concerns.

Thanks,

Authors

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Looking forward to your further comments!
comment: Dear reviewer YecH,

Thank you very much for your further valuable comments on our revisions, which help us to improve the paper. We would appreciate it if you could review our latest responses and re-evaluate our paper. If you have any further concerns, we are willing to discuss them with you.

Thanks,

Authors

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Looking forward to your further comments!
comment: Dear reviewer YPf3,

Thank you very much for your valuable comments, which help us to further improve the paper. We would appreciate it if you could check our responses and re-evaluate our paper. We are also willing to discuss with you if you have any further concerns.

Thanks,

Authors

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Response to reviewer YecH's four questions
comment: Thank you for the further comments. Our replies are listed below.


1. The deadline for updating submissions has passed, and we can not upload any revised manuscripts. Please click the "Show Revisions" button on this page to see all the revisions and use the "compare revisions" button for version comparison. An online tool called pdf-differ can also be used for checking the differences between two pdf documents (https://www.diffchecker.com/pdf-diff).


2. The correctness of cryo-EM 3D reconstructions on the experimental dataset can be evaluated in three ways, as listed below.

- The most reliable way for assessing correctness without the ground truth is to use the "half-map FSC resolution", which is the standard criterion in the field of cryo-EM. It has been shown that the half-map FSC resolution is equivalent to the full-map FSC resolution used for simulated datasets (see P. Rosenthal and R. Henderson. J. Mol. Biol. 2003, 333:721-745 https://doi.org/10.1016/j.jmb.2003.07.013). The key difference is that half-map FSC resolution calculation removes the need for the ground truth by splitting the dataset into two equal half subsets. When reconstructing from two independent half datasets, it is very difficult for the high-frequency signals to match between the two half-map reconstructions due to the high level of noise in experimental data. Therefore, one can have high confidence about the correctness of the reconstruction when the half-map FSC resolution is high. The half-map FSC resolution of ACE-EM reconstructions on the experimental dataset has reached the highest possible resolution, which provides strong evidence for the correctness of the reconstruction result. The half-map FSC resolution is also the evaluation criterion in all previous cryo-EM reconstruction methodology papers, such as cryoDRGN and cryoAI.

-  Another way to assess the correctness is through visual inspection. In practice, when a reconstruction fails, the resulting volume has abnormal features. A good example is shown in Figure 4 of the manuscript, where long streaks and smeared densities were observed in the failed reconstruction, which is absent in the correct reconstruction and the ground truth. No such abnormal features were found in the ACE-EM reconstructions on the experimental dataset. Additionally, common knowledge about the prominent features of the target molecules (such as secondary structures in proteins, DNA, and RNA) can be used for assessing the correctness. A successful reconstruction should contain these features, as is the case in the ACE-EM reconstructions.

- Finally, the correctness of ACE-EM can be further corroborated through comparison with other methods. As shown in Figure 5 of our revised manuscript, the reconstructed structure from ACE-EM is qualitatively consistent with the reconstruction using cryoSPARC, especially for the detailed features. It is very unlikely for results from different methods to be highly consistent but incorrect. To approximately quantify the similarity, we calculated the FSC resolution (threshold: 0.5) using the half-map reconstructions from ACE-EM and cryoSPARC. The resulting high resolution (2.7 pixels) indicates that the two reconstructions are highly similar. However, this can not be regarded as a precise quantitative comparison for two reasons. First, a conventional method like cryoSPARC has its own limitations in reconstruction quality, as shown in Table 1 (simulated datasets) of our manuscript, and therefore can not be regarded as the ground truth. Also, cryoSPARC is one of the baseline methods, and its results cannot be used for assessing the method itself and other methods. The only quantitative criterion is the "half-map FSC resolution" as mentioned above.

    Based on the three criteria discussed above, the correctness of the reconstruction results from ACE-EM is justified to the best knowledge we know about the experimental dataset.


3. Please let us know which notations need to be clarified. We will explain them here and also make revisions to our manuscript.


4. The applicability of our method on the real experimental dataset is justified by its good performance as evaluated by the gold standard half-map FSC-resolution, a common criterion used for assessing the reconstruction quality for all experimental datasets. As we mentioned before, ML-based methods of cryo-EM 3D reconstruction are still at an early exploratory stage. Methods like cryoAI failed to handle the 80S ribosome experimental dataset of a certain size (L=128). ACE-EM is the only amortized-inference-based method that successfully reconstructed the target volume for this experimental dataset. It is also the only method that has achieved the highest possible resolution on this experimental dataset (see Table 2 of our revised manuscript).

From: ICLR.cc/2023/Conference/Paper5784/Reviewer_YecH
title: Comments on revision
comment: Thank you for your response.

1. Please highlight the changes in the paper with different font color.

2. For real data experiments, how to justify that the reconstruction is correct without the ground truth? 

3. Some notations after revision are not clearly defined.

4. The applicability of the method on the real dataset needs further justification.

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Response to Reviewer sowP
comment: Thank you for the previous comments. We have thoroughly improved our manuscript. Please take a look and we look forward to your further comments.

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: We look forward to your further comments
comment: Our manuscript has been thoroughly revised according to your previous comments. We sincerely look forward to your further comments on this revised version.

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Response to Reviewer REBB (part 2)
comment: **2. Loss definition: In Section 3.4 and 3.5, for all the loss definitions, a common issue is that the given parameters and unknown parameters are not clear in these formula definitions. It may be better to clarify what are the optimization targets or parameters in the formula definition of each loss objective for the model training.**

  Thank you for the suggestions. The unknown parameters only include the trainable parameters in the encoder and decoder. All other parameters in the loss definitions are given parameters. In the revised manuscript, we specifically discussed the trainable parameters for the encoder and decoder in Appendix A.


**3. Experiments and results: ... If the accuracy of pose estimation of the proposed method is not outperforming explicitly, why could the reconstructed 3D volume be better than previous methods? It is better to visualize the qualitative results of all these methods with some specific examples to see the difference between these different reconstruction methods.**

  - We provide visualizations of our 3D reconstruction in Figure 4, 5, and 6. In Figure 6, the results from cryoAI, ACE-EM, and the ground-truth volumes are given.  Results from ACE-EM show more detailed features and are closer to the ground truth.
  
  - In Table 1 of our initial submission, we adopted the Median Rotation Error metric used in the previous baseline (cryoAI). However, the median error cannot reflect the actual error, as some large long-tailed errors are not well-reflected. To explicitly show this, we plotted the distribution of Rotation Error. As shown in Figure 8, although the median is close, there is a significant number of incorrect predictions without the PIP task.
  - For a better assessment of the Rotation Error, we have added the Mean Rotation Error in Table 1 in the revised paper. We find that ACE-EM outperforms the baselines significantly.
  - In addition, Figure 2 shows that the mean rotation error of the model without PIP is significantly higher compared to that model with PIP task.

**4. Moreover, it would also be interesting to visualize the generated projection images output from the decoder in the PIP cycle. How is it similar or different from the real projection images?**

  We have added Figure 7 to show the input projection images, intermediate images generated by the decoder, and the predicted images with CTF and noise applied. The predicted images closely match the input images. From another perspective, if the output images from the decoder in the PIP task were dissimilar to the original images, the PIP task wouldn't have boosted the training performance.

**5. More comments: From another point of view, I think the proposed method is actually quite similar to many previous works in style translation such as MUNIT [1]... This is not mentioned or discussed by the author in this paper, but it may be an interesting perspective relating to these relevant works.**

  We added a short paragraph (due to the page limit) in the related work section to discuss other cyclic training methods, including MUNIT. MUNIT and our method share a similar high-level theory but with 4 main differences: architecture setting, data format, aims, and challenges.

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Response to Reviewer REBB (part 1)
comment: Thank you so much for supporting our work and valuable review comments!  We have revised the paper according to your suggestions. 
1. New results on the real-world experimental dataset have been added in Sec. 4.3 where ACE-EM outperformed previous baselines like cryoAI.
2. We have fully revised our manuscript to provide a clearer description. Besides, the related work section has been improved with more citations.
3. For an intuitive understanding of our results, we have added Figure 7, showing the original and predicted 2D projection images from the spliceosome dataset. The reconstructed volumes can be found in Figure 5 and 6.

Please let us know if further clarification is needed.

Our responses to your specific questions are listed below.

**1. Method: The writing and notations in Sec. 3.3 is kind of confusing. It may need to be more clear in this Section what is the decoder, what is the reference volume, and what are trainable parameters. These also need to be correctly corresponding to the illustration in Figure 2. Moreover, if the pose estimation decoder and the reference image are both learned, they may influence each other during the simultaneous training, is there any constraint to guarantee the correctness of both?**

  We rewrote this subsection and replaced it with a more general and shorter description. Here, we will answer your questions based on the previous version of our manuscript.

   - The definition of the decoder can be rewritten as $ D_{PI} := P(G_{pose(i)}; \theta) = P(T_i G_{ref}; \theta) $. All the trainable parameters are represented as $\theta$.  $T_i$ is the combination of $R_i$ and $t_i$. The coordinate grid $G_{ref}$ stores fixed parameters that are constant throughout the training, and $T_i$ is a linear transformation operator on $G_{ref}$ which is updated based on the encoder outputs. 
    - To be specific, the decoder can be implemented using neural networks or just a trainable variable with deterministic functions. For example, $\mathrm{VoxelGrid_{R}}$ decoder is not a neural network. It is implemented using a cubic 3D tensor of fixed size $L^3$ (equivalent to a discrete cryo-EM density map). For a given pose, the corresponding projection image can be obtained by applying a series of deterministic functions to the 3D tensor, such as the PyTorch grid_sample and affine_grid functions. Then, the tensor values are updated based on the backpropagation of losses. On the other hand, the FourierNet decoder is implemented using neural networks.
    - Volume $V$ is the 3D reconstruction output volume, a discrete representation of the reconstruction target. It can be obtained from the parameters of the decoder after training by some methods.

   - There are no other constraints except for the loss functions, which are designed to reduce the differences between the input and output images (IPI task) and between the input and output poses (PIP task). In practice, these loss functions are sufficient to guide the encoder and decoder to convergence.

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Response to Reviewer YecH (part 2)
comment: 
**5. Novelty of the proposed method need to be further justified. ... The authors need to state and prove clearly the theoretical or computational advantages of using the ACE-EM method, such as the computational cost of the modified method, the training stability, the rate of convergence. What is the defect of the method?**

  - Please see our previous answer about the advantages of ACE-EM.
  - Experimental results demonstrate that ACE-EM can perform well in both simulated and real-world experimental datasets.
    - In particular,  ACE-EM outperformed all the baseline methods and reached 2 pixels, the Nyquist resolution (the highest possible resolution), in spliceosome and spike simulated datasets with an SNR at 200 dB and -10 dB. On the 80S ribosome experimental dataset, ACE-EM also reached the Nyquist resolution, which is the only amortized-inference-based method that has reached the highest possible reconstruction resolution.
  - We also explicitly highlight our contributions in the Introduction section in the revised version of the paper.
  - We hope the reviewer can re-evaluate our contributions based on the new results and the revised paper.

**6. How do you obtain the ground truth in real data? Has the method helped discover new features and structures in biology?**

  - Strictly speaking, there are no ground-truth volumes for real cryo-EM datasets. The convention in the field of cryo-EM is to use the half-map FSC-resolution to assess the reconstruction quality in the statistical sense.  The input dataset is split evenly into two half-datasets and trained independently. Half-map FSC resolution is defined as the FSC-resolution calculated using these two half-map reconstructions. Generally speaking, the volumes obtained from reliable traditional software like cryoSPARC can sometimes be regarded as the approximated ground truth.

  - For structure discovery, thanks to the good performance of ACE-EM on the experimental dataset, if a set of 2d images of an unknown object is given, our proposed ACE-EM can uncover the underlying 3D structure.

**7. Reproducibility: The paper does not include the link to codes for reproducing the work. The network structure and parameters for the current paper are not included.**

  We have included a reproducibility statement at the end of the revised manuscript, and the code to reproduce our experiments will be open-sourced upon publication.

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Response to Reviewer YecH (part 1)
comment: 
Thank you so much for your valuable review comments! We have revised the paper according to your suggestions. 
1. The result on the real-world experimental dataset is added in Sec. 4.3, and the result demonstrates that the proposed ACE-EM can outperform previous baseline methods like cryoAI.
2. We have revised our manuscript to provide a clearer description. More relevant citations have been added in the related work section. We also added additional analyses and results on experimental datasets. 
3. For a direct understanding, we added Figure 7, which demonstrates the images from the spliceosome dataset. The reconstructed volumes can be seen in Figure 5 and 6.

Please let us know whether any part of the revised manuscript requires clarification.

Our responses to your other questions are listed below.

**1. Lack of traditional reconstruction methods for comparison**

  The cryoSPARC is one of the commonly used traditional reconstruction methods, and its results are shown in Table 1 and Table 2.

**2. The results based on the synthetic data experiments are not good enough to show the advantage of using the ACE-EM method.** 

  The advantages of ACE-EM can be shown in 4 aspects based on the synthetic data experiments.
   - Better reconstruction resolution. ML-based methods of cryo-EM 3D reconstruction are in the primary exploration stage. The highest possible resolution(2.0 pixels) at FSC threshold 0.5 is not reached by previous ML-based methods, even in simulated datasets without noise. ACE-EM outperformed all the baseline methods and reached 2 pixels in simulated datasets with SNR 200 dB and -10 dB. It is noticeable that the correlation of resolution 2.03 pixels is still high at 0.87 in spliceosome (200dB) with FourerNet. The FSC resolution of spliceosome (-20dB) with ACE-EM reached 2.1 pixels, which is even higher than the results of -10 dB provided by other baselines.

   - Smaller rotation error, larger pose coverage. From Section 4.4.1 and 4.2, compared to methods without the PIP task, both the mean rotation error and pose coverage have been significantly improved when using ACE-EM.

   - Faster convergence. With the PIP task in ACE-EM, the convergence is much faster and a better final performance could be achieved (shown in Figure 2). The training time is increased to 1.25-1.50 times the original duration per epoch with the PIP task, but the resulting faster convergence and better performance are worth the extra cost.

   - Some decoders that failed in previous autoencoder architectures can be successfully used in ACE-EM (see the ablation performance of $ \mathrm{VoxelGrid_{R}} $ in Section 4.4.2). 


**3. Have you looked at the AlphaFold database?**

  We actually do not understand this question. AFDB is the 3D structure prediction results by AlphaFold from amino acid sequences, which is noise-free, and our work is to extract a 3D density map from noisy images. We cannot find a clear connection between them.

**4. How is the 2D image illustration of the 3D reconstruction results?**

   - The reconstruction results for the spliceosome dataset are shown in Figure 4 and 6, and the spike and 80S ribosome reconstruction results are in Figure 5 (appendix). The ACE-EM 3D reconstructions closely match the ground-truth volumes in all three cases, especially for the high-spatial-frequency features.
   - And Figure 6 shows that our reconstruction has more details than the result of cryoAI.

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Response to Reviewer YPf3
comment: Thank you so much for your valuable review comments! We have revised the paper according to your suggestions. 
1. The result on the real-world experimental dataset is added in Sec. 4.3, and the result demonstrates that the proposed ACE-EM can outperform previous baselines like cryoAI.
2. We have rearranged the whole paper to provide a clearer description. Besides, the related work section is improved with more citations, and more results are added in Appendix. 
3. For a direct understanding, we added Figure 7, which demonstrates the images from the spliceosome dataset. The reconstructed volumes can be seen in Figure 5 and 6.

If there are still unclear parts or missing references, please let us know.

Following are our responses to the questions.

**1. The contribution is minor.**

  - The result on the experimental dataset is added in the paper. Experimental results demonstrate that ACE-EM can perform well in both simulated and real-world experimental datasets.
    - In particular,  ACE-EM outperformed all the baseline methods and reached 2 pixels, the Nyquist resolution (the highest possible resolution), in spliceosome and spike simulated datasets with SNR 200 dB and -10 dB. And ACE-EM also reached the Nyquist resolution (2 pixels or 7.54 angstroms) in the 80S ribosome experimental dataset, which is the only architecture that reached the Nyquist resolution with amortized inference methods.
  - We also explicitly highlight our contributions in the Introduction section in the revised version of the paper.
  - We hope the reviewer can re-evaluate our contribution based on the new result and the revised paper.


**2. The experimental design could be improved, e.g. it is not clear why both the FourierNet and VoxelGrid model were tested.**

  - We add a paragraph in Section 3.2 to describe the choices of decoders more clearly. In short, we want to highlight that ACE-EM is a general framework that can boost performance regardless of the decoder type. Therefore, the two widely used decoders, real-space voxel grid decoder $\mathrm{VoxelGrid_{R}}$ and frequency-space decoder $\mathrm{FourierNet}$, are both tested.
  - Also, in Section 4, the experimental results on both simulated and experimental datasets explicitly prove that ACE-EM can boost performance regardless of decoder type.

From: ICLR.cc/2023/Conference/Paper5784/Authors
title: Thanks all reviewers! 
comment: We appreciate the reviewers for spending time reviewing our paper and offering valuable suggestions, and we have revised the paper according to the comments. 
In particular, we made the following changes:

- The result on the real-world experimental dataset has been added in Sec. 4.3, and the results indicate that the proposed ACE-EM can outperform previous baselines like cryoAI.
- We have revised the entire manuscript to provide a clearer description. The related work section has been improved with more citations, and more results have been added in Appendix. 
- To provide an intuitive understanding of our results, we have added Figure 7 which shows the original and predicted 2D projection images from the spliceosome dataset. The reconstructed 3D volumes can be seen in Figure 5 and 6.

Please let us know if further clarification is needed.

From: ICLR.cc/2023/Conference/Paper5784/Reviewer_YPf3
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper presents an autoencoder that extends existing neural methods for homogeneous cryo-EM reconstruction.  The image encoder takes images and outputs a predicted pose. The volume decoder renders posed images as either 2D slices out of the FourierNet volume model (Levy et al) or projections out of a real-space voxel grid model. Training is augmented with a secondary task where the order of the encoder and decoder are reversed to check the consistency of synthesized poses. The method was tested on two synthetic datasets.

strength_and_weaknesses: This paper presents an auxiliary task that may speed up the convergence of an amortized inference approach for homogeneous reconstruction. However, the contribution is minor and not evaluated very comprehensively, e.g. only a few synthetic datasets.

clarity,_quality,_novelty_and_reproducibility: The paper is not clear enough and there are some inaccuracies in the related works and incomplete citations. The experimental design could be improved, e.g. it is not clear why both the FourierNet and VoxelGrid model were tested.

summary_of_the_review: A potentially interesting extension of recent works that learn a model to predict unknown image pose, however the work is too preliminary at this point for publication.

correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 1: The contributions are neither significant nor novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper5784/Reviewer_YecH
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper presents the ACE-EM model for Cryo-EM 3D reconstruction. It shows the approach of ACE-EM method, and shows experimental results on the accuracy of pose estimation in the synthetic datasets. 
strength_and_weaknesses: Strength:
It proposes a modified deep learning method for Cryo-EM data reconstruction, which is an important application.

Weaknesses:
1. Not enough literature review. The authors need to have more literature review of the methods used in Cryo-EM data reconstruction. The paper has reference some GAN related methods. It is better to refer more related work. 

2. Experimental results are not sufficient. The paper is lack of real-data experiments, and lack of traditional reconstruction methods for comparison. Have you looked at the AlphaFold database? How is the 2D image illustration of the 3D reconstruction results?

3. Nolvety of the proposed method need to be further justified.





clarity,_quality,_novelty_and_reproducibility: 1. Clarity and quality: 
More literature review for the Cryo-EM research is needed. Citations are missing in the paper, and there are spelling errors. 

2. Novelty:
The authors need to state and prove clearly the theretical or computational advantages of using the ACE-EM method, such as the computational cost of the modified method, the training stability, the rate of convergence. What is the defect of the method?

3. Experimental validation:
Real data experiments on public datasets are missing. The results based on the synthetic data experiments are not good enough to show the advantage of using the ACE-EM method. How do you obtain the ground-truth in real data? Has the method help discover new features and structures in biology? 

4. Reproducibility: The paper does not include the link to codes for reproducing the work. The network structure and parameters for the current paper are not included. 
summary_of_the_review: This paper presents the ACE-EM method for Cryo-EM data reconstruction. The paper shows clearly how the ACE-EM method work. However, the advantage and nolvety of using the ACE-EM method are not well supported. The real-data experiments are missing, and the synthetic data experiments should be further improved. The paper need to have more literature review of the methods used in the Cryo-EM data analysis and 3D reconstruction. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper5784/Reviewer_sowP
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper presents a modified autoencoder architecture called ACE (asymmetric complementary autoencoder) and designed the ACE-EM method to solve the issue of local optimum. It consists of two tasks. The first task takes projection images and outputs predicted images using an image encoder followed by a pose decoder. The second task reverses the order of encoder and decoder, which takes randomly sampled poses and outputs predicted poses. Experiments show the proposed  ACE-EM can reach higher pose space coverage within the same training time and has better 3D reconstruction results for several benchmark datasets. 
strength_and_weaknesses: +: The paper proposes an asymmetric complementary autoencoder architecture for cryo-EM 3D reconstruction, which efficiently reaches high pose space coverage.
+ The proposed ACE-EM framework can reach the Nyquist frequency in Spliceosome and
Spike simulated datasets for the construction resolution.
clarity,_quality,_novelty_and_reproducibility: The paper is generally well written.
summary_of_the_review: Overall this is an interesting paper with good results. The application is important.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper5784/Reviewer_REBB
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper studies the Cryo-EM 3D reconstruction problem, and proposes an autoencoder framework with a pose decoder and an image encoder for learning both pose estimation and 3D reconstruction tasks. The two cyclic training strategies are used with image-pose-image and pose-image-pose learning.
strength_and_weaknesses: Strength:
+ The paper is well written and organized, which is easy to understand.
+ The paper introduces an interesting unsupervised learning approach for Cryo-EM 3D reconstruction by learning pose estimation simultaneously with two cycles.

Weakness:
- Method: The writing and notations in Sec. 3.3 is kind of confusing. Based on the description, it looks like the image decoder is a deterministic function based on either voxel grid or Fourier slice theorem. Then what are the trainable parameters here? Is that the reference image/volume G_{ref}? But there is another notation V. And this trainable parameter is not shown in Figure 1, which may make it not very clear how the 3D construction task is completed. It may need to be more clear in this Section that what is the decoder (network or deterministic function?), what is the reference volume, and what are trainable parameters. These also need to be correctly corresponding to the illustration in Figure 2. Moreover, if the pose estimation decoder and the reference image are both learned, they may influence each other during the simultaneous training, is there any constraint to guarantee the correctness of both?

- Loss definition: In Section 3.4 and 3.5, for all the loss definitions, a common issue is that the given parameters and unknown parameters are not clear in these formula definitions. It may be better to clarify what are the optimization targets or parameters in the formula definition of each loss objective for the model training.

- Experiments and results: Although the quantitative results in Table 1 show the improvement in broth pose estimation and 3D reconstruction, the difference in results looks small especially for pose estimation. Some of the previous methods may also get better results. If the accuracy of pose estimation of the proposed method is not outperforming explicitly, why could the reconstructed 3D volume be better than previous methods? It is better to visualize the qualitative results of all these methods with some specific examples to see the difference between these different reconstruction methods. Moreover, it would also be interesting to visualize the generated projection images output from the decoder in the PIP cycle. How does it similar or different from the real projection images?

- More comments: From another point of view, I think the proposed method is actually quite similar to many previous works in style translation such as MUNIT [1], especially for the encoder-decoder framework and two cyclic training. In this paper, the pose is regarded as one kind of “style” information that could be learned from the image through an unsupervised way. This is also relevant to previous works that leverage such framework for novel-view synthesis for medical image reconstruction, which regard the view angle of projection images as one kind of “style” information. This is not mentioned or discussed by the author in this paper, but it may be an interesting perspective relating to these relevant works.

- Minor comments:
1. On page 3 Sec. 3.2, “The filtered images are given to a ResNet-18 [cite] network with two MLP networks…”. Seems as missing citations.
2. Some writing and description need to be carefully checked. For example, in the abstract, it claims: “ The first task employs a pose decoder to take projection images and outputs predicted images with an image encoder ……” But in the Figure 1, the IPI shows the pose “encoder” and image “decoder”, which may make the reader confused. 

[1] Huang, Xun, Ming-Yu Liu, Serge Belongie, and Jan Kautz. "Multimodal unsupervised image-to-image translation." In Proceedings of the European conference on computer vision (ECCV), pp. 172-189. 2018.


clarity,_quality,_novelty_and_reproducibility: The paper is well written and organized, which is easy to understand. The proposed method shows some novelty especially for Cryo-EM community, but could also be relevant to some previous works as aforementioned. The paper does not include the codebase to reproduce the work. The information included in the current paper may not be fully complete for reproducing the work such as the specific  network structure and parameters.
summary_of_the_review: The paper studies the problem of Cryo-EM reconstruction and proposes an interesting problem. But there may still be some questions about the method, experiments and results as aforementioned.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold