Paper: Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models (ICLR-2024)

Reviewer comment (Novelty):
The paper does not clearly define its novelty. The contributions are formulated very vaguely. For example, "we propose a novel inpainting conditional diffusion model to explore the dense correspondence between source and target images" - what does "explore" really mean in this sentence? "we introduce a new refining conditional diffusion model by further using post hoc image-to-image techniques to enhance the quality and fidelity of synthesized images." - what is new in this model? Is it a typical diffusion model for image refinement, or is there something special? All in all, it is hard to say whether the theoretical/technical novelty is significant enough.

Author response:
**Q1: What is novel in the submission? I do not assume there is no novelty at all, but I'd like to see a clear formulation of the novelty.**

**Response:** Pose-guided image synthesis, which involves generating realistic images closely aligned with given pose and appearance information, is a highly challenging task. In existing research, most methods focus on directly generating pedestrian images that match the target pose from the source image and target pose. See: [Figure1](https://drive.google.com/file/d/1cAIz1CR0o8rh1abPlfG7p2GtxWyXiLdB/view?usp=drive_link). However, directly capturing the complex structure of spatial transformations is quite challenging, especially when it is necessary to infer obscured body areas from the given source image, which often results in noticeable artifacts. See: [https://drive.google.com/file/d/16lBLXR6wcb5KRAfW3rLVaelQ0c9cfk0r/view?usp=drive_link](https://drive.google.com/file/d/16lBLXR6wcb5KRAfW3rLVaelQ0c9cfk0r/view?usp=drive_link) .
Then, the pose of the source image and the target image do not align, leading to misalignment in image-to-image generation. Especially when facing significant pose changes, these methods usually fail outright. See [https://drive.google.com/file/d/1ynlhikPzw6Nnp7T2x9eQa3ZMwMds-tqA/view?usp=drive_link](https://drive.google.com/file/d/1ynlhikPzw6Nnp7T2x9eQa3ZMwMds-tqA/view?usp=drive_link). Lastly, existing methods lack a mechanism for texture repair, resulting in generated results that are often not realistic enough. See [https://drive.google.com/file/d/12PECjus_hKl7KxIp4HK4eZ_tIWr1bLef/view?usp=drive_link](https://drive.google.com/file/d/12PECjus_hKl7KxIp4HK4eZ_tIWr1bLef/view?usp=drive_link). Therefore, current methods face the following problems: 

**1. Generation of invisible areas easily produces artifacts;**

**2. Unaligned generation results fail outright when dealing with a wide range of pose changes;**

**3. Lack of a mechanism for detail texture repair.**

To address these issues, PCDMs decompose this task and propose a progressive conditional diffusion model. See [https://drive.google.com/file/d/1H2ssD3oUTPRu-R9GQurXjePPq_ol3Vx6/view?usp=drive_link](https://drive.google.com/file/d/1H2ssD3oUTPRu-R9GQurXjePPq_ol3Vx6/view?usp=drive_link).
To solve problem 1, we propose a prior conditional diffusion model that directly predicts the features of the target image based on the source pose, target pose, and source image. This prediction is much simpler than directly generating the target image because the prior model only needs to focus on one task. 

To solve problem 2, we propose an inpainting conditional diffusion model that uses the global features predicted in the first stage to achieve alignment at the pose, feature, and image levels. The generated results will inevitably correspond densely, thus maintaining a high degree of consistency. 

Finally, to solve problem 3, we propose a refining conditional diffusion model that repairs the generated image in the second stage through the texture details of the source image. 

PCDMs incrementally bridge the gap between person images under the target and source poses through three stages, and qualitative, quantitative, and user studies demonstrate the consistency and photorealism of our proposed PCDMs compared to previous methods.

Reviewer comment (Writing):
The method is described quite inaccurately. Only red arrows in Fig. 2 indicate that there is some iterative process of image generation. The paper should describe in detail the whole process.

Author response:
**Q2: How does the whole process of image generation look like? How many iterations are used?**

**Response:** Firstly, the prior conditional diffusion model predicts the features of the target image based on the pose of the source image, the pose of the target image, and the features of the source image. Secondly, the inpainting conditional diffusion model generates a coarse-grained target estimate based on the pose pairs of the source and target images, the source image and the mask of the target image, the features of the source image, and predicted features of the target image (which obtained by the first stage). Finally, the refining conditional diffusion model generates the fine-grained target estimate using the coarse-grained target estimate obtained from the previous stage and the source image.
Besides, the PCDMs require 400k iterations, with the number of iterations for each stage as follows.

|Stages  |Iterations |
|:--|:--|
|Stage1    |100k |
|Stage2    |200k |
|Stages    |100k |

Reviewer comment (Novelty):
What is novel in the submission? I do not assume there is no novelty at all, but I'd like to see a clear formulation of the novelty.

Author response:
**Q1: What is novel in the submission? I do not assume there is no novelty at all, but I'd like to see a clear formulation of the novelty.**

**Response:** Pose-guided image synthesis, which involves generating realistic images closely aligned with given pose and appearance information, is a highly challenging task. In existing research, most methods focus on directly generating pedestrian images that match the target pose from the source image and target pose. See: [Figure1](https://drive.google.com/file/d/1cAIz1CR0o8rh1abPlfG7p2GtxWyXiLdB/view?usp=drive_link). However, directly capturing the complex structure of spatial transformations is quite challenging, especially when it is necessary to infer obscured body areas from the given source image, which often results in noticeable artifacts. See: [https://drive.google.com/file/d/16lBLXR6wcb5KRAfW3rLVaelQ0c9cfk0r/view?usp=drive_link](https://drive.google.com/file/d/16lBLXR6wcb5KRAfW3rLVaelQ0c9cfk0r/view?usp=drive_link) .
Then, the pose of the source image and the target image do not align, leading to misalignment in image-to-image generation. Especially when facing significant pose changes, these methods usually fail outright. See [https://drive.google.com/file/d/1ynlhikPzw6Nnp7T2x9eQa3ZMwMds-tqA/view?usp=drive_link](https://drive.google.com/file/d/1ynlhikPzw6Nnp7T2x9eQa3ZMwMds-tqA/view?usp=drive_link). Lastly, existing methods lack a mechanism for texture repair, resulting in generated results that are often not realistic enough. See [https://drive.google.com/file/d/12PECjus_hKl7KxIp4HK4eZ_tIWr1bLef/view?usp=drive_link](https://drive.google.com/file/d/12PECjus_hKl7KxIp4HK4eZ_tIWr1bLef/view?usp=drive_link). Therefore, current methods face the following problems: 

**1. Generation of invisible areas easily produces artifacts;**

**2. Unaligned generation results fail outright when dealing with a wide range of pose changes;**

**3. Lack of a mechanism for detail texture repair.**

To address these issues, PCDMs decompose this task and propose a progressive conditional diffusion model. See [https://drive.google.com/file/d/1H2ssD3oUTPRu-R9GQurXjePPq_ol3Vx6/view?usp=drive_link](https://drive.google.com/file/d/1H2ssD3oUTPRu-R9GQurXjePPq_ol3Vx6/view?usp=drive_link).
To solve problem 1, we propose a prior conditional diffusion model that directly predicts the features of the target image based on the source pose, target pose, and source image. This prediction is much simpler than directly generating the target image because the prior model only needs to focus on one task. 

To solve problem 2, we propose an inpainting conditional diffusion model that uses the global features predicted in the first stage to achieve alignment at the pose, feature, and image levels. The generated results will inevitably correspond densely, thus maintaining a high degree of consistency. 

Finally, to solve problem 3, we propose a refining conditional diffusion model that repairs the generated image in the second stage through the texture details of the source image. 

PCDMs incrementally bridge the gap between person images under the target and source poses through three stages, and qualitative, quantitative, and user studies demonstrate the consistency and photorealism of our proposed PCDMs compared to previous methods.

Reviewer comment (Reproducibility):
How does the whole process of image generation look like? How many iterations are used?

Author response:
**Q2: How does the whole process of image generation look like? How many iterations are used?**

**Response:** Firstly, the prior conditional diffusion model predicts the features of the target image based on the pose of the source image, the pose of the target image, and the features of the source image. Secondly, the inpainting conditional diffusion model generates a coarse-grained target estimate based on the pose pairs of the source and target images, the source image and the mask of the target image, the features of the source image, and predicted features of the target image (which obtained by the first stage). Finally, the refining conditional diffusion model generates the fine-grained target estimate using the coarse-grained target estimate obtained from the previous stage and the source image.
Besides, the PCDMs require 400k iterations, with the number of iterations for each stage as follows.

|Stages  |Iterations |
|:--|:--|
|Stage1    |100k |
|Stage2    |200k |
|Stages    |100k |

Reviewer comment (Writing):
"Additionally, we add an extra embedding to predict the target global embedding." - what is the target global embedding exactly? Is it a feature vector representation of the target image?

Author response:
**Q4: "Additionally, we add an extra embedding to predict the target global embedding." - what is the target global embedding exactly? Is it a feature vector representation of the target image?**

**Response:** Yes, the target global embedding is a feature vetcor of target image extracted by CLIP image encoder. We will rewrite and update it in the reversion for better understanding.

Reviewer comment (Presentation):
Fig. 3 shows some noisy image labelled as ‘Target’ fed into the Image Encoder - what is that exactly?

Author response:
**Q5: Fig. 3 shows some noisy image labelled as ‘Target’ fed into the Image Encoder - what is that exactly?**

**Response:** “Target” represents the target image with noise. During the training phase of the diffusion model, a noisy target image is provided to guide the model on how to recover the original image from the noise. We add noise to the global image embedding for the prior diffusion model, and the diffusion model is trained to predict the original global image embedding.

Reviewer comment (Writing):
In 3.3., “To prevent confusion between black and white in the source and target images,” - unclear.

Author response:
**Q6: In 3.3., “ To prevent confusion between black and white in the source and target images,” – unclear**

**Response:**  We thank the reviewer for pointing out this issue. In fact, we merge the source image and the masked image, with the masked image using all-zero pixels to indicate the area to be generated. If there are black pixels in the source image that match the mask, it could easily mislead the model into thinking these areas are to be generated. To tackle this issue, we introduce a single-channel marker symbol, the size of which matches the width and height of the input image (not shown in the diagram). We choose to use 0 and 1 to represent pixels that are masked and unmasked, respectively. This method aids in reducing model confusion and ensures the accurate identification of the generation area.

Reviewer comment (Writing):
In (5), x_0 is under the expectation symbol, but not in the formula - why?

Author response:
**Q8: In (5), x_0 is under the expectation symbol, but not in the formula - why?**

**Response:**  As $\boldsymbol{x}_t = \alpha_t\boldsymbol{x}_0+\sigma_t\boldsymbol{\epsilon}$, $\boldsymbol{x}_t$ is sampled based on $\boldsymbol{x}_0$, so $\boldsymbol{x}_0$ should be under the expectation symbol.

Reviewer comment (Presentation):
In Fig. 4, what do the noisy Source and Target images actually represent?

Author response:
**Q9: In Fig. 4, what do the noisy Source and Target images actually represent?**

**Response:** In Fig. 4, we concatenate the source and target images, here the noisy Source and Target image means the concatenated image with Gaussian noise. The diffusion model is trained to predict the noise added on the concatenated image.

Reviewer comment (Writing):
In general, to disambiguate the text, I would suggest to reserve a term ‘target’ only for the dataset target image, and call a generated image aimed to resemble the target as ‘target estimate’.

Author response:
**Q10: In general, to disuambiguate the text, I would suggest to reserve a term ‘target’ only for the dataset target image, and call a generated image aimed to resemble the target as ‘target estimate’.**

**Response:** We appreciate the reviewer's attention to this. In order to improve readability, we have revised several details of our presentations and will continue to update our manuscript.

Reviewer comment (Presentation):
In the experiments section there are no images for ‘Market-1501’ results. The results for DeepFashion show unnatural faces, and all faces look the same. This makes me think that the performance of the model for person re-identification will be limited, however the experiments show good performance on Market-1501. Would be nice to see some illustrations.

Author response:
**Q11: In the experiments section there are no images for ‘Market-1501’ results. The results for DeepFashion show un-natural faces, and all faces look the same. This makes me think that the performance of the model for person re-identification will be limited, however the experiments show good performance on Market-1501. Would be nice to see some illustrations.**

**Response:** We appreciate your kind reminder.  In fact, person re-identification primarily relies on appearance and posture for matching, due to the long shooting distance of surveillance cameras, which results in low resolution of pedestrian images and blurred facial features. Market-1501 is a standard dataset for person re-identification, which also exhibits these traits. Additionally, according to your suggestion, we have added result images from Market-1501 for a better understanding. See:  [https://drive.google.com/file/d/1Au19kw-IODWdwPbmaKlGBAHHaFxxYMEC/view?usp=drive_link](https://drive.google.com/file/d/1Au19kw-IODWdwPbmaKlGBAHHaFxxYMEC/view?usp=drive_link).

Reviewer comment (Evaluation):
How long does it take to generate a new image using the proposed pipeline?

Author response:
**Q12: How long does it take to generate a new image using the proposed pipeline?**

**Response:** Thanks for the question. PCDMs can generate a 128x64 image in 2.181 seconds, a 256x176 image in 2.288 seconds, and a 512x352 image in 3.701 seconds. Moreover, we also offer the time consumption for each stage as a reference. All the experiments were carried out on the a same V100 GPU. 

|Size    |PCDMs     |Stage1    |Stage2     |Stage3 |
|:----------|:----------|:---------|:---------|:------|
|128 x 64    |2.181    |0.414    |0.899    |0.868 |
|256 x 176    |2.288    |0.426    |0.945    |0.917 |
|512 x 352    |3.701    |0.458    |2.125    |1.118 |

Under the same configuration, the previous sota method PIDM (which also employs the diffusion model) needs 11.746s at 512x352. Despite PCDMs being a multi-stage pipeline, PCDMs is faster than PIDM as we adopt efficient latent diffusion architecture.

Reviewer comment (Evaluation):
It would be good to show the computation overhead of each stage as well as the training/inference time.

Author response:
**Response:** We appreciate your kind reminder. Based on your advice, on 256 x176 sized DeepFashion, we have added a comparative experiment on training and inference time with the previous sota method PIDM (which also employs the diffusion model). All experiments were executed on the same machine, with training utilizing 8 V100 GPUs and inference conducted on a same V100 GPU for a fair comparison. **The results indicate that our PCDMs significantly outperform in training and inference time.** This is because our model is based on latent space diffusion, while PIDM is a pixel-level diffusion model.

|Methods  |Training Time (H)| Inference Time (s)| 
|:------------|:-------------|:------------|
|PIDM    | 197.7|**9.377** | 
|PCDMs    |103.8 |**2.288**| 

Furthermore, according to your suggestion, we have detailed the training and inference time, as well as the memory overhead for each stage.
|Stages | Training Memory (G) |Training Time (H)|  Testing Memory (G)| Inference Time (s)
|:------------|:------------|:-------|:------|:-------|
|Stage1    | 9.7 x 8 GPUs|  9.6| 9.4 x 1 GPUs| 0.426 | 
|Stage2    | 29.9 x 8 GPUs| 64.8 | 13.8 x 1 GPUs| 0.945|
|Stage3    | 16.5 x 8 GPUs| 29.4 | 10.9 x 1 GPUs| 0.917|

Reviewer comment (Evaluation):
Section 4.2 shows qualitative results of the ablation study. However, quantitative comparisons of these variations are missing.

Author response:
**Response:** We appreciate your insightful suggestions. In response to your advice, we have incorporated the quantitative ablation results from **Figure 8** into the updated version.

|Settings |SSIM  (↑)   |LPIPS  (↓)   |FID  (↓) |
|:-------|:--------|:-------|:-------|
|B1 |0.7209     |0.1693     |8.4238 |
|B2   |0.7357    |0.1426    |7.7815 |
|B3 |0.7378    |0.1419    |7.6924 |
|PCDMs  |0.7444    |0.1365    |7.4738 |

The experimental findings reveal that, initially, B1 can attain highly competitive performance on the DeepFashion dataset solely by employing the inpainting conditional diffusion model. Subsequently, with the integration of the prior conditional diffusion model and refining diffusion, B2 and B3 surpass B1 in terms of SSIM, LPIPS, and FID. These outcomes underscore their significant contribution to the success of our PCDMs. We extend our gratitude once again for your insightful guidance.

Reviewer comment (Evaluation):
What is the overall training and inference time and how is it compared to the prior methods?

Author response:
**Response:** We appreciate your kind reminder. Based on your advice, on 256 x176 sized DeepFashion, we have added a comparative experiment on training and inference time with the previous sota method PIDM (which also employs the diffusion model). All experiments were executed on the same machine, with training utilizing 8 V100 GPUs and inference conducted on a same V100 GPU for a fair comparison. **The results indicate that our PCDMs significantly outperform in training and inference time.** This is because our model is based on latent space diffusion, while PIDM is a pixel-level diffusion model.

|Methods  |Training Time (H)| Inference Time (s)| 
|:------------|:-------------|:------------|
|PIDM    | 197.7|**9.377** | 
|PCDMs    |103.8 |**2.288**| 

Furthermore, according to your suggestion, we have detailed the training and inference time, as well as the memory overhead for each stage.
|Stages | Training Memory (G) |Training Time (H)|  Testing Memory (G)| Inference Time (s)
|:------------|:------------|:-------|:------|:-------|
|Stage1    | 9.7 x 8 GPUs|  9.6| 9.4 x 1 GPUs| 0.426 | 
|Stage2    | 29.9 x 8 GPUs| 64.8 | 13.8 x 1 GPUs| 0.945|
|Stage3    | 16.5 x 8 GPUs| 29.4 | 10.9 x 1 GPUs| 0.917|

Reviewer comment (Experiments):
Why use different image encoders (CLIP and DINOv2) in different stages?

Author response:
**Response:**  In the first stage, we need a global embedding of the target image. We chose the CLIP image encoder as CLIP image embedding can capture rich image content and style information, which can be used to guide subsequent target image synthesis. In the next two stages, we need fine-grained features of the source image. We use DINOv2 as an image encoder since previous work [1] found that DINOv2 performs better than CLIP.

[1] Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zeroshot object-level image customization. arXiv preprint arXiv:2307.09481, 2023.

Reviewer comment (Reproducibility):
Are the MLP networks in the inpainting stage (Figure 4) and refining stage (Figure 5) the same?

Author response:
**Response:** In the inpainting and refining stages, we all use the same MLP network to map features, but they do not share weights and are trained independently.

Reviewer comment (Presentation):
Figures 1 and 2 seem to have a lot of redundancy. Maybe consider consolidating them into one?

Author response:
**Response:** We sincerely appreciate the constructive comments from the reviewers. Following your suggestions, we have integrated them into the revised version, and simplified the painting to make it more intuitive. See:  [https://drive.google.com/file/d/1cAIz1CR0o8rh1abPlfG7p2GtxWyXiLdB/view?usp=drive_link](https://drive.google.com/file/d/1cAIz1CR0o8rh1abPlfG7p2GtxWyXiLdB/view?usp=drive_link) and [https://drive.google.com/file/d/1H2ssD3oUTPRu-R9GQurXjePPq_ol3Vx6/view?usp=drive_link](https://drive.google.com/file/d/1H2ssD3oUTPRu-R9GQurXjePPq_ol3Vx6/view?usp=drive_link).

Reviewer comment (Writing):
Typo in user study paragraph 2: “J2b” -> “Jab”

Author response:
**Response:** We apologize for the error in the manuscript here and greatly appreciate the reviewer's patient reading. Furthermore, we have made every effort to check the reversion thoroughly.

Reviewer comment (Writing):
[minor] "Although PCDMs have a lower FID score than PIDM" -> "worse FID score"?

Author response:
**Response:** We apologize for the error in the manuscript here and greatly appreciate the reviewer's patient reading. Furthermore, we have made every effort to check the revised version thoroughly.

Reviewer comment (Experiments):
In Figure 6 visualization, it seems the proposed method is much better at the reconstruction of text patterns. Is this because the proposed method utilize a pretrained stable diffusion v2? How much performance gain over prior works could be due to having a pretrained stable diffusion v2?

Author response:
**Response:** We greatly appreciate your question. We have added a set of comparisons between using pretrained and not using pretrained at the reconstruction of text patterns.  See:  [https://drive.google.com/file/d/1VMxDBfT-vQfaVXcPeMyzKFDJbXN6lT3i/view?usp=drive_link](https://drive.google.com/file/d/1VMxDBfT-vQfaVXcPeMyzKFDJbXN6lT3i/view?usp=drive_link).
The results show that even without pretrained, the outcomes are still remarkably realistic. This indicates that the improvement in text reconstruction is primarily due to the rationality and superiority of our framework. Additionally, we have added a set of qualitative metrics, and we found that the pretrained PCDMs show a slight improvement in these metrics.

|Settings |SSIM  (↑)   |LPIPS  (↓)   |FID  (↓) |
|:--|:------|:--------|:--------|
|w/o pretrained |0.7428     |0.1389     |7.6946 |
| w/ pretrained  |0.7444    |0.1365    |7.4734 |

Reviewer comment (Experiments):
The qualitative ablation in Figure 8 is good to show the effectiveness of each component of the proposed method. However, the quantitive ablation is also very important. Besides, I would expect adding results of PCDMs w/o refining in Table 1.

Author response:
**Response:** Thank you for your  thoughtful advice.  Following your suggestions, we have added the quantitative ablation results of Figure 8 to the revised version. 


|Settings | SSIM  (↑)   | LPIPS  (↓)   | FID  (↓) |
|:---------|:-------|:-------|:----------|
|B1 |0.7209     |0.1693     |8.4238 |
|B2   |0.7357    |0.1426    |7.7815 |
|B3 |0.7378    |0.1419    |7.6924 |
|PCDMs  |0.7444    |0.1365    |7.4738 |

The experimental results demonstrate that, firstly, B1 (only utilizing the inpainting conditional diffusion model) can achieve highly competitive performance on the DeepFashion dataset. Secondly, B2 (Prior + Inpainting) and B3 (Inpainting + Refining) outperform B1 on the SSIM, LPIPS, and FID. These results show they are also constructive to the success of our PCDMs. 

Besides, according to your advice, we have added the w/o refining results to Table 1. We appreciate your thoughtful reminder once again. See: [https://drive.google.com/file/d/1F3JJ9bWHdXCdrrHr_avV5CGqjrOTk-e2/view?usp=drive_link](https://drive.google.com/file/d/1F3JJ9bWHdXCdrrHr_avV5CGqjrOTk-e2/view?usp=drive_link).

Reviewer comment (Experiments):
The refining results reported on different methods are on different samples. To better understand the refining benefits for different methods, it would be good to show the results of before-refining and after-refining on the same samples both quantitatively and qualitatively. For example, authors can add visual samples in Figure 10, and add quantitative results of PIDM + refining. I wonder if PIDM + refining can also achieve close performance.

Author response:
**Response:** We wholeheartedly agree with your perspective and have redrawn Figure 10 and updated it in the revised version for better comprehension.  See: [https://drive.google.com/file/d/1ooijhEL_1VcCvxqBKMtTQZhxuxJe4cvS/view?usp=drive_link](https://drive.google.com/file/d/1ooijhEL_1VcCvxqBKMtTQZhxuxJe4cvS/view?usp=drive_link).
Furthermore, we have added the quantitative results of PIDM combined with refining. 


|Methods    |SSIM  (↑)    |LPIPS  (↓)     |FID (↓) |
|:---------------|:-------|:-------|:----------|
|PIDM     |0.7312     |0.1678    |6.3671 |
|PIDM + Refining    |0.7397    |0.1462 |    6.7384 |
|PCDMs     |0.7444     |0.1365    |7.4738 |

The experimental results show that PIDM+Refining significantly improves visual consistency, SSIM, and LPIPS metric, although there is a slight deterioration in the FID metric. Besides, PCDMs also surpass PIDM+Refining on SSIM and LPIPS metrics.

Reviewer comment (Evaluation):
According to the ablation study, the features predicted by the first stage is important for pose-level alignment. I would expect more analysis of the progressive alignment procedure. For example, the authors may make some visual analysis based on the features, such as pixel correspondence or warp results similar to CoCosNet.

Author response:
**Response:** Thanks for your advice. CoCosNet introduces an explicit correspondence module to establish a dense correspondence, thereby enabling the visualization of pixel correspondence. In our first stage, we use a transformer diffusion model to predict the global embedding of the target image and the global embedding is just a global representation, so it cannot visualize the pixel correspondence. Thank you once again for your kind suggestions.

Reviewer comment (Evaluation):
In the metric section, it would be good to add references for R2G, G2R, and Jab to make the paper more self-contained.

Author response:
**Response:** We appreciate the suggestion.  Our revised version has added references for R2G, G2R, and Jab.

Reviewer comment (Presentation):
I would suggest adding some samples of source image with invisible logo and target image with visible logo to the main paper. Otherwise, one may be concerned about the overfitting issue.

Author response:
**Response:** We appreciate this point and have added examples of source images with invisible logo and target images with visible logo in Figure 6 of the main paper.  See:  [https://drive.google.com/file/d/1R0o4rjNZjOdlduhWf4TdZPF7fmv_L3r7/view?usp=drive_link](https://drive.google.com/file/d/1R0o4rjNZjOdlduhWf4TdZPF7fmv_L3r7/view?usp=drive_link).
The results indicate that PCDMs do not overfit, and our results demonstrate better visual consistency than other SOTA methods.

Reviewer comment (Evaluation):
Since the proposed method has three diffusion stages, I wonder if its time cost will be much higher. Some analysis of the inference time could be discussed, which is important for practical applications.

Author response:
**Response:** We thank the reviewer for pointing out this issue.  We have added an experiment comparing the inference time with PIDM. All experiments were conducted on the same V100 GPU to ensure a fair comparison. 

|Size    |PIDM    |PCDMs  |
|:---------|:-----------|:------------|
|256x176     |9.377 (s)   |**2.288 (s)** |
|512x352    |11.746 (s)  |**3.701 (s)**|

The results show that PCDMs outperform PIDM, another diffusion model, regarding inference time despite PCDMs being a multi-stage pipeline. For instance, PCDMs are over twice as fast as PIDM at a resolution of 256x176, and even three times faster at 512x352. This is because our model is based on latent space diffusion, while PIDM is a pixel-level diffusion model. Furthermore, we can consider employing techniques such as quantization and distillation to achieve more efficient inference in practical applications.

Reviewer comment (Novelty):
The pipeline contains stages and such a coarse-to-fine paradigm has been adopted in other works in different ways. For example, [a,b] also adopt a human prior + inpaint + refinement strategies. More discussions on the whole pipeline and related works could make the paper more comprehensive.

Author response:
**Response:** Thank you for your thoughtful advice. In the related work, we go into detail about these methods, and we updated them in our revised version.  See:  [https://drive.google.com/file/d/16Se4EDH_ahscbNJI-SR-GPz9cK1At4Kk/view?usp=drive_link](https://drive.google.com/file/d/16Se4EDH_ahscbNJI-SR-GPz9cK1At4Kk/view?usp=drive_link).

Reviewer comment (Experiments):
A quantitive ablation study is needed.

Author response:
**Response:** Thank you for your  thoughtful advice.  Following your suggestions, we have added the quantitative ablation results of Figure 8 to the revised version. 


|Settings | SSIM  (↑)   | LPIPS  (↓)   | FID  (↓) |
|:---------|:-------|:-------|:----------|
|B1 |0.7209     |0.1693     |8.4238 |
|B2   |0.7357    |0.1426    |7.7815 |
|B3 |0.7378    |0.1419    |7.6924 |
|PCDMs  |0.7444    |0.1365    |7.4738 |

The experimental results demonstrate that, firstly, B1 (only utilizing the inpainting conditional diffusion model) can achieve highly competitive performance on the DeepFashion dataset. Secondly, B2 (Prior + Inpainting) and B3 (Inpainting + Refining) outperform B1 on the SSIM, LPIPS, and FID. These results show they are also constructive to the success of our PCDMs. 

Besides, according to your advice, we have added the w/o refining results to Table 1. We appreciate your thoughtful reminder once again. See: [https://drive.google.com/file/d/1F3JJ9bWHdXCdrrHr_avV5CGqjrOTk-e2/view?usp=drive_link](https://drive.google.com/file/d/1F3JJ9bWHdXCdrrHr_avV5CGqjrOTk-e2/view?usp=drive_link).

Reviewer comment (Experiments):
The qualitative refining comparisons with PIDM + refining is necessary.

Author response:
**Response:** We wholeheartedly agree with your perspective and have redrawn Figure 10 and updated it in the revised version for better comprehension.  See: [https://drive.google.com/file/d/1ooijhEL_1VcCvxqBKMtTQZhxuxJe4cvS/view?usp=drive_link](https://drive.google.com/file/d/1ooijhEL_1VcCvxqBKMtTQZhxuxJe4cvS/view?usp=drive_link).
Furthermore, we have added the quantitative results of PIDM combined with refining. 


|Methods    |SSIM  (↑)    |LPIPS  (↓)     |FID (↓) |
|:---------------|:-------|:-------|:----------|
|PIDM     |0.7312     |0.1678    |6.3671 |
|PIDM + Refining    |0.7397    |0.1462 |    6.7384 |
|PCDMs     |0.7444     |0.1365    |7.4738 |

The experimental results show that PIDM+Refining significantly improves visual consistency, SSIM, and LPIPS metric, although there is a slight deterioration in the FID metric. Besides, PCDMs also surpass PIDM+Refining on SSIM and LPIPS metrics.

Reviewer comment (Evaluation):
A visual analysis of the progressive alignment procedure at each stage is needed.

Author response:
**Response:**
I'm concerned that there might be a misunderstanding between us regarding **progressive alignment**. Methods like CoCosNet align at the pixel level through a differentiable warping function. In contrast, the goal of our first stage is to "predict the global features of a target image" to compensate for the image's invisible areas. Simultaneously, this global feature provides alignment at the feature, pose, and image levels in the second stage (considering the left and right source and target images as a whole). **Therefore, the outputs of our three stages are feature vectors, coarse-grained images, and fine-grained images, respectively. So we cannot directly compare the three stages at the pixel level.** To address this issue, we have added an experiment where we compare the similarity of the features from all three stages with the GT.

|  Stages |  Scores   |
|:-----|:----|
|Stage1| 0.8916|
|Stage2|  0.9237|
|Stage3|   0.9365|

The experimental results show that the results of our three stages are increasingly close to the GT.