Paper: Stochastic Adaptive Sequential  Black-box Optimization for  Diffusion Targeted Generation (ICLR-2024)

Reviewer comment (Writing):
The methodology, as currently presented, might be heavy and unfriendly to the general audience. The authors should aim to articulate their primary contributions and the uniqueness of their proposed methodology more clearly, focusing on how it addresses the identified challenges.

Author response:
In this paper, we focus on the problem of how to employ diffusion models to generate user-preferred content with black-box target scores while avoiding re-training. This problem is essential but unexplored in the literature. The related DDOM needs a reweighted sampling of training samples to re-train a conditional diffusion model from scratch.  


Moreover, the black-box optimization methods in the literature can not be directly applied for diffusion model fine-tuning due to ignoring the sequential nature of the diffusion sampling process. To address this problem, we $ \textbf{are the first to}$ formulate the diffusion model black-box fine-tuning problem as a sequential black-box optimization problem with an adaptive diffusion noise process. 
We propose a novel covariance adaptive sequential black-box optimization algorithm to solve this problem. Furthermore, we prove a $O(\frac{d^2}{\sqrt{T}})$ convergence rate of our algorithm for convex functions without smooth and strongly convex assumptions. To the best of our knowledge, our algorithm is the $\textbf{first}$  covariance adaptive black-box optimization method that achieves a provable $O(\frac{d^2}{\sqrt{T}})$ convergence rate for general non-smooth convex functions. 


We firmly believe that our work explores a crucial area underexplored and provides solid technique contributions to this area.

Reviewer comment (Experiments):
The rationale for using text-guided generation as a demonstration of the proposed method is not entirely clear. The authors should elaborate on why they believe this setting poses a challenging black-box optimization score. Does this scorer belong to those touch non-smooth cases?

Author response:
The black-box targeted image generation is challenging due to the very high dimension of images.  Specifically,   the dimension of the images employed in our work is $256\times256\times3 = 196608$.  In contrast, most of the tasks employed in the DDOM paper are less than $100$-dimension, and the highest dimension is $5152$.  The image generation task is more challenging than the one studied in the DDOM paper. 

"Does this scorer belong to those touch non-smooth cases?"

Most scorers in real applications are non-smooth cases. For example, the scorers based on neural networks with ReLU function are non-smooth. To be more precise,  note that the ReLU function is non-smooth. Thus, the neural network using the ReLU function is also non-smooth. As a result,  the score functions that rely on  the neural network  using the ReLU function are also non-smooth.

Reviewer comment (Evaluation):
The quality of generated images is damaged. As evidenced in Figure 2, the generated images of baseline methods are more natural and clear than those of the proposed method. Is there any solution to resolve this problem?

Author response:
We have solved the loss of visual quality issue and include additional experiments.  Please check our overall response and Section A in the Appendix of our revised paper.

Reviewer comment (Experiments):
The single-domain experiment on text-guided generation does not sufficiently prove the effectiveness of the method proposed. The authors should consider introducing more diverse black-box scorers to exhibit the method's versatility. For example, DDOM conducted experiments across multiple domains; perhaps a similar approach could be beneficial here.

Author response:
Thanks for your suggestion. We include additional experiments on the tasks studied in the DDOM. Please check our overall response and Section A in the Appendix of our revised paper.

Reviewer comment (Theory):
The derivation process in Equation (10) and how this objective aligns with Equation (8) is not clear. Can the authors provide further clarification on this?

Author response:
Note that $J(\bar{\theta}_K^t)$ is a constant when we minimize $J(\bar{\theta}_K)$ w.r.t.  $\bar{\theta}_K$. Thus, minimizing Eq.(10) (Eq.(9) in the revised paper)  is equivalent to minimizing objective Eq.(8) (Eq.(7) in the revised paper)

Reviewer comment (Writing):
The major weakness lies in the clarity of writing, which made it challenging for me to understand the exact mechanism of the method and to check the claim on the convergence rate. In particular, the indexing of \(k\) and \(t\) are confusing.

Author response:
"(1) the indexing of k and t are confusing."

$\textbf{A(1)}$: Thanks for your suggestion. We have revised the paper accordingly.  The index $k$ denotes the sequential step of the diffusion sampling.  The index  $t$ denotes the number of iterations for parameter updates in the black-box optimization process (see Alg.1 and Alg.2). 
To clarify the confusion, we revised section 2.1 to avoid the double definition of $t$ in  Eq.(1) $\sim$ (2).

Reviewer comment (Evaluation):
The empirical evaluation is limited and unconvincing. For example, in Figure 2, the extended dataset (last row) seems to have a higher CLIP score, but lower visual quality. While the author briefly touched upon this point and leaves it to future work, this behavior definitely limits the practical use of the proposed method, and I think it is a concern that needs to be well addressed within the scope of this work in order for it to be well-received in the community.

Author response:
$\textbf{W2}$:  Loss of visual quality issue.  "The generated images have higher CLIP scores but lower visual quality. "

$\textbf{A2}$: We have solved this issue and included additional experiments.  Please check our overall response and Section A in Appendix of our revised paper. 

$\textbf{Q18:}$ "Loss of visual quality issue."

$\textbf{A18:}$ We have solved the loss of visual quality issue and included additional experiments. Please check our overall response and Section A in the Appendix of our revised paper.

Reviewer comment (Evaluation):
I have reservations regarding whether the proposed method has effectively demonstrated superior practical utility, given that visually the samples from competing methods also align with the generation goal and can have better visual quality despite a lower target score.

Author response:
$\textbf{W2}$:  Loss of visual quality issue.  "The generated images have higher CLIP scores but lower visual quality. "

$\textbf{A2}$: We have solved this issue and included additional experiments.  Please check our overall response and Section A in Appendix of our revised paper. 

$\textbf{Q18:}$ "Loss of visual quality issue."

$\textbf{A18:}$ We have solved the loss of visual quality issue and included additional experiments. Please check our overall response and Section A in the Appendix of our revised paper.

Reviewer comment (Novelty):
What do you mean by a "full matrix update" and an "adaptive update"? When making claims such as "algorithm is the first full matrix adaptive black-box optimization with x convergence rate," it is important to explain the qualifiers and the relevant literature.

Author response:
Our algorithm updates the full covariance matrix at each step during the optimization process. The “adaptive update” means that the covariance matrix is updated according to the function values of the quires at each step instead of a fixed one. 

In the literature, NES,  CMAES and INGO are methods that update the full covariance matrix for black-box optimization, which has been discussed in our related work section.  In the NES and CMAES papers,  the authors did not provide convergence analysis.  In INGO, the convergence rate is analyzed for standard black-box problems under strongly convex assumptions.  Our convergence rate holds for sequential black-box optimization problems without strongly convex assumptions.  In particular, when the total step of the sequential problems is one, i.e., K=1.  We achieve a convergence rate for standard black-box problems.