SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We present the projection theorem here to draw connections between the above boundary search and different operational latent space levels (i.e.,  $\epsilon$ -space and  $h$ -space). As we describe in the main paper, the classification results from Tab. 6 show that even though the accuracy score is generally lower in  $\epsilon$ -space, it does carry meaningful semantic boundaries. This above observation and claim differ from the previous literature [44], where the latent spaces of DDMs are considered to lack semantic meaning. In fact, given the recent study from [35], which first reveals the semantic behaviors of pre-trained DDMs in  $h$ -space, it provides evidence to imply that the same semantic meanings might also exist in the higher-dimensional  $\epsilon$ -space. As  $h$ -space is a subspace of corresponding  $\epsilon$ -space with higher dimensionality.
F BoundaryDiffusion with Mixing Trajectory
We show the algorithm implementation for our proposed BoundaryDiffusion with mixing trajectory under the conditional application scenario in Algo. 1. For the unconditional scenario, the only difference is that we can directly sample the latent encodings from the Gaussian distribution as the initial  $\mathbf{x}_T$ , and get the corresponding  $h$ -level latent encoding  $\mathbf{h}_T$  from the given DDPM at  $T$  step.
Algorithm 1 BoundaryDiffusion (Conditional)
Input: input image  $\mathbf{x}_0$ , target boundaries  $\mathbf{b}_{\epsilon}$  and  $\mathbf{b}_h$  for the editing attribute  $m$ , pre-trained DDM  $p$ , inversion steps  $S_{inv}$ , denoising steps  $S_{gen}$ , mixing step  $t_m$ , user defined editing distance  $\zeta_{\epsilon}$  and  $\zeta_h$ , and editing space steps  $K$ .  
// Step 1: Inversion via DDIMs to get the latent encoding at  $t_m$   
Define  $\{\tau_s\}_{s=1}^{S_{inv}}$  s.t.  $\tau_1 = 0$ ,  $\tau_{S_{inv}} = t_m$   
for  $s = 1,2,\dots,S_{inv} - 1$  do  
 $\epsilon \gets p(\mathbf{x}_{\tau_s},\tau_s)$ $\mathbf{x}_{\tau_{s+1}} = \sqrt{\alpha_{\tau_s}}\mathbf{x}_{\tau_s} + \sqrt{1 - \alpha_{\tau_s}}\epsilon$   
end for  
 $\mathbf{h}_{t_m} \gets$  extract  $h$  feature map from  $\epsilon$   
// Step 2: Boundary guidance  
// Step 2.1: Define initial editing space in  $\epsilon$  and  $h$  latent levels  
 $\{d_{\epsilon}^j\}^K$  s.t.  $d_{\epsilon}^1 = -\zeta_{\epsilon}$ ,  $d_{\epsilon}^K = \zeta_{\epsilon}$ $\{d_h^j\}^K$  s.t.  $d_h^1 = -\zeta_h$ ,  $d_h^K = \zeta_h$   
// Step 2.2: Compute projection distance to the boundaries  
 $\{d_{p,\epsilon}\} = \{d_{\epsilon}\} - \mathbf{b}_{\epsilon}^T\mathbf{x}_{t_m}$ $\{d_{p,h}\} = \{d_h\} - \mathbf{b}_h^T\mathbf{h}_{t_m}$   
// Step3: Denoising with mixing trajectory  
for  $k = 1,2,\dots,K$  do  
 $\mathbf{x}_{t_m}' = \mathbf{x}_{t_m} + d_{\epsilon}^k\mathbf{b}_{\epsilon}$ $\mathbf{h}_{t_m}' = \mathbf{h}_{t_m} + d_h^k\mathbf{b}_h$ $\mathbf{x}_s \gets \mathbf{x}_{t_m}'$ $\mathbf{h}_s \gets \mathbf{h}_{t_m}'$   
for  $s = S_{gen}, S_{gen} - 1,\dots,2$  do  
 $\epsilon \gets p_s(\mathbf{x}_s,\mathbf{h}_s,s)$ $z \sim \mathcal{N}(0,I_d)$ $\mathbf{x}_{s-1} = \sqrt{\alpha_{s-1}}(\frac{\mathbf{x}_s - \sqrt{1 - \alpha_s}\epsilon}{\sqrt{\alpha_s}}) + \sqrt{1 - \alpha_{s-1} - \sigma^2}\epsilon + \sigma_s z$   
end for  
end for
G More Experimental Results
We present more experimental results and discussion in this section.
G.1 Pre-trained Models and Datasets
The pre-trained DDMs we use for experiments mainly include the DDPM [21] and the improved DDPM (iDDPM) [41]. The main difference between the original DDPM and the improved version lies within the fact that iDDPMs use a hybrid learning objective that obtains better log-likelihoods than directly optimizing it.
We conduct experiments on multiple datasets, which includes CelabA-64 [39], CelebA-HQ-256 [27], AFHQ-dog-256 [10], LSUN-church-256 [66], LSUN-bedroom-256 [66]. Different from existing works that usually pay little attention to the image resolutions in the experiments, the resolutions play an important role in our experiments since they define the actual dimensionality of the latent spaces for pre-trained DDMs. However, the  $64^{2}$  resolution model is mainly used in the high-dimensional analysis and interpolation observations, for the image editing and semantic control experiments, we use  $256^{2}$  as the default resolution for visualization quality.
G.2 Semantic Boundary Validation
We search the semantic boundaries via linear classifiers on both  $\epsilon$ -space and  $h$ -space using 100 images, and we show the semantic behaviors via the testing classification accuracy on different attributes in Tab. 6. We observe from the classification results that the boundaries in  $h$ -space are in general better defined compared to the  $\epsilon$ -space, which is consistent with previous findings from [35]. Notably, for certain attributes such as glass and mustache, both space levels perform well in defining the
boundaries, which implies and aligns with our empirical finding that guidance on both levels of latent spaces helps for more effective semantic control.
Table 6: Classification accuracy on separation boundaries in different latent spaces at the mixing step.
Latent space	Smile	Glass	Age	Mustache
ε	0.86	0.95	0.87	0.96
h	0.98	0.95	0.93	0.96
G.3 User Study
As subjective evaluations, we conduct a user study to compare our proposed method with Asyrp [35] and DiffusionCLIP [31] on CelebA-HQ-256 [39]. We use the official codebases from previous works and follow the exact default commands, using the smile attribute as the editing target, either to add or remove smiles from 100 raw images that are randomly selected from the dataset. Our human evaluators include 20 graduate students volunteers, with 15 males and 5 females. Specifically, we show the original images together with edited images to the human evaluator and ask them to pick the best result with respect to general quality and attribute editing effectiveness.
More non-cherry picky editing results from three different methods are included in Fig. 5 and Fig. 6.
Fig. 7 shows more interpolation results from different combinations of latent encoding sources and sampling methods on CelebA-64 during our investigation on the distance effect. More qualitative samples for different editing settings are included in Fig. 8, Fig. 9, Fig. 10 and Fig. 11.
Add smiling
Figure 5: Non-Cherry-Picked randomly selected results for add smiling and remove smiling editing operations from our proposed BoundaryDiffusion, Asyrp [35], and DiffusionCLIP [31].
Remove smiling
Raw image
DiffusionCLIP
BoundaryDiffusion
Figure 6: More Non-Cherry-Picked randomly selected results for LSUN-Church dataset given "red brick church" as editing text prompt. Comparing against learning-based SOTA method DiffusionCLIP, our learning-free method preserves better image details and achieve better fidelity and more natural editing effects.
Raw image
DiffusionCLIP
BoundaryDiffusion
Figure 7: More interpolation results from different combinations of latent encoding sources and sampling methods on CelebA-64.
Editing strength
Figure 8: More qualitative results for editing strength modification. We use the CelabA-HQ-256 dataset and the attributes glass and age as examples. In particular, we also show samples with distortions and lower quality when the editing distance becomes too large. The optimal editing distance range is also related to the properties of the high-dimensional spaces.
Figure 9: More qualitative results for text-based conditional editing on the LSUN-Bedroom-256 dataset.
Figure 10: More qualitative results for text-based conditional editing on the LSUN-Church-256 dataset.
Figure 11: More qualitative results for unconditional semantic control on AFHQ-Dog-256 and CelabA-HQ-256 datasets with smile semantic.