SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As shown in Fig 1(d), we iterate this process continuously to obtain  $\hat{x}_0$  from coarse to fine. Note that in the last step we directly use  $f_{\theta}(\hat{x}_1,1,F)$  as  $\hat{x}_0$ . In  $\hat{x}_0$ , we choose the span with the highest confidence score in  $\hat{z}_0$  as the final prediction. To reduce inference overhead, we do not employ any post-processing techniques, such as box renewal in DiffusionDet [61] and self-condition [71].
4.1 Datasets, Metrics and Implementation Details
Datasets. We evaluate the efficacy of our model by conducting experiments on three representative datasets: Charades-STA [23], QVHighlights [37] and TACoS [74]. The reason is that the above three datasets exhibit diversity. Charades-STA comprises intricate daily human activities. QVHighlights contains a broad spectrum of themes, ranging from everyday activities and travel in lifestyle vlogs to social and political events in news videos. TACoS mainly presents long-form videos featuring culinary activities. The training and testing divisions are consistent with existing methods [28, 38].
Metrics. To make fair comparisons, we adopt the same evaluation metrics as those used in previous works [38, 37, 23, 75, 29, 19, 27], namely R1@n, MAP@n, and  $\mathrm{MAP}_{avg}$ . Specifically, R1@n is defined as the percentage of testing queries that have at least one correct retrieved moment (with an intersection over union (IoU) greater than n) within the top-1 results. Similarly, MAP@n is defined as the mean average precision with an IoU greater than n, while  $\mathrm{MAP}_{avg}$  is determined as the average MAP@n across multiple IoU thresholds [0.5: 0.05: 0.95].
Implementation details. For a fair comparison [44, 20, 30], we freeze the video encoder and text encoder and use only the extracted features. For VGG [63], C3D [64] or SlowFast+CLIP (SF+C) [66, 65], we extract video features every 1/6s, 1s or 2s. So the frame number  $N_{v}$  is related to the length of the video, while the max text length  $N_{t}$  is set to 32. Note that since the videos in TACoS are long, we uniformly sample the video frame features and set the max frame number  $N_{v}$  to 100 in TACoS. We set the hidden size  $D = 256$  in all Transformer layers. In SCG, we also use a variant of the pairwise margin loss called InfoNCE loss [76]. The number of random spans  $N_{r}$  is set to 10 for QVHighlights, 5 for Charades-STA and TACoS. We use the cosine schedule for  $\beta$ . For all datasets, we optimize MomentDiff for 100 epochs on one NVIDIA Tesla A100 GPU, employ Adam optimizer [77] with 1e-4 weight decay and fix the batch size as 32. The learning rate is set to 1e-4. By default, the loss hyperparameters  $\lambda_{\mathrm{L1}} = 10$ ,  $\lambda_{\mathrm{iou}} = 1$  and  $\lambda_{\mathrm{ce}} = 4$ . The weight values for  $\mathcal{L}_{sim}$  and  $\mathcal{L}_{vmr}$  are 4 and 1. To speed up the sampling process during inference, we follow DDIM [69] and iterate 50 times.
Table 1: Performance comparisons (\%) on the Charades-STA dataset. "★" denotes that we re-implement the method under the same training scheme. "A" stands for using audio data.
Method	Type	Charades-STA
R1@0.5	R1@0.7	MAP@0.5	MAP@0.75	MAPavg
MAN [33]	VGG, Glove	41.21	20.54	-	-	-
RaNet* [75]	VGG, Glove	42.91	25.82	53.28	24.41	28.55
2DTAN* [27]	VGG, Glove	41.34	23.91	54.68	24.15	29.26
DORi [78]	VGG, Glove	43.47	26.37	-	-	-
CBLN [44]	VGG, Glove	47.94	28.22	-	-	-
DCM [40]	VGG, Glove	47.80	28.00	-	-	-
MMN* [28]	VGG, Glove	46.93	27.07	58.85	28.16	31.58
MomentDETR* [37]	VGG, Glove	50.54	28.01	57.39	25.62	29.87
MomentDiff	VGG, Glove	51.94	28.25	59.86	29.11	31.66
UMT [38]	VGG+A, Glove	48.44	29.76	58.03	27.46	30.37
MomentDiff	VGG+A, Glove	52.62	29.93	60.69	29.74	31.81
DEBUG[79]	C3D, Glove	37.39	17.69	-	-	-
LPNet[36]	C3D, Glove	40.94	21.13	-	-	-
VSLNet* [30]	C3D, Glove	48.67	30.33	56.88	25.79	30.16
MomentDETR* [37]	C3D, Glove	50.49	29.95	56.27	26.08	29.92
MomentDiff	C3D, Glove	53.79	30.18	59.32	29.85	31.89
MomentDETR* [37]	SF+C, C	53.22	30.87	58.86	26.43	30.43
MomentDiff	SF+C, Glove	55.42	32.17	60.93	32.47	32.59
MomentDiff	SF+C, C	55.57	32.42	61.07	32.51	32.85
4.2 Performance Comparisons
Comparison with state-of-the-art methods. To prove the effectiveness of MomentDiff, we compare the retrieval performance with 17 discriminative VMR methods. Tab. 1, Tab. 2, and Tab. 3 show the R1@n, MAP@n, and  $\mathrm{MAP}_{avg}$  results on Charades-STA, QVHighlights and TACoS. Compared with SOTA methods [28, 38, 75, 44, 30, 37], MomentDiff achieves significant improvements on Charades-STA regardless of whether 2D features (VGG), multimodal features (VGG+A), 3D features (C3D), or multimodal pre-trained features (SF+C) are used. This proves that MomentDiff is a universal generative VMR method. In the other two datasets (QVHighlights and TACoS), we still have highly competitive results. Specifically, compared to MomentDETR [37], MomentDiff obtains  $2.35\%$ ,  $3.86\%$ , and  $13.13\%$  average gains in R1@0.5 on three datasets. It is worth noting that TACoS contains long videos of cooking events where different events are only slightly different in terms of cookware, food and other items. The learnable queries in MomentDETR may not cope well with such fine-grained dynamic changes. We attribute the great advantage of MomentDiff over these methods to fully exploiting similarity-aware condition information and progressive refinement denoising.
Table 2: Performance comparisons (%) on QVHighlights Table 3: Performance comparisons (%) with SF+C video features and CLIP text features. "★" de-on TACoS. We adopt C3D features to notes that we re-implement the method with only segment encode videos. MDE is the abbreviation moment labels. "†" stands for using audio data. MDE is the of MomentDETR [37].
abbreviation of MomentDETR [37].
Method	QVHighlights
R1@0.5	R1@0.7	MAP@0.5	MAP@0.75	MAPavg
MCN [25]	11.41	2.72	24.94	8.22	10.67
CAL [80]	25.49	11.54	23.40	7.65	9.89
XML [81]	41.83	30.35	44.63	31.73	32.14
XML+ [81]	46.69	33.46	47.89	34.67	34.90
MDE* [37]	53.56	34.09	53.97	28.65	29.39
MomentDiff	57.42	39.66	54.02	35.73	35.95
UMT*† [38]	56.26	40.31	52.77	36.82	35.79
MomentDiff†	58.21	41.48	54.57	37.21	36.84
Method	TACoS
R1@0.1	R1@0.3	R1@0.5
CTRL [23]	24.32	18.32	13.30
SCDM [32]	-	26.11	21.17
DRN [31]	-	-	23.17
DCL [41]	49.36	38.84	29.07
CBLN [44]	49.16	38.98	27.65
FVMR [42]	53.12	41.48	29.12
RaNet [75]	-	43.34	33.54
MDE* [37]	41.16	32.21	20.55
MMN* [28]	51.39	39.24	26.17
MomentDiff	56.81	44.78	33.68
Transfer experiments. To explore the location bias problem, we first organize the Out of Distribution (OOD) experiment following [82], which repartitions Charades-STA [23] and ActivityNet-Captions [83] datasets according to moment annotation density values [82]. In Tab. 4, we exceed
Table 4: Transfer experiments (%) on Charades-CD (VGG, Glove) and ActivityNet-CD (C3D, Glove).
Method	Charades-CD	ActivityNet-CD
R1@0.3	R1@0.5	R1@0.7	MAPavg	R1@0.3	R1@0.5	R1@0.7	MAPavg
2DTAN [27]	49.71	28.95	12.78	12.60	40.04	22.07	10.29	12.77
MMN [28]	55.91	34.56	15.84	15.73	44.13	24.69	12.22	15.06
MomentDETR [37]	57.34	41.18	19.31	18.95	39.98	21.30	10.58	12.19
MomentDiff	67.73	47.17	22.98	22.76	45.54	26.96	13.69	16.38
Table 5: Transfer experiments (%) on anti-bias datasets with VGG and Glove video-text features.
Method	Charades-STA-Len	Charades-STA-Mom
R1@0.3	R1@0.5	R1@0.7	MAPavg	R1@0.3	R1@0.5	R1@0.7	MAPavg
2DTAN [27]	39.68	28.68	17.72	22.79	27.81	20.44	10.84	17.23
MMN [28]	43.58	34.31	19.94	26.85	33.58	27.20	14.12	19.18
MomentDETR [37]	42.73	34.39	16.12	24.02	29.94	22.16	11.56	18.66
MomentDiff	51.25	38.32	23.38	28.19	48.39	33.59	15.71	21.37
MomentDETR by a large margin on Charades-CD and ActivityNet-CD. These results prove the robustness of the model in dealing with OOD scenarios.
To further explore the impact of a single factor  $(\boldsymbol{w}_0$  or  $c_{0}$ ) on the location bias problem, we organize moment retrieval on two anti-bias datasets with location distribution shifts: ① Charades-STA-Len. We collect all video-text pairs with  $w_{0} \leq 10s$  and randomly sample pairs with  $w_{0} > 10s$  in the original training set of Charades-STA, which account for  $80\%$  and  $20\%$  of the new training set, respectively. On the contrary, we collect all pairs with  $w_{0} > 10s$  and randomly sample pairs with  $w_{0} \leq 10s$  from the original test set, accounting for  $80\%$  and  $20\%$  of the new test set. ② Charades-STA-Mom. Similarly, we collect all video-text pairs with the end time  $c_{0} + w_{0} / 2 \leq 15s$  and sample pairs with the start time  $c_{0} - w_{0} / 2 > 15s$  as the training set, which accounts for  $80\%$  and  $20\%$  , respectively. Likewise, the construction rules for the test set are the opposite of those for the training set. Dataset statistics can refer to Train/Test in Fig. 4 and the supplementary material.
In Tab. 5, the proposed MomentDiff shows much more robustness than previous state-of-the-art method MomentDETR [37]. Concretely, compared with the experiment in Tab. 1, the performance gap between MomentDiff and MomentDETR gets larger on Charades-STA-Len and Charades-STA-Mom. Fig. 4 also demonstrates that the distribution of our prediction is closer to the one of the test set. We conjecture that it is because MomentDiff discards the learnable proposals that fit the prior distribution of the training set. Moreover, 2DTAN [27] and MMN [28] perform worse than MomentDETR on the original Charades-STA dataset, but they achieve better or comparable results than MomentDETR in Tab. 5. This shows that predefined proposals [27, 28] may be better than learnable proposals in dealing with the location bias problem, but they take up more time and space overhead. Differently, our method performs well on both public datasets and anti-bias datasets.
To provide further insight into MomentDiff, we conduct critical ablation studies on Charades-STA.
Figure 4: Statistical distributions on anti-bias datasets with location distribution shifts. We count the length or moment of each video-text pair, and finally normalize the counts (# Samples) for clarity. Train/Test: the count distribution of true spans on the training or test set. MDE: the count distribution of top-1 predicted spans by using MomentDETR on the test set.
Table 6: Ablation study (%) on the Charades-STA dataset with SF+C video features and CLIP text features. We report R1@0.5, R1@0.7 and  $\mathrm{MAP}_{avg}$ . Default settings are marked in blue.
(a) Different span embedding types.
Type	R1@0.5	R1@0.7	MAPavg
ROI	50.21	28.85	29.12
FC	55.57	32.42	32.85
(d) Different loss designs.
Lsim	Lvmr	R1@0.5	R1@0.7	MAPavg
✓	31.53	16.52	17.89
✓	40.94	24.65	23.76
✓	✓	55.57	32.42	32.85
(b) Effect of scale  $\lambda$
scale	R1@0.5	R1@0.7	MAPavg
1.0	54.33	32.17	31.39
2.0	55.57	32.42	32.85
3.0	50.12	28.08	30.47
(e) Effect of span number  $N_{r}$ .
Nr	R1@0.5	R1@0.7	MAPavg
1	50.83	26.17	26.98
5	55.57	32.42	32.85
10	53.89	30.48	30.27
20	50.42	27.62	27.53
(c) VMD and noise intensity  $m$ .
Type	R1@0.5	R1@0.7	MAPavg
w/o VMD	46.69	24.03	23.28
w/o m	50.41	29.73	28.72
Ours	55.57	32.42	32.85
(f) Model performance vs. speed.
Step	R1@0.5	R1@0.7	MAPavg	FPS
1	53.31	29.78	29.54	531.4
2	55.62	31.92	32.99	465.0
10	55.41	32.16	33.54	338.2
50	55.57	32.42	32.85	148.8
100	55.39	32.39	32.93	97.89
Query: The person starts drinking a glass of coffee.
Figure 5: Visualization of the diffusion process on Charades-STA (Left) and QVHighlights (Right). For clarity, we show 3 random spans  $(x_{50})$  with Gaussian initialization, and progressively get the top-1 result  $(x_0)$  according to the confidence score. Green box: right segment. MDE: MomentDETR [37].
Query: Kids checking out their goodies and chocolates during Easter.
Span embedding type. Regarding the way discrete spans are mapped to the embedding space, we compare the ROI strategy [61] with our linear projection (FC) in Tab. 6(a). For the ROI strategy, we slice the fusion embeddings  $\pmb{F}$  corresponding to random spans, followed by mean pooling on the sliced features. Tab. 6(a) shows that ROI does not work well. This may be due to two points: 1) ROI is a hard projection strategy, while the importance of each video frame is quite different. FC is similar to soft ROI, and its process can be trained end-to-end. 2) FC is decoupled from  $\pmb{F}$ , which allows the model to focus on modeling the diffusion process and avoid over-dependence on  $\pmb{F}$ .
Scale  $\lambda$ .  $\lambda$  is the signal-to-noise ratio [61] of the diffusion process, and its effect is shown in Tab. 6(b). We find that the effect of larger  $\lambda$  drops significantly, which may be due to the lack of more hard samples for denoising training when the proportion of noise is small, resulting in poor generalization.
Video Moment Denoiser (VMD) and noise intensity  $m$ . In Tab. 6(c), we first remove the denoiser and the diffusion process (w/o VMD). After training with the same losses, we find that predicting with only fusion embeddings  $F$  leads to a drastic drop in results, which reveals the effectiveness of denoising training. Then we remove the noise intensity  $m$  (w/o  $m$ ), and the result is reduced by  $5.16\%$  on R1@0.5. This shows that explicitly aggregating noise intensity with random spans improves noise modeling. Combined with VMD and  $m$ , the diffusion mechanism can fully understand the data distribution and generate the real span from coarse to fine.
Loss designs. In Tab. 6(d), we show the impact of loss functions. In  $\mathcal{L}_{sim}$ , we use pointwise and pairwise constraints to guide token-wise interactions between multimodal features, while ensuring reliable conditions for subsequent denoising. In  $\mathcal{L}_{vmr}$ , the model can learn to accurately localize exact segments. Adequate multimodal interaction and denoising training procedures are complementary.
Span number. In Tab. 6(e), we only need 5 random spans to achieve good results. Unlike object detection [61], the number of correct video segments corresponding to text query is small. Therefore, a large number of random inputs may make the model difficult to train and deteriorate the performance.
Model performance vs. speed. In Tab. 6(f), we explore the effects of different diffusion steps. When step=2, good results and fast inference speed have been achieved. Subsequent iterations can improve the results of high IoU (i.e., R1@0.7), which shows the natural advantages of diffusion models.
We show two examples of the diffusion process in Fig. 5. We can find that the retrieved moments by MomentDiff are closer to the ground truth than those by MomentDETR. The diffusion process can gradually reveal the similarity between text and video frames, thus achieving better results. Besides, the final predictions corresponding to spans with multiple random initial locations are close to the ground truth. This shows that our model achieves a mapping from arbitrary locations to real segments.
**Limitation.** Compared to existing methods [27, 37], the diffusion process requires multiple rounds of iterations, which may affect the inference speed. As shown in Tab. 6(f), we reduce the number of iterations, with only a small sacrifice in performance. In practical usage, we suggest choosing a reasonable step number for a better trade-off between performance and speed.
Conclusion. This paper proposes a novel generative video moment retrieval framework, MomentDiff, which simulates a typical human retrieval style via diffusion models. Benefiting from the denoising diffusion process from random noise to temporal span, we achieve the refinement of prediction results and alleviate the location bias problem existing in discriminative methods. MomentDiff demonstrates efficiency and generalization on multiple diverse and anti-bias datasets. We aim to stimulate further research on video moment retrieval by addressing the inadequacies in the framework design, and firmly believe that this work provides fundamental insights into the multimodal domain.
This work is supported by the National Key Research and Development Program of China (2022YFB3104700), the National Nature Science Foundation of China (62121002, 62022076, 62232006).
[1] Han, T., W. Xie, A. Zisserman. Self-supervised co-training for video representation learning. In NeurIPS, pages 5679-5690. 2020.
[2] Chen, Y., Y. Tsai, M. Yang. End-to-end multi-modal video temporal grounding. In NeurIPS, pages 28442-28453. 2021.
[3] Kim, M., H. Kwon, C. Wang, et al. Relational self-attention: What's missing in attention for video understanding. In NeurIPS, pages 8046–8059. 2021.
[4] Xie, C.-W., S. Sun, X. Xiong, et al. RA-CLIP: Retrieval augmented contrastive language-image pre-training. In CVPR, pages 19265–19274. 2023.
[5] Zhao, L., K. Zheng, Y. Zheng, et al. RLEG: vision-language representation learning with diffusion-based embedding generation. In ICML, pages 42247-42258. 2023.
[6] Jin, P., H. Li, Z. Cheng, et al. Text-video retrieval with disentangled conceptualization and set-to-set alignment. In *IJCAI*. 2023.
[7] Ging, S., M. Zolfaghari, H. Pirsiavash, et al. Coot: Cooperative hierarchical transformer for video-text representation learning. In NeurIPS, pages 22605-22618. 2020.
[8] Li, P., C.-W. Xie, L. Zhao, et al. Progressive spatio-temporal prototype matching for text-video retrieval. In ICCV, pages 4100-4110. 2023.
[9] Li, P., H. Xie, J. Ge, et al. Dual-stream knowledge-preserving hashing for unsupervised video retrieval. In ECCV, pages 181-197. 2022.
[10] Miech, A., J.-B. Alayrac, L. Smaira, et al. End-to-end learning of visual representations from uncurated instructional videos. In CVPR, pages 9879-9889. 2020.
[11] Jin, P., J. Huang, F. Liu, et al. Expectation-maximization contrastive learning for compact video-and-language representations. In NeurIPS, pages 30291-30306. 2022.
[12] Jin, P., H. Li, Z. Cheng, et al. Diffusionret: Generative text-video retrieval with diffusion model. In ICCV. 2023.
[13] Jin, P., J. Huang, P. Xiong, et al. Video-text as game players: Hierarchical banzhaf interaction for cross-modal representation learning. In CVPR, pages 2472-2482. 2023.
[14] Li, P., H. Xie, S. Min, et al. Deep fourier ranking quantization for semi-supervised image retrieval. TIP, pages 5909-5922, 2022.
[15] Zhang, A., H. Fei, Y. Yao, et al. Transfer visual prompt generator across llms. In NeurIPS. 2023.
[16] Li, P., Y. Li, H. Xie, et al. Neighborhood-adaptive structure augmented metric learning. In AAAI, pages 1367-1375. 2022.
[17] Yuan, Y., T. Mei, W. Zhu. To find where you talk: Temporal sentence localization in video with attention based location regression. In AAAI, pages 9159-9166. 2019.
[18] Cao, M., T. Yang, J. Weng, et al. Locvtp: Video-text pre-training for temporal localization. In ECCV, vol. 13686, pages 38-56. 2022.
[19] Zhang, M., Y. Yang, X. Chen, et al. Multi-stage aggregated transformer network for temporal language localization in videos. In CVPR, pages 12669-12678. 2021.
[20] Soldan, M., A. Pardo, J. L. Alcazar, et al. MAD: A scalable dataset for language grounding in videos from movie audio descriptions. In CVPR, pages 5016-5025. 2022.
[21] Kim, D., J. Park, J. Lee, et al. Language-free training for zero-shot video grounding. In WACV, pages 2539-2548. 2023.
[22] Moon, W., S. Hyun, S. Park, et al. Query-dependent video representation for moment retrieval and highlight detection. In CVPR. 2023.
[23] Gao, J., C. Sun, Z. Yang, et al. Tall: Temporal activity localization via language query. In ICCV, pages 5267-5275. 2017.
[24] Liu, M., X. Wang, L. Nie, et al. Attentive moment retrieval in videos. In SIGIR, pages 15-24. 2018.
[25] Anne Hendricks, L., O. Wang, E. Shechtman, et al. Localizing moments in video with natural language. In ICCV, pages 5803-5812. 2017.
[26] Chen, J., X. Chen, L. Ma, et al. Temporally grounding natural sentence in video. In EMNLP, pages 162-171. 2018.
[27] Zhang, S., H. Peng, J. Fu, et al. Learning 2d temporal adjacent networks for moment localization with natural language. In AAAI, pages 12870-12877. 2020.
[28] Wang, Z., L. Wang, T. Wu, et al. Negative sample matters: A renaissance of metric learning for temporal grounding. In AAAI, pages 2613-2623. 2022.
[29] Zhao, Y., Z. Zhao, Z. Zhang, et al. Cascaded prediction network via segment tree for temporal video grounding. In CVPR, pages 4197-4206. 2021.
[30] Zhang, H., A. Sun, W. Jing, et al. Span-based localizing network for natural language video localization. In ACL, pages 6543-6554. 2020.
[31] Zeng, R., H. Xu, W. Huang, et al. Dense regression network for video grounding. In CVPR, pages 10287-10296. 2020.
[32] Yuan, Y., L. Ma, J. Wang, et al. Semantic conditioned dynamic modulation for temporal sentence grounding in videos. NeurIPS, 32, 2019.
[33] Zhang, D., X. Dai, X. Wang, et al. MAN: moment alignment network for natural language moment retrieval via iterative graph adjustment. In CVPR, pages 1247-1257. 2019.
[34] Liu, D., X. Qu, J. Dong, et al. Adaptive proposal generation network for temporal sentence localization in videos. In EMNLP, pages 9292-9301. 2021.
[35] Xu, H., K. He, B. A. Plummer, et al. Multilevel language and vision integration for text-to-clip retrieval. In AAAI, pages 9062–9069. 2019.
[36] Xiao, S., L. Chen, J. Shao, et al. Natural language video localization with learnable moment proposals. In EMNLP, pages 4008-4017. 2021.
[37] Lei, J., T. L. Berg, M. Bansal. Qvhighlights: Detecting moments and highlights in videos via natural language queries. In NeurIPS. 2021.
[38] Liu, Y., S. Li, Y. Wu, et al. UMT: unified multi-modal transformers for joint video moment retrieval and highlight detection. In CVPR, pages 3032-3041. 2022.
[39] Otani, M., Y. Nakashima, E. Rahtu, et al. Uncovering hidden challenges in query-based video moment retrieval. In BMVC. 2020.
[40] Yang, X., F. Feng, W. Ji, et al. Deconfounded video moment retrieval with causal intervention. In SIGIR, page 1-10. 2021.
[41] Nan, G., R. Qiao, Y. Xiao, et al. Interventional video grounding with dual contrastive learning. In CVPR, pages 2765-2775. 2021.
[42] Gao, J., C. Xu. Fast video moment retrieval. In ICCV, pages 1523-1532. 2021.
[43] Wang, H., Z.-J. Zha, L. Li, et al. Structured multi-level interaction network for video moment localization via language query. In CVPR. 2021.
[44] Liu, D., X. Qu, J. Dong, et al. Context-aware biaffine localizing network for temporal sentence grounding. In CVPR, pages 11235-11244. 2021.
[45] Zhang, H., A. Sun, W. Jing, et al. Towards debiasing temporal sentence grounding in video. arXiv preprint arXiv:2111.04321, 2021.
[46] Zeng, Y., D. Cao, X. Wei, et al. Multi-modal relational graph for cross-modal video moment retrieval. In CVPR, pages 2215-2224. 2021.
[47] Mun, J., M. Cho, B. Han. Local-global video-text interactions for temporal grounding. In CVPR, pages 10807-10816. 2020.
[48] Sohl-Dickstein, J., E. A. Weiss, N. Maheswaranathan, et al. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, vol. 37, pages 2256-2265. 2015.
[49] Ho, J., A. Jain, P. Abbeel. Denoising diffusion probabilistic models. In NeurIPS. 2020.
[50] Austin, J., D. D. Johnson, J. Ho, et al. Structured denoising diffusion models in discrete state-spaces. In NeurIPS, pages 17981-17993. 2021.
[51] Ho, J., T. Salimans, A. A. Gritsenko, et al. Video diffusion models. In NeurIPS. 2022.
[52] Gu, S., D. Chen, J. Bao, et al. Vector quantized diffusion model for text-to-image synthesis. In CVPR, pages 10686-10696. 2022.
[53] Hao, S., K. Han, S. Zhao, et al. ViCo: Detail-preserving visual condition for personalized text-to-image generation. arXiv preprint arXiv:2306.00971, 2023.
[54] Xue, Z., G. Song, Q. Guo, et al. Raphael: Text-to-image generation via large mixture of diffusion paths. arXiv preprint arXiv:2305.18295, 2023.