SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 9: Statistics of JourneyDB. We provide 4 million generated image-prompt pairs, 1 million captions and over 8 million VQA annotations.
Dataset	Labeled Image	Labeled Prompt	Style QA	Content QA
VQ-Diffusion [6]	1,888	1,869	3,067	3,346
VQGAN + CLIP [57]	1,888	1,869	3,067	3,346
GLIDE [58]	1,898	1,878	3,101	3,349
CogView2 [59]	1,914	1,896	3,135	3,387
Latent Diffusion [14]	1,942	1,942	3,159	3,438
Versatile Diffusion [60]	1,953	1,935	3,179	3,427
Stable Diffusion v1.4 [14]	2,028	2,010	3,301	3,621
DeepFloyd-XL [61]	2,052	2,028	3,334	3,655
Epic Diffusion [62]	2,066	2,047	3,366	3,685
DALL-E mini [63]	2,097	2,075	3,415	3,739
Dreamlike Photoreal 2.0 [64]	2,100	2,080	3,393	3,737
Stable Diffusion v2.0 [14]	2,104	2,084	3,405	3,756
Deliberate [65]	2,122	2,101	3,447	3,781
LAFITE [66]	2,124	2,105	3,439	3,804
Realistic Vision [67]	2,144	2,119	3,475	3,832
FuseDream [68]	2,176	2,157	3,536	3,893
SDXL Refiner 0.9 [8]	2,184	2,161	3,540	3,915
MajicMix Realistic [69]	2,189	2,167	3,568	3,910
ChilloutMix [70]	2,207	2,185	3,571	3,923
DALL-E 2 [7]	2,220	2,197	3,593	3,967
Openjourney [71]	2,237	2,214	3,630	3,992
SDXL Base 0.9 [8]	2,270	2,246	3,686	4,062
Total	45,803	45,365	74,407	81,565
Table 10: Evaluation results of Prompt Inversion and Image Captioning on the extension test set of JourneyDB.
Models	Prompt Inversion	Image Caption	VQA
BLEU-4	METEOR	ROUGE-L	CIDEr	BLEU-4	METEOR	ROUGE-L	CIDEr	Style	Content
BLIP-2 OPT [4]	3.46	8.06	24.82	51.81	3.99	9.00	26.25	55.64	-	-
BLIP-2 FlanT5 [4]	5.15	9.41	26.06	54.57	4.56	9.77	27.57	59.28	68.38%	66.57%
D Cross-model Test Set
As listed in Table 9, we additionally introduce another 22 text-to-image generative models into JourneyDB, such as VQ-Diffusion [6] DALL-E 2 [7], StableDiffusion-XL [8], etc., which significantly improves the diversity of JourneyDB, making it a comprehensive benchmark for evaluating the comprehension of generated images. For each generative model, we originally generated 3,200 images, and a group of 60 annotators helped clean up the pairs without consistency to obtain the final cross-model test set containing 45,803 images in total.
We evaluate the BLIP models on the new dataset on the image caption and prompt inversion tasks. Results are shown in Table 10. This additional text-image dataset, with manually cleaned text prompt, image captions, and VQA annotations, serves as a divergent and comprehensive benchmark for the evaluation of the visual understanding model for generated images.