SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Baselines and Metrics. We use two automated variants from §3.1: Cap3D and Cap3D (QA), which uses a two-stage prompt captioning to ask more about the input 3D geometry; and compare to crowdsourced human descriptions, Human, detailed in §4.1, and ABO metadata, Meta.
Our primary metric of comparison is similar human A/B testing to §5.1, since automated metrics such as CLIPScore do not accurately represent the distance between fine-grained captions and images as shown in [78].
Results. In stark contrast to Objaverse, Human captions beat automated (Cap3D) in Table 4. Automated captions alone contain little geometric detail (e.g., Figure 4), making Cap3D unsuited for this setting. However, by using the two-stage prompt engineering, Cap3D (QA) is preferred to Human. Shown in Figure 4, Cap3D (QA) produces significant fine-grained geometric detail as well as longer captions in general. In contrast, Metadata is clearly the weakest baseline.
Table 4: ABO Fine-Grained Geometry Cap-tions. Cap3D (QA) performs best; crowd-sourced beats captioning alone.
Method	A/B Score (1-5)	A/B Win %	A/B Lose %
Human v. Cap3D	3.09±0.02	47.3±1%	41.4±1%
Cap3D(QA) v. Human	3.08±0.02	50.2±1%	44.0±1%
Cap3D(QA) v. Cap3D	3.27±0.02	56.0±1%	37.4±1%
Cap3D(QA) v. Meta	4.27±0.02	88.2±1%	10.0±1%
Table 5: Text-to-3D: Human Captions. Cap3D captions are better than human on the 30k set. Finetuning on Cap3D full set performs best.
Finetune Dataset	FID↓	CLIP Score	CLIP R@1	R-Precision (2k)
R@5	R@10
Point-E	Pretrained	36.1	72.4	6.0	16.2	22.4
30k (Human)	34.6	74.4	8.2	21.3	29.1
30k (Cap3D)	33.7	75.0	10.4	24.3	32.1
350k (Cap3D)	32.8	75.6	12.4	28.1	36.9
Shap-E	Pretrained	37.2	80.4	20.3	39.7	48.7
30k (Human)	36.0	79.6	18.6	36.3	45.3
30k (Cap3D)	37.2	79.4	19.1	37.5	46.1
350k (Cap3D)	35.5	79.1	20.0	38.8	47.3
5.3 Large-Scale Text-to-3D Generation
Dataset. We evaluate text-to-3D generation on three subsets of Objaverse: (1) a 30k split of objects containing human-authored captions, to measure if finetuning on Cap3D captions outperform human-authored ones; (2) a 350k split of Objaverse objects paired with Cap3D captions, for finetuning state-of-the-art text-to-3D methods – obtaining high-density point cloud and latent codes to finetune Point-E and Shap-E for all 785k objects is prohibitively expensive (20k GPU days); and (3) a 300 object split for optimization-based baselines, which typically take  $>30$  mins per object to optimize. Pretrained and Finetuned models are evaluated on 8 views across a held-out test set of 2k objects.
Methods. We consider several recent SOTA methods in three general categories: text-to-3D diffusion, cascaded text-to-image then image-to-3D diffusion, and optimization-based. We use the direct text-to-3D variant of Point-E [89], as well as two variants of Shap-E [90]: STF [102] and NeRF [103]. We use Stable Diffusion cascaded with Point-E (Im-to-3D), adapting ControlNet [64] and LoRA [104] for Stable Diffusion finetuning. We use optimization-based baselines DreamField [85], the publicly available implementation of DreamFusion [3], Stable DreamFusion [105]; and 3DFuse [106], using their implementation based on Karlo [24, 107].
Metrics. We use standard metrics from prior work [3, 85, 89, 90] to evaluate. Primarily, these are CLIP Score and CLIP R-Precision. CLIP R-Precision ranks a rendered image against all text pairs in the test set by CLIP cosine similarity, and computes precision upon true text-image correspondence. Since we have ground truth images, we calculate the FID [108] of 3D rendered images against ground truth images, as well as assess CLIP Score on these reference images. We also use ViLT Retrieval R-Precision, used in 5.1, which has the same evaluation procedure as CLIP R-Precision with a different model.
Results. Table 5 lists the results of finetuning using human-authored and Cap3D captions. Point-E improves after finetuning upon human captions. However, performance is further improved using our captions on the same dataset; and improved most by training upon the full dataset. This result strongly defends Cap3D captioning at scale. Shap-E does not improve on CLIP metrics after finetuning in any dataset, but performs the least bad on the full dataset using our captions; and FID improves most.
Table 6 presents results from several state-of-the-art pretrained and finetuned models using Cap3D-generated captions. The models finetuned on our captions generally outperform pretrained models under the FID metric. For CLIP-related metrics, the finetuned models of Point-E (Text-to-3D) and StableDiffusion + Point-E (Im-to-3D) also beat their pretrained counterparts. Point-E and Stable Diffusion have been trained on massive datasets, so improvement from finetuning is strong evidence Cap3D captions are effective. The observed downturns in Shap-E could be attributed to at least two factors. First, our replication of their privately-available train code is unstable, often resulting in NaN loss during finetuning. We restart from earlier checkpoints upon crashing, but the result alone is concerning. Second, we exclusively finetune the diffusion model in Shap-E's two-stage approach.
Qualitative results in Figure 6 validate quantitative findings. Point-E and Stable Diffusion baselines show large improvements from finetuning, while Shap-E can better fit the Objaverse data distribution (corresponding to improved FID).
Table 6: Text-to-3D on Objaverse. Finetuning improves FID over pretrained performance across models. CLIP metrics of Stable Diffusion increase; CLIP metrics of Point-E increase significantly.
FID↓	Pretrained	FID↓	Finetuned on Cap3D
CLIP Score	CLIP R@1	R-Precision (2k) R@5	R@10	CLIP Score	CLIP R@1	R-Precision (2k) R@5	R@10
Ground Truth Images	-	81.6	32.7	55.1	64.3	-	81.6	32.7	55.1	64.3
Point-E (Text-to-3D) [89]	36.1	72.4	6.0	16.2	22.4	32.8	75.6	12.4	28.1	36.9
S. Diff. [22] (CNet) [64]+ [89](Im-to-3D)	54.7	73.6	11.0	23.4	30.0	53.3	74.6	12.4	26.2	33.8
S. Diff. [22] (LoRA) [104]+ [89](Im-to-3D)	54.7	73.6	11.0	23.4	30.0	53.7	74.4	11.6	24.6	31.4
Shap-E [90] (STF) [102]	37.2	80.4	20.3	39.7	48.7	35.5	79.1	20.0	38.8	47.3
Shap-E [90] (NeRF) [103]	48.7	79.4	19.0	37.7	46.8	48.2	78.1	18.3	35.1	43.5
Figure 6: Text-to-3D results. Finetuning on Cap3D captions can significantly improve results. Additional examples are available in Appendix C.
Optimization baselines, shown in Table 7, perform very well upon CLIP-based metrics, consistent with prior work [90]. In fact, DreamField outperforms ground truth images in CLIP metrics. This demonstrates DreamField overfits to the CLIP metric, which is the standard protocol for text-to-3D evaluation. We propose to also consider ViLT precision (see §5.1). This helps mitigate the bias of CLIP, though DreamField performance on this metric is still strong.
Table 7: Text-to-3D: Optimization Baselines. Overfitting via CLIP leads to higher CLIP-based scores than ground truth; ViLT score is more fair.
FID↓	Score	CLIP	ViLT
R@1	R@5	R@1	R@5
True Images	-	83.2	53.2	77.8	41.3	69.0
D. Field [85]	106.1	83.7	61.8	83.6	32.3	56.0
D. Fusion [3]	127.8	72.4	28.4	46.1	23.7	45.3
3DFuse [106]	91.1	77.0	38.6	58.5	26.3	53.0
6 Limitations and Future Works
As described in §3, Cap3D consists of four steps: (1) 3D objects rendering; (2) captioning via BLIP2; (3) filtering captions via CLIP; (4) consolidate multiview information via GPT4. To effectively capture comprehensive information through 2D renderings, cameras are strategically placed above or below objects. However, this occasionally results in unconventional 2D views, making BLIP2 susceptible to errors that CLIP fails to rectify. This, in turn, hampers GPT4's ability to merge variegated information across views, culminating in vague and verbose descriptions, as illustrated in Figure 7. The system also falters with certain complex indoor 3D scans, as depicted in Figure 8, thus requiring more robust image_captioning models [30] and potentially benefiting from additional view incorporations beyond the current eight.
Our method's provision of extensive 3D-Caption pairs for Objaverse [9] could foster the advancement of 3D-LLM models [109, 110], facilitating 3D-caption centric tasks like captioning, dialog, and language-based navigation. Additionally, the geometric descriptions generated for ABO [35] enable
compositional structure analysis of fine-grained 3D objects [111, 112]. Our developed method assists in scaling up of 3D-text pairs for expansive 3D datasets [113].
Figure 7: An failed case. The caption under each rendered image are generated by BLIP2 + filtered by CLIP. The inaccurate content are highlighted with colors. GPT4 + CLIP cannot fix the error generated by BLIP2 and result in a fuzzy description.
Figure 8: An failed case. The caption under each rendered image are generated by BLIP2 + filtered by CLIP. The inaccurate content are highlighted with colors. The various views contain inaccurate information. The associated details, roughly described, fail to accurately depict the indoor scene.
In this work, we collect (1) 3D object captions at scale, creating the largest publicly available high-quality 3D-text by an order of magnitude. To do so we propose Cap3D, an automated pipeline leveraging several models pretrained on large datasets, and show design choices are important to performance. In addition, we collect (2) a dataset of geometric captions upon fine-grained 3D objects. This helps analyze shortcomings of automated captioning and study the potential of question answering, while yielding geometric descriptions for 3D assets of real objects paired with real images. These datasets serve as benchmarks for text-to-3D tasks (1) at scale and (2) in geometric detail.
This work is supported by two grants from LG AI Research and Grant #1453651 from NSF. We greatly thank Kaiyi Li for his technical support. We thank Mohamed EI Banani, Karan Desai, and Ang Cao for their helpful discussions. Thanks Matt Deitke for helping with Objverse-related questions. Thanks to Haochen Wang for helping notice some incorrect rendering.
[1] Heewoo Jun and Alex Nichol. Shap-e: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023.
[2] Anchit Gupta, Wenhan Xiong, Yixin Nie, Ian Jones, and Barlas Oğuz. 3dgen: Triplane latent diffusion for textured mesh generation. arXiv, 2023.
[3] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv, 2022.
[4] Yuk Ming Tang and Ho Lun Ho. 3d modeling and computer graphics in virtual reality. In *Mixed Reality and Three-Dimensional Computer Graphics*. IntechOpen, 2020.
[5] Rick Parent. Computer animation: algorithms and techniques. Newnes, 2012.
[6] Afsoon Afzal, Deborah S Katz, Claire Le Goues, and Christopher S Timperley. A study on the challenges of using robotics simulators for testing. arXiv preprint arXiv:2004.07368, 2020.
[7] Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on Robot Learning, pages 80–93. PMLR, 2023.
[8] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In Conference on robot learning, pages 1-16. PMLR, 2017.
[9] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objverse: A universe of annotated 3d objects. 2023.
[10] Karan Desai, Gaurav Kaul, Zubin Aysola, and Justin Johnson. Redcaps: Web-curated image-text data created by the people, for the people. NeurIPS, 2021.
[11] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACL, 2018.
[12] Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3558-3568, 2021.
[13] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv, 2021.
[14] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. 2022.
[15] Mohamed El Banani, Karan Desai, and Justin Johnson. Learning Visual Representations via Language-Guided Sampling. In CVPR, 2023.
[16] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.
[17] Norman Mu, Alexander Kirillov, David Wagner, and Saining Xie. Slip: Self-supervision meets language-image pre-training. In ECCV, 2022.
[106] Junyoung Seo, Wooseok Jang, Min-Seop Kwak, Jaeoon Ko, Hyeonsu Kim, Junho Kim, Jin-Hwa Kim, Jiyoung Lee, and Seungryong Kim. Let 2d diffusion model know 3d-consistency for robust text-to-3d generation. arXiv, 2023.
[107] Donghoon Lee, Jiseob Kim, Jisu Choi, Jongmin Kim, Minwoo Byeon, Woonhyuk Baek, and Saehoon Kim. Karlo-v1.0.alpha on coyo-100m and cc15m. https://github.com/kakaobrain/karlo, 2022.
[108] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NeurIPS, 2017.
[109] Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds. arXiv preprint arXiv:2308.16911, 2023.
[110] Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. arXiv preprint arXiv:2307.12981, 2023.
[111] Tiange Luo, Kaichun Mo, Zhiao Huang, Jiarui Xu, Siyu Hu, Liwei Wang, and Hao Su. Learning to group: A bottom-up framework for 3d part discovery in unseen categories. arXiv preprint arXiv:2002.06478, 2020.
[112] Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 909–918, 2019.
[113] Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objverse-xl: A universe of  $10\mathrm{m} + 3\mathrm{d}$  objects. arXiv preprint arXiv:2307.05663, 2023.
Appendix A Price Breakdown Details
This section provides our details computation for Table 1. Using a single A40 GPU, BLIP2 runs at \(\sim\) 2700 iterations per hour, enabling it to process around \(\sim\) 337.5 objects hourly given the eight-run requirement for generating captions for 8 rendering views. This translates to about 2.96 hours to process 1k objects, costing \(2.96 \times \\(1.28 = \\)3.79\) with the rate \(\$1.28/hr\) on the cloud platform, CoreWeave. On the same A40 GPU, CLIP operates at \(\sim\) 27000 iterations per hour, incurring a cost of \\(0.38. Importantly, utilizing eight A40s costs the same as using one, due to the parallel processing capacity across multiple GPUs for multiple rendering views.
We compute our GPT4 cost by averaging input token numbers, as OpenAI GPT4 API (8k context) costs \(0.03 / 1k\) tokens. Our input prompt is: "Given a set of descriptions about the same 3D object, distill these descriptions into one concise caption. The descriptions are as follows: 'captions'. Avoid describing background, surface, and posture. The caption should be:", which consists of (1) text prompt and (2) captions generated by BLIP2 or BLIP2 + CLIP. Without CLIP's filtering, our input prompt contains 40 captions which have \(\sim\) 511.1 tokens on average, cost \(511.1 / 1000 \times 0.03 \times 1000 = \\(15.33\) for \(1k\) objects. With CLIP, our input prompt contains 8 captions which have \(\sim\) 139.3 tokens on average, cost \(139.3 / 1000 \times 0.03 \times 1000 = \\)4.18$ for \(1k\) objects.
The average cost per  $1k$  objects for human-authored annotation is computed as the average expenditure on the crowdsourcing platform, Hive. The human annotation speed is computed by averaging the annotation progress across our whole annotation process.
We do not report the average cost of Cap3D (QA) in the main paper, as we only use it on ABO. For completeness, we report it here. The one distinction is BLIP2 is run twice instead of once for the two-stage question answering (QA). The cost of BLIP2 thus doubles, from  $3.79 to$ 7.58; and total cost increases from $8.35 to $12.14 per 1k objects.
Appendix B Additional 3D Captioning Results
Figure 9: Random 3D captioning examples generated by Cap3D. Two views of 3D objects (Objaverse [9]) are shown here, Cap3D uses eight.
Figure 10: Random 3D captioning examples generated by Cap3D. Two views of 3D objects (Objaverse [9]) are shown here, Cap3D uses eight.
Figure 11: Random 3D captioning examples generated by Cap3D. Two views of 3D objects (Objaverse [9]) are shown here, Cap3D uses eight.
Figure 12: Random 3D captioning examples generated by Cap3D. Two views of 3D objects (Objaverse [9]) are shown here, Cap3D uses eight.
Figure 13: Random 3D captioning examples generated by Cap3D. Two views of 3D objects (Objaverse [9]) are shown here, Cap3D uses eight.
A 3D model of a colorful, multi-colored boat with a flag, oars, and paddles.
A 3D model of a wooden door with a design, featuring a wooden sign, curved wall, and clock.
A 3D model featuring a bird, twigs, fishing poles, a dragonfly, and skis with poles.
A 3D blue box with a hole and sand on it.
A purple and yellow toy lantern with a handle.
3D model of a cricket bat, royalty-free vector illustration.
A 3D rendering of a row of yellow poles.
White 3D spiral staircase model
Figure 14: Random 3D captioning examples generated by Cap3D. Two views of 3D objects (Objaverse [9]) are shown here, Cap3D uses eight.
3D model of a small white and blue toy robot teddy bear.
3D model of a white axe
Wooden coat rack in a 3D model.
A 3D model of a green plant with leaves and an eye.
A 3D model of an armored hand wearing gloves with a leather strap and metal cuff.
3D model of a white cloud.
White plastic ring
3D model of a small white building with stairs, featuring a cube and ceiling light fixture.
3D model of a wooden power pole with wires.
3D model of a white plastic bottle with a lid.
A 3D model of a small boat and house on a yellow platform.
A knife and sliced bread on a wooden cutting board.
A 3D model of a two-story house with a roof structure.
Figure 15: Random 3D captioning examples generated by Cap3D. Two views of 3D objects (Objaverse [9]) are shown here, Cap3D uses eight.
A 3D rendering of a fish next to a Rubik's cube.
A 3D model of a house on a grassy field with a road in front.
3D model of a flower in a black box with a purple container and blue corner shelf.
Figure 16: Comparative Analysis: Cap3D Generated Caption vs Human-Annotated Caption vs Objverse Metadata [9]. Two views of 3D objects are shown here, Cap3D and human use eight.
Figure 17: Comparative Analysis: Cap3D Generated Caption vs Human-Annotated Caption vs Objverse Metadata [9]. Two views of 3D objects are shown here, Cap3D and human use eight.
Appendix C Additional Text-to-3D Results
In this section, we provide several text-to-3D results for all of our compared methods. We include Shap-E and Point-E pretrained models and the models finetuned on our data, as well as optimization baselines, including DreamFusion, DreamField, and 3D Fuse.
Figure 18: Text-to-3D results. The top text prompt and "Reference" are from our test set. We fine-tune the left 5-column methods on Cap3D-generated captions. The detailed setting and methods are described in §5.3.
Figure 19: Text-to-3D results. The top text prompt and "Reference" are from our test set. We fine-tune the left 5-column methods on Cap3D-generated captions. The detailed setting and methods are described in §5.3.
Figure 20: Text-to-3D results. The top text prompt and "Reference" are from our test set. We fine-tune the left 5-column methods on Cap3D-generated captions. The detailed setting and methods are described in §5.3.
Figure 21: Text-to-3D results. The top text prompt and "Reference" are from our test set. We fine-tune the left 5-column methods on Cap3D-generated captions. The detailed setting and methods are described in §5.3.
Figure 22: Text-to-3D results. The top text prompt and "Reference" are from our test set. We fine-tune the left 5-column methods on Cap3D-generated captions. The detailed setting and methods are described in §5.3.
Figure 23: Text-to-3D results. The top text prompt and "Reference" are from our test set. We fine-tune the left 5-column methods on Cap3D-generated captions. The detailed setting and methods are described in §5.3.
Appendix D ABO Captioning: Automated Metrics
In §5.2, we report human A/B judgments on ABO. We do not report automated metrics, which are poor measures of performance for at least two reasons. First, ABO contains a large number of objects that are very similar, meaning it would be challenging for captions to distinguish their differences. Thus, retrieval metrics such as ViLT Image or Text Retrieval will show very poor scores across metrics. Second, we show automated captioning performs poorly at describing geometry well, meaning it is likely automated image-caption alignment will not align based on geometry well. For completeness, we report automated metrics in Table 8. As expected, all retrieval scores are very low. Automated captioning scores best across automated metrics, however we caution against drawing conclusions from this result. Human studies in Table 4 suggest the opposite, and qualitative results agree with this finding, e.g. Figure 4.
Table 8: ABO Automated Caption Evaluations. Automated captions are a poor measure of performance on ABO as (1) many objects are similar, making retrieval difficult; (2) automated captioning does not describe geometry well, so we should not expect automated image-caption alignment to describe geometrically correct captions well.
Method	CLIP Score	ViLT Img Retr.	ViLT Text Retr.
R@5	R@10	R@5	R@10
Meta	61.9	0.8	1.7	0.8	1.7
Human	75.2	2.6	4.4	2.3	4.2
Cap3D	89.9	4.2	7.2	3.2	5.6
Cap3D(QA)	82.7	2.9	5.3	2.4	4.3
In contrast with A/B tests, which take place on the full 6.4k objects of ABO, this table is computed on a random 5k object subset of ABO to follow standard retrieval benchmarks (performance drops considerably as dataset size increases. Using 5k instead of the full 6.4k makes it much easier to contextualize retrieval numbers). A/B performance on this 5k subset is very close to the full 6.4k dataset, meaning the sample is highly representative, and one can compare the results from this table in combination with Table 4 in the main paper.
Appendix E Cap3D, BLIP2, Human Caption Comparisons
A comparative analysis on the number of n-grams was conducted to shed light on the distinct vocabulary sizes among Cap3D, BLIP2, and human captions for Objaverse [9] objects. As elucidated in Table 9, Cap3D exhibits a considerably larger vocabulary as compared to BLIP2. Although human-generated captions encompass a higher number of phrases, the disparity is notably lesser than that observed between BLIP2 and Cap3D. Moreover, it is hypothesized that the slightly elevated dictionary size in human captions could be attributed to the inclusion of typographical errors, typically arising from crowdsourced platforms.
Table 9: N-gram Comparison among Cap3D, BLIP2, and Human Captions
Occ. in 5k captions
Unigrams	Bigrams	Trigrams
BLIP2	1,928	6,616	10,899
Cap3D	3,108	15,274	25,883
Human	3,762	17,818	27,316
Appendix F Additional Details
F.1 Prompt used in Cap3D