GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
Authors: name: Tong Wu; name: Guandao Yang; name: Zhibing Li; name: Kai Zhang; name: Ziwei Liu; name: Leonidas Guibas; name: Dahua Lin; name: Gordon Wetzstein
Published: 2024-01-08 18:52:09

Abstract: Despite recent advances in text-to-3D generative methods, there is a notable
absence of reliable evaluation metrics. Existing metrics usually focus on a
single criterion each, such as how well the asset aligned with the input text.
These metrics lack the flexibility to generalize to different evaluation
criteria and might not align well with human preferences. Conducting user
preference studies is an alternative that offers both adaptability and
human-aligned results. User studies, however, can be very expensive to scale.
This paper presents an automatic, versatile, and human-aligned evaluation
metric for text-to-3D generative models. To this end, we first develop a prompt
generator using GPT-4V to generate evaluating prompts, which serve as input to
compare text-to-3D models. We further design a method instructing GPT-4V to
compare two 3D assets according to user-defined criteria. Finally, we use these
pairwise comparison results to assign these models Elo ratings. Experimental
results suggest our metric strongly align with human preference across
different evaluation criteria.