The Photographer Eye: Teaching Multimodal Large Language Models to See
  and Critique like Photographers
Authors: name: Daiqing Qi, name: Handong Zhao, name: Jing Shi, name: Simon Jenni, name: Yifei Fan, name: Franck Dernoncourt, name: Scott Cohen, name: Sheng Li

While editing directly from life, photographers have found it too difficult
to see simultaneously both the blue and the sky. Photographer and curator,
Szarkowski insightfully revealed one of the notable gaps between general and
aesthetic visual understanding: while the former focuses on identifying the
factual element in an image (sky), the latter transcends such object
identification, viewing it instead as an aesthetic component--a pure color
block (blue). Such fundamental distinctions between general (detection,
localization, etc.) and aesthetic (color, lighting, composition, etc.) visual
understanding present a significant challenge for Multimodal Large Language
Models (MLLMs). Although some recent works have made initial explorations, they
are often limited to general and basic aesthetic commonsense. As a result, they
frequently fall short in real-world scenarios (Fig. 1), which require extensive
expertise--including photographic techniques, photo pre/post-processing
knowledge, and more, to provide a detailed analysis and description. To
fundamentally enhance the aesthetics understanding of MLLMs, we first introduce
a novel dataset, PhotoCritique, derived from extensive discussions among
professional photographers and enthusiasts, and characterized by the large
scale, expertise, and diversity. Then, to better learn visual aesthetics from
PhotoCritique, we furthur propose a novel model, PhotoEye, featuring a
languageguided multi-view vision fusion mechanism to understand image
aesthetics from multiple perspectives. Finally, we present a novel benchmark,
PhotoBench, a comprehensive and professional benchmark for aesthetic visual
understanding. On existing benchmarks and PhotoBench, our model demonstrates
clear advantages over existing models.