ICML 2025

SCENIR: Visual Semantic Clarity through
Unsupervised Scene Graph Retrieval
Nikolaos Chaidos 1 Angeliki Dimitriou 1 Maria Lymperaiou 1 Giorgos Stamou 1
Abstract
Despite the dominance of convolutional and
transformer-based architectures in image-to-
image retrieval, these models are prone to biases
arising from low-level visual features, such as
color. Recognizing the lack of semantic under-
standing as a key limitation, we propose a novel
scene graph-based retrieval framework that em-
phasizes semantic content over superficial im-
age characteristics. Prior approaches to scene
graph retrieval predominantly rely on supervised
Graph Neural Networks (GNNs), which require
ground truth graph pairs driven from image cap-
tions.
However, the inconsistency of caption-
based supervision stemming from variable text en-
codings undermine retrieval reliability. To address
these, we present SCENIR, a Graph Autoencoder-
based unsupervised retrieval framework, which
eliminates the dependence on labeled training
data. Our model demonstrates superior perfor-
mance across metrics and runtime efficiency, out-
performing existing vision-based, multimodal,
and supervised GNN approaches.
We further
advocate for Graph Edit Distance (GED) as a
deterministic and robust ground truth measure
for scene graph similarity, replacing the inconsis-
tent caption-based alternatives for the first time
in image-to-image retrieval evaluation. Finally,
we validate the generalizability of our method
by applying it to unannotated datasets via auto-
mated scene graph generation, while substantially
contributing in advancing state-of-the-art in coun-
terfactual image retrieval. The source code is
available at https://github.com/nickhaidos/scenir-
icml2025.
1Artificial Intelligence and Learning Systems Laboratory, Na-
tional Technical University of Athens. Correspondence to: Niko-
laos Chaidos <nchaidos@ails.ece.ntua.gr>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada....