ICML 2025

Attributes Shape the Embedding Space of Face Recognition Models
Pierrick Leroy* 1 Antonio Mastropietro* 2 Marco Nurisso* 1 3 Francesco Vaccarino 1
Abstract
Face Recognition (FR) tasks have made signif-
icant progress with the advent of Deep Neu-
ral Networks, particularly through margin-based
triplet losses that embed facial images into high-
dimensional feature spaces. During training, these
contrastive losses focus exclusively on identity
information as labels. However, we observe a
multiscale geometric structure emerging in the
embedding space, influenced by interpretable fa-
cial (e.g., hair color) and image attributes (e.g.,
contrast). We propose a geometric approach to
describe the dependence or invariance of FR mod-
els to these attributes and introduce a physics-
inspired alignment metric. We evaluate the pro-
posed metric on controlled, simplified models and
widely used FR models fine-tuned with synthetic
data for targeted attribute augmentation. Our find-
ings reveal that the models exhibit varying degrees
of invariance across different attributes, provid-
ing insight into their strengths and weaknesses
and enabling deeper interpretability. Code avail-
able here: https://github.com/mantonios107/attrs-
fr-embs.
1. Introduction
Deep Learning (DL) models have achieved state-of-the-art
performance for Face Recognition (FR) tasks, due to in-
creased availability of curated datasets (Deng et al., 2019b)
and improved recognition architectures and losses (Deng
et al., 2022). In the challenging open-set scenario, novel
face identities can appear at testing time, hence the problem
is framed as a metric learning task (Liu et al., 2017). There
are three main innovation to reach outstanding FR results in
this scenario. The first consists of learning an embedding
*Equal contribution
1Department of Mathematical Sciences,
Politecnico di Torino, Turin, Italy 2Department of Computer
Science, University of Pisa, Pisa, Italy
3CENTAI Institute,
Turin, Italy. Correspondence to:...