Understanding videos such as TV series and movies requires analyzing who the characters are and what they are doing. We address the challenging problem of clustering face tracks based on their identity. Different from previous work in this area, we choose to operate in a realistic and difficult setting where: (i) the number of characters is not known a priori; and (ii) face tracks belonging to minor or background characters are not discarded. To this end, we propose Ball Cluster Learning (BCL), a supervised approach to carve the embedding space into balls of equal size, one for each cluster. The learned ball radius is easily translated to a stopping criterion for iterative merging algorithms. This gives BCL the ability to estimate the number of clusters as well as their assignment, achieving promising results on commonly used datasets. We also present a thorough discussion of how existing metric learning literature can be adapted for this task. 1 We consider three types of characters based on their roles: primary or recurring characters have major roles in several episodes; secondary or minor characters are named and play an important role in some episodes; and background or unknown (Unk) characters are unnamed and uncredited.
Among recent works, REF aim to estimate the number of clusters and their assignment and learn an embedding space that creates a fixed-radius balls for each character.
Video Face Clustering With Unknown Number of Clusters
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Computer Science