Title: Smoothing Slot Attention Iterations and Recurrences
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Object-Centric Learning, Slot Attention, Object Discovery, Object Recognition, Dynamics Modeling

Abstract: Slot Attention (SA) and its variants lie at the heart of mainstream Object-Centric Learning (OCL).
Objects in an image can be aggregated into corresponding slot vectors, by \textit{iteratively} refining cold-start query vectors, typically three times, via SA on image features.
For video, this aggregation is \textit{recurrently} shared across frames, with queries cold-started on the first frame while transitioned from the previous frame's slots on non-first frames.
However, cold-start queries lack sample-specific cues thus hinder precise aggregation on the image or video's first frame;
Also, non-first frames' queries are already sample-specific thus require aggregation transforms different from the first frame.
We address these issues for the first time with our \textit{SmoothSA}:
(1) To smooth SA iterations on the image or video's first frame, we \textit{preheat} the cold-start queries with rich information of input features, via a tiny module self-distilled inside OCL;
(2) To smooth SA recurrences across all video frames, we \textit{differentiate} the homogeneous transforms on the first and non-first frames, by using full and single iterations respectively.
Comprehensive experiments on object discovery, recognition and downstream benchmarks validate our method's effectiveness.
Further analyses illuminate how our method smooths SA iterations and recurrences.
Our source code and training logs are provided in the supplement.

TL;DR: We address issues of query cold-start in slot attention iterations on the image or video's first frame and transform homogeneity in slot attention recurrences on the video frames, improving image and video OCL significantly.