Recurrent Complex-Weighted Autoencoders for Unsupervised Object Discovery
Authors: Anand Gopalakrishnan
Aleksandar Stanić
Jürgen Schmidhuber
Michael Curtis Mozer
Venue: NeurIPS 2024 poster (2024)
Keywords: complex-weights
recurrence
autoencoders
objects
binding
synchrony
temporal correlation hypothesis
Primary area: neuroscience_and_cognitive_science

Abstract: Current state-of-the-art synchrony-based models encode object bindings with complex-valued activations and compute with real-valued weights in feedforward architectures. We argue for the computational advantages of a recurrent architecture with complex-valued weights. We propose a fully convolutional autoencoder, SynCx, that performs iterative constraint satisfaction: at each iteration, a hidden layer bottleneck encodes statistically regular configurations of features in particular phase relationships; over iterations, local constraints propagate and the model converges to a globally consistent configuration of phase assignments. Binding is achieved simply by the matrix-vector product operation between complex-valued weights and activations, without the need for additional mechanisms that have been incorporated into current synchrony-based models. SynCx outperforms or is strongly competitive with current models for unsupervised object discovery. SynCx also avoids certain systematic grouping errors of current models, such as the inability to separate similarly colored objects without additional supervision.

Recurrent Somplex-Weightee Autoencoders for Unsupervised Object Discovery
Anand Gopalakrishnan' Aleksandar Stanicaf Jürgen Schmidhuber Michael Curtis Mozer
The Swiss AI Lab, IDSIA, USI & SUPSI, Lugano, Switzerland Google DeepMind
Al Initiative, KAUST, Thuwal, Saudi Arabia
Abstract
Current state-of-the-art synchrony-based models encode object bindings with complex-valued activations and compute with real-valued weights in feedforward architectures. We argue for the computational advantages of recurrent architec- ture with complex-valle weights. We propose a fully convolutional autoencoder, SynCx, that performs iterative constraint satisfaction: at each iteration, hidden layer bottleneck encodes statistically regular configurations of features in particular phase relationships; over iterations, local constraints propagate and the model converges to a globally consistent configuration of phase assignments. Binding is achieved simply by the matrix-vector product operation between complex-valued weights and activations, without the need for additional mechanisms that have been incorporated into current synchrony-based models. SynCx outperforms is strongly competitive with current models for unsupervised object discovery. SynCx also avoids certain systematic grouping errors of current models, such as the inability to separate similarly colored objects without additional supervision.
Introduction
When shown arrays of simple visual elements, people have natural tendency to perceive the arrays in disjoint groups based on their color, shape, spatial configuration, etc. Grouping behavior was first studied and systematized by Gestalt psychologists [T33] who proposed a set of principles according to which perception operates, e.g., grouping by proximity, similarity, closure, good continuation and common fate. Although these Gestalt principles-also called laws-were assumed to be innate, statistics of the environment may explain some forms of grouping 44, and people rapidly learn new grouping principles when they exposed to novel environments [5]. Grouping abilities acquired early in an infant's ' developmental cycle 6 and serve as building block in the human ability to form concepts 77. build abstractions and categorize
g and reason about the physical world 9
Perceptual grouping has been modeled in deep learning under the guise of object-centric learning. (See Greff et al. [10] for an overview.) Object-centric models discover modular and compositional representations that can facilitate stronger generalization and relational reasoning capabilities on downstream visual tasks such question answering (1., game playing [2H14 and robotics (15177..
determining how to integrate visual information in a flexible, dynamic manner to form unified wholes. The predominant design strategy for implementing binding in deep learning systems to achieve perceptual
Both human and AI object-centric learning aim solve the binding problem [8 19
"Correspondence anand@idsia. ch Work done at IDSIA. Offficial code
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

grouping is to maintain a set of latent activation vectors (slots), each of which encodes features of just one object. Slot-based models differ from one another in the procedures used to partition information from the inputs to each slot [10] Far less investigation has gone into an alternative paradigm for perceptual grouping based on synchrony. In this paradigm, bindings between features are expressed in terms of the relative phases of complex-vall -aauue neural activities. The earliest demonstration of a synchrony-basee model in neural networks focused on a supervised setting with teacher-specified target phases for hard-coded one-hot features of contour types [20], although consideration was given to an unsupervised extension obtained by phase clustering [21]. Recent synchrony-basee models (22 (5] using complex- complex-valued valued activations to learn suitable features have made progress in unsupervised grouping performance on synthetic and more naturalistic scenes.
However, current state-of-the th--the--rt unsupervised surcrrony-basee models such as CAE ((23). CICAE (244), and RF ((25) have a number of limitations. They employ real-valued weights to process complex-valued activations by weight sharing across the real and imaginary components. As result, the models do not exploit the constructive or destructive interference of complex-valuce activations that occur naturally via multiplication and addition operations (cf., (26)) And the models require use
of additional inductive biases such as gating mechanisms (x-binding 122 23 25 24) to implement binding. Further, it remains unclear how x-binding mechanism works in more general scenarios where the complex-valee valued activations are not exactly out-of-phase or norms of weights and features do not satisfy certain conditions as noted by Löwe al. [27]. Cosine binding [27] was proposed as a more computationally motivated and interpretable alternative that easily handles such scenarios. However, this model continues to use real-valued weights to process complex- valued activations. Therefore, it uses cosine distance between inputs and intermediate outputs to implement binding which has large memory overhead.
These state-of-the-art saschrony-basedd models [25] 24 are not as robust as one might hope. As we show later, these models tend to exploit color as a shortcut feature, a helpful heuristic when different objects have different colors but a harmful one when color is an unreliable shortcut feature. And on more naturalistic datasets, RF largely learns semantic-level groups (see, e.g., Figures 2 & 20 in (25) rather than instance-level groups which is the central focus of object-centric learning. These ofte-offthhe-at models require either additional contrastive loss (see, e.g., Figure 8 in (24)) or pre-trained features from supevvupe vision transformers or 'depth masks' even on synthetic datasets (see, e.g., Figure 6 in (25) to partially resolve such grouping errors.
Many of the complexities introduced in recent models seem to be moving the field away from the core intuition that initially motivated a synchrony-based approach to binding [20]. We step back and describe the original conception of synchrony- -based grouping mechanisms from Mozer et al. 20].
Consider the shapes composed of horizontal and vertical bars in Figure the letters T and H and a pair of overlapping squares with occlusion. Perceptual grouping involves determining which of the bars belong together. The pairs highlighted in green are part of the same object; the pairs highlighted in red are parts of different objects. A feedforward convolutional architecture is not well suited to this task because spatially local patches are ambiguous with regard to grouping. While a feedforward, fully connected architectures may work in principle, it is also problematic because statistical regularities in images are local and fairly low order (i.e., involve a subset of image features). However, if we drop the feedforward restriction, a convolutional model can iteratively converge solution: each iteration suggests soft constraints on how features should be grouped, and over iterations, constraints can propagate from one region in the image adjacent regions. This methodology was adopted in classical AI vision models [2X
Figure 1: Local feature config- urations are insufficient to de- termine whether features belong to the same object: highlighted horizontal and vertical configu- rations sometimes belong to the same object (green) and at oth- ers to different objects (red).
and in early synchrony-base based models (20] In the latter, phases
representing object binding-an initialized to random values and each iteration, the phases are updated to be consistent with constraints from neighboring patches. After multiple iterations the model converges on an interpretation that assigns each image location to a particular phase (object).
Complex weights are a natural choice to instantiate this function in our synchrony formulation outlined above. Just as hidden units in a standard architecture detect configurations of features,
2

---

hidden units in complex-valuce analogue detect configurations of features in a particular relative phase arrangement. A hidden unit that tuned to the top of T will expect the horizontal and vertical bar to have the same phase; a hidden unit that is tuned to occlusion (the red patches in Figure D will expect the horizontal and vertical bars to be out of phase. In this manner, the hidden units can encode alternative hypotheses concerning object groupings, and iterative processing will attempt to find global configurations that are locally consistent.
Drawing on these intuitions we design a model and training strategy for the fully unsupervised setting. Figure 2 shows our proposed model, Synchronous Complex Network (SynCx), which uses complex-valueds weights (detectors) to process complex-vai vvalue activations (features) in every layer of fully convolutional autoencoder. The matrix vector product between complex-valuee and activations ensures that weights process inputs not only based on their features (magnitudes + features) but also their phase relationships (phases - object bindings). The addition and multiplica- tion operators in complex algebra mechanistically implement the binding mechanism in our model removing the need for additional gating mechanisms (x [22 cosine binding
weights
27 or contrastive
training [24]. The spatial locality of constraints are ensured via 2D convolutions used at every layer in the autoencoder. The model is initialized with a random phase map (prior on object bindings) and learns to (de)synchronize groups of features based on their (dis (diiiirtyy iteratively. The autoencoder weights are shared across iterations and predicted output phases (top-down feedback) from the previous iteration are used as the imput phases at the next iteration while the pixel values the input magnitudes at each step. Iteratively updating the phases allows spatially local constraints to be slowly propagated across the spatial axes and in principle should lead to the system settling to a fixed point. The autoencoder is trained to simply reconstruct the input image at each iteration.
In contrast to the synchrony formulation of Mozer et al. 20] all recent state-of-the-an models use purely feedforward processing of inputs. Therefore, these models lack the ability to propagate information about local constraints to align phases which we show (T and HI junction examples) plays a key role in achieving binding via synchrony.
We show that binding in synchrony based models can be mechanistically implemented simply by matrix-vector products between complex-valle valued weights and activations iteratively process inputs. We do not need any additional mechanisms like x- x-binding [22 23 25] or cosine binding [27] or con- trastive training [24 as in prior work to achieve the same. Our conceptually simpler model (SynCx) designed from first principles outperforms (on Tetrominoes) or competitive with (on dSpri tes and CLEVR) state-of-thh-aar synchrony- -based models for unsupervised object discovery. RF groups objects using largely color cues requiring supervision via additional features ('depth masks') to segregate ob- jects with similar colors. Whereas SynCx more gracefully separates objects of the same color as it re-
Iteration
Iteration 2
Magnitude
Phase
Magnitude
2
(complex output)
tcomplex output)
Complex Conv. Decoder
Complex Conv. Decoder
Complex Conv Encoder
Complex Conv Encoder
(complex input) Phase
(complex input)
Magnitude
Magnitude
Phase
P
zr (phase init)
Figure 2: SynCx is a fully convolutional autoencoder that iteratively processes an input image. starts with randomly initialized phase Φ and the input image fh in the magnitude-componee updates the phases in a stateful manner, i.e., output phase at iteration fed as input at iteration (02 <- Φb.) and so on. The magnitude-component the input is always clamped to the input image Hz. SynCx is trained to reconstruct μ*z using the output
μ. at every step.
3

---

lies on features (color/shapettexturr) and the local spatial context removing the need for 'depth masks'. We visualize the phase maps across iterations to qualitatively inspect the binding process. Lastly, we discuss some remaining practical limitations of current state-of-the-art synchrony-based models.
2
Method
We describe the intuitions behind our model, its architecture and training procedure below.
Basic Intuition. Our autoencoder model is trained to reconstruct the input image using represen- tational bottleneck in the hidden layer(s). This requires the model learn an efficient coding scheme to compress the input image. Images are typically composed of objects which are modular (reusable) units of feature information (shape, color, texture etc.) For a network with complex-values activa- tions, one intuitive coding scheme is using the phase components to express relationships between lower-order features (edges/texturrss/happp features (objects/parts). As the image processed by the encoder it progressively compresses the input by storing and processing modular 'parts' together as higher-order features while the decoder learns the inverse mapping recover the input from this compressed format. It is then possible to characterize objects as the collection of features with similar phases across the entire spatial map.
in order to compress them into appropriate higher-order
Model. Our model (SynCx) is fully convolutional autoencoder architecture. It uses complex- valued weights to manipulate complex-valued activations at each layer. Let h, w, h', w/, c, diay doat and denote positive integers. Every layer of the network is parametric function fw w Ch 'ww' xda which maps complex-valus valued inputs to complex-valus valued outputs h using complex-valued weights W E CP. First, we compute the complex-valueee pre-activation response y:
y = [w(x), where fw denotes 2D convolution
Then, we apply the modeeL activation rule (29] element- only on the magnitude component of y (i.e., Myy to obtain the complex-
layer output h as:
h = modReLU(y) = ReLU u + b) O edoy with y My O e
(2)
where @ denotes Hadamard product and b E Rd is a learnable parameter. We use mo instead of ReLU because the nonlinearity is applied only to the magnitude component (strictly non- negative). Consequently, the standard ReLU activation rule would operate only in its linear region. The modeeeU activation rule [29] was introduced specifically for setting such as ours where the activation function is applied only to the magnitude components of complex-valued activations. The encoder module progressively downsamples the resolution of the complex-van vaaledd spatial feature map using strided 2D convolutions. The decoder module upsamples the resolution of the spatial feature map using the upsampling function independently on its magnitude and phase components.
Training and Loss Function. Given an image μe, E h xwxe of height h, width w, and c channels (3 for color images) with non-negative pixel values. We construct complex-valued input with magnitudes f4 and phases &b which are randomly initialized at every spatial location and channel of the image (see Figure 2}. For each vector-valued feature at given location, its phase component encodes the network's 's current hypothesis about its object binding: random initialization reflects lack of knowledge initially. The autoencoder processes the input image to update its hypotheses about object bindings at every iteration n E {I,..., N). The complex-valued input x" E Chxwxe to the autoencoder at the th iteration is:
(3) Notice in Equation (3) that the magnitude components are always clamped to the image (μ²z) while the phase components (Ø") are fed back. At the th iteration, x" is processed by the encoder and decoder layers, denoted as Net(x"), to compute complex-valued output 2 E Chxwxe The magnitude component of zz,, i.e. μ*, is the reconstruction of input image ft, The phase component of z², i.e. Φ", is used to initialize the complex-valued input x * at the next iteration:
x"
z" Net(x")
Φn+1 t
This way the phase maps act as the state of a recurrent function which processes a clamped input image at all steps as shown in Figure 2) After N such iterative updates to the phase maps by the

---

autoencoder, it is trained to minimize the average pixel-wise reconstruction loss across all iterations:
N
C
U - 111
(5)
12
We use the weight initialization scheme outlined by Trabelsi et al. [30] for the complex-valuee weights W at every layer of the encoder and decoder modules. This initialization scheme derives the variances of complex-valued weights to satisfy the initialization criterion of He et al. [31] The magnitude com- ponents of the complex-vale valued weights at every layer are sampled from a Rayleigh distribution with 1/fanm while the phase components are sampled from a uniform distribution between 0 to 2r.
Through ablation studies in the following section, we quantify the effects of key components such as random initialization of the phase map, presence of a bottleneck at latent layer(s) and the number of iterative updates have on the performance of our system.
Extracting Object Assignments. Here, we outline the processing steps used to extract discrete object assignments for every pixel given its continuous phase values. We start by taking the output complex-vatue
feature map h E Chxuxdaa from the penultimate decoder layer at the last iteration N. We use latent-level complex- valued features instead of output-level high-order features (texture/shape/color etc.) than simply color cues (RGB space) to determine their object constituency. Then, we construct complex-valued feature map with unit magnitudes (i.e., 14 < E gxxxxxday and phases the same as that of h N (i.e., & ). Background regions in this constructed feature map are masked out by setting their magnitudes to zero and then converted the Cartesian form element-wise. Finally, these features in Cartesian form are clustered using k-Means to compute the object assignments for every location in the spatial map. Using unit magnitudes for complex-valued activations ensures that object assignments are solely determined by their orientations. We note that it would be most natural to cluster the phases from the output layer each location to assign it to an object. However, we observed that the object separation shown in the output phases is weaker than that from the high-dimensional latent layer. Therefore, for all results presented here we choose to cluster the higher dimensional latent representations (from the penultimate decoder layer) by location. For more details on the object assignment process refer to Appendix A
to use
3 Results
First, we describe details of the datasets, our model/baselines, training and evaluation procedures. We visualize the phase maps and qualitatively characterize the grouping behavior shown by our model. We quantitatively compare the grouping performance on the unsupervised object discovery task of our model (SynCx) against state-of-the-art synchrony-based baselines (CAE, CAE++, CICAE, and RF). We also quantify the effects of key components of our model using ablation experiments. We conclude with some practical limitations shown by the binding mechanisms in these synchrony-basee models.
Datasets. We evaluate models on three datasets from the multi-object suite B2 namely Tetrominoes, dSprites and CLEVR used by prior work in object-centric learning 33 For CLEVR, we use filtered version (35) which consists of images containing less than seven ob- jects. In all experiments we use image resolutions identical to Emami et al. (35], i.e., 35x35 for Tetrominoes, 64x64 for dSprites and 96x96 for CLEVR (center crop of 192x192 resized to 96x96). In Tetrominoes and dSprites the number of training images is 60K whereas in CLEVR it is 50K. All three datasets have 320 test images on which we report the evaluation metrics, For further details about datasets and preprocessing, we refer to Appendix A
Model & Training Details. For more details about the encoder and decoder architecture of our model we refer to Appendix A] We train our model for 40K steps on Tetrominoes, and 100K steps dSprites tes and CLEVR with Adam optimizer (36] with constant learning rate of 5e-4, i.e., no warmup schedules or decay (all hyperparameter details are given in Appendix A). The phase initialization each spatial location and channel of the input image are independent samples from von-Mises distribution with mean of 0 and concentration of 1.
Evaluation Metrics.
We use the same evaluation protocol as prior work K3 35 23 which com-
pares the grouping performance of models using the Adjusted Rand Index (ARI) 37 38
The ART
scores are measured only for the foreground pixels common practice in the object-centric literature.
5

---

Input
Iteration 1
Iteration 2
lteration 3
Tetrominoes
dSprites
Figure 3: Evolution of phase maps in radial and heatmap form (colors matched) across iterations in SynCx for two inputs from Tetrominoes (row and dSprites (row 2).
Phase Map Visualization. We seek to qualitatively inspect phase components of the output complex-valued feature map h E Ch 'ww x from intermediate decoder layer to see if our model has indeed learned phases specialized in an object-centric manner. We visualize the phase component of the high-dimensiooal feature map heatmap in two dimensions. We assign unit magnitudes and retain the phase components of and elementwise convert these to the cartesian form as we did previously to extract object assignments. We apply t-SNE 39] to perform dimensionality reduction on the complex-valus
feature map of an input image to recover scalar 'composite" phase value at every spatial location (see (ppennixx A and |Figure 10 10]ror for more details). This 'composite' phase map visualized as a two dimensional heatmap and radial plot with colors in the former corresponding to orientations the latter (see columns 2 & 3 in Figure 3). From Figure shows how the phase maps evolve across the iterations in SynCx grouping process. In the sample image from Tetrominoes (row 1 in Figure 3). we can see how the phase 'bands' corresponding each tetris block are progressively separated in their orientations. Such a visualization gives an interpretable artefact that allows qualitative inspection of the phase synchronization process.
compares the performance of recent state-of-the-ar synchrony-based baselines (CAE, CAE++, CICAE, and RF) against ours (SynCx) on the unsupervised object discovery task. On Tetrominoes, we observe that SynCx outperforms all baselines on grouping performance. SynCx more gracefully separates objects of the same color (see FFigure 4) compared to RF which in addition to complex-valued activations requires x x-binding mechanism. SynCx also outperforms the CíCAE baseline which in addition to complex- valued activations requires x-binding and contrastive training to separate similarly colored objects. On dSprites. SynCx significantly outperforms CAE, CAE++ and CiCAE baselines while being strongly competitive with RF.. Similarly CLEVR, SynCx strongly outperforms CAF, CAE++ and CtCAE baselines while being competitive with RF. Overall, SynCx despite its simple binding mechanism (no gates) and training strategy (no contrastive training) outperforms is strongly competitive with all recent state-of-the-art synchrony-based models. However, there still remains a significant gap in grouping performance between synchrony-basee especially on CLEVR. We refer to Appendix for the model comparisons that includes the MSE loss well. We refer to Appendiix for additional qualitative grouping examples from our model.
Unsupervised Object Discovery. Table I
models w.r.t dominant slot-based approaches such SlotAttention [34]
Table I: ARI scores (mean + standard deviation across seeds) for CAE, CAE++, CICAE, RF. SynCx and SlotAttention on Tetrominoes, dSprites and CLEVR. Results for CAE, CAE++ and CíCAE baselines taken from Stanić et al. 24 and for SlotAttention taken from Locatello al. 34
Model
Tetrominoes
dSprites
CLEVR
CAE
0.00 i 0.00
0.05 0.02
0.04 0.03
CAE++
0.78 + 0.07
0,51 0.08
0.27 0.13
CICAE
0.84 + 0.09
0.56 0.11
0.54 0.02
RF
0.42 + 0.09
0.84 0.03
0.65 0.01
SynCx
0.89 + 0.01
0.82 0.01
0.59 0.03
SlotAttention
0.99 + avt
0.91 0.01
0.99 000

---

Tetrominces
dSprites
CLEVR
Ground truth
RF
Syn
Figure 4: Comparison between RF and SynCx grouping on Tetrominoes, dSprites and CLEVR. RF tends to systematically group similarly colored objects together while SynCx more adept at separat- ing them such as blue tetris blocks (left), green square and heart (middle) and yellow cylinders (right).
Below, we quantify the effects of key components of model such as representational bottlenecks, number of iterations and phase initialization have on its grouping performance.
Effect of Bottlenecks. We measure the effect of having representational bottlenecks (i.e. spatial resolution of feature maps) in the hidden layers of our model. To do so, we design a variant of model (denoted SynCx w/o bottleneck in Table 2} that preserves the spatial resolution of feature maps in the encoder (using stride of in all layers) therehy removing all bottlenecks. We ensure that both these variants (SynCx and SynCx w/o bottleneck) have the same number of parameters. Instead we only vary the spatial resolution of feature maps, so the bottleneck w.r. spatial resolution and the number of model parameters. We observe sharp drop in grouping performance for our ablated model variant without any bottlenecks despite it achieving a lower test loss (see Figure 5). This result supports our initial intuition that representational bottlenecks force an autoencoder with complex-valued activations to compress spatial regularities in features by using phase components to capture relationships between them. Since objects are modular units containing highly regular features within their boundaries, this leads to phases strongly specializing (synchronizing) towards them to facilitate better image compression.
Table Bottleneck ablation on Tetrominoes.
Model
MSE
ARI
SynCx w/o bottleneck
6.29c-5 9.0e-
0.10 0.06
SynCx 2.07c-3 1.0044
0.89
SynCx w/o bottl
SynCx
Input
Recon.
GT Mask
Mask
Phase Radial
Phase Heatmap
Figure Reconstruction, object masks, radial phase plot and phase heatmaps (colors matched between columns 5 & 6) for SynCx without the bottleneck (row 1) and the full SynCx model (row 2).
7

---

Effect of Iterations. We measure the effect of multiple iterations to update phases on the grouping performance of our model. [Table 3 shows the grouping performance of our model with increasing number of iterations used during training. We observe a general trend of increas- ing phase specialization towards objects and bet- ter reconstructions as we increase the number of iterations. This supports our qualitative charac- terization of the models' operation starting with random object assignments it iteratively refines the hypotheses for the assignment for each pixel.
Table 3: Ablation on number of iterations used for training on dSprites.
No. of Iterations
MSE +
ARI t
1
3,49c-3 # 1.1e44
0.57 = 0.01
2
2.20e-3 1: 6.24e.5
0.81 0.03
3
1.73e-3 d: 1.42e44
0.82
4
1.73e-3 # 3.85-4
0.79 0.03
We also measure the effect of increasing the number of iterations at test-time compared to that used during training. [Table 4 shows the grouping performance of our model as we in- crease the number of iterations from to while during training the model used 3 itera- tions. We observe no drop in object separation performance of our model when extrapolate the number of iterations used at test-time.
Table 4: Ablation on number of iterations used at test-time dSprites.
No. of Iterations
MSE +
ARI t
4
1.71e-3 # 1.38-4
0.81 0.01
5
1.71e-3 +: 1.40e4
0.82 0.01
6 1.70c-3
0.82 0.01
Effect of Phase Initialization. We measure the effect that the initialization of phases has on
the grouping performance of model. To achieve this, compare (see Table 5) the grouping performance of three variants - i) initial phases sampled from uniform distribution between -т and ı ii) initial phases sampled from a von-Mises distribution with mean of and concentration of and iii) all values in the initial phase map are set to zero.
Rest of the model and training hyperparameters are kept the same between these two variants. We observe that the variant using von-Mises dis- tribution to sample initial input phases achieves the best test loss and improved grouping scores among all variants. This suggests the variance of the noise distribution is an important factor to tune for phase synchronization. The von-Mises distribution with mean of and concentration of is the circular analogue of the Normal distribution and therefore samples "noisy" phase values with an intermediate level of variance compared to the other two alternatives which have maximum and zero variance. We use the von-Mises variant as the default phase initialization for input phases in all experiments.
Table 5: Phase init. ablation on Tetrominoes.
Phase Init.
MSE +
ARI
Zero
2.46c-3 5.08-7
0.74 + 0.04
Uniform
1.02c-2 807e-3
0.75 0.05
von-Miscs
2.07c-3 = 1.09-4
0.89 0.01
Table 6: Comparison of parameter counts (rounded up to the nearest thousand) of various synchrony- based models. Total number of parameters expressed in terms of number of real-valued floats.
Model
Tetrominoes
dSprites
CLEVR
CAE++ 5,372,000
11,713,000
22,281,000
CICAE
5,372,000
11,713,000
22,281,000
RF 6,630,000
11,861,000
22,592,000
SynCx
966,000
974,000
974,000
Parameter and Training Efficiency Table 6 shows the pa- rameters counts for the various synchrony-based models. We see that our model, SynCx, is 6-23x more parameter efficient compared to the other synchrony-ben based alternatives. TTall 7 shows the wall-clock training times for convergence of SynCx against the strongest synchrony baseline, RF, on Tetrominoes and dSpri tes. We see that our model on these datasets consis- tently takes lesser wall-clock time for training in these settings.
Table 7: Wall-clock training times for SynCx and RF models on P100 GPU.
Model
Tetrominoes
dSprites
RF SynCx
7h 20m
4h 50m
1h 35m
4h 10m

---

RF
SynCx
Image
Recon.
GT Mask
Mask
Figure 6: Reconstruction and object masks for RF and SynCx on grayscale CLEVR. While both
models reconstruct well, RF struggles to group the objects without access to color unlike SynCx.
These results highlight the computational benefits of the simple ideas of complex-valued weights and recurrence that are the essence of our model design.
Limitations. We test how reliant color models from Table I] i.e., RF and SynCx, for unsupervised object discovery. We train and eval- uate them a grayscale version of the CLEVR dataset. To separate objects in this dataset, the models' would need to rely more on other features like shape, texture, etc. From Table 8) we see that the grouping performance of RF on grayscale CLEVR drops sharply compared the orig- inal (see Table indicating its heavy reliance on color cues, Our models' grouping performance also drops but much lesser degree compared to RF. This indicates weaker dependence on color cues in the binding mechanism of our model. In fact, on grayscale CLEVR SynCx signifi- cantly outperforms RF despite having slightly worse performance on the original colored version. The binding mechanism (i.e., matrix-vector product) in our model updates phases based on agreement between feature detectors (complex-valued weights) and high-dimensional attributes coomlexxvalluee activations) that model color, edges, texture, shape etc. while taking it account their phase relationships (local context). This facilitates it to bind to objects more robustly and not simply rely color. Overall, we can see that despite its simple architecture and training procedure our model shows good grouping performance wider range of visual environ- ments. However, binding mechanisms state-of-the-arr synchrony models are far from perfect. They unable to reliably capture objects even synthetic datasets like CLEVR containing simple 3D) shapes with metallic or matte textures under camera lighting with a non-tex non-textured background.
the two best performing synchrony-
Table 8: Grayscale CLEVR.
Model
MSE
ARI 1
RF
5.64e-4 3.76e4
0.22 : 0.1
SynCx
2.49e-4 3766-
0.45 + 0.11
Related Work
Slot-based Binding. Recent years have seen a growing number models for unsupervised per-
ceptual grouping (see Greff et al. [10] for a summary). These models maintain the separation of information using separate of activations ('slots') but differ in the segregation mechanism to
infer their contents. Representational symmetry in 'slots' is broken via temporal ordering [40
spatial ordering 442
specificiiy [45], iterative routing procedures [46] 47
B3
34
48
(cf.
earlier work 49 or combinations thereof 50-52) Extensions of slot-based grouping models for
videos (5356) 56), novel view synthesis (57 59 as well as other modalities than vision including speech
[60]. music (61] and actions [62) have been explored. Conceptual limitations of slot-based models
include separation only maintained at one level, binding information stored in hard-wired architectural components unsuited for gradient-based adaptation, uniform capacity, inability to store object-level relational factors and high computational cost for training as noted by Stanić et (24)
Synchrony-based Binding. Synchrony-based models could in principle address the above short- comings of slot-based models but have received comparatively little attention. They resolve the binding problem by augmenting each activation with additional grouping features. They can

---

be broadly divided into two classes temporal and complex, based on the type of coding strategy used for the augmentation. Models using temporal codes rely on spiking neurons with rhythmic firing behavior 65..55. In such models, features of an object are expressed by neurons that fire in-syne. However, spiking neurons are non-differentiable and therefore incompatible with gradient- based learning, requiring specialized learning algorithms difficult to scale with current hardware accelerators for training. Models using complex-valuee codes 20 21 66 [22 (23 are based on complex- valued activations suitable for scalable training using backpropagation. These models have been empirically benchmarked only on binarized images of simple geometric shapes or MNIST digits. More recent models [5 24 show improved unsupervised grouping performance on more visually challenging color images from the multi -bject suite [3]. However, these recent models heavily rely supervision in the form of 'depth masks', gating mechanisms, contrastive training or a combination thereof. In contrast, our model is simply a fully convolutional autoencoder with complex-valued weights that iteratively updates phases reconstruct an input image.
Temporal Correlation Hypothesis. Synchrony- models draw functional inspiration from the temporal correlation hypothesis [67] 688 in neuroscience. It posits that the brain uses synchronized dynamics of neuronal firings to bind together distributed feature information computed in parallel at different areas into coherent percepts. Neural synchrony is also believed to convey information about relationships between features needed for dynamic and context-dependee
binding by 'relational coding' [69]. This in contrast to 'labeled line coding' where each unit has a fixed label attached to it indicating the static feature conjuction being active. Our use of complex-valus valued weights to process complex-valued activations akin to the 'relational coding' scheme.
5 Conclusion
We explored synchrony-base binding with an architecture, SynCx, that differs from current models in three respects: i) SynCx's weights are complex valued, allowing it to encode joint reature-phase configurations in the weights; ii) SynCx is recurrent and stateful, allowing it to perform iterative constraint propagation; and iii) SynCx has an internal representational bottleneck, which require it to use the phases of complex-valued activations to encode statistical regularities in images. These three properties are crucial to extending object-centric phase synchronization behavior to the fully unsupervised setting. Our conceptually elegant model outperforms all state-of of-the--rt baselines Tetrominoes and overcomes a common failure mode of these baselines in which similarly colored objects are grouped together. Overreliance color cues for grouping by current models is further highlighted on grayscale variant of the CLEVR dataset where we see that the RF model struggles to group objects using other features (shape and texture), in contrast to SynCx. Our model shows strong performance compared to more sophisticated synchrony-ban -based baselines [25] [2] on dSprites and CLEVR without the need for additional supervision such as 'depth masks", gating mechanisms or contrastive training. Starting with our model as a simple template, future directions to explore include the use of temporal difference style loss to weight reconstruction targets and incorporate spatial priors into the binding mechanism. The process of extracting discrete object assignments from continuous phase maps by clustering could be improved by accounting for outliers, -au-ssas models
distributed phase values with unequal sizes and variances. While recent synchrony-base have been steadily closing the gap grouping performance to estabissabl slot-based approaches such as SlotAttention [34] on simple synthetic datasets (Tetrominoes and dSpr i tes) the gap remains significant on CLEVR. Further, they are yet to be scaled up to more challenging multi-object visual benchmarks (e.g. MOVi (70) which offers open challenge for future work.
10

---

Acknowledgments. We thank Kazuki Irie for numerous discussions and guidance. We also thank Thomas Kipf for giving valuable feedback on our manuscript. This research was funded by Swiss National Science Foundation grant: 200021 200021_192356 project NEUSYM and partially by the ERC Advanced grant no: 742870, AlgoRNN. This work was also supported by the following grants from the Swiss National Supercomputing Centre (CSCS) under project ID s s1205 and s1294. We also thank NVIDIA Corporation for donating DGX machines as part of the Pioneers of AI Research Award.
References
[1] Max Wertheimer. Untersuchungen zur lehre von der gestalt. ii, Psychologische Forschung, 4(1): 301-350, 1923.
[2] Kurt Koffka. Principles of gestalt psychology. Philosophy and Scientific Method, 32(8), 1935.
[3] Wolfgang Köhler. Gestalt psychology. Psychologische Forschung, 31(1), 1967.
[4] B. Kim, E. Reif, M Wattenberg, S. Bengio, and M. C. Mozer. Neural networks trained on natural scenes exhibit gestalt closure. Computational Brain & Behavior, 4:251-263, 2021.
[5] Richard Zemel, Marlene Behrmann, Michael C Mozer, and Daphne Bavelier. Experience- dependent perceptual grouping and object-based attention. Journal of Experimental Psychology: Human Perception and Performance, 28( 10:20, 2002.
[6] Elizabeth S. Spelke. Principles of object perception. Cognitive Science, 14(1):29-56, 1990.
[7] Elizabeth S Spelke and Katherine D Kinzler. Core knowledge. Developmental science, 10(1):
89 99-96,2200 2007.
[8] Charles Kemp and Joshua B. Tenenbaum. The discovery of structural form. Proceedings of the National Academy of Sciences, 105631):10887-109) 2008
[9] Peter W. Battaglia, Jessica B. Hamrick, and Joshua B. Tenenbaum. Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences, 110(45): 1 18327-18332, 2013.
[10] Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial neural networks. Preprint 20t.:::002205200.. 2020.
[11] David Ding, Felix Hill, Adam Santoro, Malcolm Reynolds, and Matthew Botvinick. Attention over leamned object embeddings enables complex visual reasoning. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2021.
[12] Victor Bapst, Alvaro Sanchez-Gonzaler Carl Doersch, Kimberly L. Stachenfeld, Pushmeet Kohli, Peter W. Battaglia, and Jessica B. Hamrick. Structured agents for physical construction. In Proc. Int. Conf. on Machine Learning (ICML), 2019.
[13] Aleksandar Stanié, Yujin Tang, David Ha, and Jürgen Schmidhuber. Learning to generalize with object-centric agents in the open world survival game crafter. IEEE Transactions on Games, pages 1-20, 2023.
[14] Jaesik Yoon, Yi-Fu Wu, Heechul Bae, and Sungjin Ahn. An investigation into pre-training object-centric representations for reinforcement learning. In Proc. Int. Conf. on Machine Learning (ICML), 2023.
[15] Priyanka Mandikal and Kristen Grauman. Learning dexterous grasping with object-centric visual affordances. In 2021 IEEE international conference on robotics and automation (ICRA), 2021.
[16) Yizhe Wu, Oiwi Parker Jones, Martin Engelcke, and Ingmar Posner. Apex: Unsupervised, object- centric scene segmentation and tracking for robot manipulation. In JEFF/RSI International Conference Intelligent Robots and Systems (IROS), 2021.
11

---

[17] Andrii Zadaianchuk, Maximilian Seitzer, and Georg Martius. Self-supervised visual reinforce- ment learning with object-centric representations. In Int. Conf. on Learning Representations (ICIRR), 2021.
[18] Anne Treisman. The binding problem. Current opinion in neurobiology, 6(2):171-178, 1996.
[19] Adina L Roskies. The binding problem. Neuron, 24(1):7-9, 1999.
[20] Michael C. Mozer, Richard Zemel, Marlene Behrmann, and Christopher I. Williams. Learning to Segment Images Using Dynamic Feature Binding. Neural Computation, 4(5): 650-665, 1992.
[21] Michael C Mozer. A principle for unsupervised hierarchical decomposition of visual scenes. In Proc. Advances in Neural Information Processing Systems (NIPS), 1998.
[22] David P Reichert and Thomas Serre. Neuronal synchrony in complex- valued deep networks. In Int. Conf. on Learning Representations (ICLR), 2014.
[23] Sindy Löwe, Phillip Lippe. Maja Rudolph, and Max Welling, Complex-valued autoencoders for object discovery. Transactions on Machine Learning Research, 2022.
[24] Aleksandar Stanić, Anand Gopalakrishnan, Kazuki Irie, and Jürgen Schmidhuber. Contrastive training of complex-valued autoencoders for object discovery. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2023.
[25] Sindy Löwe, Phillip Lippe. Francesco Locatello, and Max Welling. Rotating features for object discovery. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2023.
[26] Geoffrey Hinton. How to Represent Part-Whole Hicrarchies in a Neural Network. Neural Computation, 35(3):413 2023.
[27] Sindy Löwe, Francesco Locatello, and Max Welling. Binding dynamics in rotating features. ArXiv, abs/2402.05627,, 2024.
[28] David Waltz. Generating semantic descriptions from drawings of scenes with shadows. Techni- cal report, Massachusetts Institute Technology, MIT MAC-TR 271, 1972.
[29] Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary evolution recurrent neural networks. In Proc. Ini. Conf. on Machine Learning (ICML), 2016.
[30] Chiheb Trabelsi, Olexa Bilaniuk, Ying Zhang, Dmitriy Serdyuk, Sandeep Subramanian, Joao Fe- lipe Santos, Soroush Mehri, Negar Rostamzadeh, Yoshua Bengio, and Christopher J Pal. Deep complex networks. In Int. Conf. on Learning Representations (ICLR), 2018
[31] Kaiming Hc, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-leve. performance on imagenet classification. In Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2015.
[32] Rishabh Kabra, Chris Burgess, Loic Matthey, Raphael Lopez Kaufman, Klaus Greff, Malcolm Reynolds, and Alexander Lerchner. Multi-object datasets. object-datasets/, 2019.
[33] Klaus Greff, Raphaël Lopez Kaufman, Rishabh Kabra, Nick Watters, Christopher Burgess, Daniel Zoran, Loie Matthey, Matthew Botvinick, and Alexander Lerchner. Multi-object repre- sentation learning with iterative variational inference. In Proc. Int. Conf. on Machine Learning (ICML), 2019.
[34] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2020.
[35] Patrick Emami, Pan He, Sanjay Ranka, and Anand Rangarajan. Efficient iterative amortized inference for learning symmetric and disentangled multi-object representations. Proc. Int. Conf. on Machine Learning (ICML), 2021.
12

---

[36] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Int. Conf. on Learning Representations (ICLR), 2015.
[37] William M Rand. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association, 66(336):846880) 1971.
[38] Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of classificution, 2:193-218.
1985.
[39] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):257-- 2008.
[40] SM Ali Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, David Szepesvari, Geoffrey E Hinton, et al. Attend, infer, repeat: Fast scene understanding with generative models. In Proc. Advances in Neural Information Processing Systems (NIPS), 2016.
[41] Adam Kosiorek, Hyunjik Kim, Yee Whye Teh, and Ingmar Posner. Sequential attend, infer, repeat: Generative modelling of moving objects. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2018.
[42] Eric Crawford and Joelle Pineau. Spatially invariant unsupervised object detection with convo- lutional neural networks. In Proc. AAAI Conf. on Artificial Intelligence, 2019.
[43] Aleksandar Stanić, Sjoerd van Steenkiste, and Jürgen Schmidhuber. Hierarchical relational inference. In Proc. AAAI Conf. on Artificial Intelligence, 2021.
[44] Zhixuan Lin, Yi-Fu Wu, Skand Vishwanath Peri, Weihao Sun, Gautam Singh, Fei Deng, Jindong Jiang, and Sungjin Ahn. SPACE: unsupervised object-oriented scene representation via spatial attention and decomposition. In Int. Conf. on Learning Representations (ICLR), 2020.
[45] Geoffrey E. Hinton, Sara Sabour, and Nicholas Frosst. Matrix capsules with EM routing, In Iut. Conf. on Learning Representations (ICLR), 2018
[46] Klaus Greff, Antti Rasmus, Mathias Berglund, Tele Hao, Harri Valpola, and Jürgen Schmidhuber. Tagger: Deep unsupervised perceptual grouping. In Proc. Advances in Neural Information Processing Systems (NIPS), 2016.
[47] Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber. Neural expectation maximization. In Proc. Advances in Neural Information Processing Systems (NIPS), 2017.
[48] Yi-Fu Wu, Klaus Greff, Gamaleldin Fathy Elsayed, Michael Curtis Mozer, Thomas Kipf, and Sjoerd van Steenkiste. inverted-attention transformers can learn object representations: Insights from slot attention. In UniReps: the First Workshop on Unifying Representations in Neural Models, NeurIPS, 2023.
[49] Jürgen Schmidhuber.
Learning to control fast-weight memories: An alternative to recurrent nets. Neural Computation, 4(1 4(1):131-139, 1992.
[50] Christopher P Burgess, Loic Matthey, Nicholas Watters, Rishabh Kabra, Irina Higgins, Matt Botvinick, and Alexander Lerchner. Monet: Unsupervised scene decomposition and representa- tion. arXiv preprint 19Xiv 2019.
[51] Martin Engelcke, Adam R. Kosiorek, Oiwi Parker Jones, and Ingmar Posner. Genesis: Genera- tive scene inference and sampling with object-centric latent representations. In Int. Conf. on Learning Representations (ICLR). 2020.
[52] Gautam Singh, Yeongbin Kim, and Sungjin Ahn. Neural systematic binder. In Int. Conf. on Learning Representations (ICLR), 2022.
[53] Thomas Kipf, Gamaleldin Fathy Elsayed, Aravindh Mahendran, Austin Stone, Sara Sabour, Georg Heigold, Rico Jonschkowski, Alexey Dosovitskiy, and Klaus Greff. Conditional object- centric learning from video. In Int. Conf. Learning Representations (ICLR), 2022.
13

---

[54] Gamaleldin F. Elsayed, Aravindh Mahendran, Sjoerd van Steenkiste. Klaus Greff, Michael C. Mozer, and Thomas Kipf. SAVi++: Towards end-to-end object-centric learning from real-wordh videos. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2022.
[55] Gautam Singh, Yi-Fu Wu, and Sungjin Ahn, Simple unsupervised object-centric learning for complex and naturalistic videos. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2022.
[56] Andrii Zadaianchuk, Maximilian Seitzer, and Georg Martius. Object-centric learning for real-world videos by predicting temporal feature similarities. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2023.
[57] Nanbo Li, Cian Eastwood, and Robert Fisher. Learning object-centric representations of multi- object scenes from multiple views. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2020.
[58] Karl Stelzner, Kristian Kersting, and Adam R Kosiorek. Decomposing 3d scenes into objects via unsupervised volume segmentation. arXiv:2104.01148, 2021.
[59] Mehdi S. M. Sajjadi, Daniel Duckworth, Aravindh Mahendran, Sjoerd van Steenkiste, Filip Pavetié, Mario Lučié, Leonidas J. Guibas, Klaus Greff, and Thomas Kipf. Object Scene Representation Transformer. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2022.
[60] Pradyumna Reddy, Scott Wisdom, Klaus Greff, John R. Hershey, and Thomas Kipf. Audioslots: slot-centric generative model for audio separation. In IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023.
[61] Joonsu Gha, Vincent Herrmann, Benjamin Grewe, Jürgen Schmidhuber, and Anand Gopalakr- ishnan. Unsupervised musical object discovery from audio. In Workshop Machine Learning for Audio, NeurlPS, 2023.
[62] Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber, and Sjoerd van Steenkiste. Unsuper- vised Learning of Temporal Abstractions With Slot-Based Transformers. Neural Computation, 35(4):593-626, 2023.
[63] Christoph von der Malsburg and Werner Schneider. A neural cocktail-party processor. Biological cybernetics, 54( 1:29-40, 1986.
[64] Christoph von der Malsburg and Joachim Buhmann. Sensory segmentation with coupled neural oscillators. Biological cybernetics, 67(3):233-242, 1992.
[65] DeLiang Wang. The time dimension for scene analysis. IEEE Transactions Neural Networks, 1411::440 2005.
[66] A. Ravishankar Rao, Guillermo A. Cecchi, Charles C. Peck, and James R. Kozloski. Unsu- pervised segmentation with dynamical units. IEEE Transactions on Neural Networks, 19(1): 168-182, 2008
[67] Christoph von der Malsburg. Binding in models of perception and brain function. Current opinion in neurobiology. 5(4):520-526, 1995.
[68] Wolf Singer and Charles M Gray. Visual feature integration and the temporal correlation hypothesis. Annual review of neuroscience, 18(1):555-58 586, 1995.
[69] Wolf Singer. Neuronal synchrony: A versatile code for the definition of relations? Neuron, 24 (1):49-65, 1999.
[70] Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh-Ti (Derek) Liu, Henning Meyer, Yishu Miao, Derek Nowrouzezahrai, Cengiz Oztireli, Etienne Pot, Noha Radwan. Daniel Rebain, Sara Sabour, Mehdi S. M. Sajjadi, Matan Sela, Vincent Sitzmann, Austin Stone, Deqing Sun, Suhani Vora, Ziyu Wang, Tianhao Wa, Kwang Moo Yi, Fangcheng Zhong, and Andrea Tagliasacchi. Kubrie: a scalable dataset generator. In Proc. IEFE Conf. on Computer Vision and Pattern Recognition (CVPR), 2022.
14

---

[71] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python, Journal of Machine Learning Research, 12:2825-2830, 2011.
[72] Leland MeInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint axXiv:1802.03426 2018.
[73] Leland Mclnnes, John Healy, Nathaniel Saul, and Lukas Grossberger. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861, 2018.
15

---

A Experimental Details
Datasets. We evaluate all models on a subset of three datasets from the multi-object suite namely: Tetrominoes which consists of colored tetris blocks on a black background, dSpri tes with colored sprites of various shapes like heart, square, oval, etc., on a grayscale background. and lastly, CLEVR, dataset from a synthetic 3D environment. For CLEVR, we use the filtered version [35] which consists of images containing less than seven objects sometimes referred to as CLEVR6 as in Locatello et al. [34] We normalize all input RGB images to have pixel values in the range [0, 1] consistent with prior work [23]. To generate the grayscale variant of the CLEVR dataset we apply the color to grayscale conversion function from the Pillow library part of the data preprocessing pipeline.
shows the architecture specifications such number of layers, kernel sizes, stride lengths, number of channels etc., for the convolution layers used by the encoder and decoder modules in SynCx. We reproduce unsupervised object discovery results of the RF baseline by adapting the source code released by the authors. Similarly, we reproduce unsupervised object discovery for the CAE, CAE++ and CICAE by adapting the source code released by the authors.
Models. Table 9
Table 9: Encoder and Decoder architecture specifications for SynCx.
Encoder
3 x 3 conv, 64 channels, stride 2, modReLU
3 x 3 conv, 128 channels, stride 2, modReLU
3 x 3 conv, 128 channels, stride 2, modRel.U
Decoder
Nearest neighbor upsample x2
3 x 3 conv, 128 channels, stride 1, modReLU
Nearest neighbor upsample x2
3 x 3 conv, 64 channels, stride 1, modReLU
Nearest neighbor upsample x2
3 x 3 conv, 64 channels, stride 1, modReLU
For 64x64 and 96x 96 inputs, I additional decoder layer:
x conv, 64 channels, stride 1, modRel.U
Output Layer
1 x I conv, 3 channels, stride 1
Training Details. Table 10 shows the hyperparameter configurations used to report unsupervised object discovery results (Table for SynCx. To reproduce the unsupervised object discovery results for RF in Table we use the hyperparameter configurations reported by the authors 25 listed in Appendix D.6.
Table 10: Training hyperparameters for SynCx.
Hyperparameter
Tetrominoes
dSprites
CLEVR
Training Steps
40,000
100,000
100,000
Batch size
64
16
32
Learning rate
5e-4
5e-4
5e-4
Gradient Norm Clipping
1.0
1.0
1.0
Number of iterations
4
Phase initialization
Von-Mises
von-Mises
von-Mises
https///ttthiibbcco//ppppp////op
16

---

Computational Efficiency. We report the training and inference time (wall-clock) for our models across 3 image resolutions for the 3 datasets used in this work from the multi-object suite. Inference on the test set containing 320 images of 35x35 resolution takes 7.87 seconds using 3 iterations, for 64x64 images takes 38.96 seconds using 3 iterations and for the 96x96 images it takes 112.75 seconds using 4 iterations on single NVIDIA GTX 1080Ti GPU. Training time(s) on the other hand differs depending on the image resolution, number of iterations and model size. To train our model for 40k steps on 35x35 resolution images from Tetrominoes took 1.7 hours To train our model for 100k steps on 64x64 resolution images from dSprites took 4.25 hours on a NVIDIA Tesla P100 GPU. To train model for 100k steps on 96x96 resolution images from CLEVR took 17.87 hours on a NVIDIA Tesla and std-dev across 5 seeds) reported in this work we estimate the compute requirement to be 390 GPU hours in total for training models (RF and SynCx). Further, we estimate that the total compute used in this project is roughly 10-15 times more than the above figure, used for experiments in the prototyping phase. Note, that the figure above does not account for the phase map visualization and evaluation (clustering) procedures to extract object masks using the trained models primarily executed CPUs.
NVIDIA Tesla P100 GPU.
GPU. To reproduce all the results/tables (mean
Phase Map Visualization. We seek to visualize the complex-valued feature map h C* qp''ww x at some intermediate layer of the decoder module as an image. We apply dimensionality reduction along the channels of h so to represent as an one dimensional feature map. We start by constructing a complex-valued feature map in polar form with phase components (ox)) and magnitude components xw'
being the identity, i.e., Ph <- 1 E RA'xw' xdai, Then, we convert the feature map obtained in the previous step from polar to Cartesian form. Next, we apply t-SNE to project the Cartesian domain feature map in two dimensions at each spatial location. We use the t-SNE implementation in the scikit-learn library [7] (sklearn. www.mmaaiiiooo.ommmmm TSNE) with n_ iter set to 500, metric set to Euclidean and perplexity set to 20. We also experimented with UMAP (72 for the dimensionality reduction computation but found it to inferior to I-SNE qualitatively (see Figure 10) and processing time per image (UMAP takes 24 seconds whereas t- t-SNE takes 8 seconds). We use the UMAP implementation in the umap-learn library 773 (umap. UMAP) with default arguments. Therefore, we use t-SNE for dimensionality reduction in the phase visualization process. We recover 'composite' phase o E R xw' by converting the two dimensional feature back to the polar form. The 'composite' 1q''xw' each spatial location of the map is the Euclidean norm (along channels) magnitude F E
of the vector- features (i.e., magnitude components of h) at that location. The 'composite' phase every spatial location is visualized as the heatmap value and the "composite" magnitude its distance from the center in the radial plot. The colors of points are matched across the heatmap and the radial plot such that the color of a pixel in the former correspond to its orientation the latter. Lastly, the magnitude of background regions are masked out as was the case while extracting object assignments from phase maps.
Evaluation Details. To evaluate our model for the unsupervised object discovery task we require discrete object assignments at every location in the image. We cluster the continuous phase compo- nents of the complex-valued feature maps from an intermediate layer (second to last by default) of the decoder to compute these object assignments for every pixel. We use the k-means implementation in the scikit-learn library [71] (sklearn. clustering, KMeans) with n_ clusters set using the ground-truu value for each dataset and n_ init set to
17

---

B Additional Results
Table 11: MSE and ARI scores (mean + standard deviation across 5 seeds) for CAE, CAE++ CtCAE, RF and SynCx models for Tetrominoes, dSprites and CLEVR. Results for CAE, CAE++ and CICAE baselines are taken from Stanić et al. [24].
Dataset
Model
MSE
ARI f
Tetrominoes
CAE
4.57e-2
1.08e-
0.00 +
CAE+
5.07e-5
2.800-5
0.78 = 0.07
CICAE
9.73e-5 4.646-5
0.84 = 0.09
RF
5.27e-6 2.60e-6
0.42 + 0.09
SynCx
2.07e-3 1.09e-4
0.89 + 0.01
dSprites
CAE
8.16e-3 2.54e-5
0.05 0.02
CAE++
1.60e-3
1.33--
0.51 0.08
CiCAE
1.56e-3
1.58e-4
0.56 0.11
RF
5.96e-4 1.62e
0.84 0.03
SynCx
1.73e-3 1.42e
0.82 1: 0.01
CLEVR
CAE
1.50e-3
4.53e
0.04 1 0.03
CAE+
2.41e-4 3.45e
0.27 1: 0.13
CiCAE
3.39e-4
3.65e-5
0.54 +:2 0.02
RF
2.69e-4 9.376-5
0.65 - 0.01
SynCx
3.30e-4
5.30e-5
0.59 : 0.03
18

---

C
Additional Visualizations
Input
Recon.
GT Mask
Mask
Radial
Heatmap
Figure 7: Collection of images grouped by SynCx from the Tetrominoes dataset. Rows 1-3 show cases where SynCx perfectly groups the tetris blocks even with mulitple objects of the same color. Row 4 shows a failure mode where it imperfectly partitions the two red blocks into one straight parts and two L-shaped parts. It is plausible decomposition since the dataset contains many tetris blocks with such L-shaped bends. Rows and show another failure mode where it fails to decompose the two similarly colored tetris blocks all. Rather SynCx learns specialized phases for the edges and interior portions of each of the similarly colored tetris blocks. This could be a caused by particularly poor initialization of phase maps by random sampling.
19

---

Input
Recon.
GT Mask
Mask
Radial
Heatmap
Figure 8: Collection of images grouped by SynCx from the dSprites dataset. Rows 1-3 show cases where SynCx perfectly groups images containing 3-5 objects. We also that the pairs of phase map and radial plot visualizations show discernible specialization phase values towards an object. Row 4 shows a grouping failure where the square and heart in light blue incorrectly grouped together: However, on closer inspection of the corresponding phase plots we can see that the phases do show specialization (see 4 column 6) towards both the square (light blue) and heart (dark blue). This grouping failure is limitation in the clustering process (KMcans)) to extract object assignments and assumes clusters have equal angular variance which need hold true in practice. Similarly, grouping failures in rows and 6, where blue oval and purple square (row 5) or light blue heart and cobalt square (row are incorrectly grouped together despite showing enough phase specialization in each case.
20

---

Input
Recon.
GT Mask
Mask
Radial
Heatmap
Figure 9: Collection of images grouped by SynCx from the CLEVR dataset. Rows 1-4 show cases where SynCx groups scenes containing three to six objects reasonably well. Row shows failure mode where the gray, red and purple objects noticeable (see column 6). This can be attributed to the limitations the clustering process similar to the ones highlighted previously with regards to assumption of equal angular variance of clusters. Similar effects can be observed with the grouping errors in row
incorrectly grouped together despite showing
21

---

tSNE
UMAP
tSNE
UMAP
tSNE
UMAP
ISNE
UMAP
UMAP tSNE
Figure 10: Collection of samples comparing the use of t-SNE versus UMAP for the dimensionality reduction computation the visualization process. Panels and 5 show cases where the clusters of projected phase maps of both t-SNE and UMAP correlate with the number of objects in the image. Panels 2, and show cases where the clusters of phases projected using t-SNE correlate with the number of objects in the image whereas the clusters of phases projected using UMAP do not. Overall, we find that t-SNE produces qualitatively better projections than UMAP, i.c., phase clusters correlate better to the number of objects in the image.
22

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract make the following claims: "recurrent architecture with complex-valued weights perform equally or better at unsupervised object discovery com- pared to architectures that have complex activations but real-valued weights" which is supported by superior performance in the experiments; we claim that SynCx (our model) avoids certain systematic grouping errors of current models, such as the inability to sepa- rate similarly colored objects without additional supervision, which is also supported by experiments with grayscale data.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
We discuss limitations of our method dedicated paragraph (last paragraph of the Experiments section), in particular we show its robustness to grouping in the absence color information.
Justification:
Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide all architectural and hyperparameter details in the paper.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Data we used already public and upon acceptance we will publish our code.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All training and test details are provided in Appendix A.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All experimental results are provided with mean and standard deviation across 5 seeds.
8. Experiments Compute Resources
23

---

For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide sufficient information (see Appendix A) of the hardware resources required reproduce the main results of the paper as well resources used during the prototyping phase.
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurIPS Code of Ethics httpps///oorrg...cc/oaaa/////c/p
Answer: [Yes]
Justification: We have reviewed the NeurlPS Code of Ethics and confirm that work respects it, i.e. does not involve human subjects, the data we use synthetic (no privacy ethical concerns) and we foresee harmful consequences of our work as the models small scale and training and evaluation data is all synthetic.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We foresee neither negative nor positive societal impacts of work as the visual scenes we experiment on are highly synthetic (toy tasks) nowhere near the complexity scale of naturalistic images/videos. We also only use very small-scale models (less than 1M params) which are not mature yet for use in production.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We not use language models, image generators or datasets scraped from the internet in our work, so we foresee no risk of misuse.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The SynCx model is original contribution of ours. The baseline models CAE, CAE++, CtCAE and RF have been properly credited with citations the research papers and code repositories. We also properly cite the data source for the multi-object suite.
Justification:
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We don't introduce any assets.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: We do not have any crowd sourcing research with human subjects.

---

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: We do not have any research with human subjects our work.
25


Reviews:
[{"review_id": "J0Mfb6amIS", "reviewer": "Reviewer_Ta5A", "summary": "The authors propose SynCx: a model of synchrony-based perceptual grouping. SynCx is a recurrent autoencoder with complex-valued parameters. By design, phase-information in the model is carried through across model iterations allowing it to be \"stateful\". SynCx demonstrates competitive/comparable performance on dSprites and CLEVR while outperforming currently available synchrony-based methods on Tetrominoes. Its worth noting that SynCx operates in a fully unsupervised manner and does not require sophisticated hueristics, or augmentations to its training objectives, or additional forms of supervision to perform object discovery.", "questions": "Please refer to weaknesses above.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper targets an interesting problem in perceptual science -- what are the cues (at varying levels of abstraction) that are required to execute successful feature binding to represent \"objectness\". They expose a problem in current state-of-the-art methods in their overreliance on a low-level cue: color. The authors perform empirical analysis on a color-lesioned version of CLEVR and find that SynCx's reliance on color is lesser compared to the other baseline. SynCx is comparable in performance to more sophisticated methods on two challenging benchmarks.", "weaknesses": "Despite the promise of this approach, there are several outstanding questions to be addressed by this manuscript.\n\nL178 \"We use latent-level complex-valued features ... than simply color cues (RGB space)\": This seems contradictory to the authors primary claim. If only the phase information is used for the clustering analysis (and this is stateful) then it is unclear why the authors suggest the use of latents vs outputs? Can the authors clarify?\n\nThe color-lesion experiment on CLEVR has potential low-level confounds apart from color (like shadows, shading, etc.) that models can use as shortcuts for binding. \n\nWhy did the authors have to retrain SynCx on the grayscale dataset? If binding-by-synchrony is faithfully implemented by their model, shouldn't this work zero shot? If this is not the case, then what is the rationale for it?\n\nThe authors raise a point about instance-level groups being the focus of object-centric learning (L59). Given that, there are very few (maybe 2) examples presented where the scene contains multiple instances of the same object (same color, size, orientation, etc.) and the corresponding outputs from SynCx. Can the authors present these?\n\nThe \"lesioning expermients\" part of the manuscript needs more thorough work.\n\n\"Effect of Bottlenecks\": It is understandable that the test loss is lower for the model without bottlenecks since it has a greater number of parameters. However, the explanation for why bottlenecks are important seems misleading. The importance of bottlenecks, in my opinion, originates due to the effective receptive field sizes in these layers and has nothing to do with complex-valued units or synchrony per se. It is unclear what this analysis adds to the manuscript.\n\n\"Effect of Iterations\": From the results presented, it seems like there is diminishing returns after just 2 iterations? If that's the case, then wouldn't this approach be very comparable to feedforward methods? Moreover, can the authors train SynCx on a large number of timesteps (say 10) and observe the dynamics of convergence in the phases during test time? Claims about \"iterative constraint propagation\" needs further backing. Perhaps these datasets are too easy?\n\n\"Effect of Phase Initialization\": I found it hard to grasp the take away of this section. The Von-Mises distribution (with mean 0, and concentration 1) biases most of the units in the initialization to have similar phases. The random initialization performed pretty poorly. What does this suggest though computationally?\n\nFigure 2 is slightly misleading in its depiction. The magnitude of the output $\\mu_z^1$ is shown to be the same as $\\mu_x$. This derails the reader from understanding the flow of information across timesteps. Notationally, denoting time as a superscript and outputs as $z$s is confusing from a readers perspective. A clearer version could be something along the lines of $\\hat{x}(t) = \\mu_{\\hat{x}}(t) e^{i\\phi_\\hat{x}(t)}$\n\nMinor (typos):\n\nFigure 5 caption: \"Sync without the bottleneck\"\n\nL135: Grammatical. \"... or strongly competitive\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for acknowledging the simplicity of our approach which does not require sophisticated heuristics or additional training objectives or forms of supervision. We address your specific concerns below.\n\n``` L178 \"We use latent-level complex-valued features ... than simply color cues (RGB space)\": This seems contradictory to the authors primary claim. If only the phase information is used for the clustering analysis (and this is stateful) then it is unclear why the authors suggest the use of latents vs outputs? Can the authors clarify? ```\n\nThe short answer to your good question is that we tried to read out from the output phases but they were inferior to using latent phases to segment objects. We found that the richer latent-level phases associated with higher-order features are useful for grouping. We also experimented with variants of the model that used a single (shared) phase for all outputs, and where each output feature used its own phase. In the classic supervised learning approach to segmentation with complex-valued units (model trained to predict image and segmentation mask), we could cluster the output phases, but the training signal supported meaningful information in the output phases. We are continuing to try to understand why phase information is lost at the output in the fully unsupervised setting, but one reason is likely that a unidimensional phase representation is too restrictive. In support of this conjecture is the fact that, in RF, Lowe et. al [25] have incorporated multiple ‘phase’ (rotation) components for each neuron. Our alternative to this, is to use the high-dimensional phases from latent features. We will comment on these aspects in the updated version. \n\n``` The color-lesion experiment on CLEVR has potential low-level confounds apart from color (like shadows, shading, etc.) that models can use as shortcuts for binding. ```\n\nWhile it is true that there are these low-level differences in the image, this is far less of a cue than color signal. Colors can be accessed directly from the pixel values whereas these other cues require integration of multiple pixels. In short, the color cues are a shortcut feature that has high availability (Hermann et al. 2024) because individual pixels can specify the output grouping. Removing these cues (shortcut features) allows us to evaluate how much the other models rely on them (and to show that ours does to a lesser extent). Additionally, shadows make it perhaps even harder to do correct binding, and shading inside one object (e.g. for shiny metallic objects) arguably also makes the task harder.\n\nOn the Foundations of Shortcut Learning, Hermann et. al 2024.\n\n``` Why did the authors have to retrain SynCx on the grayscale dataset? If binding-by-synchrony is faithfully implemented by their model, shouldn't this work zero shot? If this is not the case, then what is the rationale for it? ```\n\nThe experimental setup as suggested by the reviewer would introduce a sort of test-time out-of-distribution evaluation that challenges all object-centric models, as illustrated by the fact that SlotAttention [33] which performed similar grayscale experiments required retraining from scratch (see Figure 4 in [33]). Since the statistics of features belonging to any object have changed due to the grayscale transformation it is reasonable to expect the previously learned binding mechanisms on the colored images requires retraining the model weights. As such, it would be quite unreasonable to expect current state-of-the-art synchrony or slot-based approaches (eg. SlotAttention [33]) to zero-shot be able to group grayscale versions if it was only trained on RGB counterparts. Lastly, this experiment on grayscale CLEVR is designed to test whether the models have the capability of leveraging other image statistics other than color. And we show that our model (SynCx) significantly outperforms previous synchrony-based models. \n\n``` The authors raise a point about instance-level groups being the focus of object-centric learning (L59). Given that, there are very few (maybe 2) examples presented where the scene contains multiple instances of the same object (same color, size, orientation, etc.) and the corresponding outputs from SynCx. Can the authors present these? ```\n\nThe point about instance-level groups was intended to highlight a commonly observed failure mode of state-of-the-art baseline model RF [25] which groups all instances of a semantic category such as ‘cow’, ‘train’, ‘bread slice’ or ‘banana’ as one group. This can be seen in Figures 2 & 20 in RF [25] where it groups different cows or bananas or trains as one entity. Note here, that the two cows or bananas or two bread slices are not exact copies (replicas) rather these object instances only partially share some features such as color, texture, shape as they belong to same semantic category. We can also see this failure mode of RF surfaces again on our datasets when it fails to disambiguate between two blue tetris blocks or two yellow cylinders in Figure 3. It primarily seems to use color as a shortcut feature to discriminate between groups on these datasets leading to systematic grouping errors. This failure mode of RF is exacerbated on Tetrominoes leading to very poor ARI scores (in Table 1). \n\nIn contrast, we see that our model is capable of disambiguating the separate entities in groups of two or three tetris blocks with same color and texture (row 2, 3 in Figure 7) or two hearts (row 2 in Figure 4 and row 1 in Figure 8) or three squares (row ) or two red balls (row 2 in Figure 9) or two blue cubes (row 3 in Figure 9). \n\nend of part 1/3", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ooFumMNajV", "reviewer": "Reviewer_g2JM", "summary": "Summary:\n\nThe authors introduce SynCx, a fully convolutional recurrent complex-valued autoencoder (AE) designed for unsupervised object discovery. SynCx operates by iteratively refining a randomly initialized phase array, which is the same shape as the input image, to reconstruct the input image. At each stage, the AE decoder produces a complex-valued output, where the magnitude represents the reconstructed image and the phase encodes object binding information. The decoder output is computed as a function of the input image, gated by the most recent phase (binding state). The model is trained using pixel-wise reconstruction error. Experimental results demonstrate that SynCx excels at unsupervised object discovery in comparison to relevant compared baselines and is especially good at binding in the presence of conflicting stimuli with identical low-level features.", "questions": "- What happens if SynCx is run with more recurrent iterations than what was used during training? Is the model's binding encoding stable at RNN $t > t_{train}$, or does the object discovery performance degrade during this \"extrapolation\" phase?", "limitations": "The authors have adequately addressed limitations in this submission.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths:\n- The proposed SynCx model comprises of a notably simple and elegant architecture + training. A key advantage over other unsupervised object discovery techniques is that SynCx does not rely on a fixed number of discrete object slots or buckets, allowing it to scale more flexibly to scenes with varying numbers of elements.\n- The experimental results, presented with standard deviation, demonstrate the effectiveness of SynCx in unsupervised object discovery, outperforming relevant baselines (Table 1).\n- The authors provide valuable insights through ablation studies on the bottleneck, number of iterations, and phase initialization distribution, validating the chosen configuration of SynCx.\n- The writing is clear and concise, and the figures are well-organized and easy to understand, making the submission a pleasure to read and greatly contributing to its overall clarity.", "weaknesses": "Weaknesses: \n- The paper's illustrations primarily demonstrate SynCx's performance on scenes with non-overlapping objects or overlapping objects with distinct low-level features (e.g., Figure 4 in dSprites). However, it is unclear how well SynCx would perform on scenes with overlapping objects that share similar low-level features (e.g., same color or shape). Specifically, would the phase representation effectively encode the presence of multiple distinct objects at overlapping pixels, mimicking human vision's ability to separate them?\n- The paper does not provide clear guidance on determining the optimal number of recurrent iterations for running SynCx, particularly in the absence of ground truth object identity information. For instance, if the number of objects in the scene increases significantly during test time, would SynCx still perform effectively with the same number of iterations used during training, where the number of objects was lesser?\n- While SynCx shows promise, it is uncertain whether the approach would generalize to binding on naturalistic stimuli. This limitation is shared with existing baselines too, but addressing this gap in future work would be a valuable extension of SynCx.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive comments and support for the simplicity and elegance of our model/training, quality of our experimental validation/analysis and the overall presentation. We’re happy that the reviewer enjoyed reading our work.  \n\n``` However, it is unclear how well SynCx would perform on scenes with overlapping objects that share similar low-level features (e.g., same color or shape). Specifically, would the phase representation effectively encode the presence of multiple distinct objects at overlapping pixels, mimicking human vision's ability to separate them? ```\n\nThis is an interesting observation about the phase representation. In Figure 9 (row 3) you can see two blue cubes (same shape / similar color) which partially occlude each other and our model has achieved good grouping of the two objects. Looking at the clustered phase maps (‘Mask’ column 4) the discretized phase at the overlapping regions encodes the presence of the object in the front and does not encode the occluded object as well. Looking at the t-SNE reduced phase map (‘Heatmap’ column 6) also we see that overlapping regions encodes the presence of the object in front (shades of blue) and does not encode the occluded object in the back (shades of purple). \n\nAnother example, in Figure 9 (row 1) you can see a yellow cylinder and purple cube both metallic (same texture) which partially occlude each other. Again we see the same behavior in the phase representation where in the overlapping regions has specialized towards the object in front rather than the occluded object at the back. \n\n``` if the number of objects in the scene increases significantly during test time, would SynCx still perform effectively with the same number of iterations used during training, where the number of objects was lesser? ```\n\nThat’s an interesting question. We suspect that what matters more for convergence is the density of objects. Because of the convolutional architecture, patches are processed in parallel, and more objects won’t necessarily be harder to resolve if they are dispersed in the scene. We would like to investigate convergence time as a function of number and density of objects, but unfortunately the rebuttal period was too short to provide you with a concrete answer.\n\n``` While SynCx shows promise, it is uncertain whether the approach would generalize to binding on naturalistic stimuli. This limitation is shared with existing baselines too, but addressing this gap in future work would be a valuable extension of SynCx. ```\n\nWe share the reviewers view on the scalability of this class of models to more visually complex scenes. We have shown the current limits of these synchrony-based models even on simple datasets like CLEVR which is far from perfect decomposition (ARI score of 1.0). We hope to continue research on these synchrony-based models to bridge this gap in future work. We view the contributions in this work as an initial template towards this end goal due to the simplicity of the model and its training procedure. \n \n``` What happens if SynCx is run with more recurrent iterations than what was used during training? Is the model's binding encoding stable at RNN 𝑡>𝑡𝑡𝑟𝑎𝑖𝑛, or does the object discovery performance degrade during this \"extrapolation\" phase? ```\n\nTable below shows the object discovery performance for our model on dSprites when trained with 3 iterations and evaluated using 4, 5 and 6 iterations. We can see that there is no degradation in object discovery performance of our model when we extrapolate the number of iterations used at test time. \n\n| No. of Iterations | ARI          |\n|-------------------|--------------|\n| 4                 | 0.81 +- 0.01 |\n| 5                 | 0.82 +- 0.01 |\n| 6                 | 0.82 +- 0.01 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ryXE5ALcOY", "reviewer": "Reviewer_nb8Q", "summary": "The paper introduces SynCx, a fully convolutional auto-encoder with complex values that is applied iteratively. This recurrent model takes advantage of the ability of complex-valued weights to not only compute features but also bind similar features that can be encoded in the phase, creating a grouping effect. This approach enhances previous attempts at using synchrony to solve the binding problem by employing recurrence to propagate local knowledge. This explains its success in circumventing previous issues with synchrony that relied on shortcuts such as color to solve the problem.", "questions": "Maybe a minor question, in figure 4 there is a visualization of the iterative process, showing how every iteration seem to provide a better grouping result. However, in the phase step is not clear to me how every step is contributing to the propagation of the local context.", "limitations": "Maybe good to add a sentence clarifying the limitations when visualizing the process.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper offers a clear and precise introduction that allows the user to understand the method but also the rationale of the choices taken. The method albeit simple, seems to provide an effective way to advance the state of the art of the binding problem. There are qualitative and quantitative evaluations that well support the main claims of the paper.", "weaknesses": "It has been shown that TSNE have some limitations, specifically results can change dramatically based on the choice of iterations and perplexity. How were these values selected? Also perhaps good using another dimensionality reduction technique to perform the visualization such as  UMAP.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive comments regarding the simplicity of our model, the ease of understanding the rationale behind our design choices and for noting that the evaluations support the main claims in our paper.  \n\n``` It has been shown that TSNE have some limitations, specifically results can change dramatically based on the choice of iterations and perplexity. How were these values selected? Also perhaps good using another dimensionality reduction technique to perform the visualization such as UMAP. ```\n\nWe thank the reviewer for these helpful recommendations regarding our phase visualization process. Regarding the t-SNE hyperparameters used (listed in lines 564-566) we tried out a few values for n_iters=[250, 500, 1000] and perplexity=[15, 20, 30] in the prototyping phase. We settled on the values of n_iter=500 and perplexity=20 for the visualizations shown in the paper since they gave good initial results (visual inspection) within a reasonable amount of processing time per image. During the prototyping phase, we compared UMAP against t-SNE for performing dimensionality reduction in our visualization process (download PDF from https://file.io/gMEMsZWRKNZX). We used the default UMAP hyperparameters and t-SNE hyperparameters listed above for this comparison. We find that t-SNE produces qualitatively better projections than UMAP, i.e., phase clusters correlate better to the number of objects in the image (see panels 2,3 and 4). Lastly, the visualization process takes around 8 seconds per image with t-SNE compared to 24 seconds per image for UMAP. Therefore, we decided that t-SNE is the better choice for our use case here. \n\n``` Maybe a minor question, in figure 4 there is a visualization of the iterative process, showing how every iteration seem to provide a better grouping result. However, in the phase step is not clear to me how every step is contributing to the propagation of the local context. ```\n\nThank you for your question. We agree with the reviewer that it is difficult to visually interpret from sequences of phase maps and radial plots to see how local context gets propagated. The illustrative example in Figure 1 was meant to serve as an idealized mode of operation of such a system. Regarding a way to visually show how local context is propagated. We think a possible approach would be to visualize/inspect local activation patches to show some measure of ‘agreement’ in their phases across successive iterations. Currently, we do not know a simple visualization method to visually show this phenomenon. We are curious to know if the reviewer has some recommendations for the same.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ULr9OUbo2p", "reviewer": "Reviewer_sSNp", "summary": "The paper aims to solve the problem of unsupervised object segmentation, which can be viewed as a feature binding problem when feature detectors process images at different locations in parallel. It proposes an improvement to synchrony-based approach which uses complex numbers as activation value for neurons: the weights are also complex numbers. In addition, the model processes images in recurrent way and let phases for neurons at different locations of an object incrementally group together. It also finds that a representational bottleneck in an auto-encoder architecture is required for the model to work.", "questions": "In the demo for dSprite in Fig 4, it appears that the phases are already well separated after the 1st iteration, but got more spread out in the second iteration and got back better in the third iteration. So I wonder whether such fluctuation in clustering is common.", "limitations": "The authors properly addressed an important limitation of the model, although I think there are some more as I wrote in the weakness. There is no negative societal impact as I can see.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper illustrated superior or competitive performance on several common datasets against other synchrony-based binding algorithms for unsupervised object segmentation.\n\nThe design of complex weights are well motivated and makes perfect sense for a reader after reading it.\n\nIt gets rid of contrastive training, $\\chi$-binding, and other tricks used by similar models.", "weaknesses": "I appreciate that the authors acknowledges that the model still relies on color cues to group pixels to a large degree, which is true to previous models such as RF, and its limitation in dealing with metallic or matte texture. It is possible that this is a fundamental problem for unsupervised object detection model that overly relies on a bottleneck to group pixels, as shown by https://arxiv.org/abs/2403.03730. Color is simply the easiest cue when a bottleneck is enforced, but there is no drive for such a model to learn cues relevant to real-world scenarios with complex lighting conditions and textures. Therefore, it is unlikely for solution proposed in the current model to be an ultimate solution for object perception. One may even ask whether the performance is mainly gained by the bottleneck or by the phase synchrony being proposed. To answer this, it seems that a comparison with earlier models not in the synchrony-based binding family, such as MONet or slot-attention might provide a clue. But such comparison is unfortunately not provided.\n\nNonetheless, I think the complex weight idea is a good addition to the subdomain of synchrony-based binding models. \n\nAnother limitation is that the detection of object relies on post-hoc clustering of phase. So it remains to be seen how it works when there are many objects, i.e., whether the $2\\pi$ space of phase is sufficient to separate different objects.\n\nLastly, if we think about the brain, especially of animals without language, their ability of learning object detection in unsupervised way should ultimately help them develop some sort of invariance: the same object should be represented similarly across different situations such that a squirrel will know that a new acorn is similar to other acorns and should still edible, unlike a plastic bottle. Although the invariance has been less focused by most of the object-centric models, it can be interesting to evaluate whether the phase is similar for similar object across images.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their support of the motivation behind the use of complex weights and its rationale. We thank them for their support of the simplicity of our model which eliminates the need for contrastive training, $\\chi$-binding and other heuristics.\n\n``` I appreciate that the authors acknowledges that the model still relies on color cues to group pixels to a large degree, which is true to previous models such as RF, and its limitation in dealing with metallic or matte texture. ```\n\nWe thank the reviewer for the positive comments about the honesty in our evaluation of current state-of-the-art synchrony models. We would like to emphasize that our method (SynCx) relies far less on solely color cues compared to current state-of-the-art synchrony models such as RF (see lines 285-287). \n\n``` Another limitation is that the detection of object relies on post-hoc clustering of phase. So it remains to be seen how it works when there are many objects, i.e., whether the  space of phase is sufficient to separate different objects. ```\n\nIn our experiments we show that our model is capable of handling up to 5 or 6 objects (see Figure 8 row 3 and Figure 9 row 4). Note, that the phase space is still not fully covered indicating that it could potentially accommodate more objects. We will comment on this as well in the limitations section. \n\nRegarding the post-hoc phase clustering, this ability of synchrony-based models to maintain a continuous phase representation for binding adaptable by gradient-based learning can be viewed as a strength compared to slot-based models where the binding information, i.e. number of slots and slot capacity are set apriori and not adaptable by gradient-based learning. This makes it difficult for slot-based models to adaptively change the coarseness of grouping at test-time without more fine-tuning or even re-training from scratch. Whereas our model (and synchrony-based models in general) maintain a continuous representation of object bindings adaptable by gradient-based learning. And we can post-hoc get different groupings with varying levels of granularity by modulating the clustering procedure applied on the continuous phase map. This gives the phase representation produced by synchrony-based models greater flexibility in chunking up the input image.   \n\n``` it is unlikely for solution proposed in the current model to be an ultimate solution for object perception. One may even ask whether the performance is mainly gained by the bottleneck or by the phase synchrony being proposed. ```\n\nWe appreciate the comments from the reviewer. We agree with the reviewer that we don’t believe that the contributions presented here are the ultimate solution and we have not made such claims in our work either. Instead, we advocate for a simpler approach (lesser inductive biases and simpler training procedure) compared to all other synchrony-based models (CAE, CAE++, CtCAE and RF). \n\n``` it seems that a comparison with earlier models not in the synchrony-based binding family, such as MONet or slot-attention might provide a clue. But such comparison is unfortunately not provided. ```\n\nThank you for this suggestion. We acknowledge the gap in unsupervised grouping performance between current state-of-the-art synchrony-based models and slot-based approaches such as SlotAttention as noted by the reviewer. We will add the following object discovery results of SlotAttention to Table 1 in the updated version to provide more context. \n\n| Model         | Tetrominoes    | dSprites       | CLEVR          |\n|---------------|----------------|----------------|----------------|\n| SlotAttention | 0.995 +- 0.002 | 0.913 +- 0.003 | 0.988 +- 0.003 |\n\nIt is noticeable how the performance gap between recent synchrony-based models (esp. SynCx) and SlotAttention has been narrowing on Tetrominoes and dSprites. We will also discuss this performance gap between SlotAttention and synchrony-based models on CLEVR dataset in the limitations section. Synchrony-based models have received far less attention by the research community and are making strides very recently to bridge the gap to predominant slot-based models which have been widely improved by the community efforts over a longer timeframe. Therefore, we view synchrony-based models as an exciting research direction with open problems to solve.\n\n``` Although the invariance has been less focused by most of the object-centric models, it can be interesting to evaluate whether the phase is similar for similar object across images. ```\n\nThis is an interesting question, regarding representational invariance for an object across viewpoints. When using synchrony-based models, what matters for separating objects is the relative phase (phase difference) between object phases rather than the absolute phase values assigned to pixels of any particular object. In fact, it’s plausible that different objects could take the same phase value (eg. $\\dfrac{\\pi}{3}$) across different iterations or when starting with different random initializations for the phase map. In the reviewer’s example different views of the acorn are not encoded by some absolute phase value (eg. $\\dfrac{\\pi}{4}$) thereby leading to invariant representations of the acorn. \nLastly, we believe that this approach of using multiple views of an object for prediction provides useful training signals for an object-centric model. However, this is outside the focus (scope) of the research questions addressed here and an interesting avenue for future work. \n\nend of part 1/2", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y0FiAW9JPL", "reviewer": "Reviewer_3hk1", "summary": "The authors propose a new way to obtain binding through complex-valued activations in an autoencoder for object discovery. They introduce recurrence in the generated phase map and perform the operations using complex weights. \nTheir model reaches competitive performance on a SOTA benchmark for unsupervised object discovery, including datasets such as Tetrominoes, Multi dSprites, and Clevr, in terms of reconstruction and object separation.", "questions": "- The example in Figure 1 is used to motivate the need for recurrence and complex-valued weights; have the authors tried to evaluate their model on similar stimuli to confirm that it can group the objects accordingly?\n\n- The recurrent dynamic is done only on the phase: at each iteration t, only the phase at t-1 is fed as an input, and not the magnitude (the magnitude remains the input image at each iteration). Have the authors tried a closed-loop recurrence where the whole complex output at iteration t-1 constitutes the input at iteration t? This solution would seem more elegant and more in line with neuroscience as it is unlikely that the phase and firing rate of each neuron can be split in such a way. \n\n- The initial modReLU equation seems to include a normalization of the amplitude by z/|z|. Do the authors include it as well, and if yes, can they add this term to equation (2)? If not, can the authors explain why not include this term? Also, is there any constraint applied to the bias to prevent the amplitude from being negative? If not, do the authors make sure the amplitude remains positive or switch the phase value accordingly?\n\n- Table 4 shows the benefit of von-Mises initialization compared to Uniform. Have the authors tried to initialize the phases at zero, as done in the CAE?\n\n- The colors in the polar plot visualization are supposed to reflect the phase value matched with the colors in the phase maps. However, why do some phases of similar angles have different colors? For example, in Fig 8, row 5, the yellow phases are represented close to angle 0, but in row 2 this same color represents phases close to -pi/2. Is it possible to use a standard colormap applied to each plot to visualize the phase distribution and compare between images?\n\n- The t-SNE to visualize the phase map first ensures that the magnitude of the activity is set to 1 (L562). However, the result from the clustering exhibits an activity with magnitude differing in their amplitude value. Why is this the case?", "limitations": "The limitations focus only on the reliance of SynCx on color cues, even though this model performs better than the baseline in this experiment.\n\nSome additional limitations should be included in the limitation section:\n- Even though this model does not need an additional objective to perform binding, it employs twice as many parameters as the existing baselines. Additionally, the number of flops is also doubled and then multiplied by the number of iterations needed. This results in a model requiring significantly more computational resources than the baselines, to reach performance almost on par with RF and CtCAE. \n- The weaknesses of the evaluation procedure should also be mentioned in the limitation section.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is very clearly written and nice to read.\n\nThe authors propose an interesting addition to existing complex-valued autoencoders in the recurrent dynamic of the phases. This approach is well motivated by neuroscience and shows a clear improvement in the object separation metrics. \n\nThe control experiment on greyscaled images highlights an interesting bias in existing models, which seems partially alleviated in the current model.", "weaknesses": "**The authors need to provide a more solid justification for the use of complex weights:** The previous models use real weights, as justified by Reichert & Serre, 2017 to align with neuroscience evidence. Indeed, the conjunction of multiplication by real weights followed by a sum of the complex activity is suggested as a natural inductive bias for synchrony: a complex addition between complex numbers with desynchronized phases will result in reduced activity in the amplitude. Conversely, in-phase activity will be enhanced, serving as an elegant way to reach synchrony in the population. \nApplying complex weights will give more degrees of freedom to modify the phase value of each neuron, but it leads to less control in terms of resulting phase value: they can be potentially shifted to any angle, even opposite to their initial values. This behavior does not seem to align with neuroscience evidence, contrary to what the authors claim.\n\nRelated to these points, the authors claim, in lines 86 and 96,  that complex weights are *critical* to detect particular phase arrangements. But this behavior can potentially also be obtained with real-valued weights. In fact, the weights are applied to the real and imaginary parts of a complex activity. The phase arrangement is therefore present in the activity, and can also play a role in the detection of such features. This observation is even more valid when a specific phase bias is applied separately to the phase after the convolution, as proposed by Lowe et al, 2022. \n\n**The evaluation pipeline of object assignment and phase map visualization differ and highlight a lack of good binding abilities:** The recurrence on the phases dynamic is supposed to enhance phase separation at the last iteration and in the last layer of the decoder. However, the authors mention that the object assignment (L177) and the phase map visualization (L212) come from *an appropriate/intermediate layer of the decoder*. Does this mean that the phases of the last layer of the decoder do not exhibit grouping properties? And for each model and dataset, do the object assignment and the phase map result from the same layer, or can they result from different layers even though they are from the same model and dataset? And why do the clustering metrics differ between object assignment (k-means) and phase map (t-SNE)? These results should be at the center of the binding abilities of the model, and all these differences seem to highlight a lack of clear ability of the model to perform such grouping. Ideally, all the metrics should be computed at the last iteration and in the last layer of the decoder. But if there is confound related to a color bias in the amplitude, the object assignment and the phase map should at least results from the same clustering algorithms and the same decoder layer.\nThis choice of layer is particularly critical since the grouping metrics are applied on the last layer of the decoder for all other baseline models. \n\n**Are the baselines 100% comparable with SynCx?** The introduction of complex weights doubles the number of parameters compared to baseline models using real weights. Are the baseline models still comparable in terms of number or parameters? If not, how could the authors explain that the MSE loss of SynCx is always higher compared to RF and CtCAE (except on dSprites)?\nAlso, Slot-based models are mentioned in the related work but their performance on this benchmark is not included in the results. It would also be interesting to evaluate whether they might also be biased towards colors by reproducing the results of Table 5 with this family of models.\n\n**Minor weaknesses**:\n- Missing citation: even though modReLU was mentioned in Trabelsi et al, 2018; the modReLU was initially proposed by Arjovsky et al., 2015.\n- The authors use the term \"robustness\" (L55) to express the excessive reliance of models on color cues. I'd suggest replacing this term as robustness represents something different in the DL literature.\n- L100: parentheses missing after citation 24.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback on our work and appreciate the positive feedback on the quality of our presentation. \n\n``` But this behavior can potentially also be obtained with real-valued weights. In fact, the weights are applied to the real and imaginary parts of a complex activity. The phase arrangement is therefore present in the activity, and can also play a role in the detection of such features. ```\n\nPlease note that simply using shared real-valued weights and applying it to compute matrix-vector products with real and imaginary parts of a complex-valued activation followed by summation alone does not lead to phase synchronization in the complex-valued activations in previous models (CAE++, CtCAE and RF). Phase synchronization in these models crucially relies on the use of an additional inductive bias i.e. the gating mechanism ($\\chi$-binding) as noted by us (see lines 46-50) and echoed by Loewe et. al [23, 25]. Please refer to Table 2 in [23] and Appendix section D.3 in [25] where they perform an ablation on CAE and RF models without the $\\chi$-binding mechanism. This ablated model variant fails to phase synchronize towards objects. Lastly, regarding our usage of the word “critical” we mean that the use of complex-valued weights is a natural choice to instantiate this function in our formulation of synchrony outlined (Introduction section). We do not mean that this is the only way to achieve phase synchronization behavior in general. We’re happy to adjust the corresponding text to avoid any misconceptions.\n\n``` This behavior does not seem to align with neuroscience evidence, contrary to what the authors claim. ```\n\nWe would like to clarify that our work does not claim that the SynCx model design and training procedure are biologically plausible. Instead we motivate the use of complex-weights and recurrence from a computational perspective as follows.\n \nUsing real valued weights allow you to activate a hidden feature if two input features are aligned in phase, but you may wish to activate a hidden feature that indicates that the two input features should be grouped apart (e.g., 180 degrees out of phase).  This is a principled computational motivation for using complex weights. As for whether complex valued weights have biological plausibility, it's easy to think of mechanisms that effectively behave like complex-valued weights.  The critical mechanism is to detect features operating at different relative phases, which in the biological case, will correspond to different firing times.  Dendritic tree connectivity can implement synchrony that is achieved at arbitrary phase lags. We don't emphasize this in our paper (and certainly can, if the reviewer would find it helpful) because we are focused on showing the computational improvements and elegance that results from complex-valued weights. We hope this resolves any misinterpretations of our motivations. \n\n``` And for each model and dataset, do the object assignment and the phase map result from the same layer, or can they result from different layers even though they are from the same model and dataset? ```\n\nPlease note, that for grouping performance (ARI scores) of our model in all Tables (across all datasets) in the paper, we always cluster the phases from the penultimate decoder layer at the last iteration N. Similarly, for all phase map visualization we apply t-SNE to perform dimensionality reduction to the d-dimensional phases from the penultimate decoder layer. In Figure 4, we simply showed how these phases evolve across iterations. The visualization of phase maps (Figure 4 and associated text) is simply a qualitative tool for inspection. We do not use it to make any claims about the grouping performance of our model against baselines. We will edit the corresponding lines in text to make this point clear and avoid any confusion.  \n\n``` And why do the clustering metrics differ between object assignment (k-means) and phase map (t-SNE)?```\n\nPlease note, that the visualizations of phase maps (Figure 4) are simply the 1D projections (dimensionality reduction done using t-SNE) of 64-D continuous phase vectors at each location in the spatial feature map. The visualization procedure does not apply any clustering procedure. The 1-d projected phase maps shown in Figure 4 are not similar to the discrete object segmentation masks (like columns 2, 4, 6 in Figure 3).  \n\n``` Slot-based models are mentioned in the related work but their performance on this benchmark is not included in the results. ```\n\nThank you for this suggestion, this was noted by another reviewer as well. We will add the following object discovery results of SlotAttention to Table 1 in the updated version to provide more context. Slot-based models do not show this color bias on these datasets tested here.\n\n| Model         | Tetrominoes    | dSprites       | CLEVR          |\n|---------------|----------------|----------------|----------------|\n| SlotAttention | 0.995 +- 0.002 | 0.913 +- 0.003 | 0.988 +- 0.003 |\n\nWe will also discuss this performance gap between SlotAttention and state-of-the-art synchrony-based models in the limitations section. \n\n``` Missing citation: even though modReLU was mentioned in Trabelsi et al, 2018; the modReLU was initially proposed by Arjovsky et al., 2015. ```\n\nWe thank the reviewer for this correction, we will update the citation as suggested. \n\n``` The authors use the term \"robustness\" (L55) to express the excessive reliance of models on color cues. I'd suggest replacing this term as robustness represents something different in the DL literature. ```\n\nWe thank the reviewer for pointing this out. We will use another term to avoid any misinterpretation. We would like to ask if the reviewer has any suggestions for alternatives?\n\n``` L100: parentheses missing after citation 24. ```\nWe thank the reviewer for this correction and will update it. \n\nend of part (1 / 3)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Please find attached a PDF containing samples of the UMAP v/s t-SNE comparison for dimensionality reduction in our phase map visualization process.

Meta-review: This paper proposes a novel approach for unsupervised object discovery using a recurrent model with complex-valued weights that is validated against several standard benchmarks. The use of recurrence and complex-valued operations makes this paper potentially interesting to neuroscientists and cognitive scientists who seek to understand how temporal dynamics of neural populations might lead to synchrony-based operations that support cognition. Eventually, insights about binding-by-synchrony could also support the development of a new class of deep neural networks for complex sensory and cognitive tasks.

These factors led to three of the reviewers giving this paper above-threshold scores. However, there was also sentiment that the contributions of this paper were (1) not significant, and (2) not clearly translatable to neuroscience, cognitive science, or AI. In terms of the contributions of the paper, one of the reviewers noted that the effect of recurrence in this model is not clear. Indeed, the authors wrote in their introduction that the recurrence of the model is a key advancement beyond prior works, but the impact of recurrent steps of processing is minimal (performance caps out after 2 or 3 iterations). Moreover, the impact of complex valued units should in principle be more interpretable representations by multiplexing different computations in the phase vs. magnitudes of units. However, the authors note that directly interpreting phase information can be challenging at the output of the model. I will also note that the fact that the authors did not provide source code with their submission makes it more challenging to directly evaluate claims of interpretability or the effect of recurrence.

Overall, this is a paper with interesting ideas and potential impact by advancing the state-of-the-art on unsupervised object discovery, albeit on toy datasets. There is however a very limited impact on the cognitive sciences or AI more broadly, so I recommend the authors dial back that rhetoric. This is a true borderline paper that the reviewers were split on and could not resolve their issues despite a vigorous discussion period, but given two of the reviewers saw high potential impact, I lean towards accepting this.
Decision: Accept (poster)