SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 7: Hyperparameters for the Rotating Features models that are trained on the Multi-dSprites and CLEVR datasets.
Dataset	Multi-dSprites	CLEVR
Training Steps	100k	30k
Batch Size	16	64
LR Warmup Steps	5000	5000
Peak LR	0.001	0.001
Gradient Norm Clipping	0.1	0.1
Feature dim d	128	128
Rotating dim n	10	10
Image/Crop Size	64	128
Cropping Strategy	Random	Random
Augmentations	-	-
k	6	5
Table 8: Performance of Rotating Features and CAE* on the Multi-dSprites and CLEVR datasets. While the CAE* largely fails to separate objects in these datasets, Rotating Features achieve strong results in terms of ARI-BG and MBO $_i$ .
Dataset	Model	MSE ↓	ARI-BG ↑	MBOi ↑
Multi-dSprites	CAE*	3.540e-03 ± 1.990e-04	0.371 ± 0.056	0.426 ± 0.042
Rotating Features	7.082e-04 ± 1.173e-04	0.888 ± 0.015	0.863 ± 0.011
CLEVR	CAE*	1.191e-03 ± 2.998e-05	0.289 ± 0.042	0.234 ± 0.074
Rotating Features	6.643e-04 ± 5.564e-05	0.664 ± 0.013	0.608 ± 0.017
D.7 Additional Results
We provide supplementary qualitative and quantitative results for the experiments discussed in the main part of this paper. For the 4Shapes, 4Shapes RGB(-D), and 10Shapes datasets, additional qualitative results can be found in Figs. 17 to 19, with corresponding quantitative results detailed in Tables 9 to 11. Additional qualitative results for the Pascal VOC dataset are presented in Fig. 20, and extra results for the FoodSeg103 dataset are provided in Fig. 21. Lastly, we compare the performance of Rotating Features on the FoodSeg103 dataset using  $k$ -means and agglomerative clustering with various hyperparameters in Fig. 22.
Table 9: Detailed quantitative performance comparison on the 4Shapes (mean ± SEM across 4 seeds). As the number of rotation dimensions  $n$  in the Rotating Features increases, both reconstruction (MSE) and object discovery (ARI-BG and MBO $_i$ ) performance improve. This allows Rotating Features to significantly outperform the standard autoencoder (AE) and Complex AutoEncoder (CAE).
Model	n	MSE ↓	ARI-BG ↑	MBOi ↑
AE	-	5.492e-03 ± 9.393e-04	-	-
CAE	-	3.435e-03 ± 2.899e-04	0.694 ± 0.041	0.628 ± 0.039
Rotating Features	2	2.198e-03 ± 1.110e-04	0.666 ± 0.046	0.589 ± 0.026
3	1.112e-03 ± 2.698e-04	0.937 ± 0.018	0.861 ± 0.020
4	5.893e-04 ± 4.350e-05	0.944 ± 0.016	0.908 ± 0.012
5	5.439e-04 ± 6.984e-05	0.975 ± 0.003	0.934 ± 0.006
6	2.526e-04 ± 1.416e-05	0.991 ± 0.002	0.970 ± 0.003
7	1.642e-04 ± 1.810e-05	0.992 ± 0.002	0.974 ± 0.003
8	1.360e-04 ± 7.644e-06	0.987 ± 0.003	0.968 ± 0.008
Table 10: Detailed results on the 4Shapes RGB(-D) datasets (mean ± SEM across 4 seeds). As we increase the number of potential colors that objects may take on, the performance of Rotating Features worsens on the RGB dataset. However, this effect can be mitigated when providing depth information for each object (RGB-D).
Dataset	Number of colors	MSE ↓	ARI-BG ↑	MBOi ↑
RGB	1	1.017e-05 ± 2.541e-06	0.982 ± 0.009	0.874 ± 0.008
RGB	2	1.597e-04 ± 2.185e-05	0.726 ± 0.035	0.595 ± 0.015
RGB	3	1.044e-04 ± 6.547e-06	0.850 ± 0.008	0.767 ± 0.016
RGB	4	1.166e-04 ± 5.985e-06	0.682 ± 0.041	0.626 ± 0.050
RGB	5	1.653e-04 ± 3.065e-05	0.494 ± 0.056	0.474 ± 0.031
RGB-D	1	4.920e-06 ± 5.421e-07	1.000 ± 0.000	0.875 ± 0.015
RGB-D	2	4.693e-05 ± 2.166e-05	0.996 ± 0.004	0.798 ± 0.007
RGB-D	3	2.436e-05 ± 3.930e-06	0.936 ± 0.029	0.785 ± 0.017
RGB-D	4	4.562e-05 ± 4.567e-06	0.909 ± 0.039	0.759 ± 0.013
RGB-D	5	8.850e-05 ± 3.780e-05	0.922 ± 0.040	0.727 ± 0.014
Table 11: Detailed quantitative results on the 10Shapes dataset (mean ± SEM across 4 seeds). When  $n$  is sufficiently large, Rotating Features learn to separate all ten objects, highlighting their capacity to separate many objects simultaneously.
n	MSE	ARI-BG	MBOi
2	1.672e-02 ± 7.893e-04	0.341 ± 0.030	0.169 ± 0.013
10	1.918e-03 ± 1.315e-04	0.959 ± 0.022	0.589 ± 0.039
Figure 17: Qualitative performance comparison on the 4Shapes dataset. Rotating Features with a sufficiently large rotation dimension  $n$  create sharper reconstructions and significantly better object separations, despite similar numbers of learnable parameters.
Figure 18: Rotating Features applied to the 4Shapes RGB(-D) datasets. While the model fails to create accurate object separations when applied to the RGB version of this dataset, it can accurately distinguish objects when given additional depth information for each object (RGB-D). Nonetheless, it tends to group one of the four objects with the background.
Figure 19: Rotating Features applied to the 10Shapes dataset. With a sufficiently large rotation dimension  $n$ , Rotating Features are capable to separate all ten objects in this dataset, highlighting their increased capacity to represent many objects simultaneously.
Figure 20: Rotating Features applied to the Pascal VOC dataset. By leveraging higher-level input features created by a pretrained vision transformer, Rotating Features can learn to separate objects in real-world images.
Figure 21: Rotating Features applied to the FoodSeg103 dataset. By utilizing input features generated by a pretrained vision transformer, Rotating Features can learn to distinguish objects in real-world images.
Figure 22: Clustering methods applied to Rotating Features after training on the FoodSeg103 dataset (mean ± SEM across 4 seeds). Since Rotating Features automatically learn to separate the appropriate amount of objects, we can test various clustering methods and hyperparameters efficiently after training. While agglomerative clustering (right) achieves slightly worse results than  $k$ -means (left), it eliminates the requirement to set the number of objects to be separated.