SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Augmentation effects We provide the canonical orientation predicted by Learned Canonicalization (LC) [30] for Rotated MNIST [52] at the start of the training and after training along with original images. The progression of canonized images at the start and after training demonstrates the diminishing effect of augmentation effects when trained with learned canonicalization.
Effect of prior regularization. We present a comprehensive analysis of the output distribution over 8 discrete angles predicted by the canonicalization function, both before and after training, on the test set. These findings are depicted in Figure 4 for Learned Canonicalization (LC) [30], and Figure 5 for Prior-Regularized LC. Here, the numbers 0 to 7 correspond to angles that are multiples of 45 degrees, ranging from 0 to 315 degrees, respectively.
We demonstrate that incorporating prior regularization into the canonicalization function aids in mapping the images to the identity prior (represented by 0 angle). This improvement positively impacts the accuracy on the original test set, as evidenced by the results in Table 2. Conversely,
Original Images
Canonized images at the start of the training
Figure 3: Visualization of the diminishing augmentation effect introduced by learning canonicalization [30] during training for rotated MNIST dataset. In this visualization, the leftmost image represents the original training images. Moving towards the center, we present the canonized images at the beginning of the training process. Finally, the rightmost image unveils the transformation of the canonized images after training the model for 100 epochs.
Canonized images after training
relying solely on the classification task loss yields no significant alteration in the angle distribution, as the post-training test set distributions remain random.
Additionally, we provide valuable insights into the fraction of images mapped to the identity element in Table 5. It is important to note that the expressivity of the canonicalization function, specifically employing lightweight equivariant networks, contributes to the inability to map all images to the identity elements. This observation calls for further exploration in understanding the role of expressivity and generalization within canonicalization networks with known prior orientations.
Table 5: Fraction of images mapped to identity.
Training Completed
Dataset ↓	Model	×	✓
CIFAR10 [31]	Learned Canonicalization (LC) [30]	0.11	0.23
Prior-Regularized LC (ours)	0.11	0.76
Distribution of angles at the start of training
Figure 4: Distribution of angles output from canonicalization function in  $C8$  for Learned Canonicalization [30] for CIFAR10 [31] before and after training. We use indices on the  $x$ -axis instead of angle values to represent the corresponding multiple of  $45^{\circ}$ . Frequency denotes the number of images mapped to a particular multiple of  $45^{\circ}$ .
Distribution of angles after training
Distribution of angles at the start of training
Distribution of angles after training
Figure 5: Distribution of angles output from canonicalization function in  $C8$  for Learned Canonicalization [30] with prior regularization for CIFAR10 [31] before and after training. We use indices on the  $x$ -axis instead of angle values to represent the corresponding multiple of  $45^{\circ}$ . Frequency denotes the number of images mapped to a particular multiple of  $45^{\circ}$ .
D Optimization challenges on continuous 2D rotations for Images
We investigate the prior introduced in eq. (7) specifically for images. To achieve this, we employ an E-2 steerable network [34] to devise a canonicalization function that generates rotation matrices. Our approach involves initially generating two vector fields, which are subsequently averaged across the spatial dimension to obtain two vectors. By performing Gram-Schmidt orthonormalization on these vectors, we derive a rotation matrix.
In Figure 6, we present the distributions of predicted angles ranging from  $-180$  to  $+180$ . Through our analysis, we observe that the mean and standard deviation of the predicted angles on the test set are  $-0.54$  and  $80.23$ , respectively. This observation signifies that while the prior guides the canonicalization function to output angles close to  $0$ , there exist instances where angles significantly deviate from this central value. This phenomenon warrants further investigation and a complete understanding of this left as future work.
Distribution of angles at the start of training
Figure 6: Distribution of angles output from steerable canonicalization function in  $SO2$  for Learned Canonicalization [30] with prior regularization (eq. (7)) for CIFAR10 [31] before and after training.  $x$ -axis denotes angles from -180 to +180. Frequency denotes the number of images mapped to a particular angle.