SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[70] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. "Mastering the game of Go with deep neural networks and tree search." Nature 529.7587 (2016), pages 484-489.
[71] Ilya Sutskever. An observation on Generalization. Simons Institute workshop on Large Language Models and Transformers. 2023. URL: https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023-08-14.
[72] Gábor J. Székely, Maria L. Rizzo, and Nail K. Bakirov. "Measuring and testing dependence by correlation of distances." The Annals of Statistics 35.6 (2007), pages 2769-2794.
[73] Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L Dyer, Remi Munos, Petar Velicković, and Michal Valko. "Large-Scale Representation Learning on Graphs via Bootstrapping." ICLR. 2022.
[74] Naftali Tishby, Fernando C Pereira, and William Bialek. “The information bottleneck method.” Allerton Conference on Communication, Control, and Computing. 1999. arXiv: physics/0004057.
[75] Aaron van den Oord, Oriol Vinyls, and Koray Kavukcuoglu. "Neural discrete representation learning." NeurIPS. 2017.
[76] Ankit Vani, Max Schwarzer, Yuchen Lu, Eeshan Dhekane, and Aaron Courville. "Iterated learning for emergent systematicity in VQA." *ICLR*. 2021.
[77] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. "Extracting and composing robust features with denoising autoencoders." ICML. 2008.
[78] Ronald J Williams. "Simple statistical gradient-following algorithms for connectionist reinforcement learning." Reinforcement Learning (1992), pages 5-32.
[79] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Genisses, Aneesh S Pappu, Karl Leswing, and Vijay Pande. "MoleculeNet: a benchmark for molecular machine learning." Chemical Science 9.2 (2018), pages 513-530.
[80] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. "How Powerful are Graph Neural Networks?" *ICLR*. 2019.
[81] Zhenlin Xu, Marc Niethammer, and Colin A Raffel. "Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language." NeurIPS. 2022.
[82] Hattie Zhou, Ankit Vani, Hugo Larochelle, and Aaron Courville. "Fortuitous Forgetting in Connectionist Networks." *ICLR*. 2022.
[83] Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. "Image BERT Pre-training with Online Tokenizer." ICLR. 2022.
Algorithm 1: Proposed IL-SEM algorithm
Split the network into  $h(\mathbf{x})$  and  $g(\mathbf{z})$ , then add an SEM bottleneck to discretize  $\mathbf{z}$   
for  $t = 0$  to  $T_{gen}$  do  
Initialize the student speaker  $h_t^S (\mathbf{x})^5$   
# Imitation Phase (start from the second generation)  
if  $t > 0$  then  
for  $i = 0$  to  $I_{imit}$  do  
Sample a batch  $\mathbf{x}$  from the training set  $\mathcal{D}_{train}$   
Sample the pseudo labels from teacher's prediction (one-hot vectors)  $\mathbf{g}_{1:m} \sim h_t^T (\mathbf{x})$   
Calculate the student's prediction  $\mathbf{z} = [\hat{z}_1, \dots, \hat{z}_m] = h_t^S (\mathbf{x})$   
Update  $h_t^S (\cdot)$  with multi-label cross-entropy loss  $\mathcal{L}_{ml} = \sum_{i=1}^{m} \mathbf{g}_i^\top \cdot \log \hat{z}_i$   
end for  
end if  
# Interaction Phase (regular training on the downstream task)  
for  $i = 0$  to  $I_{int}$  do  
Initialize the listener  $g_t(\mathbf{z})$  randomly  
Sample a batch  $(\mathbf{x}, \mathbf{y})$  from the training set  $\mathcal{D}_{train}$   
Calculate the downstream prediction  $\hat{\mathbf{y}} = (g_t \circ h_t^S)(\mathbf{x})$   
Update the parameters of  $h_t^S$  and  $g_t$  to minimize the downstream loss  $\mathcal{L}_{ds}(\hat{\mathbf{y}}, \mathbf{y})$   
end for  
The student becomes the teacher for the next generation:  $h_{t+1}^T \gets h_t^S$   
end for  
Return the last (or the best)  $(g_t \circ h_t^S)(\mathbf{x})$  for the downstream task
A The Ladder of Compositionality
Table 3: What information contained in  $\mathbf{z}$  on the ladder and their corresponding capabilities.  $H\left( \cdot \right)$  is the entropy,  $\Leftrightarrow$  means bijection,  $\Leftrightarrow$  means isomorphism bijection.
Infor. in z	Train acc.	ID-gen	OOD-gen with seen concepts	Comp-gen
Stage I	H(G | z) > 0	×	×	×	×
Stage II	H(G | z) = 0	✓	✓	×	×
H(z | G) > 0
Stage III	z ↔ G	✓	✓	✓	×
Stage IV	z ⇌ G	✓	✓	✓	✓
To figure out what  $\mathbf{z}$  we need in order to generalize well compositionally, we propose the "ladder of compositionality" in Figure 1. To justify our claims, this appendix will discuss how we could climb the ladder step by step by analyzing how the corresponding requirements are generated. A formal definition of the compositional mapping in terms of group theory, which is necessary for reaching the final stage of the ladder, is also provided. In short, we find only relying on the mutual information between  $\mathbf{z}$  and  $\mathbf{G}$  (or between  $\mathbf{x}$  and  $\mathbf{y}$ ) cannot reach the final rung of the ladder: we need other inductive biases, which is the main motivation of this paper.
A.1 Stage I: z misses some important information in G
The learned representation would have  $H(\mathbf{G} \mid \mathbf{z}) > 0$  at this stage. As  $Y = \mathrm{GenY}(\mathbf{G},\epsilon)$  and  $\hat{Y} = g(\mathbf{z})$  are assumed to be invertible (here  $Y$  and  $\hat{Y}$  are random variables), this condition can be rewritten as  $I(Y;\hat{Y}) < H(Y)^6$ . Hence following the analysis in [69], a model with such an encoder  $\mathbf{z} = h(\mathbf{x})$  even cannot achieve high enough training performance, let alone generalizing to unseen
In practice, we can choose to randomly initialize the speaker (usually when the model is small), copy the pretrained checkpoint (when the model is large), or copy the parameters of the teacher in previous generations (the seed iterated learning variant mentioned in [50]).
Using the fact that  $H(\mathbf{G} \mid \mathbf{z}) = H(Y \mid \hat{Y}) = H(Y) - I(Y; \hat{Y})$ .
test distributions. This condition might occur when the model underfits the training data, e.g., at the beginning of training or the model's capacity is too small. To make an improvement, one can increase the model size or train longer.
A.2 Stage II: z not only contains all information in G but also some in O
At this stage, the learned representation would have  $H(\mathbf{G} \mid \mathbf{z}) = 0$  and  $H(\mathbf{z} \mid \mathbf{G}) > 0$ . This means  $h(\mathbf{x})$  remembers additional information that  $\mathbf{G}$  doesn't have, e.g., noisy information in  $\mathbf{O}$ . From  $H(\mathbf{G} \mid \mathbf{z}) = 0$ , we know  $H(Y \mid \hat{Y}) = 0$  and hence  $I(Y; \hat{Y}) = H(Y)$ . Then the model would have perfect training performance and could also be able to generalize well when training and test datasets share the same distribution. However, when facing the out-of-distribution generalization problem, especially when a spurious correlation exists between some factors in  $\mathbf{O}$  and  $\mathbf{G}$ , the extra information learned by  $\mathbf{z}$  can mess the predictions up. Such a phenomenon is named "short-cut learning" and is quite common in many deep-learning systems [20]. For example, if the background strongly correlates with the object in the training set (e.g., a cow usually co-occurs with the grass while a seagull usually co-occurs with the beach), the DNN then tends to rely more on these "short-cut" features (e.g., the background rather than the object) during training. If such correlations disappear or reverse in the test set, the models relying on factors in  $\mathbf{O}$  cannot generalize well to a new distribution.
Making improvement - loss based on information-bottleneck To make an improvement, the model should eliminate the task-irrelevant information as much as possible. Based on this principle, authors of [74] propose to minimize the following information bottleneck equations:
$$
\max  I (Z; Y) - \beta I (Z; X), \quad \beta > 0, \tag {3}
$$
which means the learned  $\mathbf{z}$  should extract as much as information from  $Y$  (or equivalently,  $\mathbf{G}$ ) and forget as much as irrelevant information about  $X$  (i.e., those in  $\mathbf{O}$ ). This method is also widely applied in other relevant tasks, like domain adaptation [46], invariant risk minimization [1], and etc.
Making improvement – data augmentation Another simple and efficient way to make improvements is data augmentation: one can identify some task-irrelevant factors in  $\mathbf{O}$  and design specific data augmentation methods to teach the model to be insensitive to them. For example, if we believe that the label of an image should be irrelevant to color jittering, random cropping, rotation, flipping, etc., we can apply random augmentations during training and treat the differently augmented  $\mathbf{x}$  and  $\mathbf{x}^{\prime}$  as the same class. Then the model would inherently learn to be insensitive to such factors and hence forget the corresponding information. One interesting thing about data augmentation is that it can be designed and applied in a reverse direction, i.e., we can break some semantic factors and train the model to be insensitive to the broken samples. For example, believing the shape of the image and order of the words are semantic factors in  $\mathbf{G}$ , the authors of [56] propose to randomly rotate the image patches or words to make negative samples. Those models that perform well on such negative samples are more likely to rely on the factors in  $\mathbf{O}$ .
Making improvement - auxiliary task design, e.g., SSL Furthermore, one can also consider designing auxiliary tasks in addition to the downstream task, e.g., pretrain using self-supervised learning (SSL) and finetune on the target task. In [67], the authors empirically show that the representations learned via SSL usually generalize better than those learned via supervised learning when facing OOD downstream problems, even though the models are trained using a similar amount of data samples. There are also some works demonstrating that SSL representations encode more semantic information about the input image [10], which is a sign that auxiliary tasks like SSL can introduce extra biases that favor information in  $\mathbf{G}$ .
Consider the first group of SSL methods, which are usually based on contrastive loss, e.g., SimCLR [11], MoCo [30], etc. These methods usually require  $h(\mathbf{x}_i)$  and  $h(\mathbf{x}_i')$  to be similar while  $h(\mathbf{x}_i)$  and  $h(\mathbf{x}_j), i \neq j$  to be distinct, where  $\mathbf{x}_i$  is the anchor input,  $\mathbf{x}_i'$  is the augmentation of it, and  $\mathbf{x}_j$  is another different image. The carefully designed augmentation can encourage the model to ignore some task-irrelevant factors that belong to  $\mathbf{O}$ . Imagine  $\mathbf{x}_i'$  is generated by deleting the background of  $\mathbf{x}_i$ . As the training enforces  $d_z(h(\mathbf{x}), h(\mathbf{x}'))$  to be small, the learned model will then become insensitive to the information in the background, and hence avoid relying on this "short-cut" feature. Note that the contrastive SSL is utilizing the bias from data augmentation in a more aggressive way: the SSL algorithm will tell the model that  $\mathbf{x}$  and  $\mathbf{x}'$  are the same image while the supervised learning only informs the model that  $\mathbf{x}$  and  $\mathbf{x}'$  belong to the same class.
Another line of SSL is built on reconstruction tasks, like denoising auto-encoder (DAE [77]) and masked auto-encoder (MAE [29]). The  $h(\mathbf{x})$  in these methods is usually trained in an auto-encoder fashion: using a reconstruction network  $r(\mathbf{z}): \mathcal{Z} \to \mathcal{X}$ , we require the reconstructed  $\mathbf{x}_{\text{recon}} = (r \circ h)(\mathbf{x})$  to be similar to the original input  $\mathbf{x}$ . As the above equation has a trivial solution, i.e.,  $(r \circ h)(\cdot) = \text{identity}$ , which means  $h(\mathbf{x})$  copies every details about  $\mathbf{x}$  (including all  $\mathbf{O}$  and  $\mathbf{G}$ ), some early works like denoising auto-encoder propose to add noise on  $\mathbf{x}$  to encourage non-trivial solutions. Depending on the noise we introduce, the model will learn to ignore different factors accordingly, which seems quite similar to the data augmentation mentioned in contrastive SSL methods. To encourage  $h(\mathbf{x})$  to extract more useful semantic information in  $\mathbf{G}$ , methods like MAE [29] propose to mask most of the patches of the input image and try to make reconstructions based on the remaining patches. Such methods exhibit amazing reconstruction performance (not in terms of high resolution, but the precise semantic reconstruction), which also implies that the  $h(\mathbf{x})$  trained in this way is capable of extracting high-level semantic generating factors (those are likely in  $\mathbf{G}$ ). Furthermore, methods like BEiT [4] and iBOT [83] also patchify and mask the input images, and concurrently, impose extra constraints on  $\mathbf{z}$  by comparing them with the descriptions generated by the big language model. Such designs also encourages  $h(\mathbf{x})$  to extract high-level semantic information, as illustrated in [10].
In summary, as the SSL algorithms learn good  $\mathbf{z}$  by designing loss or tasks on it, we can introduce extra inductive bias via auxiliary tasks. By requiring  $\mathbf{z}$  to be invariant when adding noise or conducting data augmentation on  $\mathbf{x}$ , the task-irrelevant information can be ruled out during learning. By requiring  $\mathbf{z}$  contains the necessary information for reconstruction when only part of  $\mathbf{x}$  is observable, the task-relevant semantic information can be highlighted during learning. By combining these principles, it is possible for us to learn good  $h(\mathbf{x})$  that only extracts information in  $\mathbf{G}$ . With the help of these methods, our  $\mathbf{z}$  might learn exactly all information in  $\mathbf{G}$ , which means the third rung is achieved.
A.3 Stage III:  $\mathbf{z}$  learns exactly all information in  $\mathbf{G}$ , i.e.,  $h(\cdot)$  leads to a bijection
Starting from Stage II, if we can design clever training methods (e.g., adding regularization, data augmentation, auxiliary task, etc.) and make our  $h(\mathbf{x})$  to be insensitive to some  $\mathbf{O}$ , ideally, we can learn an almost perfect encoder that extracts exactly all information contained in  $\mathbf{G}$ . In this case, we have  $H(\mathbf{G} \mid \mathbf{z}) = 0$  and  $H(\mathbf{z} \mid \mathbf{G}) = 0$ , i.e.,  $h(\cdot)$  leads to a bijection between  $\mathbf{z}$ -space and  $\mathbf{G}$ -space, which is denoted as  $\mathbf{z} \leftrightarrow \mathbf{G}$ . Ideally, such  $\mathbf{z}$  can generalize well even when  $P_{train} \neq P_{test}$ , as long as all the concepts in the test set are seen by  $h(\mathbf{x})$  during training, i.e.,  $\text{supp}[P_{test}(\mathbf{G})] \subseteq \text{supp}[P_{train}(\mathbf{G})]$ . However, we speculate that even  $h(\mathbf{x})$  on Stage III will struggle in the comp-gen problem, as the problem assumes  $\text{supp}[P_{test}(\mathbf{G})] \cap \text{supp}[P_{train}(\mathbf{G})] = \emptyset$ . In other words, we need the model to decompose and recombine the learned concepts in a systematic way, which is similar to the ground-truth-generating mechanism. To achieve this goal, we need to consider how to achieve Stage IV.
A.4 Stage IV:  $h(\cdot)$  leads to a isomorphism bijection between  $\mathbf{z}$  and  $\mathbf{G}$
To generalize well compositionally, we not only need  $\mathbf{z}$  contains exactly all information in  $\mathbf{G}$ , the structure of  $\mathbf{G}$  should also be embodied in  $\mathbf{z}$ , which means  $h(\cdot)$  should lead to an isomorphism bijection between  $\mathbf{z}$ -space and  $\mathbf{G}$ -space (i.e.,  $\mathbf{z} \Leftrightarrow \mathbf{G}$ ). Specifically, we need:
Hypothesis 1 in detail. To generalize well compositionally, we need  $\mathbf{z} \Leftrightarrow \mathbf{G}$ , which requires:
1.  $\mathbf{z} \leftrightarrow \mathbf{G}$ , i.e.,  $h(\cdot)$  leads to a bijection between  $\mathbf{z}$  and  $\mathbf{G}$ ;
2.  $P(\mathbf{z})$  and  $P(\mathbf{G})$  factorize in a similar way;
3. Each  $z_{i}$  maps to some  $G_{\mathbf{w}_i}$ , where  $\mathbf{w}$  is a permutation vector of length  $m$ , and such a mapping is invariant<sup>7</sup>;
4. For each  $z_{i}$ , the mapping between  $z_{i} = k$  and  $G_{\mathbf{w}_{i}} = \mathbf{u}_{k}$  is invariant<sup>8</sup>, where  $\mathbf{u}$  is a permutation vector of length  $v_{\mathbf{w}_{i}}$ .
We call such mappings compositional and call the other bijections holistic. To better understand the difference between these two types of mappings, we consider a simple example where the generating factors are  $\mathbf{G} = [G_1, G_2]$ ,  $G_1 = \{\text{blue, red}\}$  and  $G_2 = \{\text{circle, box}\}$ , and the representations are
For example,  $z_{1}$  always encode color and  $z_{2}$  always encode shape.
For example,  $z_{1} = 0$  always means blue and  $z_{1} = 1$  always means red.
(a). Compositional Mapping
(b). Holistic Mapping
Figure 5: A compositional mapping and a holistic mapping. Although holistic mapping in this example seems to have fewer "crossings", it has lower topological similarity.
Table 4: An example of coding the mappings, where  $\alpha$  is how many characters (including space and unique symbol, e.g.,  $\rightarrow$  and :) are used to express the grammar.
A compositional mapping
5 rules, α = 43	A holistic mapping
4 rules, α = 56
S	→	z2, z1
z2:	0 →	blue	S: →	blue circle
z2:	1 →	red	S: →	red circle
z1:	0 →	circle	S: →	red box
z1:	1 →	box	S: →	blue box
$\mathbf{z} = [z_1, z_2]$ ,  $z_1 = \{0, 1\}$  and  $z_2 = \{0, 1\}$ . Hence the space of  $\mathbf{G}$  is  $\{\text{blue circle, blue box, red circle, red box}\}$ , and the space of  $\mathbf{z}$  is  $\{00, 01, 10, 11\}$ . To generate a compositional mapping, we can first decide the meaning of  $z_i$ , e.g.,  $z_1$  represents the shape and  $z_2$  represents the color. After that, we assign vocabularies in  $z_i$  to denote different meanings, e.g.,  $z_1 = 0 \rightarrow \text{circle}$ ,  $z_1 = 1 \rightarrow \text{box}$ ,  $z_2 = 0 \rightarrow \text{blue}$ , and  $z_2 = 1 \rightarrow \text{red}$ . Combining these two steps, we generate a compositional mapping, as illustrated in Figure 5(a). Any bijections that cannot be decomposed in this way are holistic mappings, like Figure 5(b). Obviously, the compositional mappings can generalize compositionally while the holistic ones cannot. However, finding a compositional mapping from all possible bijections is a hard problem, as the number of compositional mappings is much smaller than the holistic ones: assuming we have  $m = 3$  different  $G_i$  and each with  $v = 3$  possible values, there are roughly  $10^3$  compositional mappings  $(m!(v!)^m)$ , and  $10^{28}$  holistic mappings  $((v^m)! - m!(v!)^m)$ .
Another important characteristic of compositional mappings is the "distance-preserving" ability. In other words, they will map two  $\mathbf{x}$  with similar  $\mathbf{G}$  to two similar  $\mathbf{z}$  in the representation space, i.e.,  $\mathbf{z}$  preserve the topological structure of  $\mathbf{G}$ . We can use topological similarity  $(\rho, \rho)$  defined in Equation 2 to quantify how compositional a mapping is. Usually, for all bijections, mappings with larger  $\rho$  are more compositional.
To sum up the aforementioned four stages of the learned  $\mathbf{z}$ , we list the conditions that  $\mathbf{z}$  must satisfy and their capabilities in Table 3.
A.5 Relationship with disentangled representations
Readers might notice that our problem settings and the requirements for  $\mathbf{z}$  mentioned in Hypothesis 1 are quite similar to those discussed in disentanglement representation learning [32]. Here we explain the connections and differences between them.
We care more about downstream tasks. In disentanglement representation learning, people focus more on a general property of the learned representation by assuming the existence of independent semantic generating factors. Disentanglement is then considered a desired property of such representations. Works in this line try to formalize this property and believe this property is beneficial for multiple downstream tasks. However, as the downstream task is not incorporated in such analysis, it is hard to conclude whether specific factors are semantic or not (remember the split of  $\mathbf{O}$  and  $\mathbf{G}$  highly depends on the task). On the other hand, this paper focuses more on the generalization ability of the learned representations on specific downstream task(s). We directly optimize the loss of the downstream task in the interaction phase, which we believe can regularize the representations more efficiently.
We don't strictly require a disentangled  $\mathbf{z}$ . Although representations with properties like consistency or restrictiveness mentioned in [68] could be beneficial for the downstream tasks, it seems not necessary to have all of them. In other words, disentanglement is a sufficient but not necessary requirement for a model to generalize well in downstream tasks. That is because the factorization of  $\mathbf{G}$  could be non-trivial (as nature is not simple). We believe that capturing the hidden structure of  $\mathbf{G}$  and  $\mathrm{GenY}(\cdot)$  using  $\mathbf{z}$  and  $g(\cdot)$  is more important than mapping an involved generating mechanism to a disentangled system. Additionally, implicitly splitting  $\mathbf{O}$  from  $\mathbf{G}$  is rather crucial in our settings, which is rarely discussed in the fields of disentanglement representation learning.
We mainly consider discrete factors. Inspired by how human language evolves, it is natural to start from the discrete factors and representation due to the discreteness of human language. Our experimental results also benefit a lot from the discreteness, e.g., using cross-entropy loss to amplify the learning speed advantage, sampling pseudo labels to strengthen the inductive bias, using group theory to formalize the compressibility and Kolmogorov complexity, etc. However, as nature might not be purely discrete, incorporating the continuous latent space is crucial to enlarge the scope of our study. We would leave this in our future work.
B Compositionality, Compressibility, Kolmogorov complexity, and number of active bases
This appendix links several key concepts related to compositional mappings together, i.e., compressibility, Kolmogorov complexity, and number of active bases. The analysis here provides good intuition on why we might expect iterated learning to be helpful in comp-gen.
B.1 Higher compositionality, lower Kolmogorov complexity
We first complete the proof of Proposition 1.
Proof. Recall the fact that any bijection from  $\mathbf{z}$  to  $\mathbf{G}$  can be represented by an element in the symmetry group  $S_{v^m}$ . From the definition of the symmetry group, we know each element in  $S_{v^m}$  can be represented by a permutation matrix of size  $v^m$ . As there is only one 1 in each row and column of a permutation matrix, any permutation matrix can be uniquely represented by a permuted sequence of length  $v^m$ . Specifically, assume we have a sequence of natural numbers  $\{1,2,\dots,v^m\}$ , each permuted sequence  $\mathrm{Perm}(\{1,2,\dots,v^m\})$  represents a distinct permutation matrix, and hence represents a distinct bijection from  $\mathbf{z}$  to  $\mathbf{G}$ . In other words, we can encode one bijection from  $\mathbf{z}$  to  $\mathbf{G}$  using a sequence of length  $v^m$ , i.e.,  $\mathrm{Perm}(\{1,2,\dots,v^m\})$ , and bound the corresponding Kolmogorov complexity (in bits) as
$$
\mathcal {K} (\text {b i j e c t i o n}) \leq v ^ {m} \cdot \log_ {2} v ^ {m} = v ^ {m} \cdot m \cdot \log_ {2} v, \tag {4}
$$