Emergence of symbolic abstraction heads for in-context learning in large language models (2025)

Abstract: Large Language Models (LLMs) based on self-attention circuits are able to perform, at inference time, novel reasoning tasks, but the mechanisms inside the models are currently not fully understood. We assume that LLMs are able to generalize abstract patterns from the input and form an internal symbolic internal representation of the content. In this paper, we study this by analyzing the performance of small LLM models trained with sequences of instantiations of abstract sequential symbolic patterns or templates. It is shown that even a model with two layers is able to learn an abstract template and use it to generate correct output representing the pattern. This can be seen as a form of symbolic inference taking place inside the network. In this paper, we call the emergent mechanism abstraction head. Identifying mechanisms of symbolic reasoning in a neural network can help to find new ways to merge symbolic and neural processing.

## introduction recognizing abstract patterns is a fundamental ability that humans have, allowing them to generalize from a few instances and make inferences on unseen scenarios. llms seem to be able to perform similar reasoning tasks and even exceed human performance in some cases @xcite . symbolic machine reasoning systems have a long history @xcite @xcite but the emergence of the capability in current machine learning systems is not fully understood. large transformers @xcite and state-space models @xcite exhibit intriguing emergent properties. extremely large models appear capable of executing tasks like in-context learning @xcite and chain-of-thought reasoning, which are not directly derivable from their training data. the reasoning abilities of llms with tabular, non-text data have been illustrated recently, as noted in @xcite . furthermore, the reasoning prowess of llms has been highlighted in robotic control @xcite , autonomous vehicle navigation @xcite , and the processing of iot sensor data @xcite . the mechanistic interpretability of large language models (llms) remains a highly active field of research, with numerous recent theories about how specific behaviors manifest in such extensive models @xcite @xcite . this line of research is driven by the common understanding that current llms are computationally extremely expensive and environmentally unsustainable, for most use cases, and still from their theoretical capacity @xcite . the current methods for the minimization of the models, e.g., using distillation techniques, produce only relatively small gains @xcite . a better understanding of the mechanisms can help to improve the design of llm architectures and training paradigms. the induction head mechanism is considered a key factor behind in-context learning, enabling a language model to identify a recurring pattern from its input and either replicate it in the output or merge it with previously stored knowledge @xcite . other theories explaining the emergence of certain behaviors in large language models include the concept of task-vectors @xcite and bayesian inference occurring during the model's inference phase @xcite . in this paper, we investigate the ability of small transformer models to recognize, learn, and generalize abstract sequential symbolic patterns, or templates. a template refers to an abstract sequential symbolic pattern that follows a defined structure but can be instantiated with different symbolic elements. for example, the template abcabcab represents a repeated sequence where a, b, and c are symbolic placeholders that can take on specific values. templates represent the underlying structure of patterns, allowing us to explore whether models can learn these abstract patterns and generalize to new instances that the model has never seen, such as abacabac, that follow similar structural rules. these templates can be instantiated into specific sequences,such as 12312312 or 45645645, by assigning values to the placeholders. understanding whether models can generalize across unseen instantiations and solve such patterns dynamically during inference is in the focus of this work. furthermore, we aim to determine whether a model can recognize such instantiations in-context, that is, whether it can recognize the symbolic mappings during inference without retraining and by using this understanding to solve new instances of the same abstract template. this ability would indicate that the model is not only learning patterns from its training data but also reasoning dynamically based on the input it encounters during inference. our experiments demonstrate that small transformer models with two or three layers can successfully solve the task of abstract pattern matching, whereas single-layer models fail to solve the abstract task, which aligns with the theory of singlelayer models inability to perform the induction head task @xcite . additionally, we observed the emergence of an abstraction head. we define an abstraction head as an attention mechanism in transformer models that attends to previous instantiations of a pattern in an abstract manner, identifying the structural relationships between symbolic placeholders, and using this information to perform pattern matching on unseen instances with a similar abstract structure. the identification of the mechanisms of symbolic reasoning emerging in the training of neural networks can help to build new types of neurosymbolic processing paradigms. moreover, it may also help to train neural networks with an internal visible symbolic reasoning mechanism. the recent survey by @xcite gives a taxonomy of different neurosymbolic systems. the explicit training of abstraction head configurations could be seen as novel way to implement type 6 neurosymbolic systems in their taxonomy. ## methodology to achieve our primary objective of understanding how llms perform on abstract sequential symbolic patterns, we designed a controlled experimental setup involving synthetic datasets of templates and their instantiations. to generate the data, we define abstract patterns of length eight using three symbolic variables: a, b, and c. for example, consider the abstract pattern abcabcab. by assigning specific values to a, b, and c, we generate different instantiations of the same template such as 12312312, 45645645, 78978978, and 15915915. we generated all possible permutations of length 8, resulting in 6561 patterns. during this process, we observed that some patterns did not include all three variables, such as the pattern acacacac which only contains variables a and c, leaving out b. to ensure uniformity, we excluded such patterns, requiring that all three variables are present. additionally, we excluded patterns where the last token appeared only once at the end (e.g. abababac), as the last digit in each pattern serves as an evaluation metric in our experimental setup. to avoid duplication, we also treated patterns that are equivalent after instantiation as duplicates. for example, abcabcab and acbacbac were considered the same pattern and only one was used. this resulted in 1,806 unique patterns, which we split into 80% for training and 20% for testing to evaluate whether the models can learn new abstract patterns they have never encountered before from context. after generating the abstract patterns, the next step is to concretize them using instantiations. this is achieved by generating all possible unique combinations of variables a, b, and c, ensuring that each variable is different. per pattern, we obtain 504 different instantiations. to prepare the data for training and evaluation, we combined every four instantiation in each input sequence, where every instance should represent the same abstract pattern. this choice ensures that the model is exposed to different representations of the same abstract structure, providing sufficient context for generalization. furthermore, it ensures that the model is exposed to at least one instantiation of a pattern early in the sequence, before generalizing on the rest of the input sequence, which is a crucial pattern in determining the model's ability of learning during inference time. for instance, the abstract pattern abcabcab could produce the following input sequence: ## models to evaluate the ability of small llm models to generalize and learn abstract patterns, we experiment with several transformer @xcite architectures, designed to test the models capabilities. the primary task for training these models is autoregressive sequence prediction @xcite . in this setup, the model predicts the n th token based on the n-1 previous tokens, using them as context to predict the next output. as for the main experiment, we focus on experimenting with models with one, two, and three layers consisting of eight, four and two attention heads, respectively. meanwhile, we focus on the 2-layer, 4-head architecture for the second experiment. the hidden size dimension is set to 128 across all architectures, and each model includes a single feed-forward layer. absolute positional embeddings are used to encode positional information in the input sequences. in this paper, we used the python library x-transformer to build the transformers @xcite . as for the training, the models were trained on a batch size of 64 using adam's optimizer (kingma and ba, 2017), publicly available in pytorch with a learning rate of 1 × 10 -3 for a total of 100,000 steps. ## evaluation to assess the ability of the models to generalize and learn new patterns, we designed two evaluation tasks for the main experiment: the last token prediction and variable matching. accuracy is the primary evaluation metric for both tasks. for the last token prediction task, accuracy measures the proportion of correct predictions for the last digit in the 4th instantiation. for example, given the input: for the complementary experiment, we focus on measuring the accuracy of the second, third, and fourth instantiations, since the model would have seen both the variables and one instantiation, allowing it to generalize on the rest of the instantiations. ## results in this section, we present the results of training the models on instantiations based on abstract patterns, and testing them on the instantiations of the abstract patterns in the test set, which the model has not been exposed to. we will include the training loss of the models, in addition to the last token prediction and variable matching metrics. in addition, we visualize specific attention heads based on their importance and contribution to solving the tasks. ## discussion this section analyzes the results presented in the previous section, focusing on the model's performance on the training loss and the accuracy metrics we defined, attention mechanisms, and the models' ability to generalize on abstract patterns during inference time. ## conclusions in this study, we investigated the ability of small transformer models to recognize, learn, and generalize abstract sequential symbolic patterns through controlled experiments. we demonstrated that models with two or three layers successfully perform this task, unlike single-layer models. a key finding was the emergence of the abstraction head, an attention mechanism that directs its focus toward the first instantiation in an abstract manner. our findings on the emergence of abstraction heads provide a foundation for advancing neuro-symbolic processing paradigms, potentially enabling the development of new type 6 neuro-symbolic systems which contain symbolic processing inside a trained neural network.