C og LM : Tracking Cognitive Development of Large Language Models (2025)

Abstract: Piaget’s Theory of Cognitive Development (PTC) posits that the development of cognitive levels forms the foundation for human learning across various abilities. As Large Language Models (LLMs) have recently shown remarkable abilities across a wide variety of tasks, we are curious about the cognitive levels of current LLMs: to what extent they have developed and how this development has been achieved. To this end, we construct a benchmark CogLM (Cognitive Ability Evaluation for Language Model) based on PTC to assess the cognitive levels of LLMs. CogLM comprises 1,220 questions spanning 10 cognitive abilities crafted by more than 20 human experts, providing a comprehensive testbed for the cognitive levels of LLMs. Through extensive experiments across multiple mainstream LLMs with CogLM, we find that: (1) In our testing framework, advanced LLMs (such as GPT-4) have demonstrated human-like cognitive abilities, comparable to those of a 20-year-old human. (2) The parameter size and optimization objective are two key factors affecting the cognitive levels of LLMs. (3) The performance on downstream tasks is positively correlated with the level of cognitive abilities. These findings fill the gap in research on the cognitive abilities of LLMs, tracing the development of LLMs from a cognitive perspective and guiding the future direction of their evolution.

## introduction large language models (llms) have recently achieved impressive performance on a wide variety of natural language processing (nlp) tasks, including text comprehension (kenton and toutanova, 2019), reasoning @xcite , code generation @xcite , and mathematical problems @xcite . however, few studies have explored the reasons behind the evolutionary relationship among various abilities, which makes it difficult to understand the development of llms' capabilities as a whole and may pose potential risks to their further development. to this end, we introduce piaget's theory of cognitive development (ptc) @xcite @xcite , which posits that the development of cognitive levels forms the foundation for human learning across various abilities. inspired by this, we think that studying the cognitive development of llms can assist us in better understanding the current performance of llms on downstream tasks and illuminate the path for future enhancements of their capabilities. as the most authoritative theory in the development of psychology, ptc suggests that human children move through four different stages of learning, including the sensorimotor stage (0-2 years old), the preoperational stage (2-7 years old), the concrete operational stage (7-12 years old), and the formal operational stage (above 12 years old). children in different cognitive stages exhibit significantly distinct patterns of thinking and cognitive abilities, which affects their learning of other skills. examining llms from the perspective of ptc, some natural and crucial questions are: at what stage has the cognitive ability of llms developed compared to humans at present? what are the key factors that affect the cognitive abilities of llms? is the emergence of advanced abilities and performance bottlenecks in current llms related to their cognitive levels? to explore the above questions, we construct a benchmark based on the scenario experiments used in ptc for evaluating the cognitive abilities of llms, denoted as coglm. a large-scale human trial was conducted involving 207 participants aged between 6 and 20 years to ensure the alignment be-tween the coglm and ptc. we then perform extensive experiments on coglm over several series of language models, including opt @xcite , llama-2 @xcite , @xcite . our results indicate that: (1) under our testing framework, advanced llms, such as gpt-4, have developed human-like cognitive abilities, matching those of a 20-year-old individual. (2) two primary factors influencing these cognitive capacities in llms are the size of their parameters and their optimization objectives. (3) there is a positive correlation between the cognitive level of llms and their performance in downstream tasks. we believe that our findings can present a clear understanding of the current cognitive level of llms and provide insights into the emergence of advanced abilities in llms, shedding light on the future development of them. our contributions can be summarised as follows: ## coglm benchmark development to comprehensively and accurately assess the cognitive abilities of llms, we undertake the following efforts: (1) we revisit 12 cognitive abilities proposed by ptc, 10 of which are selected and redefined to construct coglm according to the characteristics of llms (section 2.1). (2) we create standardized data construction guidelines to ensure the quality of coglm (section 2.2). (3) we conduct extensive human testing to ensure the alignment between coglm and ptc (section 2.3). ## conclusions piaget's theory of cognitive development (ptc) as a tool to track the development of cognitive abilities of llms. we construct coglm based on the scenerio experiments used in ptc, and conduct thorough human testing to ensure the alignment between coglm and ptc. through extensive experiments on multiple series of llms, we show that: (1) in our testing framework, human-like cognitive abilities have emerged in advanced llms (such as gpt-4), comparable to those of 20-year-old humans. (2) the parameter size and optimization objective are two key factors affecting the cognitive abilities of llms. (3) the ability of performing downstream tasks is positively correlated with the level of cognitive abilities. we believe that our findings can provide a novel insight into the emergence of abilities in llms, and shed light on the future development advanced abilities of llms.