Title: United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theory
Venue: NeurIPS 2025
Accepted: no
Keywords: Large Language Models

LLM Agents

Multi Agent Systems

Cognitive Load Theory

Collective Intelligence

Collaborative Reasoning

Working Memory

Complex Problem Solving

Emergent Cognition
Abstract: Large Language Models (LLMs) exhibit a notable performance ceiling on complex, multi-faceted tasks, as they often fail to integrate diverse information or adhere to multiple constraints. 
We posit that such limitation arises when the demands of a task exceed the LLM's effective cognitive load capacity. This interpretation draws a strong analogy to Cognitive Load Theory (CLT) in cognitive science, which explains similar performance boundaries in the human mind, and is further supported by emerging evidence that reveals LLMs have bounded working memory characteristics.
Building upon this CLT-grounded understanding, we introduce ***CoThinker***, a novel LLM-based multi-agent framework designed to mitigate cognitive overload and enhance collaborative problem-solving abilities. ***CoThinker*** operationalizes CLT principles by distributing intrinsic cognitive load through agent specialization and managing transactional load via structured communication and a collective working memory. We empirically validate *CoThinker* on complex problem-solving tasks and fabricated high cognitive load scenarios, demonstrating improvements over existing multi-agent baselines in solution quality and efficiency. Our analysis reveals characteristic interaction patterns, providing insights into the emergence of collective cognition and effective load management, thus offering a principled approach to overcoming LLM performance ceilings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces CoThinker, a novel multi-agent Large Language Model (LLM) framework that draws a strong analogy from Cognitive Load Theory (CLT) in human cognitive science to explain and mitigate performance ceilings observed in LLMs on complex, multi-faceted tasks. The authors posit that these limitations arise when task demands exceed an LLM's effective cognitive load capacity, similar to human cognitive overload. CoThinker operationalizes CLT principles by distributing intrinsic cognitive load through agent specialization (Agent Parallel Thinking), managing transactional load via structured communication (Communication Moderator), and employing a collective working memory (Transactive Memory System).\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper's core strength lies in its innovative approach of drawing inspiration from Cognitive Load Theory (CLT) to understand and address LLM performance limitations. This analogy between human cognitive load and LLM capacity provides a credible and well-grounded theoretical foundation for the CoThinker architecture.\n2. The CoThinker design is logically structured, with each module (Agent Parallel Thinking, Transactive Memory System, Communication Moderator, Synthesizer) directly motivated by principles from CLT and human collective intelligence. This theoretical grounding makes the design choices seem principled and effective.\n3. The empirical performance is decent. Also, the inclusion of interesting ablation studies further strengthens the paper by providing insights into the impact of key architectural parameters and their relation to cognitive load management.\n4. It is interesting to see systematical operationalization of CLT within a multi-agent LLM framework\n\n**Weaknesses**\n1. While Section 2 critiques limitations of existing multi-agent designs in their motivation, it lacks a detailed architectural comparison with prominent existing frameworks (e.g., AutoGen). This makes it challenging to clearly discern CoThinker's unique architectural contributions and how its specific design choices directly overcome the architectural flaws, rather than just motivational shortcomings, of prior work. A more explicit comparative analysis of architectural elements would clarify the work's distinct advancements.\n2. A critical limitation is the exclusive focus on the Gemini series of LLMs for experimentation. As noted, Table 1 reveals that the comparative performance of different architectures varies across different Gemini models (e.g., gemini-1.5-flash vs. gemini-1.5-flash-8b), suggesting model-specific sensitivities. This raises significant questions about the generalizability of CoThinker's observed benefits and optimal parameter settings to other LLM families (e.g., OpenAI's GPT series, Claude, Llama, Qwen etc). The paper's conclusions might be constrained to the specific characteristics of Gemini models.\n\nMinor suggestions: \n\ni. While the paper provides an analysis of LiveBench results, a more detailed breakdown for Table 1 is needed. For instance, for gemini-1.5-flash and gemini-1.5-pro models, CoThinker is only slightly better than DMAD on average. Conversely, for gemini-1.5-flash-8b, CoThinker shows a more substantial improvement over DMAD but is only slightly superior to CoT. A deeper, more granular discussion on why these varying degrees of improvement are observed across different model capacities and baselines would significantly enrich the analysis and demonstrate a more thorough understanding of the results.\n\nii. In the main text, I would suggest authors to specify which part of Appendix are referred rather than claiming generally \"See details in Appendix\"\n\nQUESTIONS:\nJust out of curiosity, how does the theoretical \"cognitive load ceiling\" you propose relate to or interact with the practical limitations imposed by context windows in LLMs? Is there a direct correlation (i.e., does exceeding the context window automatically imply cognitive overload), or are these distinct but possibly related constraints?\n\nLIMITATIONS:\nyes, in Appendix E", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes that the performance limitations of LLMs on complex tasks arise from cognitive overload, drawing an analogy with Cognitive Load Theory (CLT) in human cognition. To address this, the authors introduce CoThinker, a multi-agent framework that distributes cognitive load among specialized LLM agents through shared memory and structured communication. Empirical results demonstrate that CoThinker significantly improves performance and reasoning efficiency on high-load tasks compared to existing multi-agent and single-agent baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n- The grounding of LLM capability into the cognitive load theory itself is novel and insightful, providing a good motivation for the design in the paper\n- The paper’ analysis in agent network is insightful and inspiring, especially the ablation part (Section 5.4)\n\nWeakness:\n- The paper’s writing on the network construction part could be further improved and be more concrete. Currently from the writing it’s still not very clear how the information is passed over different agents and how they exactly collaborate in problem-solving\n- The model used for testing only involves gemini model which is limited. Also, it’s possible to further conduct ablation study on different LLM configuration and each component’s role. For instance, how would the ability of the synthesizer affect the final agent system’s performance?\n\nQUESTIONS:\n- Is the working memory described in Section 3.1 also incorporates LLM’s own parametric knowledge? From the discussion seems the analogy is only related to the in-context information. What’s the position of parametric knowledge in this analogy? Seems the “elicitation” (not memorization) of parametric knowledge won’t be affected by something like LLM’s own capacity.\n- How is your theory and agent design related to the concept of “cognitive offloading”? Instead of think in a “parallel” way, it’s more like LLM with self-awareness of its capabilities could offload some burden to other external model ore resources, in oder to cooperate and achieve success.\n- For the network construction in method, is it constructed by agent proactively during problem solving, or it is settled before any interaction? Could you provide an example on how the information is generated and passed in the system? For example, does the agent only generates by gathering information from certain agent, or it also proactively broadcast information to certain agent, using TMS as the media?\n- Is it possible to mix agents of different possibilities in the same network? Will this make an abolition study and provide insights on each agent’s role?\n\nLIMITATIONS:\nLimitations could be discussed from the role of each component in the framework, what are some misaligned aspect between the original CL theory and the LLM side, and what are some error cases that still needs improvements.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors address a failure mode of LLMs when presented with complex tasks. Specifically, tasks that require integration of large amounts of information. They draw inspiration from Cognitive Load Theory in cognitive science in order to analyze this scenario and hipothesize that LLMs' failures are mainly due to an excess of cognitive load. Following the same analytical framework, the authors propose an architecture aimed at lowering the cognitive load of LLMs upon collaboration, and they test its impact on two complex benchmarks. The authors also compare the performance of LLMs under the proposed architecture vs other 5 single- and multi-agent baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts:\n- The inspiration from CLT in cognitive science is solid, highly interesting and has potential.\n- Revision of related work seems thorough.\n- Claims are clear.\n- Explanation of the cognitive background and proposed method is very clear and easy to follow, including figures and formal descriptions when needed.\n- Empirical evaluation seems correct, and the use of relative metrics makes the comparison against the baselines easier.\n- Good analysis of results and ablation studies.\n\nWeaknesses:\n- Reasons for the selection of the two benchmarks are a bit scarce. Could elaborate more on the motivations and criteria for their selection.\n- Only evaluated using gemini models, and other LLMs might react differently (addressed in limitations).\n- Ablation studies have very low resolution (only 3 values per hyperparameter).\n- The authors do not explicitly address the limitations of their work in the main paper, although they do so in the appendix.\n\nQUESTIONS:\nI have the following questions / suggestions:\n1. What were the motivations or selection criteria for the evaluation benchmarks? I can guess roughly from the rest of the paper, but it would be nice to have it explicitly estated, maybe also referencing one or two benchmarks that might seem suitable to inexpert eyes (this last bit is just a suggestion, so highly optional).\n2. You only evaluate gemini models. Could you elaborate on that decision and how these results might or might not transfer to other model families?\n3. The limitations addressed in the appendix should be in the main paper. I also believe that the future work is always inspiring and should be addressed in the main paper.\n\nLIMITATIONS:\nAddressed, but only in the appendix. Should be included in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a multi-agent in-context learning framework, CoThinker, to improve large language models' reasoning capabilities for complex tasks. The framework draws inspiration from human cognitive load theory (CLT) and collective intelligence observed in human teams. Empirical results indicate that CoThinker outperforms baseline agentic frameworks on highly complex reasoning tasks.\n\nSTRENGTHS AND WEAKNESSES:\n1. The theoretical analogy drawn in Section 3 between human cognitive load theory (CLT) and the limited reasoning capabilities of LLMs is overgeneralized. The concept of limited cognitive resources is imposed on LLMs based solely on observations of liomited reasoning capabilities similar to those in humans. No formal proof or empirical evidence is provided to link CLT to specific internal representations or mechanisms within LLMs. For instance, it is unclear whether the context window of in-context learning or certain intermediate neuron activations should be considered the \"working memory\" of an LLM.\n2. The proposed implementation of collective intelligence is distinct from how actual human teams function. Specifically, submodules like the \"transactive memory system\" and \"communication moderator\" are not actual entities in human teams but rather are concepts that emerge from the interactions (e.g., negotiation) among multiple autonomous agents. It is not clear from a theoretical standpoint how the proposed collective intelligence analogy differs from existing Multi-Agent Systems (MAS) that employ similar mechanisms like communication, shared memory, and theory of mind. Consequently, the paper's first claimed contribution of drawing a “strong” analogy between CLT and the limited reasoning capabilities of LLMs is not convincing.\n3. The key assumption of the proposed CoThinker architecture is that LLMs can better handle sub-tasks divided from the original complex task due to a lower cognitive load. However, this is not verified quantitatively or qualitatively in the paper. \n4. The retrieval process shown in Figure 2 is not explained in the main text. It appears that the communication moderator only considers an agent’s output from the last round without taking the state of the transactive memory system into account. This design decision seems counter-intuitive and requires further explanation, as the main purpose of maintaining shared memory (i.e., \"who knows what\") is to guide effective information sharing among team members.\n5. The communication topology and algorithm are determined by a set of pre-defined rules. More explanation is needed to justify the rationale for generating these rules and their connection to CLT.\n6. The benchmark baseline section is missing state-of-the-art models trained with native reasoning capabilities (e.g., DeepSeek-R1, GPT-o3, Gemini 2.5 pro). The value of the proposed agentic framework with a fixed workflow is less clear given new training methods based on reinforcement learning and internal reasoning chains.\n7. As shown in the ablation studies, CoThinker is sensitive to hyperparameters, such as the number of agents. This raises the question of how to tune these hyperparameters and prompts when applying the proposed framework to a new, unseen domain.\n\nQUESTIONS:\n1. How do the authors quantitatively or qualitatively verify that subdividing complex tasks into simpler subtasks indeed reduces cognitive load for LLMs?\n2. What metrics or indicators were used to measure the cognitive load of each module within the architecture?\n3. Could the authors clarify why the communication moderator relies solely on agent outputs from the previous round rather than considering the state of the transactive memory system?\n4. How were the communication rules and topology decisions derived from cognitive load theory, and what justifications support these specific design choices?\n5. How would the proposed framework generalize when implemented using state-of-the-art reasoning models such as Deepseek-r1, GPT-o3, or Gemini-2.5?\n6. Given CoThinker’s sensitivity to hyperparameters (e.g., number of agents), how would one systematically tune these parameters and prompts for effective deployment in new, unseen domains?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}