INDOORWORLD : Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environment (2025)

Abstract: Virtual environments are essential to AI agent research. Existing environments for LLM agent research typically focus on either physical task solving or social simulation, with the former oversimplifying agent individuality and social dynamics, and the latter lacking physical grounding of social behaviors. We introduce IndoorWorld, a heterogeneous multi-agent environment that tightly integrates physical and social dynamics. By introducing novel challenges for LLM-driven agents in orchestrating social dynamics to influence physical environments and anchoring social interactions within world states, IndoorWorld opens up possibilities of LLM-based building occupant simulation for architectural design. We demonstrate the potential with a series of experiments within an office setting to examine the impact of multi-agent collaboration, resource competition, and spatial layout on agent behavior.

## introduction the emergence of large language model (llm)based agents has extended llms beyond traditional one-off interactions, equipping them with long-term memory, planning capabilities, and embodied actions @xcite @xcite . among them, multi-agent systems leverage distinct agent roles to achieve greater collective intelligence for problem-solving @xcite @xcite or to enable more realistic cognitive and psychological modeling in social simulations. like in traditional ai research @xcite , virtual environments are essential for llm-based agents, enabling them to perceive and act. these environments provide external sensory input and introduce a world state, allowing agent actions to be grounded as operators that modify the environment. these environments serve as low-cost testbeds that accelerate llm-agent development, typically falling into two categories: physical tasksolving environments and social simulation environments. physical task-solving environments such as vir-tualhome @xcite and alfworld @xcite enable agents to interact with external objects to manipulate the world state towards specific objectives. however, these environments often assume identical action spaces, homogeneous agent abilities, neglecting individual differences among agents, and the impact of social dynamics on task solving. on the other hand, social simulation environments, such as smallville @xcite , enable social interactions between agents, such as relationship building and information sharing. while these systems effectively simulate human-like social behaviors driven by individual personalities and roles, they often employ an oversimplified model of the physical world. this leads to a lack of groundings for agents' actions in the change of world state, resulting in social interactions that remain merely "plausible" without any correspondence to an external physical reality. for example, an agent may refer to non-existent physical objects in a dialog. the gaps lead to missed opportunities to use ai agents for applications that require tight integration of physical task-solving and social simulation. one such application is building occupant simulation for architectural design @xcite , where occupant behaviors are driven by both dynamic physical and social factors. to respond to the missed opportunities, we present indoorworld , a heterogeneous multi-agent environment that tightly integrates physical and social dynamics in an indoor space setting, introducing novel challenges for llm-driven agents in orchestrating social dynamics to influence physical environments and anchoring social interactions within physical world states. as a multi-agent system, in-doorworld allows for fully decentralized agent control and collective task assignment, facilitating self-regulated labor division, task prioritization and coordination. indoorworld provides a scalable and expressive testbed for advancing research on multi-agent llm systems. our key contributions are as follows: ## indoorworld environment our environment indoorworld seamlessly integrates task-solving and social simulation in an indoor space setting, offering a versatile testbed to study the interplay between social and physical agent behavior. indoorworld enables agents with rich individual differences to collaborate on tasks with complex hierarchy, involving self-regulated labor division, prioritization and coordination, while satisfying physiological needs through environmental interactions. by incorporating fine-grained modeling of agent internal structure and decision-making process, indoor-world more accurately resembles real-world human behaviors. tables 1 , 5 , and 7 summarize the key differences between indoorworld and existing platforms, highlighting its potential to advance research in llm-based multi-agent systems. ## conclusion this study introduces indoorworld , a multiagent simulation environment that integrates finegrained heterogeneous agent modeling with physical interactions. indoorworld enables agents with varied abilities to coordinate roles, satisfy physiological needs, and interact with a rich array of objects within realistic settings. we evaluate three llms in collaborative task solving using a benchmark of common office tasks that involve both social and physical interactions, highlighting the effectiveness of task prioritization. our simulation experiments on resource management and spatial layouts demonstrate the potential real-world applications of indoorworld in architectural design.