Title: ActiveVOO: Value of Observation Guided Active Knowledge Acquisition for Open-World Embodied Lifted Regression Planning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Open-world Planning

Embodied Agents

Active Knowledge Acquisition

Lifted Regression
Abstract: The ability to actively acquire information is essential for open-world planning under partial observability and incomplete knowledge. However, most existing embodied AI systems either assume a known object category or rely on passive perception strategies that exhaustively gather object and relational information from the environment. Such a strategy becomes insufficient in visually complex open-world settings. For instance, a typical household may contain thousands of novel and uniquely configured objects, most of which are irrelevant to the agent’s current task. Consequently, open-world agents must be capable of actively identifying and prioritizing task-relevant objects to enable efficient and goal-directed knowledge acquisition. In this work, we introduce ActiveVOO, a novel zero-shot framework for open-world embodied planning that emphasizes object-centric active knowledge acquisition. ActiveVOO employs lifted regression to generate compact, first-order subgoal descriptions that identify task-relevant objects, and provides a principled mechanism to quantify the utility of sensing actions based on commonsense priors derived from LLMs and VLMs. We evaluate ActiveVOO on the visual ALFWorld benchmark, where it achieves substantial improvements over existing LLM- and VLM-based planning approaches, notably outperforming VLMs fine-tuned on ALFWorld data. This work establishes a principled foundation for developing embodied agents capable of actively and efficiently acquiring knowledge to plan and act in open-world environments.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method named ActiveVOI for embodied task planning with open-world and zero-shot capabilities. ActiveVOI uses Lifted Regression to generate subgoals which then help identify task-relevant objects. Then a quantification of object utility named Value of Information (VOI) is introduced, which is generated by LLMs and VLMs. Experiments on this framework are carried out on ALFWorld benchmark, showing its outperformance over state-of-the-art LLM/VLM based planners.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n\n1. This paper presents its novel method with detailed explanation and illustration, along with clear and rigorous mathematical descriptions.\n\n2. This paper presents an organic combination of Lift Regression and VOI theory for open-world active knowledge acquisition, and cleverly makes good use of commonsense and reasoning abilities of LLMs/VLMs.\n\n3. Experiments prove this framework’s SOTA performance under ALFWorld environment, and multiple ablation studies clearly present the contribution of each component including VOI calculation and subgoal object acquisition.\n\nWeaknesses：\n\n1.  The VOI calculation highly relies on LLM and VLMs, which we know face the problems of inconsistency and prompt-dependency, especially when the scene is complicated, or the instruction is unclear.\n\n2. The experiment is conducted on ALFWorld exclusively, which lacks validation on generalizability.\n\n3. Some might have concerns about novelty, as notions of active knowledge acquisition and VOI are already established, and this paper just makes combination in a new context.\n\nQUESTIONS:\n1. Will there be experiments with other embodied environments like Habitat and iGibson?\n\n2. Will there be experiments under real-world settings?\n\n3. Is there any measure taken to overcome LLM’s problems of inconsistency and prompt-dependency, and/or help correct possible bias?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ActiveVOI, a framework for open-world embodied planning under partial observability and incomplete knowledge. The key innovation is an active, object-centric approach to knowledge acquisition: instead of passively gathering all available information, the agent uses Lifted Regression to generate compact, relational subgoal descriptions and identifies relevant objects via the theory of Value of Information (VOI). Commonsense priors from LLMs/VLMs help estimate object utility. Experimental validation is performed on the visual ALFWorld benchmark, showing substantial improvements over previous (even fine-tuned) LLM/VLM-based baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:  \n- The paper tackles a crucial challenge in embodied AI: practical and scalable planning in open worlds, where passive information gathering is infeasible due to combinatorial complexity, although it's not a new concept. \n- The proposed framework combines Lifted Regression for global, object-centric reasoning with a decision-theoretic (VOI) framework for prioritizing which objects to sense.\n- The formalization of open-world planning with lifted predicates, explicit attention to handling incomplete knowledge, and proof-based (sound and complete) guarantees for the regression process. VOI application to active perception/planning is theoretically justified and well-motivated.\n\nWeaknesses:  \n- The writing and the figures of the manuscript requires further improvement. For example, fig.1 shows complex pipelines, while the differences of the two pipelines only mainly lines in the Active Knowledge Acquisition step. \n- ALFWorld is not an Open-World Embodied setting, at least from my point of view. There are some similar works tackling the active reasoning problems in environments like Minecraft or Crafter, which I think is more suitable to call as open-world. The author should consider testing their algrithims in more scenarios. \n-  Active perception and information-driven exploration are long-standing topics in AI and robotics, especially using VOI. Recent works have begun to use LLMs (and even VLMs) for goal decomposition, subgoal extraction, and commonsense reasoning for embodied agents. I'm not sure whether the integration of VOI with LLMs for active object-centric search significantly novel, or just an incremental composition of existing tools.\n\nQUESTIONS:\n- Are there concrete differences between using LLMs for task-relevant object prediction/information gain and previous LLM- or information-theoretic approaches to active perception?\n- Do the authors anticipate that this approach would transfer smoothly from simulated benchmarks (ALFWorld) to real-world robot settings? If not, why they choosing the ALFWorld simulator rather than other Open-World Embodied tasks?\n- One of the claims of the framework is its incorporation of LLMs/VLMs in quantifying the utility of knowledge acquisition targets—i.e., using LLM-based commonsense for Value of Information (VOI) calculations and prioritization. If a traditional symbolic/planning agent were given the same—fully specified and correct—knowledge base (i.e., a list of all objects, their types, and relations), could it not also generate an optimal plan using standard planning techniques (such as PDDL/STRIPS)? In other words, is the reliance on LLM commonsense truly necessary for high performance, or is it mainly compensating for incomplete or noisy object knowledge?\n\nLIMITATIONS:\nSee ABOVE.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents ACTIVEVOI, a novel framework for active knowledge acquisition in open-world embodied planning. It addresses the challenges faced by AI agents in environments with partial observability and incomplete task-relevant knowledge. ACTIVEVOI introduces a two-step approach: (1) determining relevant object descriptions through Lifted Regression, and (2) quantifying and prioritizing knowledge acquisition based on the Value of Information (VOI) theory, using commonsense knowledge from large language and vision-language models (LLMs/VLMs). The framework is evaluated on the ALFWorld benchmark, showing substantial improvements over existing methods, especially in visual tasks. ACTIVEVOI does not require task-specific fine-tuning and demonstrates superior performance in zero-shot settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nNovelty: ACTIVEVOI offers a fresh approach to active knowledge acquisition by combining Lifted Regression with VOI, addressing a crucial gap in open-world embodied planning.\n\nEffectiveness: The framework demonstrates significant improvements over existing models, particularly in zero-shot settings where no prior data is available for training or fine-tuning. This is a noteworthy achievement in open-world environments, which are inherently challenging.\n\nPractical Application: The integration of commonsense knowledge from LLMs/VLMs enables the agent to make intelligent decisions on what information to acquire, potentially leading to more efficient planning in real-world applications.\n\nWeaknesses:\n\nLimited Benchmark Evaluation: The experiments are primarily conducted on the ALFWorld benchmark, which is a well-regarded testing environment, but additional evaluations across other benchmark tasks, such as Alfred or VirtualHome would help to confirm whether the observed performance improvements are consistent across different types of environments and tasks. Expanding the evaluation scope would provide more robust evidence of the framework's general applicability.\n\nAssumptions on Knowledge Independence: The framework makes several assumptions regarding the independence of object attributes and relational properties. In complex real-world scenarios with intricate object interactions, these assumptions may lead to inaccuracies in the VOI calculations, affecting the agent's decision-making process.\n\nScalability and Efficiency: While the method is effective in small-scale environments, it remains unclear how well the system scales to much larger, more dynamic real-world environments with a high number of objects and complex interrelationships.\n\nQUESTIONS:\nCould the authors provide further details on how the method might scale in real-world environments? Specifically, how might the approach be adapted to handle environments with significantly more objects or dynamic interactions between objects?\n\nHow does the proposed VOI calculation handle dependencies between object attributes and relations in more complex environments, where the assumptions of independence may not hold? Would relaxing these assumptions improve performance?\n\nCan the authors elaborate on potential strategies for reducing the computational cost of VOI calculations, especially when scaling the method to larger environments with many objects?\n\nLIMITATIONS:\nYes, the authors have addressed the limitations of their work, particularly in terms of the evaluation environment and the theoretical assumptions made. However, there could be more discussion on the real-world applicability of the framework and how it can be adapted to handle more complex, dynamic environments.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ActiveVOI, a zero-shot framework for open-world embodied planning that actively acquires task-relevant information using Lifted Regression and Value of Information (VOI). The agent identifies subgoals via symbolic regression, uses LLM/VLM commonsense to estimate object relevance, and then prioritizes sensing based on VOI. On the ALFWorld benchmark, ActiveVOI achieves strong performance, outperforming most baselines in a single-trial visual setting, without any fine-tuning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- **Strong empirical performance in single-trial, zero-shot settings**: ActiveVOI achieves top-tier success rates among all visual agents that operate in a single trial without fine-tuning, significantly outperforming state-of-the-art baselines like GPT-4O (vision), LLAVA-13B, and BUTLER. This highlights the practicality and robustness of the method in realistic, one-shot planning scenarios.\n\n- **Principled and novel framework**: The combination of Lifted Regression for generating symbolic subgoals and VOI-guided active sensing represents a creative and theoretically well-grounded integration of symbolic AI and modern pretrained models. The decomposition into object-centric descriptions and relational constraints allows for efficient and targeted exploration.\n\n- **Clear methodology with strong theoretical foundation**: The paper includes rigorous formalization of the planning setup, VOI estimation, and regression mechanics. The object-relevance partitioning and the assumptions underlying the belief factorization are clearly articulated.\n\n\nWeaknesses:\n- **Heavy reliance on LLM/VLM accuracy**: The method assumes that GPT-4 (or similar models) provides reliable estimates for object relevance. As shown in ablations, performance drops with weaker LLMs (e.g., GPT-3.5). This dependence may limit robustness, especially when commonsense priors are uncertain or incorrect—an issue not deeply explored in the failure analysis.\n\n- **Scalability and memory handling remain open questions**: The planning horizon is capped, and the system operates episodically without a persistent knowledge base. In lifelong or multi-task settings, the agent would likely re-discover the same information repeatedly, as no explicit mechanism exists for long-term memory, knowledge abstraction, or reuse.\n\nQUESTIONS:\n- **LLM/VLM failure handling**: In cases where the LLM or VLM provides incorrect or misleading priors (e.g., low belief for a relevant object), does ActiveVOI have any fallback or exploration mechanism? Could belief calibration, ensembling, or conservative VOI strategies improve robustness to noisy commonsense reasoning?\n\n- **Knowledge persistence and reuse**: Have you considered augmenting ActiveVOI with a memory system for storing verified object properties or relations across episodes? This could support lifelong learning and reduce redundant sensing in related future tasks. Would VOI still be usable in such a setting?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-05"}