Title: Online Feedback Efficient Active Target Discovery in Partially Observable Environments
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion Model

Active Target Discovery
Abstract: In various scientific and engineering domains, where data acquisition is costly—such as in medical imaging, environmental monitoring, or remote sensing—strategic sampling from unobserved regions, guided by prior observations, is essential to maximize target discovery within a limited sampling budget. In this work, we introduce Diffusion-guided Active Target Discovery (DiffATD), a novel method that leverages diffusion dynamics for active target discovery. DiffATD maintains a belief distribution over each unobserved state in the environment, using this distribution to dynamically balance exploration-exploitation. Exploration reduces uncertainty by sampling regions with the highest expected entropy, while exploitation targets areas with the highest likelihood of discovering the target, indicated by the belief distribution and an incrementally trained reward model designed to learn the characteristics of the target. DiffATD enables efficient target discovery in a partially observable environment within a fixed sampling budget, all without relying on any prior supervised training. Furthermore,  DiffATD offers interpretability, unlike existing black-box policies that require extensive supervised training. Through extensive experiments and ablation studies across diverse domains, including medical imaging, species discovery and remote sensing, we show that DiffATD performs significantly better than baselines and competitively with supervised methods that operate under full environmental observability.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn scanning problems such as MRI imaging, the data are locally acquired and one would actively decide the targeted scanning area (it is expensive to have a comprehensive scan). In such problems, it is difficult to train a decision maker to guide the scanning process, because fully scanned data are scarce. To this end, this paper considers the scanning steps as denoising steps in the context of a diffusion model. The authors use the diffusion model to build distribution on predictive space at different area. Then, one may leverage the distribution to guide the information gained by scanning different grids, may the information be exploration or exploitation.\n\n**Contributions:**\n- cast scanning decisions into diffusion dynamics.\n- apply exploitation and exploration heuristics leveraged by the Bayesian experimental design community.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The studied problem is important and well-introduced.\n- I find it quite interesting to cast the query steps as diffusion dynamics.\n- The method is evaluated on 3 different test problems. \n\n**Weaknesses:**\n- This paper has a lot of parameters and the notation is very complicated. I personally find section 5 (main method section) quite hard to follow. Perhaps some more description on the intuition would help?\n- If I am not wrong, this method has quite significant online computation for each scanning decision (training reward models, optimizing scores etc). For problems like species discovery, perhaps the online complexity is not a major issue. But for problems like MRI scanning (which is mentioned repeatedly in the paper), online computational time is a limitation, and the authors might really need to discuss this. Online complexity is a very common problem in Bayesian experimental design.\n- It seems to me that there are a lot of hyperparameters and models to be tuned. Like exploration weight, $\\kappa(\\cdot)$, and more. This would make it rather complicated to apply the method. For instance, table 4 shows that the SR drops to 0.8465 with $\\alpha = 0.2$ while in table 3 we see that the baseline GA is 0.8261 (I wonder whether the authors conducted statistical tests over repeated experiments? Or were all experiments only run once?).\n- It seems that the algorithm relies on the reward function to correctly identify the target area, while the reward function is trained as the scanning proceeds. One might need to discuss model misspecification problem (e.g. bad model $\\rightarrow$ bad query, bad query $\\rightarrow$ bad model).\n- In addition, if I am not wrong, the reward function is a black-box function, while in the introduction the authors claim white-box policy, which might be exaggerated.\n\n**Clarification:**\n- Could you elaborate why equation (7) and appendix line 434 are valid? I mean, if both the reward and the negative entropy are non-negative, then the product scales in the same direction of the two factors. But if either of them spans across positive and negative values, then optimizing the product is not the same as optimizing the reward and the negative entropy.\n\nQUESTIONS:\n**Clarification:**\n- Could you please elaborate how the generative $\\hat{x}_t^{i}$ conditioned on the observations $\\tilde{x}_{t-1}$? Does equation (3) indicate this or from which equation may I see this?\n- In Thm 1, why do you have [ \\hat{x}_t^{i} ]_{q_t} ? I thought $\\hat{x}_t^{i}$ is already the $i$-th grid, but do we use grid $q_t$ to index $\\hat{x}_t^{i}$?\n- In line 159, it seems like the aim here is to avoid computing all possible grids. How is this actually achieved by Thm 1?\n- In Thm 1, how strong is the assumption $\\alpha_i = \\alpha_j$ and is this assumption limiting in practice?\n- Could you elaborate why equation (7) and appendix line 434 are valid? I mean, if both the reward and the negative entropy are non-negative, then the product scales in the same direction of the two factors. But if either of them spans across positive and negative values, then optimizing the product is not the same as optimizing the reward and the negative entropy.\n\n**Minor:**\n- You use $Q_t = q_0, …, q_t$ in the paper. However, $q_0, …, q_{t-1}$ were from the past and are actually fixed while decision $q_t$ is optimized. I would suggest writing $Q_{t-1}, q_t$ instead to be clearer on the distinction.\n- Code: When you attach the code, you may consider removing .git folder. That creates unnecessary tracks and github links (your repo seems anonymous which is fine).\n- Code: you may consider adding versions to requirements.txt, e.g. scipy can sometimes have version issues.\n\nLIMITATIONS:\nI tend to say one should discuss the computation needed for each query, and maybe discuss the quality of the trained reward function.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel method, DiffATD, to guide diffusion models in task discovery. To this end, following existing ideas from experimental design, DiffATD trades off between exploration of novel areas in the search space, measured by (an approximation of) the model's entropy, to exploitation of a reward that is learned during the learning process.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n* The problem is important; DiffATD might be useful in more complex domains (such as robotics) for task discovery.\n* The experiments are very comprehensive, showing solid results across several independent tasks.\n* I am not an expert in this field, but the proposed method seems novel.\n\n## Weaknesses\n* Experiments that demonstrate task discovery beyond vision-related tasks (e.g. drug discovery or robotics) would significantly strengthen the paper.\n* The main empirical results of the paper essentially only ablate DiffATD's exploration and exploitation components, comparing against max-entropy, greedy and uniform sampling. The paper only compares DiffATD with multi-armed bandits techniques (UCB and $\\epsilon$-greedy) on MNIST in the appendix. Additional experiments showing demonstrating DiffATD's superiority compared to more competitive methods will improve the paper. For instance, comparing with a baseline that relies on supervised learning (as mentioned in the related works), and showing that DiffATD is much more sample efficient would strengthen the paper.\n* My understanding is that most experiments were carried out only on one seed, with the exception of the experiment in appendix S, which studies only two datasets and lacks comparison against other existing baselines in the literature.\n\nQUESTIONS:\n* Where do you get the rewards from in your experiments? You motivate DiffATD with problems in which labeled data (for supervised learning) is expensive; however, in your experiments it seems like the reward is given by some ground-truth data that was collected offline. Can you please clarify that?\n\nLIMITATIONS:\nThe authors did not mention any limitation of their method DiffATD, answering in the submission guideline:\n> Does the paper discuss the limitations of the work performed by the authors? We design a novel algorithm for a new task.\n\nI believe some more comprehensive discussion on how one can improve DiffATD and what were the challenges in making it work would improve the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

comments: SUMMARY:
This paper investigates the target discovery problem with a limited computational budget.
Specifically, leveraging the generative capacity of diffusion models, this paper introduces DiffATD, which utilizes diffusion dynamics for efficiently active target discovery with a limited sampling budget and better interpretability.
Experimental results show that DiffATD outperforms baselines in various benchmarks.

STRENGTHS AND WEAKNESSES:
### Strengths
1. Interesting problem settings and well-defined problem formulation from the perspective of entropy and mutual information.

### Weakness
1. Comparison with other search algorithms (e.g., MCTS and UCB).
2. Comparison with other objective detection algorithms.
3. Could the authors explain the relationship between the proposed method and information bottleneck?

### Typos
1. Lines 190: proof in ??\
2. Lines 234: Active Discovery of Species.
3. Lines 244: Active Discovery of Skin Disease.
4. Lines 271 and 293: Same issues

QUESTIONS:
### Questions
1. Could the authors provide a comparison with the other existing search algorithms?
2. Could the authors provide a comparison with the other existing objective detection methods?
3. Could the authors explain the relationship between the proposed method and information bottleneck?

LIMITATIONS:
No. Comparison with existing search algorithms (e.g., MCTS) and objective detection methods.
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-27

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies active target discovery where items have to be found across a domain. The paper proposes DiffATD, which maintains a belief distribution over unobserved states. DiffATD then selects new observations by carefully balancing exploration and exploitation. The paper evaluates DiffATD across of item \"discovery\" examples with images. Across those examples, DiffATD outperforms uniform pixel-selection and variants based around pure exploration or pure exploitation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The idea of balancing exploration and exploitation for active target discovery appears novel and interesting.\n* The proposed method significantly outperforms baselines across all experiments. The experimental methodology appears solid, although experiments are small-scale, and are not comparing to many baselines. I am not familiar enough with the subfield to say conclusively whether any particular baselines are missing.\n\nWeaknesses:\n* The key weakness in my eyes is the omission of a comparison to methods from bandits / Bayesian optimization. It seems to me that the problem could be modeled as a noise-free bandit problem. If modeled as a bandit problem, one could build upon a wide range of prior literature that studies how to effectively balance exploration and exploitation. There are a plethora of methods, for example, also some information-based methods such as \"information-directed sampling\" [1] which seems related to the proposed method.\n* The paper is not including any theoretical guarantees for the sampling scheme, as opposed to prior results from bandits / BayesOpt.\n* The term $\\kappa(B)$ from the sampling objective depends on the remaining budget $B$. This seems undesirable to me, as often a budget is not clearly defined (e.g., search and rescue from the motivation). Often, we simply want to identify the target \"as quickly as possible\". Many bandit algorithms do not require specification of a budget upfront, yet still provide anytime optimality guarantees.\n\nTypos:\n* lines 190: \"??\"\n\n[1]: Russo & Van Roy. Learning to Optimize via Information-Directed Sampling.\n\nQUESTIONS:\n* The paper does not ablate the choice of belief model experimentally. Is the choice of single-step diffusion important?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo address the challenge of identifying targets of interest in partially observable environments while minimizing costly ground-truth feedback, this paper introduces Diffusion-guided Active Target Discovery (DiffATD). The approach leverages Tweedie’s formula to construct a belief distribution over unobserved regions. Using this distribution, the method employs an exploration-exploitation strategy: exploration is guided toward regions with high entropy, while exploitation targets regions with low entropy. Experiments are conducted to demonstrate the effectiveness of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths:* \n\n* The authors effectively articulate the motivation of their approach in the introduction section, particularly in comparison to standard reinforcement learning methods, emphasizing its advantage in reducing labeling costs.\n* Figure 1 clearly illustrates the role of the exploration-exploitation strategy in efficiently identifying the target of interest.\n* The proposed methodology is grounded in  theoretical foundations and appears to offer a novel contribution.\n* The approach may offer  impact in domains such as medical imaging, species discovery, and remote sensing, where labeling is costly and limited.\n* An ablation study is conducted to demonstrate the contribution of individual components—specifically, exploration and exploitation—to the overall performance.\n\n\n*Weaknesses:* \n* The baseline comparisons are limited. Aside from random search and internal ablations (exploration-only and exploitation-only), no competitive baselines are included. The authors should compare against stronger alternatives where applicable [1–3]. If not applicable, authors need to provide the reason for that. \n* The results for the fully supervised approach such as FullSEG should be included as part of the all 3 datasets (Overhead Imagery, Species, Skin Disease).\n* Multi-armed bandit methods like Upper Confidence Bound (UCB) are not considered as baselines. Given their relevance, the authors should include such methods across all datasets to provide a more comprehensive comparison.\n* Some notations are unnecessarily long and could benefit from simplification for readability. For instance, symbols like e.g., $exploit^{score} (q_t)$, $likeli^{score}(q_t)$  in Equation 7 may be renamed for clarity.\n* Certain references appear to be missing. For example, the refer to the  proof aboveTheorem 3 is marked with “??”. \n* Figure captions lack sufficient detail. Specifically, Figure 2 would benefit from a more comprehensive explanation of the DiffATD framework, rather than the generic caption “An Overview of DiffATD.”\n* The function of the parameter $\\kappa (\\mathcal{B})$  in Equation 9, which balances exploration and exploitation, is not visualized. Including a plot or illustration of this function would improve interpretability.\n\nQUESTIONS:\nPlease refer to Strengths and Weaknesses Section\n\nLIMITATIONS:\nThe authors does not seem to include the Broader impacts and Limitations of their work. Some of the impacts of this work could be in domains like medical imaging, ecological species discovery, and remote sensing. In terms of the limitations, the proposed approach  may have higher computational cost which should be discussed in this paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}