MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated
  Experimental Feedback
Authors: name: Wanhao Liu, name: Zonglin Yang, name: Jue Wang, name: Lidong Bing, name: Di Zhang, name: Dongzhan Zhou, name: Yuqiang Li, name: Houqiang Li, name: Erik Cambria, name: Wanli Ouyang

Hypothesis ranking is a crucial component of automated scientific discovery,
particularly in natural sciences where wet-lab experiments are costly and
throughput-limited. Existing approaches focus on pre-experiment ranking,
relying solely on large language model's internal reasoning without
incorporating empirical outcomes from experiments. We introduce the task of
experiment-guided ranking, which aims to prioritize candidate hypotheses based
on the results of previously tested ones. However, developing such strategies
is challenging due to the impracticality of repeatedly conducting real
experiments in natural science domains. To address this, we propose a simulator
grounded in three domain-informed assumptions, modeling hypothesis performance
as a function of similarity to a known ground truth hypothesis, perturbed by
noise. We curate a dataset of 124 chemistry hypotheses with experimentally
reported outcomes to validate the simulator. Building on this simulator, we
develop a pseudo experiment-guided ranking method that clusters hypotheses by
shared functional characteristics and prioritizes candidates based on insights
derived from simulated experimental feedback. Experiments show that our method
outperforms pre-experiment baselines and strong ablations.