ICML 2025

Flow-of-Options: Diversified and Improved LLM Reasoning by Thinking
Through Options
Lakshmi Nair 1 Ian Trase 1 J. Mark Kim 1
Abstract
We present a novel reasoning approach called
Flow-of-Options (FoO), designed to address in-
trinsic biases in Large Language Models (LLMs).
Flow-of-Options enables LLMs to systematically
explore a diverse range of possibilities in their rea-
soning, as demonstrated by an FoO-based agentic
framework developed for autonomously solving
Machine Learning (ML) tasks.
FoO enforces
diversity in LLM solutions through compressed
and interpretable task representations, resulting in
improvements of 38.2% – 69.2% on standard data
science tasks, and 37.4% – 47.9% on therapeutic
chemistry tasks, as compared to state-of-the-art
baselines. With an overall operation cost under
$1 per task, our framework is well-suited for
cost-sensitive applications. Going beyond tabular
classification and regression, we show the broader
applicability of our FoO-based agentic system to
tasks such as reinforcement learning and image
generation. Our code is open-sourced at: https:
//github.com/flagshippioneering/
Flow-of-Options.
1. Introduction
Large Language Models (LLMs) have impacted automation
on a wide spectrum of tasks, motivating a growing body
of work in agentic system design, e.g., web-browsing (Yu
et al., 2025), idea generation (Li et al., 2024b), and data
science (Guo et al., 2024; Grosnit et al., 2024). Other ap-
proaches have focused on improving the inherent reasoning
capabilities of the LLMs, through Tree-of-Thoughts (ToT)
(Yao et al., 2024) or Chain-of-Thoughts (CoT) (Wei et al.,
2022). This paper seeks to bridge the two areas, exploring
how “thought-based” approaches can be extended to im-
prove agentic systems. Hence, we propose Flow-of-Options
1Flagship Pioneering, Cambridge MA, USA. Correspondence
to: Lakshmi Nair <lnair@flagshippioneering.com>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. ...