Title: Improving LLM-based Global Optimization with Search Space Partitioning
Primary area: optimization
Keywords: global optimization, LLMs, bayesian optimization

Abstract: Large Language Models (LLMs) have recently emerged as effective surrogate models and candidate generators within global optimization frameworks for expensive black-box functions. Despite promising results, LLM-based methods often struggle in high-dimensional search spaces or when lacking domain-specific priors, leading to sparse or uninformative suggestions. To overcome these limitations, we propose HOLLM, a novel global optimization algorithm that enhances LLM-driven sampling by partitioning the search space into promising subregions. Each subregion acts as a "meta-arm" selected via a bandit-inspired scoring mechanism that effectively balances exploration and exploitation. Within each selected subregion, an LLM then proposes high-quality candidate points, without any explicit domain knowledge. Empirical evaluation on standard optimization benchmarks shows that HOLLM consistently matches or surpasses leading global optimization methods, while substantially outperforming traditional global LLM-based sampling strategies.

TL;DR: HOLLM is a global optimization algorithm that adaptively partitions the search space and generates promising candidate solutions in them with LLMs.