# An In-depth Study of LLM Contributions to the Bin Packing Problem

Julien Herrmann, Guillaume Pallez

► To cite this version:

Julien Herrmann, Guillaume Pallez. An In-depth Study of LLM Contributions to the Bin Packing Problem. 2025. hal-05340506

HAL Id: hal-05340506

https://hal.science/hal-05340506v1

Preprint submitted on 31 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# An In-depth Study of LLM Contributions to the Bin Packing Problem

Julien Herrmann¹

Guillaume Pallez²

¹CNRS-IRIT, France

²INRIA, Rennes, France

## Abstract

Recent studies have suggested that Large Language Models (LLMs) could provide interesting ideas contributing to mathematical discovery. This claim was motivated by reports that LLM-based genetic algorithms produced heuristics offering new insights into the online bin packing problem under uniform and Weibull distributions. In this work, we reassess this claim through a detailed analysis of the heuristics produced by LLMs, examining both their behavior and interpretability. Despite being human-readable, these heuristics remain largely opaque even to domain experts. Building on this analysis, we propose a new class of algorithms tailored to these specific bin packing instances. The derived algorithms are significantly simpler, more efficient, more interpretable, and more generalizable, suggesting that the considered instances are themselves relatively simple. We then discuss the limitations of the claim regarding LLMs' contribution to this problem, which appears to rest on the mistaken assumption that the instances had previously been studied. Our findings instead emphasize the need for rigorous validation and contextualization when assessing the scientific value of LLM-generated outputs.

## 1 Introduction

Heuristics and approximation algorithms are widely used to tackle NP-hard combinatorial optimization problems arising in real-world applications. Manually designing, adapting, and tuning an effective heuristic for a given problem demand expert experience and a deep understanding of the problem's constraints and objectives. Hyper-heuristics have therefore emerged as an active research area, aiming to automate algorithm design by selecting and tuning effective heuristics

from a set of heuristic components [1, 2, 3]. Genetic algorithms, for instance, evolve a pool of candidate solutions through crossover and mutation operators [4, 5]. However, the intrinsic complexity and heterogeneity of NP-hard problems usually limit these operators to narrow classes of heuristic spaces, typically parameterized operations predefined by human experts [1].

Recent work [6, 7, 8] has proposed using Large Language Models (LLMs) to open up new possibilities for the genetic operators, enabling (i) exploration of different heuristic spaces; and (ii) a better understanding of the resulting solution. These approaches combine LLMs with evolutionary search techniques to produce heuristics in an automated manner. One of the most visible recent papers, “Mathematical discoveries from program search with large language models” [6], published in Nature in December 2023 and cited by more than 800 papers (according to Google Scholar) as of October 31st 2025, introduces the FunSearch procedure. It models heuristic design as a search problem in the space of functions, and uses LLMs in an evolutionary framework to iteratively improve the quality of the generated functions. FunSearch has been applied to three problems: cap set [9], admissible set [10], and online bin packing [11]. The best generated heuristic for each problem is reported in the paper’s appendix [6].

The key claims of Romera et al. [6] are that “FunSearch outputs programs that tend to be interpretable” and that “LLMs should be seen as a source of diverse (syntactically correct) programs with occasionally interesting ideas”. In this paper, we reassess these claims, focusing on the online bin packing problem. Can the contributions of the LLM-based evolutionary framework for this specific NP-hard problem be considered as “interesting”, and how should they be evaluated? We first show
that interpreting the proposed heuristics is nontrivial. We then show that this difficulty is avoidable by providing a simple, more efficient, two-parameter heuristic that generalizes to a much broader set of instances. While the difficulty of evaluating the heuristic is anecdotal, it is confirmed by subsequent work that still relies on LLMs instead of grasping the key information that could be obtained from Romera et al. [6].

The rest of the paper is organized as follows. We first review related work on hyper-heuristics and LLM-assisted algorithm design in Section 2. We then discuss the interpretability of the heuristics proposed in Romera et al. [6] and subsequent work in Section 3, before discussing their scientific contribution in Section 4. In particular we propose and evaluate a new, two-parameter simple heuristic for these instances. Building on these observations, we critically reassess the claim that LLM-based evolutionary frameworks can occasionally contribute scientifically interesting ideas, and argue that, for bin packing, the available evidence in Romera et al. [6] is insufficient to establish this claim.

## 2 Related work

There is a long history of *Automatic Heuristic Algorithm* design, also known as *hyper-heuristics*, for combinatorial problems [12, 2, 3]. This body of work includes hyper-heuristics for the bin packing problem [13, 14]. Among these approaches, genetic programming [15, 16] is a generic optimization principle inspired by natural evolution that provides an interpretable approach to algorithm design [17, 18]. However, it relies on handcrafted algorithmic components and domain knowledge, and is typically constrained by the heuristic space predefined by human experts.

**On the inclusion of LLMs in the design of hyper-heuristics** Standalone LLMs with prompt engineering usually prove insufficient for solving open problems as they tend to confabulate and struggle to produce novel, verifiably correct ideas beyond existing knowledge [19]. To overcome these limitations, recent work has combined them with evolutionary frameworks [8, 20, 21, 22]. It is believed that LLMs could leverage the definition of genetic operators for code generation within evolutionary frameworks [23, 24, 25, 20]. The crossover and mutation operators when evolving heuristics are handled by

LLMs that have been fine-tuned on a large corpus of code, broadening the heuristic search space.

**Machine-assisted theory discovery** Beyond designing hyper-heuristics, some studies claim that using AI in the design of hyper-heuristics allows for scientific discovery [26, 27]. This is the central claim of Romera et al. [6]. The authors introduce FunSearch, a distributed evolutionary framework that combines a frozen LLM as a code generator with an evaluation function that scores candidate heuristics. It evolves the critical parts of a predefined heuristic skeleton using best-shot prompting, island-based population management, and asynchronous sampling/evaluation to iteratively improve the quality of the generated heuristics. FunSearch is based on Codey, an LLM from the PaLM 2 model family [28], and requires millions of samples (i.e., queries to the LLM) and days of training to design a heuristic suited to a given dataset. In Romera et al. [6], the authors focus on three NP-hard combinatorial problems, including the *online bin packing* [11] problem. They claim that FunSearch allows “the discovery of new scientific results” but no in-depth analysis of the generated heuristics is provided to establish whether the discoveries are truly new. The position paper by Jiang et al. [29] identifies FunSearch as a milestone in “neuro-symbolic mathematics” — the use of statistical models to aid formal, verifiable reasoning.

**Implications of Romera-Paredes et al.' work** At the time of the writing (October 1st, 2025), Romera et al.'s work [6] has been cited by 744 research papers. Only 91 of them specifically mention *bin packing*. An evaluation of a convenience sample biased toward the most cited of the remaining 653 papers suggests that most of them cite the article primarily by title to assert that coupling LLMs with evolutionary frameworks can yield “mathematical discoveries”. Among the 91 papers that cite Romera-Paredes and mention *bin packing*, few actually study the bin packing problem. Among those that do, two main categories emerge: (i) papers that design improved LLM-based hyper-heuristics [30, 31, 32]—for instance refining code generation via more suitable evolutionary frameworks and mitigating FunSearch's inefficiency due to millions of LLM calls; and (ii) papers that evaluate the claimed efficiency [33]. *Notably, although Romera et al. [6] claim that their examples demonstrate that program search with LLMs can enable new mathematical discoveries, at this*
time, we have not found work that builds on their
generated heuristics.

Liu et al. [30] present an *Evolution of Heuristics* (EoH) framework that jointly evolves high-level natural language descriptions (*thoughts*) and corresponding code implementations using LLMs within a population-based evolutionary algorithm. The framework applies prompt-based variation operators to generate and refine heuristics for NP-hard combinatorial problems, including bin packing.

ReEvo [32] and HSEvo [31] extend FunSearch by incorporating reflective feedback loops, hybrid evolutionary operators, and diversity-preserving mechanisms. Farquhar et al. [34] build on FunSearch’s architectural choice of separating generation and verification evaluator to propose a semantic entropy metric that quantifies model uncertainty and aims to reduce the risk of hallucinated proposals.

Sim et al. [33] conduct an extensive benchmarking study on LLM-evolved heuristics for bin packing. Their experiments reveal that many LLM-evolved heuristics tend to overfit and generalize poorly beyond narrow instance clusters. This critique underscores the need for standardized benchmarking and caution in claiming broad applicability.

However, to the best of our knowledge, no studies
have examined the interpretability of LLM-evolved
heuristics for the bin packing problem to understand
*why* they outperform baseline heuristics and whether
they constitute “truly new discoveries”. Likewise, no
study has instantiated the insight attributed to the
LLM-evolutionary pipeline as a reusable algorithmic
principle for bin packing.

**Stochastic Online bin packing / Average analysis on online bin packing** When claiming a mathematical discovery, it is essential to review the relevant state of the art— something that was not done in Romera et al.’s work. The problem of online bin packing, where item sizes are independent and identically distributed (i.i.d) random variables drawn from a stochastic distribution, is known as Stochastic Online Bin Packing [35, 36], or, equivalently, as the *average case-analysis* of online bin packing [37]. For such algorithms, performance is generally assessed through the *expected competitive ratio* and the *expected wasted space*, where wasted space denotes the total amount of unused capacity in partially filled bins. Most existing studies assume that the underlying distribution is unknown and, therefore, cannot be leveraged by the algorithm [36, 37].

In contrast, Romera et al. [6], explicitly considers a known distribution that the algorithm can exploit. When such information is available, prior research has focused almost exclusively on the uniform case —either $U[0, 1]$ [38, 39, 37] or $U[0, b]$ with $b \le 1$ [40]. In the latter setting, Bentley et al. [40] showed that the *first-fit* and *best-fit* algorithms are asymptotically optimal and achieve constant wasted space with high probability when $b \le 0.5$. To the best of our knowledge, there is no prior research on the stochastic online bin packing problem where the uniform distribution admits a positive lower bound $a > 0$, nor any that considers non-uniform distributions such as the Weibull distribution, as explored in Romera et al. [6].

3 Interpretability of LLM-
evolved heuristics

One of the key claims of Romera et al. [6] is that
“FunSearch outputs programs that tend to be inter-
pretable”. In this section we focus on the technical
content of Romera et al. [6] through an in-depth anal-
ysis of the produced heuristics. Our goal is to assess
the interpretability of the reported discovery. We fur-
ther extend this analysis by considering subsequent
work, notably Liu et al. [30].

All the code used for the experiments and analyses
in this paper is available at https://gitlab.inria.fr/ai-
interpretability/llm-4-bin-packing.

**3.1 Online bin packing [11] and priority function**

Bin packing [11] is the problem of packing a set of
items of various sizes into the minimum number of
fixed-size bins. It is a well-known NP-hard combina-
torial problem with applications in many areas, from
operations research to scheduling jobs on compute
clusters. In the *online* scenario, items are packed
as they arrive, in contrast with the *offline* scenario
where all items are known beforehand. Solving *on-
line bin packing* problems therefore requires heuristics
that greedily decide to which bin each incoming item
should be assigned.

A common approach to solving the online bin pack-
ing problem is to use priority functions: at all times,
bins are sorted according to a *priority function* that
depends on their remaining space and the size of the
incoming item. The heuristic then allocates the item
to the bin with the highest priority score.
The most widely used heuristics for *bin packing* are FirstFit, BestFit, and WorstFit. In FirstFit, bins are prioritized by decreasing index. In BestFit (resp. WorstFit), bins are prioritized by decreasing (resp. increasing) remaining available space. These simple heuristics show good performance in practice [41] both in the worst case (or adversarial case) and in the average uniform case [40].

Romera et al. [6] and subsequent works considered special instances of *online bin packing* instances where incoming item sizes follow two probability distributions: UNIFORM(20, 100) and WEIBULL(3.0, 45). For each of these instances they developed specific heuristics which we analyze in Section 3.2 and Section 3.3. Subsequent work [33] has shown that these heuristics fail to generalize across different parametrization of the problem.

## 3.2 Item sizes following a Uniform(20, 100) distribution with bin capacity 150

We begin by discussing the heuristic developed for the UNIFORM(20, 100) problem. The instances consist of items with integer sizes drawn from a uniform distribution in [20, 100]. Each bin has a capacity of 150.

**Heuristic generation** The best heuristic generated by Romera et al. [6] for UNIFORM(20, 100) is denoted c12 (Algorithm 1). It was generated by evolving FunSearch on 20 training instances, randomly generated using the same parameters as those in the first of the OR-Library bin packing benchmarks, OR1 [42]. The training instances consist of 120 items.

<caption>**Algorithm 1** c12 [6] for items of distribution UNIFORM(20, 100)</caption>

1. def priority_c12(item: float, bins: np.ndarray) -> np.ndarray:
2.     def s.bin, item:
3.         if bin - item <= 2:
4.             return 4
5.         elif (bin - item) <= 3:
6.             return 3
7.         elif (bin - item) <= 5:
8.             return 2
9.         elif (bin - item) <= 7:
10.        return 1
11.    elif (bin - item) <= 9:
12.        return 0.9

```python
elif (bin - item) <= 12:
    return 0.95
elif (bin - item) <= 15:
    return 0.97
elif (bin - item) <= 18:
    return 0.98
elif (bin - item) <= 20:
    return 0.98
elif (bin - item) <= 21:
    return 0.98
else:
    return 0.99
return np.array([s.bin, item] for
                 bin in bins])
```

**Priority function and analysis** The authors of FunSearch observed that several of the FunSearch heuristics “assign[ed] items to least capacity bins only if the fit is very tight after placing the item.” [6]. We verify this claim, and provide a more precise characterization of their behavior.

c12 is the most easily interpretable heuristic in Romera et al. [6]. It consists of a list of 11 if/then statements where conditions on the remaining bin capacity after placing the item determine the bin's priority. One can notice that the priority of the bin is the highest when `bin-item` is less or equal to 7 while decreasing when `bin-item` increases. It means that if the remaining size of a bin after putting the item is below 7, the bin will be selected while prioritizing the fullest bin among them. If no such bin exists, the heuristic will select the first bin leaving more than 21 remaining size (line 23). Since empty bins are always available, bins that would leave between 7 and 21 remaining capacity are never selected by the heuristic. Note that, in practice, Lines 11-20 of c12 are never useful and can be removed. A graphical approximation of the algorithm's behavior is provided in Figure 1.

**Discussion on the performance** c12 can be simplified by smoothing its behavior: applying BestFit to bins where the remaining space after placing the item would be less than 7 (if such bins exist), and otherwise applying FirstFit to bins where the remaining space would be larger than 21. We refer to this heuristic as Smooth_c12. In border cases, these two algorithms may behave slightly differently: for instance, two bins leaving 4 and 5 remaining size will be treated identically by c12, while Smooth_c12 will prioritize the fullest one. However, our experiments show that their
<figure><img src="image_2.png" /><figcaption>Figure 1: Priority provided by c12. The scale of the y-axis is not linear to highlight the difference of priority.</figcaption></figure>

performance, relative to BestFit in terms of the number of bins used, is similar (see Figure 2).

<figure><img src="image_3.png" /><figcaption>Figure 2: Boxplots showing the heuristics performance relative to BestFit over 1000 instances of the UNIFORM(20, 100) distribution with 500 items and bin capacity 150.</figcaption></figure>

We hypothesize that the average 2.0% gain of c12 and Smooth_c12 over BestFit is due to several combined factors:

* (H1) **The large number of items to be scheduled.** We hypothesize that the performance gain arises when many items must be scheduled. This is supported by Figure 3, which shows that BestFit outperforms c12 when the number of items is small: for fewer than 90 items, the average performance of c12 is lower than that of BestFit.

* (H2) **The minimum item size imposed by the distribution.**

Indeed, since all generated items are larger than 20, placing an item in a bin that leaves less than 20 units of remaining capacity effectively *closes* that bin. Therefore, a strategy that avoids closing bins with *large* remaining capacity is advantageous when many items are expected, as it increases the chance of closing the bins more efficiently later. We discuss (H2) in Section 4 with other uniform distributions by studying a parametrized version of Smooth_c12: ab-FirstFit (Algorithm 4).

<figure><img src="image_4.png" /><figcaption>Figure 3: Average performance of c12 relative to BestFit over 1000 instances of the UNIFORM(20, 100) distribution, as the number of items increases (bin capacity = 150).</figcaption></figure>

## 3.3 Item sizes following a Weibull(3.0, 45) distribution with bin capacity 100

We now analyze the heuristics presented in Romera et al. [6] and Liu et al. [30] for the WEIBULL(3.0, 45) distribution. The instances consist of items with integer sizes drawn from a Weibull distribution with shape parameter $k = 3.0$ and scale parameter $\lambda = 45$. Each bin has a capacity of 100.

### 3.3.1 c14 heuristic in Romera et al. [6]

**Heuristic generation** The best performing heuristic by Romera et al. [6] is called c14 (Algorithm 2). It was generated by evolving FunSearch on 5 bin packing instances of 5000 items.

**Algorithm 2** c14 [6] for items of distribution WEIBULL(3.0, 45)
1 `def priority_c14(item: float, bins: np.ndarray) -> np.ndarray:`

2 `score = (bins - max(bins))**2 / (item + bins**2 / item**2 + bins**2 / item**3)`

3 `score[bins > item] *= -1`

4 `score[1:] -= score[:-1]`

5 `return score`

**Priority function and analysis** c14 is considerably less interpretable. Two aspects are particularly surprising: (i) the first formula for the score function (line 2), as well as (ii) the final update of the score function, which makes the score of a bin dependent on the score of one of its neighbor (line 4).

<figure><img src="image_5.png" /><figcaption>Figure 4: Plotting $f: s, b \mapsto score(s, b)$, where score is the function from line 2 of c14. In these plots we use: $c = 100$, $s_1 = 20$, $s_2 = 50$. There is an inflexion point at $\frac{c}{1+s+s^2}$.</figcaption></figure>

The score function defined on line 2 depends on the item size $s$, the maximum bin capacity $c$, and the current remaining space $b$ in each bin to determine each bin priority, with the formula:

$$f: (s, b) \mapsto \frac{(b-c)^2}{s} + \frac{b^2}{s^2} + \frac{b^2}{s^3}$$

Function $f$ is illustrated in Figure 4. In theory, it is a decreasing function on $[s; \frac{c}{1+s+s^2}]$, which then slightly increases on $[\frac{c}{1+s+s^2}; c]$. In practice, given that the distribution of item sizes $s$ contains few small

values, and for large bin capacities $c$, the function $f$ is almost always decreasing.

Line 3 takes the additive inverse for each bin that is not a perfect-fit (i.e. $b > s$), leaving only perfect-fit bins with a positive score. Thus, up to line 3, the priority function can be interpreted as follows: if a perfect-fit bin (i.e., a bin whose remaining capacity equals the item size) exists, it is selected; otherwise, a new bin is opened.

Line 4 is arguably the most opaque part of the LLM-evolved heuristic: it subtracts from each bin's score the score of the previous bin in the implementation order. We consider this to be the point at which the interpretability of the algorithm based solely on its code ends. Beyond this point, our analysis is experiment-based and required several sets of experiments to truly understand the actual behavior of the heuristic. In the following, we present only the successful ones.

**Experimental analysis of c14's behavior** Figure 4 provides insight into the mechanism of c14. After line 4 of the algorithm, a bin will have a high priority if its own $f$ value is low (i.e. it is almost empty) and/or its predecessor's $f$ value is high (i.e. its predecessor is “full-ish”). This observation prompted us to investigate the actual differences in behavior between c14 and WorstFit.

To this end, we conducted an item-by-item comparison between the decisions taken by c14 and those that would have been selected by WorstFit. In the following, we refer to an old bin as one that already contains at least one item, and to a new bin as one that is empty. Figure 5 shows, for each new item, the cases where both heuristics behave identically (both open a new bin, or both schedule the item in the same old bin) and the cases where they differ (selecting different old bins, or c14 opening a new bin while WorstFit would not). Note that if c14 can place an item in an old bin, then WorstFit can as well, and will do so. Consequently, there is no case where WorstFit opens a new bin while c14 uses an old one. The main observations are as follows: when neither algorithm opens a new bin, they most often schedule the item in the same bin, resulting in similar behavior. However, c14 frequently opens a new bin when WorstFit does not, i.e., when a bin with sufficient remaining space exists.

To better understand this divergence, we examine, in Figure 6, the remaining space in the bins chosen by WorstFit whenever c14 instead opens a new bin. Figure 6 illustrates c14's behavior from a different
<figure><img src="image_6.png" /><figcaption>Figure 5: Comparing the behavior item-per-items of c14 and WorstFit on the WEIBULL(3.0, 45) distribution with 50k items and bin capacity 100.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 6: Remaining size in the bins used by WorstFit after fitting the item in a non-empty bin on the WEIBULL(3.0, 45) distribution with 50k items and bin capacity 100.</figcaption></figure>

perspective: in practice, when WorstFit would schedule an item into a bin whose remaining space is less than the item size plus a certain threshold (approximately 20), c14 instead opens a new bin.

Finally, based on all these evaluations, we propose the following practical interpretation of c14's general behavior: apply BestFit if a perfect-fit bin exists; otherwise, if there exists a bin whose remaining space exceeds the item size by more than 20, apply WorstFit; if neither condition holds, open a new bin. Note that, unlike for c12, this interpretation differs nonnegligibly from the actual behavior of c14 (see Figure 6).

**Discussion on the performance** In practice c14 appears to be a more opaque implementation of the same underlying principle as c12, which relies on two key hypotheses: (H1) the presence of a large number of items to be scheduled, and (H2) a distribution containing few elements smaller than a given threshold. As in the UNIFORM(20, 100) case, (H1) can be verified by examining the relative performance of c14 compared to BestFit as the number of items increases

(Figure 7). In this case, the performance gap for small numbers of items is even more pronounced.

<figure><img src="image_8.png" /><figcaption>Figure 7: Average performance of c14 relative to BestFit over 1000 instances of the WEIBULL(3.0, 45) distribution, as the number of items increases (bin capacity = 100).</figcaption></figure>

### 3.3.2 EoH heuristic in Liu et al. [30]

Liu et al. [30] extended the work of Romera et al. [6] and developed the EoH heuristic for the WEIBULL(3.0, 45) distribution. Their methodology is largely similar, differing mainly in the evolutionary framework employed. The authors do not specify the number of instances used to evolve their heuristic nor the number of items per instance. However, since they compare their results to c14, it can be reasonably assumed that they employed the same WEIBULL(3.0, 45) training datasets as Romera et al. [6]. Their resulting algorithm is presented in Algorithm 3.

<caption>**Algorithm 3** The priority function evolved with EoH [30] for items from distribution WEIBULL(3.0, 45)</caption>

1. def priority_EoH(item: float, bins: np.ndarray) -> np.ndarray:
2.     diff = bins - item % (1 / (lenRemaining capacity))
3.     exp = np. exp(diff) % (exponent term)
4.     sqrt = np. sqrt(diff) % (square root term)
5.     ulti = 1 - diff / bins % (utilization term)
6.     comb = ulti * sqrt % (combination of utilization and square root)
7.     adjust = np. where(diff > (item * 3), comb + 0.8, comb + 0.3) % (hybrid adjustment term to)
penalize large bins
8
hybrid_exp = bins / ((exp + 0.7) *
  exp) # hybrid score based on
  exponent term
9
scores = hybrid_exp + adjust # sum
  of hybrid score and
  adjustment
10
return scores

In practice this score function $f_{\text{EOH}}$ is less opaque than that of c14 as it depends solely on the item size $s$ and the remaining bin capacity $b$, rather than on adjacent bins:

$$f_{\text{EOH}} : (s, b) \mapsto \frac{b}{(e^{b-s} + 0.7)e^{0.7s}} + \left(1 - \frac{b-s}{b}\right) \sqrt{b-s} + (0.5 \text{ if } b > 4s)$$

Figure 8 provides a visual representation of this function. One can notice that, even if the priority function is different, the underlying principle it captures remains essentially the same.

<figure><img src="image_9.png" /><figcaption>Figure 8: Plotting $s, b \mapsto f_{\text{EOH}}(s, b)$ when the bin remaining capacity $b$ varies. In these plots, we use: $c = 100, s_1 = 20, s_2 = 50$.</figcaption></figure>

## 3.4 Global discussion on interpretability

One of the main benefits of the LLM-evolved strategies described by Romera et al. [6] is their claimed interpretability, as stated by the authors: “Beyond being an effective and scalable strategy, discovered pro-

grams tend to be more interpretable than raw solutions.” In Romera et al. [6], interpretability goes beyond the mere ability to read an algorithm's behavior. It is understood as enabling humans to grasp the motivations underlying this behavior and to generate new knowledge from it. This distinction between readability and interpretability echoes a broader issue in AI research: human-readable outputs do not necessarily translate into human-understandable reasoning or transferable knowledge. Specifically, according to Kim et al. [43], “a method is interpretable if a user can correctly and efficiently predict the method’s results.”

While c12 was relatively easy to interpret, understanding c14 required scheduling expertise, multiple attempts (we only show the successful path) and several rounds of experimentation. Ultimately, our interpretation of c14 did not come from the description of the score function, but from experimental observations of the algorithm's behavior. For instance, the threshold value of 21 observed in Figure 6 is not apparent anywhere in the code. In that sense, our experimental analysis was no different from what would have been required to interpret a fully black-box algorithm. The fact that subsequent work has not attempted to reuse the knowledge supposedly created by these LLM-evolved heuristics does not bode well for their interpretability. For these reasons, we find that c14 is not interpretable in the sense defined by Romera et al. [6].

We conclude this section by noting that the approach of instantiating a parameterized priority function is not new in scheduling as a way to interpret algorithmic performance. For instance, Carastan-Santos et al. [44] proposed a similar approach to solve the problem of scheduling parallel tasks with release times, aiming to minimize the mean bounded slowdown. Their result states that tasks should be sorted according to the following priority function $f_1 : (r, s, n) \mapsto \log_{10}(r) \cdot n + 8.7 \cdot 10^2 \cdot \log_{10}(s)$, where $r$ denotes the release time of the task, $n$ the number of nodes, and $s$ its size. As with c14, this expression is readable and allows for some intuition, but it does not mean that it is (easily) interpretable.

## 4 Assessing the scientific value of LLM-evolved ideas

Having discussed interpretability, we now turn to the related question: are the ideas generated by the LLM-
based genetic frameworks scientifically interesting?

Indeed, the second main claim of Romera et al. [6] is that the use of LLMs in FunSearch "represents the first discoveries made for established open problems using LLMs". Specifically for bin packing, FunSearch is reported to have "discover[ed] a program that corresponds to a more efficient heuristic for online bin packing.". This claim is supported by performance comparisons against baseline methods, and they further emphasize that these "discoveries are truly new".

As noted in the previous sections, these LLM-based genetic studies do not address the canonical open problem of online bin packing. Instead they focus on a narrower and previously unstudied instance of the problem: online bin packing with items sizes drawn from specific parameterizations of the uniform or Weibull distributions. In addition, subsequent work [33] has shown that the generated solutions fail to generalize to other parameter settings of the uniform or Weibull distributions. In our view, one way to assess the scientific value of these heuristics is to examine whether they can serve as a basis for deriving new ones applicable to other instances of the uniform or Weibull distributions.

## 4.1 Can we generalize the idea behind c12, c14, and EoH?

A naive inspection of c12 suggests a solution with too many parameters to be directly instantiated. However, we have shown that its behavior can be approximated by a two-parameter heuristic: apply BestFit if there exists a bin where the remaining capacity would be less than 7, otherwise apply FirstFit to bins with remaining capacity greater than 21.

The main idea underlying c14 and EoH is the use of an analytical priority function. In our view, however, the parameterizations of these functions are too complex to serve as a foundation for constructing new ideas. This observation is consistent with the fact that subsequent studies have continued to rely on LLM-evolved heuristics rather than building on the underlying *new idea* itself.

## 4.2 Generalization of our analysis

Our analysis of the LLM-evolved heuristics suggests that, at a fundamental level, their performance relies on the same behavioral principle, which can be generalized as follows:

1. An item of size *s* is allocated to the best-fitting

bin if the fit is sufficiently tight, i.e., to bins whose remaining capacity is less than *s* + *a* for a threshold *a*.

2. Otherwise, it is allocated to a bin whose remaining capacity is greater than *s* + *b* for a threshold *b*, using any of the baseline heuristics.

3. If no such bin exists, the item is placed into a new bin.

In the remainder, we denote by ab-FirstFit (resp. ab-WorstFit, ab-BestFit) this heuristic using FirstFit (resp. WorstFit, BestFit) as the baseline strategy for the second step. Their priority function are described in Algorithm 4, 5, and 6.

<caption>**Algorithm 4 Priority function for ab-FirstFit for thresholds *a* and *b***</caption>

1. def priority_abFF(item: float, bins: np.ndarray) -> np.ndarray:
2.     def s bin, item):
3.         if bin <= item + a:
4.             return capacity - bin
5.         elif bin <= item + b:
6.             return 0
7.     else:
8.         return 1
9.     return np.array([s bin, item) for bin in bins])

<caption>**Algorithm 5 Priority function for ab-BestFit for thresholds *a* and *b***</caption>

1. def priority_abBF(item: float, bins: np.ndarray) -> np.ndarray:
2.     def s bin, item):
3.         if bin <= item + a:
4.             return capacity - bin
5.         elif bin <= item + b:
6.             return 0
7.     else:
8.         return 1/(bin - item)
9.     return np.array([s bin, item) for bin in bins])

<caption>**Algorithm 6 Priority function for ab-WorstFit for thresholds *a* and *b***</caption>

1. def priority_abWF(item: float, bins: np.ndarray) -> np.ndarray:
2.     def s bin, item):
3.         if bin <= item + a:
```
4      return capacity - bin
5      elif bin <= item + b :
6          return 0
7      else:
8          return -1/(x-item)
9      return np.array([s bin, item) for
                       bin in bins])
```

As discussed earlier, the performance of this class of heuristics can be explained by two main factors: (H1) a low number of elements whose sizes are below a given threshold; and (H2) a large total number of items to be scheduled.

## 4.3 Performance of ab-FirstFit, ab-WorstFit, ab-BestFit

In this section we evaluate the performance of our parametrizable heuristics across various distributions, including those studied in Romera et al. [6].

<figure><img src="image_10.png" /><figcaption>Figure 9: Heatmap showing the relative number of bins used by ab-FirstFit for various values of a and b, compared to BestFit, over 100 instances of UNIFORM(20, 100) (bin capacity = 150, number of items = 500). The best results are obtained for a = 5 and b = 24, with a relative ratio of 0.979.</figcaption></figure>

**Uniform(20, 100).** Figure 9 shows the relative number of bins used by ab-FirstFit compared to BestFit for various values of a and b, using the same UNIFORM(20, 100) distribution on which c12 was trained. Several combinations of parameters (a, b) achieve comparable performance, highlighting the robustness of the heuristic with respect to parameter selection. The best-performing parameters for ab-FirstFit (the heuristic most similar to c12) are a = 5 and b = 24. In this configuration, ab-FirstFit

outperforms BestFit by 2.1%, whereas c12 outperforms BestFit by 2.0% in the same setting.

<figure><img src="image_11.png" /><figcaption>Figure 10: Heatmap showing the relative number of bins used by ab-WorstFit for various values of a and b, compared to BestFit, over 100 instances of WEIBULL(3.0, 45) (bin capacity = 100, number of items = 5000). The best results are obtained for a = 1 and b = 21, with a relative ratio of 0.967.</figcaption></figure>

**Weibull(3.0, 45).** Figure 10 shows the relative number of bins used by ab-WorstFit compared to BestFit for various values of a and b, using the same WEIBULL(3.0, 45) distribution on which c14 was trained. Again, several parameter combinations achieve similar performance. The best performance is obtained for ab-WorstFit with parameters a = 1 and b = 22, where ab-WorstFit outperforms BestFit by 3.3% on average. On the same distribution, c14 and EOH outperform BestFit by 3.3% and 3.2%, respectively.

**Other uniform and Weibull distributions** Figure 11 evaluates the performance of ab-FirstFit, ab-BestFit and ab-WorstFit compared to the baseline strategies and the LLMs-evolved heuristics on the UNIFORM(20, 100) and WEIBULL(3.0, 45) distributions, as well as on additional distributions. The best-performing parameters a and b for ab-FirstFit, ab-BestFit, and ab-WorstFit, were selected separately for each problem instance.

Let ab-Baselines denote the best performing among the three heuristics ab-FirstFit, ab-BestFit, and ab-WorstFit. ab-Baselines outperforms all baselines and LLMs-evolved heuristics considered, even on the data distributions on which they were trained.
<figure><img src="image_12.png" /><figcaption>Figure 11: Performance of ab-FirstFit, ab-BestFit, and ab-WorstFit across various distributions. For the Weibull(7.0,75) distribution with bin capacity 500, the performance of c14 is too degraded to be visible in the graph.</figcaption></figure>

Our experiments confirm that c12, c14, and other LLM-evolved heuristics are not robust to variations in the data distribution, as previously demonstrated in detail by Sim et al. [33]). This result is not surprising: these heuristics are highly optimized for a single distribution—just as ab-Baselines is with a fixed choice of parameters.

ab-Baselines, however, generalizes the core idea
easily and effectively to distributions that exhibit
a lower bound on item sizes and contain a large
number of items. It nevertheless suffers from the
same limitation as c12 and c14 when the number of
items is small: the performance can degrade to as
much as c/(c − b) times the optimal<fnref n="1" />, where c is the
bin capacity and b the second parameter. Notably,
for WEIBULL(3.0, 45), ab-WorstFit converges much
faster to comparable performance level than c14 (Fig-
ure 13).

Finally, it is worth noting that all our experimen-
tal runs, including the meta-tuning of ab-Baselines,
can be completed within a few minutes of execu-
tion time on a personal laptop, contrasting sharply
with the thousands or even millions of LLMs queries
required to generate the LLMs-evolved heuristics.
Moreover, if the community were to pursue this line

<figure><img src="image_13.png" /><figcaption>Figure 12: Average performance of ab-FirstFit (with *a* = 5 and *b* = 24) relative to BestFit over 1000 instances of the UNIFORM(20, 100) distribution, as the number of items increases (bin capacity = 150).</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Figure 13: Average performance of ab-WorstFit (with *a* = 1 and *b* = 21) relative to BestFit over 1000 instances of the WEIBULL(3.0, 45) distribution, as the number of items increases (bin capacity = 100).</figcaption></figure>

of work, we expect that performance could be further
improved, from a computational complexity perspec-
tive, by employing dichotomic search for parameter
tuning, as an example.

## 4.4 Concluding Discussions

In their work, Romera et al. [6] proposed using LLM-
based solutions in the manner we have adopted: *Fun-
Search suggests a solution, which is examined by re-
searchers, who may note features of interest.* In this
sense, our study seems to support the process they
envisioned, in which LLMs assist researchers in ex-
ploring and refining new scientific insights.

Nevertheless, a broader question remains: can
LLMs truly contribute to solving established open

<footnote n="1">¹This can be shown by considering an instance where all arriving items are smaller than *b*, such that they would fill the bins up to *c* − *b*, then the next item would open a new bin. Note that if *b* < *x* the distribution minimum, in this case the ratio would be *c*/*(c − *x*)* > *c*/*(c − *b*)* and the result holds.</footnote>
problems, as claimed by Romera et al. [6], thereby justifying their role in scientific discovery? Indeed, their paper belongs to a broader research effort aiming to demonstrate meaningful applications of LLMs—specifically, cases where the resulting advances would not have been achievable without machine involvement. Such claims can only be supported through a rigorous validation protocol applied to a problem recognized as both challenging and significant within the scientific community.

**On the absence of related work** A key limitation of Romera et al.'s claims lies in the absence of prior studies on the online bin packing problem where item sizes follow a UNIFORM(20, 100) or WEIBULL(3.0, 45) distribution. This lack of related work raises several possible explanations.

One possibility is that no one had previously asked this particular question because it was not considered particularly important. If so, the surge of interest in these bin packing instances since 2023—mostly driven by the narrative of LLM-based discovery—may reflect an artificial increase in their perceived importance rather than genuine theoretical significance.

Another explanation is that the problem, although perhaps already examined, was solved but never formally published. If that were the case, it would raise an interesting contrast: human-generated solutions of this kind were possibly deemed too simple or insufficiently significant to merit publication, while similar LLM-generated results have recently led to a proliferation of preprints and publications [30, 31, 32, 45]. This contrast invites reflection on the expectations the research community places on AI-generated findings.

Finally, one cannot entirely rule out the possibility that relevant work does exist but has escaped our review. However, if the considered problems were perceived as significant, it would likely have achieved at least minimal visibility in the literature.

Taken together, these observations suggest that the absence of related studies is more plausibly explained by a lack of perceived importance than by genuine novelty or difficulty.

**LLMs for “mathematical discovery”** As discussed earlier, in the case of the online bin packing problem, the LLM-evolved heuristics show limited interpretability even for domain experts. Our understanding of their behavior largely relied on

experimental observation, much like one would study a black-box heuristic. By comparison, the **ab-Baselines** meta-heuristic is more interpretable, more efficient, and more generalizable. Its underlying rationale is relatively straightforward once the properties of the distribution and the large number of elements are considered.

It remains conceivable that knowing in advance that a better solution than BestFit existed helped us identify it. A central challenge of scientific research lies precisely in the uncertainty of whether better solutions exist at all. Having access to such knowledge, or to an oracle that provides it, would undoubtedly be a major advantage. Yet, this advantage does not reflect genuine insight but rather a form of informed exploration.

Indeed, the patterns we identified are not explicitly encoded in the generated heuristics but emerged indirectly from behavioral analysis. This indicates that the contribution of LLMs lies in empirical optimization rather than in genuine conceptual innovation. Consequently, the case of online bin packing provides insufficient evidence to claim that LLMs have produced insights amounting to new mathematical discovery.

**On Romera et al.'s title** The expression “mathematical discovery” in Romera et al.'s title is conceptually broad and scientifically ambiguous. It is not clear at what point an incremental improvement qualifies as a “mathematical discovery.” This ambiguity is particularly relevant in domains where empirical performance improvements and theoretical insights are tightly coupled but not equivalent.

From a technical perspective, Romera et al.'s contribution is valuable. Extending the search operators of genetic algorithms with LLMs is an original idea that is worth investigating. Automatically generating heuristics that perform competitively with standard baselines on trained datasets—without domain-specific knowledge—can prove useful in specific applied settings. However, qualifying such generated heuristics as a “mathematical discovery” requires a strong validation protocol and domain expertise. Based on our analysis, these conditions do not seem to hold for the online bin packing problem.

It is noteworthy that subsequent studies [30, 31, 32] that refined genetic frameworks using LLMs adopted the same validation protocol for their generated heuristics—comparing performance against BestFit on the WEIBULL(3.0, 45) distribution. Yet, none of
these works claimed to have made a “mathematical
discovery” when surpassing c14.

This raises questions about whether Romera et al.’s title was intended more as an attention-oriented phrasing than as a precise scientific statement. A title that overstates its claims can influence citation practices and public perception in ways that may not reflect the actual nature of the work. It would be informative to understand how this title went through the peer-review process—particularly given that Romera et al.’s article underwent an accelerated review procedure at *Nature*. Open or transparent review processes could help mitigate similar issues by promoting accountability and constructive discussion about scientific publications.

5 Conclusion

This study investigated the interpretability and sci-
tific value of heuristics evolved through LLM-based
genetic frameworks, focusing on the online bin pack-
ing problem. Despite the claim of mathematical dis-
covery, our analysis of their behavior shows that
these heuristics largely rediscover known patterns un-
der narrow experimental conditions. These patterns
can be formalized into an interpretable and more ef-
ficient two-parameter algorithm that can be easily
tuned across different distributions. This abstrac-
tion not only generalizes beyond the LLM-evolved
heuristics but also shows that their apparent nov-
elty stems primarily from reparameterizing familiar
strategies rather than introducing new algorithmic
concepts. These results suggest that the LLM-based
frameworks did not develop a conceptual understand-
ing of the problem but rather discovered effective
strategies through stochastic search and empirical
feedback, similar to a generic hyper-heuristic.

Overall, LLMs appear as another approach for ex-
tending the capabilities of genetic algorithms and
broadening the search space of hyper-heuristics. In
the case of online bin packing, these benefits remain
primarily empirical and do not yet demonstrate a ca-
pacity for autonomous theoretical insight or abstrac-
tion. A convincing validation of such a capability
would require applying LLM-based methods to sig-
nificant, well-established problems and showing that
the resulting solutions achieve demonstrable improve-
ments over the best results in the existing literature,
rather than focusing on previously unexplored in-
stances.

References

[1] Nelishia Pillay and Rong Qu. *Hyper-heuristics: theory and applications*. Springer, 2018.

[2] Edmund K Burke, Michel Gendreau, Matthew Hyde, Graham Kendall, Gabriela Ochoa, Ender Özcan, and Rong Qu. Hyper-heuristics: A survey of the state of the art. *Journal of the Operational Research Society*, 64(12):1695–1724, 2013.

[3] Thomas Stützle and Manuel López-Ibáñez. Automated design of metaheuristic algorithms. In *Handbook of metaheuristics*, pages 541–579. Springer, 2018.

[4] William B Langdon and Riccardo Poli. *Foundations of genetic programming*. Springer Science & Business Media, 2013.

[5] Fangfang Zhang, Yi Mei, Su Nguyen, and Mengjie Zhang. Survey on genetic programming and machine learning techniques for heuristic design in job shop scheduling. *IEEE Transactions on Evolutionary Computation*, 28(1):147–167, 2023.

[6] Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. *Nature*, 625(7995):468–475, 2024.

[7] Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combinatorial optimization: a methodological tour d'horizon. *European Journal of Operational Research*, 290(2):405–421, 2021.

[8] Xingyu Wu, Sheng-hao Wu, Jibin Wu, Liang Feng, and Kay Chen Tan. Evolutionary computation in the era of large language model: Survey and roadmap. *IEEE Transactions on Evolutionary Computation*, 2024.

[9] Joshua Grochow. New applications of the poly-
nomial method: the cap set conjecture and be-
yond. *Bulletin of the American Mathematical Society*, 56(1):29–64, 2019.

[10] Terence Tao and Van H Vu. *Additive combinatorics*, volume 105. Cambridge University Press, 2006.
[11] Edward G Coffman Jr, Michael R Garey, and David S Johnson. Approximation algorithms for bin-packing—an updated survey. In *Algorithm design for computer system design*, pages 49–106. Springer, 1984.

[12] Edmund Burke, Graham Kendall, Jim Newall, Emma Hart, Peter Ross, and Sonia Schulenburg. Hyper-heuristics: An emerging direction in modern search technology. In *Handbook of metaheuristics*, pages 457–474. Springer, 2003.

[13] Peter Ross, Sonia Schulenburg, Javier G Marín-Bläzquez, and Emma Hart. Hyper-heuristics: learning to combine simple heuristics in bin-packing problems. In *Proceedings of the 4th annual conference on genetic and evolutionary computation*, pages 942–948, 2002.

[14] Kevin Sim, Emma Hart, and Ben Paechter. A lifelong learning hyper-heuristic method for bin packing. *Evolutionary computation*, 23(1):37–67, 2015.

[15] Thomas Bäck, David B Fogel, and Zbigniew Michalewicz. Handbook of evolutionary computation. *Release*, 97(1):B1, 1997.

[16] Agoston E Eiben and Jim Smith. From evolutionary computation to the evolution of things. *Nature*, 521(7553):476–482, 2015.

[17] Yi Mei, Qi Chen, Andrew Lensen, Bing Xue, and Mengjie Zhang. Explainable artificial intelligence by genetic programming: A survey. *IEEE Transactions on Evolutionary Computation*, 27(3):621–641, 2022.

[18] Ya-Hui Jia, Yi Mei, and Mengjie Zhang. Learning heuristics with different representations for stochastic routing. *IEEE Transactions on Cybernetics*, 53(5):3205–3219, 2022.

[19] Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovonia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. *arXiv preprint arXiv:2302.04023*, 2023.

[20] Erik Hemberg, Stephen Moskal, and Una-May O'Reilly. Evolving code with a large language model. *Genetic Programming and Evolvable Machines*, 25(2):21, 2024.

[21] Wang Chao, Jiaxuan Zhao, Licheng Jiao, Lingling Li, Fang Liu, and Shuyuan Yang. When large language models meet evolutionary algorithms. *arXiv preprint arXiv:2401.10510*, 2024.

[22] Pengyi Li, Jianye Hao, Hongyao Tang, Xian Fu, Yan Zhen, and Ke Tang. Bridging evolutionary algorithms and reinforcement learning: A comprehensive survey on hybrid algorithms. *IEEE Transactions on evolutionary computation*, 2024.

[23] Vadim Liventsev, Anastasiia Grishina, Aki Härmä, and Leon Moonen. Fully autonomous programming with large language models. In *Proceedings of the Genetic and Evolutionary Computation Conference*, pages 1146–1155, 2023.

[24] Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. *arXiv preprint arXiv:2310.12931*, 2023.

[25] Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O Stanley. Evolution through large models. In *Handbook of evolutionary machine learning*, pages 331–366. Springer, 2023.

[26] Alex Davies, Petar Veličković, Lars Buesing, Sam Blackwell, Daniel Zheng, Nenad Tomašev, Richard Tanburn, Peter Battaglia, Charles Blundell, András Juhász, et al. Advancing mathematics by guiding human intuition with ai. *Nature*, 600(7887):70–74, 2021.

[27] Miles Cranmer, Alvaro Sanchez Gonzalez, Peter Battaglia, Rui Xu, Kyle Cranmer, David Spergel, and Shirley Ho. Discovering symbolic models from deep learning with inductive biases. *Advances in neural information processing systems*, 33:17429–17442, 2020.

[28] Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. *arXiv preprint arXiv:2305.10403*, 2023.

[29] Alice Q Jiang, Yi Wu, and Mateja Jamnik. Formal mathematical reasoning: A new frontier in ai. *AI Magazine*, 2025.
[30] Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. *arXiv preprint arXiv:2401.02051*, 2024.

[31] Pham Vu Tuan Dat, Long Doan, and Huynh Thi Thanh Binh. Hsevo: Elevating automatic heuristic design with diversity-driven harmony search and genetic algorithm using llms. *Proceedings of the AAAI Conference on Artificial Intelligence*, 39(25):26931-26938, 2025.

[32] Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. Reevo: Large language models as hyper-heuristics with reflective evolution. *Advances in neural information processing systems*, 37:43571-43608, 2024.

[33] Kevin Sim, Quentin Renau, and Emma Hart. Beyond the hype: Benchmarking llm-evolved heuristics for bin packing. In *Proceedings of EvoSTAR 2025*, 2025.

[34] Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. *Nature*, 630(8017):625-630, 2024.

[35] David S. Johnson, Alan Demers, Jeffrey D. Ullman, Michael R Garey, and Ronald L. Graham. Worst-case performance bounds for simple one-dimensional packing algorithms. *SIAM Journal on computing*, 3(4):299-325, 1974.

[36] Nikhil Ayyadevara, Rajni Dabas, Arindam Khan, and KVN Sreenivas. Near-optimal algorithms for stochastic online bin packing. *ACM Transactions on Algorithms*, 21(2):1-39, 2025.

[37] Peter W Shor. The average-case analysis of some on-line algorithms for bin packing. *Combinatorica*, 6(2):179-200, 1986.

[38] Edward G Coffman Jr, Kimming So, Micha Hofri, and AC Yao. A stochastic model of bin-packing. *Information and control*, 44(2):105-115, 1980.

[39] Edward G Coffman Jr, David S Johnson, Peter W Shor, and Richard R Weber. Bin packing with discrete item sizes, part ii: Tight bounds on first fit. *Random Structures & Algorithms*, 10(1-2):69-101, 1997.

[40] Jon Louis Bentley, David S Johnson, Frank Thomson Leighton, Catherine C McGeoch, and Lyle A McGeoch. Some unexpected expected behavior results for bin packing. In *Proceedings of the sixteenth annual ACM symposium on Theory of computing*, pages 279-288, 1984.

[41] Claire Kenyon et al. Best-fit bin-packing with random order. In *SODA*, volume 96, pages 359-364, 1996.

[42] John E Beasley. Or-library: distributing test problems by electronic mail. *Journal of the operational research society*, 41(11):1069-1072, 1990.

[43] Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. Examples are not enough, learn to criticize! criticism for interpretability. *Advances in neural information processing systems*, 29, 2016.

[44] Danilo Carastan-Santos and Raphael Y De Camargo. Obtaining dynamic scheduling policies with simulation and machine learning. In *Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis*, pages 1-13, 2017.

[45] Zijie Chen, Zhanchao Zhou, Yu Lu, Renjun Xu, Lili Pan, and Zhenzhong Lan. Qube: Enhancing automatic heuristic design via quality-uncertainty balanced evolution. *arXiv preprint arXiv:2412.20694*, 2024.