Online Knapsack with Frequency Predictions

Abstract: There has been recent interest in using machine-learned predictions to improve the worst-case guarantees of online algorithms.  In this paper we continue this line of work by studying the online knapsack problem, but with very weak predictions: in the form of knowing an upper and lower bound for the number of items of each value.  We systematically derive online algorithms that attain the best possible competitive ratio for any fixed prediction; we also extend the results to more general settings such as generalized one-way trading and two-stage online knapsack. Our work shows that even seemingly weak predictions can be utilized effectively to provably improve the performance of online algorithms.


Reviews and author responses:

 Thank you for your careful review and comments. 

**(1)**
We respectfully disagree with this assessment.  Most prior works on learning-augmented algorithms assume predictions regarding specific values of input parameters, whereas we consider a setting where the predictions only specify a range in which the parameters may lie, which may be provided by an ML model.  Concretely, instead of the predictions specifying that the total size of items having a particular value is a specific number (say 0.3), the predictions only give a range (say [0.1, 0.6]). Consequently, algorithms in this model naturally offer some resilience to poor ML predictions. 
Having said that, we do in fact show robustness, i.e., even if the actual instance does not satisfy the predictions, we obtain a bounded competitive ratio. As explained in lines 86--89 of the paper, robustness can be obtained in our model by partitioning the knapsack into two parts and using the worst-case algorithm of ZCL on one of the parts. 

**(2)**
Indeed, our algorithm essentially guarantees the best possible competitive ratio (we lose a small additive factor) for any given set of predictions. Note that the actual competitive ratio depends on the predicted $\ell_v$, and $u_v$ bounds as well as the range $[v_\min, v_\max]$ in a way that does not admit a clean closed-form expression. Nevertheless, our lower bound shows that no online algorithm can obtain a better competitive ratio. In particular, we note that for any instance that satisfies the predictions, the obtained competitive ratio is always strictly better than the worst-case ratio obtained by the ZCL algorithm.

**(3)**
We respectfully disagree with the assessment that the extensions are straightforward. In particular, the analysis for the two-stage knapsack extension requires significant technical novelty. This is because of the following reason: The algorithm is designed to remain competitive against the optimum solution for an instance that is the worst for the algorithm and is simultaneously consistent with the items that have arrived so far. Characterizing such adversarial instances in the presence of predictions is significantly more complicated in the two-stage knapsack problem as compared to the regular online knapsack version, and necessitates a different, subtle and more involved analysis.

**(4)**
The experiments demonstrate that even relatively weak predictions can be effectively exploited to obtain better than worst-case guarantees. For instance, for $\delta=2$ in the first experiment, the algorithm only knows that the number of items arriving with value $v$ is between $\ell_v$ and $3 \ell_v$ (and the actual instance number is drawn uniformly from this range); even with such a prediction, our algorithm significantly outperforms the worst-case algorithm.

 Thank you for your careful review and comments. We will address all of the issues in the revision.

**Robustness**

Our main algorithmic results require that the presented instance satisfies the predicted ranges. But, “robustness” can be achieved using a simple trick. We briefly discussed it at the end of page 2 (lines 86--89). For your convenience, we copy it here. 
“We note that by combining our algorithm and ZCL (say by using a $\gamma$ fraction of the capacity to run our algorithm and the other $1-\gamma$ fraction to run ZCL), we can simultaneously obtain a worst-case guarantee of $\frac{1-\gamma}{1 + \log(v_\max /  v_\min)}$ as well as our improved guarantees for instances that respect the prediction.”
In addition to this robustness, we can still obtain a better guarantee than the best worst-case algorithm even if the frequency predictions turn out to be slightly incorrect. Although there are multiple ways to formalize this, consider the following set up: Suppose the total size of items that actually arrive with value v is $s_v$, and it is off by $(1+\epsilon)$ factor from the prediction $[\ell_v, u_v]$. In other words, $s_v \in [\ell_v/(1+\epsilon), u_v(1+\epsilon)]$, but for simplicity, say we only underestimated, i.e., $s_v \in [\ell_v, u_v(1+\epsilon)]$. In this case, it is not difficult to verify that all claims 3, 4, 5, and all the inequalities in Section 3.1, except the last inequality, hold true. The only difference is that OPT may be allowed to collect up to $u_v(1+\epsilon)$ items of value $v \leq \tilde{v}$, which can only increase the optimum by $(1+\epsilon)$ factor. Thus, the algorithm’s competitive ratio drops by at most $(1+\epsilon)$ factor. Over-prediction is even easier to handle.

**Definition of $s_v$**

We apologize for this confusion. On page 2, we first defined the “standard” knapsack problem. When we defined our model later on the same page (lines 59--68), we unfortunately overrode some notation. Your understanding is correct: $s_v$ means the total size of items that actually arrive with value $v$. 

**Differentiability of $OPT(M(x))$ and $A(x)$**

We assumed the Continuous-Valued model (lines 138--147) to present our algorithms and their analysis transparently. Under this model, the quantities are differentiable. In a nutshell, two assumptions are made in the model: (i) every item has a “small” size compared to the knapsack capacity; and (ii)  $L(v)$ and $U(v)$ are continuous and differentiable. Here, $L(v)$ denotes the total size of items of value at most $v$ that will arrive for certain and $U(v)$ the analogous upper bound. Assumption (i) results in a small loss in the competitive ratio, which is stated in Lemma 2. Assumption (ii) is without loss of generality, and the conversion of an arbitrary instance to one satisfying these assumptions is discussed in Appendix A in the Supplementary Material. Also, for clarity, we present the discrete version of the algorithm in Appendix B, which easily is mapped from our algorithm in the continuous model.

Thanks for the suggestion.  In the revision, we will add a discussion on the limitation of our model.

 **Robustness**
In this model, robustness is easy to obtain since one can take a linear combination of two algorithms. We discuss this approach briefly in lines 86--89 of the paper. Briefly, by reserving a $(1-\gamma)$ fraction of the knapsack capacity to run the classic online algorithm (ZCL) and running our algorithm on the remaining 𝛾 fraction of the knapsack, it is possible to get the desired robustness guarantee. In particular, the parameter 𝛾 can be thought of as one’s “confidence” in the available predictions. Further, the key contribution of the paper is to demonstrate that even weak predictions that specify only a broad range in which the problem parameters should lie are sufficient to obtain better guarantees.

**Theorem 1**
Our algorithm essentially guarantees the best possible competitive ratio (we lose a small additive factor) for any given set of predictions. Note that the actual competitive ratio depends on the predicted $\ell_v$ and $u_v$ bounds as well as the value range $[v_\min, v_\max]$ in a complicated way and does not admit a clean closed form expression. However, for any instance that satisfies the predictions, the obtained competitive ratio is always strictly better than the worst-case ratio obtained by the ZCL algorithm. This is because our algorithm guarantees the best possible competitive ratio for the set of instances that are consistent with the predictions, and ZCL achieves the best competitive ratio for all possible instances. To gain some intuition regarding the optimal competitive ratio for a specific prediction, Figure 1 of the paper demonstrates the competitive ratio for a natural family of predictions.

**Steps 1-4**
For the boundary condition, please note that we assumed the Continuous-Valued model (lines 138--147) to present our algorithms and their analysis transparently. In a nutshell, two assumptions are made in the model: (i) every item has a “small” size compared to the knapsack capacity; and (ii)  $L(v)$ and $U(v)$ are continuous and differentiable. Here, $L(v)$ denotes the total size of items of value at most $v$ that will arrive for certain and $U(v)$ the analogous upper bound. Assumption (i) results in a small loss in the competitive ratio, which is stated in Lemma 2. Assumption (ii) is without loss of generality, and the conversion of an arbitrary instance to one satisfying these assumptions is discussed in Appendix A in the Supplementary Material. Also, for clarity, we present the discrete version of the algorithm in Appendix B, which easily is mapped from our algorithm in the continuous model.

The specific instance you consider, “$\ell_v = 0, u_v =\infty$, and the value density of all items is $v_\min$” doesn’t satisfy the simplifying assumptions, particularly because $S(v)$ is not differentiable at $v_\min$. After the conversion, the mass (sizes) of the items of value $v$ are distributed over values in an infinitesimal range including $v$. If we follow the discrete version, then $B(v_\min)$ will be non-zero. As mentioned above, the discrete version of our algorithm can be found in Appendix B in the Supplementary Material.

The paper considers the following variation of the online knapsack problem: Items of profit $p_i$ and size $s_i$ arrive online and upon arrival of an item it has to irrevocably be either packed to our knapsack (of unit capacity) or rejected. Let $v_i$ be the value of item $i$, and assume that $\forall I$ there holds $v_{min}\le v_i \le v_{max}$. The best known algorithm for this problem (called ZCL) obtains a competitive ratio of $1/(1+\ln\frac{v_{min}}{v_{max}}$. Let further $V$ be the set of all possible values that the arriving items may take, and $s_v$ be the total size of items of value $v$. In the problem studied in the paper it is assumed that the algorithm knows an upper and lower bound on $s_v$ for all possible $v$, and gives an $\alpha - \frac{v_{max}}{v_{min}}(|V|-1)\cdot\max s_i$-competitive algorithm for this variation, where $\alpha$ is the best possible competitive ratio that can be obtained for this problem variation.

Two natural extensions of the problem are then discussed: (i) to generalised one-way trading, and (ii) to two-stage online knapsack which is introduced in the current paper, before concluding with experimental results.  Although the algorithm itself is interesting and to the best of my knowledge also novel, I feel that the paper has a number of significant drawbacks:

1. The authors try to sell the problem as a problem within the very active area of learning augmented algorithms which in my opinion is misleading. According to my understanding the crux of this area is that although the quality/accuracy of the predictions is not known these can still be used to obtain improved results in case the predictions are accurate (consistency). The current paper assumes that the "predictions" on the range of item values are correct. In turn there is no robustness result showed or discussed and I am quite confident that the algorithm will perform arbitrarily bad in case the real instance does not satisfy this "prediction". I would therefore mostly consider the paper as an interesting algorithm for a restricted case of the online knapsack problem, which may be of interest on its own but cannot be classified as a learning augmented algorithm (and in turn I am not sure if NeuRIPS is the correct venue for such a paper).

2. It is a bit unusual to express the competitive ratio as a function of the best possible competitive ratio and I think this would deserve some more discussion.

3. The extensions are in my opinion straightforward and don't provide additional value to the paper.

4. The experiments are also not too interesting and show that naturally an algorithm with more information available will obtain a better solution.

That being said, the paper is generally well written and the intuition behind the design of the algorithm is conveyed reasonably well.

 I have no concerns about negative social impact of this paper.

This paper designs an online algorithm for the online knapsack problem (OKP) with frequency prediction and shows its competitive ratio (CR) under assumptions that (i) the value density of each item, v, is bounded within [v_min, v_max], and (ii) the demand (frequency) of the items with value density v, s_v, is bounded within [l_v, u_v], for all v. 

It also shows that the proposed algorithm is optimal (achieving the maximum CR) and can be extended to other variants of OKP (e.g., generalized one-way trading). 

In the literature, the ZCL algorithm (a threshold algorithm) in Ref [30] has been shown to achieve the optimal CR for OKP under the assumption (i). The main contribution of this paper is to show that by taking additional information in assumption (ii), it can also design a nearly optimal competitive algorithm, and it presents an interesting idea of how to design the algorithm, which has essential modifications from that of designing the ZCL algorithm.

Finally, the results have been extended to two additional variants with competitive analysis. 
  Overall, this paper studies the online knapsack problem from a novel perspective of incorporating predictions into the algorithm design. But, the theoretical approach is different from the emerging direction of online algorithms with ML advice since it does not take into account quality of prediction and hence robustness/consistency analysis (more details below). The quality and clarity of results are satisfactory and results by themselves sound solid. The significance of the results, however, is limited since it just considers a very particular type of prediction into the model, which may not be easily applicable to other related problems, and even the current model is not complete since it does not take into account error of the prediction.

Major comments:

-- This paper is of interest to the field of online algorithms with worst-case guarantees but has limited contributions to learning-augmented algorithms. It is in fact only focused on the worst-case algorithm design under perfect predictions on demand information $[l_v, u_v]$ (i.e., given accurate values of $l_v$ and $u_v$). The derived results can be considered as the consistency analysis in the works cited in the second paragraph of the introduction.  However, it lacks robustness analysis of the algorithm under malicious predictions (i.e., inaccurate $l_v$ and $u_v$ values), and the trade-off between the consistency and robustness. Thus, the current work is incomplete as a learning-augmented algorithm. The author mentions that there is a chance to combine the proposed algorithm and the ZCL algorithm to ensure robustness. However, the details of this important aspect are not elaborated and unjustified in the paper. 


-- about the lower bound result of the CR (Theorem 1). The CR of the proposed algorithm can be solved numerically, following Steps 1-4 in Section 3. But it lacks an essential understanding of how this CR depends on the given parameters. For example, the CR of the ZCL algorithm is $\Omega(1/\ln(v_{\max}/v_{\min}))$. It is important to know whether the proposed algorithm can improve this lower bound after taking additional information in assumption (ii). Thus, it is hard to evaluate the significance of the proposed algorithm.

-- about the online algorithm in Steps 1-4. The algorithm is constructed by designing a budget that can meet the increment of OPT under a special instance. The idea is interesting. However, it seems some boundary conditions are not properly set in the current version of the algorithm. The core question is what the value of the initial budget $B(v_{\min})$ is. If I understand correctly, $B(v_{\min}) = 0$ in the proposed algorithm. Then if we consider an instance with $l_v = 0, u_v = \infty$, and the value density of all items is $v_{\min}$, the proposed algorithm seems to accept no items while $OPT = v_min$ (leading to an unbounded CR). In fact, this issue also arises in equation (1). When $l(x) = 0$ for all $x$, the LHS = 0 and the RHS is non-zero. It seems there may not exist $v^*$ to make equation (1) hold. 

Minor comments:

-- in section 3, it would be clearer if an explicit online algorithm is presented first, followed by how to design in Steps 1-4.

-- in experiments, it is not fair to compare the worst-case ZCL algorithm with the prediction-based algorithm proposed in this paper, rather, it might be good to compare with a simple prediction-based variant of ZCL
 yes. 

This paper studies the online knapsack problem when the algorithm is endowed with some predictions about the input. In the online knapsack problem, a sequence items arrives online. Each item $i$ has a profit $p_i$ and a size $s_i$, and we refer to the ratio $v_i = p_i/s_i$ as the item’s value, which is assumed to be contained in an interval $[v_{min}, v_{max}]$. When each item arrives, the algorithm must make an irrevocable choice of whether to accept or reject the item. The goal is to maximize the profit of the items chosen while respecting the knapsack’s capacity constraint. The predictions have the following form: let’s say that s(v) is the total size of the items with value v. The algorithm receives as input an interval $[\ell_v, u_v]$ with the guarantee that with probability 1, s(v) lies in the interval.

The paper provides an algorithm with the following guarantee. Let’s say that V is the set of all possible values that the items may take, which we assume to be finite (an infinite set could be discretized with little loss). Let $P=\\{\ell_v, u_v \mid v \in V\\}$ be the set of frequency predictions provided to the algorithm. We say that the algorithm has a competitive ratio of $\alpha$ for this knapsack problem with predictions $P$ if the algorithm’s profit is at least $\alpha$ times the optimum for all inputs respecting the given frequency prediction $P$. Letting $\alpha^*$ be the optimal competitive ratio of this type, the paper provides an algorithm with competitive ratio $\alpha^* - \frac{v_{max}}{v_{min}} \cdot (|V| + 1) \cdot \max_i s_i$. They also provide extensions to other related problems, including the “generalized one-way trading problem” and the “two-stage online knapsack problem.”
  Overall, this paper seems to have some solid contributions with an interesting main result. The competitive ratio seems to be pretty good—it’s nice that the loss is only additive, not multiplicative. Also, I like the idea of relaxing the type of predictions that the algorithm gets.

Regarding the model, it would be helpful if the authors could say a bit more about where the predictions might come from. If the predictions are obtained via machine learning, then it’s likely that the predictions would only hold with some high probability. I wonder if the paper’s techniques might be amenable to further relaxing the predictions in this way.

I also think that the writing could be clarified. Perhaps since I’m not already familiar with approaches to the standard online knapsack problem, I found the transition to the continuous-valued online knapsack problem a bit hard to understand. In my detailed comments, I have some specific suggestions. It seems like some of the concepts in the algorithm could be visualized, so some pictures would really help. There were also some concepts that I couldn’t find the definitions of (like $B_{\alpha}(v)$ and $\alpha^*$, though the latter I picked up from context), so it seems like the paper could use a revamp for clarity’s sake.

Detailed comments:
-	Line 22: “possesses” -> “possess”
-	I found the definition of $s_v$ to be a bit confusing. In line 39, the integer $i$ indexing the value $v_i$ corresponds to the item $i$. But in line 59, the integer $i$ indexing the value $v_i$ no longer seems to correspond to the item $i$--it seems to be indexing the set $V$. Which indexing are we using in the definition of $s_v$ in line 62? I think it’s the first indexing, but I’m not sure.
-	Theorem 1: At first, I was confused that SENTINEL’s competitive ratio didn’t seem to explicitly depend on the lengths of the intervals $u_v - \ell_v$. Then I realized that $\alpha$ must depend implicitly on these lengths. It would be helpful to mention this. Perhaps it would help to write $\alpha(P)$ instead of $\alpha$ to make it clear that $\alpha$ depends on $P$ (from my understanding).
-	Line 81: Can you give an example where the maximum size approaches 0?
-	Line 175: Could you clarify why Opt(M(x)) is differentiable?
-	Line 176: “at” -> “at least”
-	Line 185: I couldn’t find the definition of $B_{\alpha}(v)$.
-	Line 187: I couldn’t find the definition of $\alpha^*$, but maybe it’s the best possible competitive ratio for all inputs respecting the given frequency prediction $P$.
-	Line 197: Could you clarify why A(x) is differentiable?
-	Lines 205-207: Could you spell out a bit more why this is the case?
-	Line 323: It looks in Section 5.1, like $\alpha^*$ is first mathematically defined in line 323, but the definition is a bit implicit, so it would be good to be more explicit.
-	Line 341: If $\ell_v$ equals 100 in expectation, shouldn’t the expected number of items that arrive be $10^6$, since $10^6 \cdot 10^{-4} = 100$?
-	Section 6: What happens for larger sizes? I think the authors could be a bit more explicit about limitations, perhaps in the future research section.

Meta-review: The paper proposes a new algorithm for the online knapsack problem. The algorithm utilizes the knowledge of upper and lower bounds for the number of items of each value, and its competitive ratio depends on the gap between the bounds. 

The reviewers found the problem formulation and the algorithm to be interesting and novel. However, there were concerns whether the proposed method can be viewed as a “learning-augmented algorithm”, given that the paper does not explicitly address the case where the predictions are inaccurate. Furthermore, some reviewers commented that the experiments demonstrating the applicability of the proposed algorithm were quite limited, and did not illustrate where the appropriate predictions would come from.


Decision: accept