Pareto-Optimal Learning-Augmented Algorithms for Online Conversion Problems

1 Introductions . An online conversion problem aims to convert one asset to another through a sequence of exchanges at varying rates in order to maximize the terminal wealth in financial markets . With limited information on possible future rates , the core challenge in an online conversion problem is how to balance the return from waiting for possible high rates with the risk that high rates never show up . A high profile example of this risk is cryptocurrency markets , e.g. , Bitcoin , where high fluctuations up and down make it challenging to optimize exchanges . Two well-known classical online conversion problems are 1-max-search [ 7 ] and one-way trading [ 8 ] , which can be considered as integral and fractional versions of the online conversion problem that trade the asset as a whole or fraction-by-fraction ( e.g. , trading stock in lot or shares ) . Beyond these two problems , a number of extensions and variants of online conversion problems have been studied with applications to lookback options [ 14 ] , online portfolio selection [ 13 ] , online bidding [ 5 ] , and beyond . Most typically , conversion problems are studied through the lens of competitive ratios and the goal is to design online algorithms that minimize the worst-case return ratio of the offline optimal to online algorithm decisions . For example , EI-Yaniv et al . [ 8 ] have shown that optimal online algorithms can be designed to achieve the minimal competitive ratios for both 1-max-search and one-way trading . However , in real-world problems , predictions about future conversion rates are increasingly available and the algorithms developed in the literature are not designed to take advantage of such information . The challenge for using such predictions is that , in one extreme , the additional information is an accurate prediction ( advice ) of future inputs . In this case , the algorithm can confidently use the information to improve performance , e.g. , [ 9 ] . However , most commonly , predictions have 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . no guarantees on their accuracy , and if an online algorithm relies on an inaccurate prediction the performance can be even worse than if it had ignored the prediction entirely . This challenge is driving the emerging area of the learning-augmented online algorithm ( LOA ) design , which seeks to design online algorithms that can incorporate untrusted machine-learned predictions in a way that leads to near-optimal performance when predictions are accurate but maintains robust performance when predictions are inaccurate . To measure this trade-off , two metrics have emerged , introduced by [ 15 ] and [ 19 ] : consistency and robustness . Consistency is defined as the competitive ratio when the prediction is accurate , i.e. , CR ( 0 ) , where CR ( ε ) is the competitive ratio when the prediction error is ε . In contrast , robustness is the worst competitive ratio over any prediction errors , i.e. , maxε CR ( ε ) . Thus , consistency and robustness provide a way to quantify the ability of an algorithm to exploit accurate predictions while ensuring robustness to poor predictions . In recent years , a stream of literature has sought to design robust and consistent LOA for a variety of online problems , such as online caching [ 15 ] , ski-rental [ 19 ] , and others . The ultimate goal is to develop algorithms that are Pareto-optimal across robustness and consistency , in the sense that for any γ , the LOA achieves the minimal consistency guarantee among all online algorithms that are γ-competitive . For the ski rental problem , recent works have derived Pareto-optimal algorithms , e.g. , [ 4 , 22 ] , but in most cases the question of where the Pareto-boundary of LOA lies is yet to be answered . In this paper , we focus on the design of LOA for online conversion problems and we seek to answer the following question : Is it possible to design a Pareto-optimal LOA for the online conversion problem ? Contributions . We show that the answer to the above question is “ yes ” , by designing an online threshold-based algorithm ( OTA ) , and proving that it is Pareto-optimal . In particular , we introduce a class of OTA that unifies the algorithmic design of both 1-max-search and one-way trading . We then incorporate predictions into OTA by parameterizing the threshold functions based on the predictions . This approach yields bounded consistency and robustness ( see Theorem 4.5 and Theorem 4.6 ) . Further , we derive lower bounds for robustness-consistency trade-offs and show that our learningaugmented OTA achieves those lower bounds , and is thus Pareto-optimal ( see Theorem 5.1 and Theorem 5.2 ) . Finally , we demonstrate the improvement of the learning-augmented OTA over pure online algorithms using numerical experiments based on real-world data tracking Bitcoin prices . The technical contributions of this paper are twofold . First , we provide a sufficient condition for design and analysis of the learning-augmented OTA with a guaranteed generalized competitive ratio . This competitive ratio is general in the sense that it not only can yield robustness and consistency guarantees , but can also potentially provide more fine-grained performance guarantees beyond robustness and consistency . Second , we provide a novel way of deriving the lower bound on the robustness-consistency trade-off , which may be of use beyond online conversion problems . The key idea is to construct a function that can model all online algorithms under a special family of instances , and the lower bound can be derived from combining the robustness and consistency requirements on this function . This constructive approach to arriving at a lower bound is distinctive . 2 Related Work . Online Conversion Problems . The online conversion problem is first introduced by El-Yaniv [ 7 ] and analyzed under the competitive analysis framework . It aims to search for the maximum rate to make conversions from a sequence of time-varying rates , which are chosen by an adversary in the worst-case . The subsequent works [ 8 , 14 ] have designed competitive algorithms for both integral and continuous versions of the online conversion problem and achieved the minimal competitive ratios . Many variants and extensions have also been studied , such as online conversion under interrelated rates [ 20 ] or with inventory constraints [ 23 , 12 ] . See [ 18 ] for a survey of other variants . In addition , the design and analysis of competitive algorithms for the online conversion problem have been shown to closely relate to those of the online knapsack problems , such as the online 0/1 knapsack with small weights [ 24 , 26 ] and online fractional knapsack [ 21 ] . Beyond the worst-case input model , another line of research considers searching for the maximum value from the time-varying rates with some form of prior knowledge . For example , the secretary problem [ 3 ] assumes that the rates arrive in a uniformly random order and the prophet inequality [ 6 ] assumes that the rates arrive in the worst-case order but with prior distributions . The prior information in these problems makes the design and analysis of algorithms essentially different from those of online conversion problems . Learning-augmented algorithms . The learning-augmented algorithm ( LOA ) takes advantages of machine learned predictions about the future input into the design of online algorithms and aims to optimize for a better competitive performance with an improved quality of predictions . The concepts of consistency and robustness are first introduced to study the online caching problem in [ 15 ] . Then the follow-up work [ 19 ] formally shows that there exist trade-offs between consistency and robustness in the ski-rental problem and its variants , and LOA needs to be designed to balance the two criteria unlike the online algorithms just optimized for the competitive ratio . Furthermore , the works [ 4 ] and [ 22 ] independently prove that the LOA designed in [ 19 ] is Pareto-optimal , i.e. , given a robustness , no online algorithms can achieve a smaller consistency . Also , in [ 11 ] and motivated by an energy optimization scenario , the authors develop Pareto-optimal algorithms for an extended version of the ski-rental problem . From the aspect of methodology , the work [ 4 ] provides a general approach to incorporate the prediction into the online primal-dual algorithm and shows it can be applied to solve multiple online problems . In terms of applications , LOA has been designed for a variety of online problems , such as online caching [ 15 ] , ski-rental [ 19 , 22 , 4 , 1 , 11 ] , online set cover [ 4 ] , online scheduling [ 19 , 10 ] , secretary and online matching [ 3 ] , metrical task systems [ 2 ] , etc . However , among those applications , the Pareto-optimality of LOA has only been rigorously shown for the ski-rental problem . 3 Problem statement and a unified algorithm . The online conversion problem . An online conversion problem considers how to convert one asset ( e.g. , dollars ) to another ( e.g. , yens ) over a trading period [ N ] : = { 1 , . . . , N } . At the beginning of step n ∈ [ N ] , an exchange rate ( or price ) , vn , is announced and a decision maker must immediately determine the amount of dollars , xn , to convert and obtains vnxn yens . The conversion is unidirectional , i.e. , yens are not allowed to convert back to dollars . The trading horizon N is unknown to the decision maker , and if there are any remaining dollars after N − 1 trading steps , all of them will be compulsorily converted to yens at the last price vN . Without loss of generality , the initial asset can be assumed to be 1 dollar , and the goal is to maximize the amount of yens acquired at the end of the trading period . The offline version of the conversion problem can be cast as maximize xn ∑ n∈ [ N ] vnxn , subject to ∑ n∈ [ N ] xn ≤ 1 . ( 1 ) If the conversion is only allowed in a single transaction , the decision xn ∈ { 0 , 1 } is a binary variable , and this integral version is called 1-max-search [ 7 ] . If the asset is allowed to convert fractionby-fraction over multiple transactions , the decision xn ∈ [ 0 , 1 ] is a continuous variable , and this fractional version is refereed to as one-way trading [ 8 ] . Following the literature , we assume the prices { vn } n∈ [ N ] are bounded , i.e. , vn ∈ [ L , U ] , ∀n ∈ [ N ] , where L and U are known parameters , and define θ = U/L as the price fluctuation . Online threshold-based algorithms . Online threshold-based algorithms ( OTA ) belong to a class of reserve-and-greedy algorithms where the idea is to use a threshold function to determine the amount of resources that need to be reserved based on resource utilization , and then greedily allocate resources respecting the reservation in each step . OTA is known to be easy-to-use but hard-to-design due to the difficulties in developing the threshold function . Prior work using OTA has often been problem specific . For example , an optimal design of the threshold function in OTA is derived in [ 21 ] for one-way trading and in [ 25 , 26 ] for the online knapsack problem , which is closely related to one-way trading . Here , we unify online algorithms for online conversion problems in an OTA framework in Algorithm 1 . Algorithm 1 takes a threshold function φ as its input , where φ ( w ) : [ 0 , 1 ] → [ L , U ] is a function of resource utilization ( i.e. , amount of traded dollar ) w and φ ( w ) can be considered as the reservation price when the utilization is w. The algorithm makes conversions only if the current price vn is at least φ ( w ( n−1 ) ) , where w ( n−1 ) = ∑ i∈ [ n−1 ] x̄i is the utilization after the previous n − 1 steps of trading . More specifically , the conversion decision x̄n in each step is determined by solving an optimization problem in Line 3 of Algorithm 1 . The OTA framework transforms the algorithmic design task in online conversion problems into the design of φ , and the challenge is to design φ such that OTA can have theoretical performance guarantees . To provide two examples , in the following we show how to recover the optimal online algorithms for 1-max-search and one-way trading . Algorithm 1 Online threshold-based algorithm with threshold function φ ( OTAφ ) 1 : input : threshold function φ ( · ) , and initial resource utilization ( i.e. , traded dollar ) w ( 0 ) = 0 ; 2 : while price vn is revealed do 3 : determine resource allocation x̄n = arg maxxn∈Xn vnxn − ∫ w ( n−1 ) +xn w ( n−1 ) φ ( u ) du ; 4 : update the utilization w ( n ) = w ( n−1 ) + x̄n . 5 : end while 1-max-search . In this integral conversion problem , OTA sets the feasible space as Xn = { 0 , 1 } and the threshold function as a constant φ ( w ) = Φ , w ∈ [ 0 , 1 ] , where Φ is also called a reservation price . Then the algorithm simply selects the first price that is at least Φ . When the reservation price is designed as Φ = √ LU , OTA is exactly the same algorithm as the reservation price policy in [ 8 ] , which achieves the optimal competitive ratio √ θ . One-way trading . OTA sets Xn = [ 0 , 1 − w ( n−1 ) ] and φ as a continuous and strictly increasing function . The conversion decisions fall into three cases based on the solution of the optimization in Line 3 : ( i ) if vn < φ ( w ( n−1 ) ) , make no conversions , i.e. , x̄n = 0 ; ( ii ) if φ ( w ( n−1 ) ) ≤ vn ≤ φ ( 1 ) , x̄n can be solved based on the first-order optimality condition , i.e. , vn = φ ( w ( n−1 ) + x̄n ) ; and ( iii ) if vn > φ ( 1 ) , x̄n = 1−w ( n−1 ) converts all its remaining dollar at the price vn . By setting the threshold function to φ ( w ) = L + ( α∗L − L ) exp ( α∗w ) , w ∈ [ 0 , 1 ] , OTA achieves the optimal competitive ratio α∗ = 1 +W ( ( θ − 1 ) /e ) , where W ( · ) is the Lambert-W function [ 21 ] .

Summary: The paper considers the following conversion problem: You start with a good (e.g. 1 bitcoin) that you can convert into another good (e.g. dollars) at a conversion rate that changes over time. Every day you have the opportunity to convert part of the good at the day's rate. On the last day, you have to convert any remaining amount of the original good that you might still own. The goal is to maximize the final amount owned. It is assumed that upper and lower bounds U and L on the conversion rate are known that hold throughout the time horizon.  Two versions of this problem are considered: 1-max search, where the good is unsplittable, i.e., one can only convert it as a whole. And a smoother version where arbitrary fractions can be converted, called one-way trading. It seems to me that one-way trading is equivalent to the randomized version of 1-max search.  Let $\theta=U/L$ be the ratio between the upper and lower bounds. In the classical online setting, for 1-max search it is known that the best competitive ratio is $\sqrt{\theta}$, and for one-way trading it is $1+W((\theta-1)/e)$, where W is the Lambert function.  This paper considers this problem through the learning-augmented lens: The algorithm receives as additional input at time 0 a prediction of the maximal conversion rate that will be observed. The paper gives algorithms for both problems and proves via upper and lower bounds that they achieve the pareto-optimal trade-off between consistency (i.e., the competitive ratio when the prediction is correct) and robustness (i.e., the competitive ratio in the worst case). For 1-max trading, the result is that if the consistency is a multiplicative factor c better than the classical competitive ratio $\sqrt{\theta}$, then the robustness is a factor c worse than $\sqrt{\theta}$. The result for one-way trading is more difficult to express and I have no clear intuition of the trade-off. The paper also includes a brief experimental evaluation.