High-dimensional Continuum Armed and High-dimensional Contextual Bandit: with Applications to Assortment and Pricing
Authors: Junhui Cai
Ran Chen
Martin J. Wainwright
Linda H. Zhao
Venue: Submitted to ICLR 2023 (2023)
Keywords: bandit
high-dimensional statistics
assortment
pricing
reinforcement learning

Abstract: The bandit problem with high-dimensional continuum arms and high-dimensional contextual covariates is often faced by decision-makers but remains unsolved. Existing bandit algorithms are impracticable due to the complexity of the double-layer high dimensionality. We formulate this problem as a high-dimensional continuum armed contextual bandit with high-dimensional covariates and propose a novel model that captures the effect of the arm and contextual on the reward with a low-rank representation matrix. The representation matrix is endowed with interpretability and predictive power. We further propose an efficient bandit algorithm based on a low-rank matrix estimator with theoretical justifications. The generality of our model allows wide applications including business and healthcare. In particular, we apply our method to assortment and pricing, both of which are important decisions for firms such as online retailers. Our method can solve the assortment-pricing problem simultaneously while most existing methods address them separately. We demonstrate the effectiveness of our method to jointly optimize assortment and pricing for revenue maximization for a giant online retailer.

Under review as a conference paper at ICLR 2023
HIGH-DIMENSIONAL CONTINUUM ARMED AND HIGH- DIMENSIONAL CONTEXTUAL BANDIT: WITH APPLI- CATIONS TO ASSORTMENT AND PRICING
Anonymous authors Paper under double-blind review
ABSTRACT
The bandit problem with high- dithemmmnsiinnl continuum arms and high- nigh-dimenssioal contextual covariates is often faced by cent developments in contextual bandit problems focus on the setting where the number of arms are small but are impracticable with high-dimensional continuous arm spaces. To bridge the gap, we propose novel model for the nigh-dimensional continuum armed and nigh-dimensional contextual bandit problem that captures the effect of the arm and covariates on the reward via a low-rank representation matrix. The representation matrix endowed with interpretability and predictive power. We further propose an efficient bandit algorithm based on a low-rank ma- trix estimator with theoretical justifications. The generality of our model allows wide applications including business and healthcare. In particular, we apply our method to assortment and pricing, both of which are important decisions for firms such online retailers. Our method can solve the coauttment-priciing problem simultaneously while most existing methods address them separately. We demon- strate the effectiveness of our method to jointly optimize assortment and pricing for revenue maximization for a giant online retailer.
but remains unsolved. Re-
INTRODUCTION
The bandit problem dates back to when Robbins 1952) formulated the problem as the sequential design of experiments and has been studied to a great extent recently due to the demand for online decision-making especially from e-commerce and health care. A decision-maker chooses an action (arm) at each round and observes a reward and the goal is to act strategically so as to find an optimal action that maximizes the long-term reward without sacrificing too much.
The bandit literature mostly focuses on the problem of finite number of independent arms, but it is often the case that infinite number of of arms and the arms share some common structure and thus can be indexed by variables as a continuum armed bandit problem. In e-commerce, the retailer needs decide the product assortment and pricing to maximize long-term profits; in mobile health, the personal device provides exercise and dietary suggestions to improve physical and mental health. The possible actions in both examples can be parameterized as continuous variables, which are possibly high dimensional. In addition, decision-makers observe other covariatesffeatures, i.e., the contextual bandit problem where the reward is modeled as a function unknown parameters and the contextual variables, and in many practical settings, the covariates are high high-dimenssonal.
As the dimensionalities of the action space (for arms) and the contextual variables grow, the tradi- tional bandit algorithms suffer from the curse of dimensionality and it is impossible or prohibitively costly to learn the optimal decision. Albeit both the arm and the contextual are sigg--dimnsional. the dimension of the underlying factors often, fortunately, small - for nigh-dimensional bandit prob- lems, one can assume a now-dimeniiona structure on the unknown parameters, such as the LASSO bandit (Bastani & Bayati 2020); and for high-dimensional continuum armed bandit problems, one can assume the reward function depends only on the low-dimensional subspace of the action space (Tyagi et al., 2016). While -iw-eemensional representation has been successfully adopted in high- dimensional bandit problems and high-ding nigh-dimensional continuum armed bandit problems respectively, a natural but important question remains open: can we efficiently solve the bandit problem with both high-dimessional continuum arms and high-dimensional contextual variables simaltaneously?

---

Under review as a conference paper at ICLR 2023
In this paper, tackle the above problem by proposing novel model that captures the effect of the arm and the contextual with an approximately low-rank matrix representation as well as an efficient algorithm (Hi-CCAB to efficiently solve the problem with theoretical justifications. Specifically, for an action that is presented as a vector a E Rdd and the corresponding contextual covariates IRd,, we yield reward
r= a Oz te
where @ E IR xda is the unknown representation matrix, which is assumed to have rank d < min(da: dz) } and E is the independent error. To learn the low-rank representation matrix, we adapt the low-rank matrix estimator the bandit setting. We further demonstrate the benefits of our methodologies in e-commerce with real sales data where the online retailer needs to decide on the product assortment and pricing jointly. The generality of our model makes it possible to learn policy on product assortment and pricing at the same time, while previous literature mostly studies the assortment and pricing problem separately.
Contributions. We highlight the following contributions of paper:
1.
We propose new model for high-dimensional continuum armed and high-dimensional contex- tual bandit problem, which is often faced by decision-maken but very little existing literature attempts to solve. The crux our model is the low-rank representation matrix that exploits the low-dimensional structure of both the high-dimensional arms and high-dimensional covariates. Our model unifies a large class of bandit models.
2.
The low-rank representation matrix is endowed with interpretability and predictive power. One can perform singular value decomposition (SVD) the representation matrix - the left singular vectors reveal the latent structure and relationships among the arms, while the right singular vectors show the latent factors of the covariate. In other words, our model implicitly performs principle component analysis (PCA) on the effect of arms and covariates on the mean reward. On the other hand, given the covariate, our model is able predict the reward of unseen arm. Both interpretability and predictive power can be tremendously useful for decision-makers.
3. We propose an efficient algorithm for the High-dimensione Contextual and High-ding figh--imenssonal Continumm Armed Bandit -CCCC) by adopting the low-rank matrix estimator. We further provide an upper bound for the convergence rate of -CCB in terms of the time-averaged expected cumulative regret.
4. The generality of our model allows for wide range of applications. Specifically, we apply Hi-CCAB to the joint assortment and pricing problem. We show that our model reveals insights for product designs, assortment, and pricing and that the assortment-pricing policy based on Hi Hi-C yields sales four times as high as the original strategy.
Literature review. Literature high-dimensional bandit problems has been expanding recently, especially after statistical tools for high-dimensional problems become mature (Negahban & Wain- wright 2011 Wainwright 2019). Lots of high-dimensional bandit literature focuses contextual bandits with high-dimension covariates, such as the LASSO bandit problem (Abbasi- Yadkori et al.]
2012 Kim & Paik] 2019 Bastani & Bayati 2020 Hao et al. 2020 Papini et | 2021) where they assume the mean reward is a linear function of a sparse unknown parameter vector, the low-rank matrix bandit where the covariate and unknown parameter are both of matrix form (Kveton et al.)
2017 Lu et al.) 2022]). and other non- con-parametric methods that learns the reward function using ran- dom forest or deep learning (Féraud et al., 2016, Zhou et al., 2020; Ban et al.) [022]] Chen et al.,
2022
Xu et al. 2022). The high-dimensional bandit models are special cases of our model.
Another stream of high-dimensional bandit literature studies representation learning in linear ban- dits, specifically for multi-task learning where several bandits are played concurrently. The arms for each task are embedded in the same space and share a common oiw-dimensional representation (Lale et al. 2019 Yang et al. 2020 Hu al., 2021 Xu & Bastani 2021). Our problem different from multi-task learning since at each time we only have one bandit and thus observe one reward while in the multi- -ask bandit problem, multiple bandits are played the same time.
For continuum armed bandits, there exists a thread of literature that assumes the mean reward func- tion is smooth and continuous on the action space in some sense, e.g., the function lies in the Lipschitz or Hölder space (Agrawal 1995] Kleinberg) 2004 Kleinberg al. 2019). Most work discretizes the arm space or adopts the non- pana-araeetric regression to estimate the reward function, which is very different from our approach. Recent literature studies continuum armed bandit with
2

---

Under review as a conference paper at ICLR 2023
contextual covariates further assumes the mean reward function is continuous the arm- covariate space (Lu et al. 2010; Slivkins, 2011 Krishnamurthy et al., 2020). Literature on sigh-dimensional continuum armed bandits, however, is scarce (Turgay et al., 2020 Majzoubi et al. 2020). Again, the techniques and assumptions therein are different from ours and their model is hard to interpret.
In terms of matrix estimation techniques, low-rank matrix estimation and recovery have been studied extensively in statistics and widely used in numerous applications (Candes & Plan 2010 Candès & Tao 2010, Negahban Wainwright [01 | [Shabalin & Nobel 2013) 3 Gavish & Donoho [014] &
Zhang 2018 Wainwright, 2019). We adapt the techniques in this literature to provide convergence
analysis for our algorithm.
Finally, in operation research, assortment and pricing are important decisions for firms and there exists voluminous literature on dynamic assortment and dynamic pricing. Most of the work on assortment is based on the multinomial logit (MNL) choice model (Caro & Gallien 2007 Kök al. 2008 Sauré & Zeevi] 2013) and recently strand of work adopt the multi-arm bandit technique to the MNL model (Chen & Wang/ 2017 Agrawal et al., 2019 Kallus & Udell 2020; Chen et al.]
2021). For dynamic pricing, the problem usually comes with demand learning. In presence of covariates, the demand can be modeled as a parametric function (Qiang & Bayati| 2016: [Ban & nonparametric function (Chen & Gallego 2021) which adopt the continuum armed bandit techniques in Slivkins (201I). However, there are relatively few papers on the joint assortment-pricinng problem. Recently, Miao & Chao (202T) provides solution using the MNL choice model with finite arms, while our model targets at infinite many arms. In addition, their model assumes the products are independent of each other and can only handle small number of products. Their model can neither incorporates contextual information nor predicts new products. Roadmap. The rest of the paper is organized as follows. Section P describes the problem formu- lation and introduces our model with two concrete examples in assortment perriiiin--riiing and health care. Section 3 presents our Hi-CCAB algorithm and its convergence result. Finally, Section 4 shows the empirical results on simulated data and a case study on real sales data from one of the largest online retailers. The proof of our theorem and additional empirical results are provided in the Appendix.
Keskin 202I)
2 PROBLEM FORMULATION
In this section, we first introduce our high- gigh--imensional continuum armed and high- nigh-dimensional contextual bandit model. Since our model novel and different from traditional bandit models, we further provide intuition and two real applications of our model in assortment- -riittiiiingggi and healthcare. Finally, we show that a large class of bandit models can be reformulated into our model.
Notation. We use bold lowercase for vectors and bold uppercase for matrices. For any vector a, use ||a|| to denote its l norm. For any matrix A, we use || == Σig azy to denote its Frobenius norm, ||A|| 22 to denote its l spectrum norm, i.e., ||All2 := sup||=||a=1 |Ax|12, and ||||| := Ed 8y to denote its nuclear norm where d is the rank and sk 's are the singular values of A. We use b) := a Tb b denote the inner product between two vectors and (A, B) (Aace(AT)) B) between two matrices.
Problem setup. At each time f, we make one decision for batch of objects of size L. Before making the decision, we observe the attributes of these L objects, which can be characterized in potentially nigh-dimensional contextual vectors: It,1. Lz Rd= Then based on all the observations we have before time and the contextual vectors at time t, we decide on an action to take (or equivalently an arm choose), which can be characterized as high-dimensional vector a, that takes value in a constraint set AA in high-dimensional space Rdd,, After we take the action each time, we observe a batch of rewards,
Tt.j = a, Oar, + Etjs 1 ... L
(1)
where is low-rank matrix and Erg is independent noise with E[era] = and Varlerol S 0².
As bandit problem, our goal is design sequential decision-making policy that maximizes the expected cumulative reward, or equivalently, minimizes the expected cumulative regret. Specifically, suppose policy governs the way we take actions a1, 022 a.,..., we have an expected cumulative regrel measuring the difference between the cumulative expected reward of the best possible action when the underlying true parameter (i.e., +) known and that we can achieve under policy n,
T
I
RJ E
Σ aE.Ac
max
Σ a T Tor, - a., -,,00.1
(2)
3

---

Under review as a conference paper at ICLR 2023
where the expectation is taken with respect to Et.)) since a,,r depends on both. We seek an optimal policy x¹ that minimizes the expected cumulative regret Rg. Note that Ri grows with T. To better measure the performance of policy, we focus the time-averaged expected cumulative regret, RT./T. We will ignore the subscript æ for a for notation simplicity in the rest of the paper.
At first sight, our model seems remote from other bandit models and hard to interpret. Our model is, in fact, a generalization of a large class of bandit models, and the generality of our model makes it applicable to wide range of decision-making problems. In the following, we will parse the model and provide more intuitions.
Let us consider the classical K-arm bandit model (without context). Each arm can be represented by a k-dimensional standard basis and the covariate vector is simply 1. Then O becomes vector where each element is the mean reward for the corresponding arm. For the multi-arm contextual linear bandit problem, we further observe covariate æ as the contextual. Then each row of θ becomes the coefficient 3 for each arm, i.e., r = B æ for k = 1...., K. Our model further unifies large class of bandit models and we will formalize the above statements later in Proposition |
The novelty of our model lies in the low-rank representation matrix O It encapsulates the effect of the arm and covariates on the reward and exploits the low-dimensional structure in the high- dimensional arm and covariates. To be more specific, let us consider @ to be exactly low-rank and of rank d. Suppose its singular value decomposition is Q = USV. where U = Ia. S is a d d diagonal matrix with positive diagonal elements, and V TTV V -a. Let the left singular vectors U = (ui..... ud), the singular values in the diagonal of S be $1 2 $2 2 2 su > 0 and the right singular vectors V (UI...., va). Then the mean reward in can be re-expressed as
B(r]= = a Oz Σ (a, U.) ())
In other words, the mean reward is the summation of inner products between the action projected on the left singular vector and the covariates projected on the right singular vector, weighted by the singular values. By assuming O to be low-rank, the mean reward is assumed to be governed by only a few linear combinations of the arm attributes and covariates. Hence our model automatically ex- plores the dow-dimensional structure of the arm vector and the contextual vector in terms of its effect on the reward, from which we can draw interpretation and insights from the effective subspaces of both the arm and covariates.
As concrete illustration of our model and to explain why our model is reasonable in real applica- tions, we provide the following use cases the joint assortment-pricet problem and health care.
Example (Assortment and Pricing). In retailing and e-commerce, the assortment problem is to decide what combination of products to present at each given time with constraints on the capacity (Kök et al. 2008), and the pricing problem is to decide the prices of the products. The goal of the two problems is to maximize certain objective such as maximizing the revenue or profit.
Products can be usually characterized by attributes such as color, pattern, and fit for apparels or technical specifications for electronics and appliance. We focus on instant noodles, which will be our case study Section 4 Each product is single-flavor or assorted with different packs and can be represented as feature vector ń (# fllwooor # flavorz, " # fllavorm) where is the number of possible flavors and is priced as p. Then the store needs to decide on what products to present and their corresponding prices. Namely, the arm (action) vector can be represented as a (pi. PL. PL.P2.P1 P2. PKK PK, 1) where K is the maximum number of slots. The arm vector is clearly in high-ddmennsooal continuous space. At the same time, we observe the contextual covariates x for each period of time, such as the location and season at the aggregated level or demographics information at the user level.
The demand and sales of products with similar attributes react similarly to the same market con- ditions. often the case that there exists latent factors of the products that governs the demand and sales. Therefore, it is reasonable to parameterize the reward function in the form of (7) ruther than ignoring the similarity between products as the literature (Miao & Chao 2027 Kallus & Udell 2020 Chen al. 2022). Our model can further suggests new products rather than only the products that has already been provided.
Example 2 (Healthcare). In healthcare, for the monitorno apps which both monitors health conditions and give suggestions on actions to take for users, the arm (a) is high dimensinnll and continuous (e.g., sleeping time, length and kind of exercise, usage of social media, diet choices
4

---

Under review as a conference paper at ICLR 2023
including energy. water. protein, minerals, and nutrition intakes), and the health outcome not only depends on our suggestions, but also depends on the user's characteristics (e.g. age, gender. weight, height, basic health status, tendency of following suggestions) both the arm and the contextual variable vector are possibly high-dimensional and the arm can take continuous values. The classical bandit models do not fit the situation. The actions usually share similar effect on health and the user's 's characteristics can be usually captured by a few latent factors. Therefore, is reasonable to assume ② be low-rank.
contextual variables (x). Clearty,
To close the section, Proposition D shows that the traditional multi-arm bandit, multi-arm high- dimensional contextual bandit, and continuum arm bandit can be written in the form of model ①
Proposition! 1. The following bandit models can be expressed as special cases of our model.
1. (multi-and bandit) For i-th arm, = (0,0,-.. ,1,... 0), where is in i-th element. Suppose x has its first element heing constant. Then @ = 14s where ps is the mean reward of the i-th arm, and G.j = 0 jf # 1. Clearly, @ has rank 1.
2. (multi-arm high-dimensional contextual bandit) For i-th arm, a = (0,0, 0, ... ,0), where the contextual vector. Then O = (B,, (,,B2.... B2..... Bmm) where Gr is the parameter vector corresponding to i-th arm (Bastani & Bayati 2020
1 is in i-th element.
3. (continuum arm bandit (without contextuals)) Suppose the arm in original continuam arm ban- dit is denoted by and the mean reward function is f(a). Since all continuous function on bounded interval can be approximated by polynomial functions
arbitrary precision, it's rea- sonable to assume (a) to be polynomial of order n, which is not known precisely and only an upper bound N known. Let a (1, a, a² ",a,, ...... .... , aa" ,.... aN and suppose the first element of x is constant 1, then O., = (a) for = and O. 0.for j + Clearly, O is rank n.
H1-CCAB ALGORITHM AND THEORETICAL RESULTS
In this section, we present our learning algorithm with a regret upper bound. Specifically, we detail the Hi-CCAB algorithm in Section 3.1 and establish upper bound for its convergence rate of the time-averaged expected cumulative regret in Section
3.1 DESCRIPTION OF THE LEARNING ALGORITHM
Our policy consists of two phases for each period E [T]: the first phase learns a low-rank repre- sentation and the second phase determines the assortment and the selling prices. In the first phase, our policy estimates O by penalized least-square estimator using (a,, Dig, ri,)) for 1,...,t and = 1,.... L. Based on θ we look for the optimal assortment and pricing within the action space A,. Algorithm describes the detailed procedure of our policy.
Low-rank representation learning. As mentioned in Section 2 both the arm and the contex- tual vectors a E dR and æ E Rd are high-dimensional, and thus O E Id xds is also high- dimensional. Fortunately, there often exists structure in both the arm and covariate space as ex- plained in Section i] To leverage the underlying structure, we impose low-rank assumption on θ, which automatically explores the effect of the low-rank structure and the relationships between the action and the contextual arms.
To estimate the low-rank representation of θ at time t, one can adopt the rank lank-penalized least square:
+ == arg min
(a @z., raa)² + d rank(O)
where d > 0 is the penalization parameter and (On is the rank of the matrix O However, the rank penalization makes (4) a non-convex problem, leading to computational challenges. To address the computational challenges, the rank penalization term is often replaced by the nuclear norm in matrix estimation and completion literature so that the optimization problem becomes a convex problem. We adopt similar idea and our objective function then becomes:
- := arg min
(a Omi - ris)² + > 1
(5)
1=1
5

---

Under review as a conference paper at ICLR 2023
The penalization parameter A is updated in each iteration such that de Au/vV where Ao is the initialized penalization parameter, which can be chosen by cross- validation or guided by J
26, Σ ΣE= la, , Diy - ri,s 1/2.
Algorithm 1: The Hi CCAB Algorithm.
Result: Actions tt ........,.00 dT.
Input: The number of steps for initialization 11, set of possible actions A action vectors based on domain knowledge {a:) covariates vector { Him1 rewards ri,j for j -- 1,...: L. and exploration parameter I;;
Initialization: Ao < 25T..
Σ a, O,, Tii Xi 1la, t tii
while ( < do
A + \o/vi;
Low-rank representation learning:
θ + arg mine
1 iL Ei= (j=(a,, Ox, - ri,)² + X 011
Policy learning:
u.+1 + arg max Dj=1 a T
Exploitation i $ {[w3] w E 2+); a141 f- 0:+1:
Exploration if [[w3] w E } a++1 t u.+1 + 8141, update action space As+1
1. 8,+1 N(.., hla,) or:
2. 8,41 ~ N(Oa,, diag (f?)) and 7 sdl [ā. se(.) calculates the standard error;
Apply action ar+1 and observe reward T1+1, for j = L; 8 &
end
Policy learning. Once we estimated the low-rank representation of θ we can proceed to the action step. The goal of the action step to exploit the knowledge we learned from the previous time, i.c., + so as to decide on the next action a,+1 that maximizes the reward, and at the same time to explore actions that better inform the true θ, which in turns will help make better decision to achieve higher long-term rewards. Specifically, given θ and the covariate #z+1, for 1, L. we look for an action 0x+1 in the action space Ar that maximizes the total rewards across L objects:
L
ā.+1 == arg max acA.
Σ a T TQ.I4115 j=1
(6)
We further perturb â.fi for the purpose of exploration by adding random noise to each coordinate when & E [[wā]: : w E }+ i.e., a++1 = âz+1 + 8.41 where 8.+1 (N,,,,, hla,) and h is tuning parameter. The intuition for [w3] J is to explore more in the initial stage and exploit more in the later stage of the algorithm. To be specific, there are around T steps for exploration before time T. The density of exploration at a small time frame around T is T 3, which goes to zero as T + co. Note that the exponent can be any number larger than 1, instead of 2. which will affect convergence rate of the regret as we will discuss later Remark [ The polynomial form can be changed as well. For each exploration step, one can also let d,+1 N . diag(t:)) where each element of + the coordinate- wise standard error of the previous actions {a,) The intuition to avoid tuning parameter h while taking the right scale. Finally update the action space Ar41 according to a,+1. For example, if the action space A₃ E Rdd can be defined by an upper limit a and lower limit a,, then we simply expand the action space by pushing the boundary of each coordinate to a++1,5 if a1t1, & [ar.g> āt.5] for j == 1....,de. dg.
Remark 1. To take advantage of the interpretability of our model, we can further explore the struc- ture of the θ, Specifically, we can apply singular value decomposition (SVD) on θ to explore the underlying latent structure of the covariates from the right singular vectors; and apply SVD on (θ, Σ xi.)) to explore the latent structure of the arms from the left singular vectors. One can further rotate the singular vectors so reveal the underlying factors using techniques in factor analysis such as Varimax (Kaiser 1958 Rohe & Zeng 2020) or to perform clustering analysis by performing K K-means on the singular vectors.

---

Under review as a conference paper at ICLR 2023
3.2 THEORETICAL RESULTS
In this section, we establish in Theorem [ that the convergence rate of time-averaged expected cumulative regret for Algorithm 1 is at least fast as T i and outline the proof strategy for [h We consider the Lime-averaee the newly incurred regret, in the long run, more directly; on the other hand, the expected cumulative regret grows with time T, which is less interpretable. This theorem implies that the newly incurred regret, roughly speaking, converges to zero fast.
expected cumulative regret since measures the trend of
Theorem 1. Suppose Tt,1 i.dd [Od,, Ia,), and the errors (Et.3) defined in reward model | follows
normal distribution: Ex.j i.d o0) Suppose ② is rank d. Suppose the exploration step in Algorithm| is ar = ā. + 6, for t E [wwå] w E 2 where 6, N( hld,), A = {a R
a
< 1}. then there is Ti such that for T Z Tú the expected cumulative regret of the
Algorithm RT. satisfies
R7
S 22 Ldg T
12TT
72
L2dd
T
60
Ld
θ
2T
90 O + 1)
T it.
5
13
13
12
(7)
where Ti I. (dr + d (log(dz + da))² and the constant Ch, depends on h, L and Xn- For T S Ti, D5 S 2v Ld>
Remark (Convergence rate). An intuitive understanding of Theorem is that the expected regret
incurred each time converges to zero at a speed at least T ts as going to infinity. The convergence rate depends on the frequency of the exploration which depends on the exponent & in the exploration set, {lwå] | w E } Recall that the exponent can be changed with any number larger than 1, which can be considered as luning parameter.
Remark 3 ("Burnout" term). The first term in inequality 7 is a "burnout" term, where the algo- rithmn is gaining knowledge of O from scratch. We not impose any assumptions on these starting steps so that we have a relative conservative "burnoul" term. However, in practice, we usually have historical data to start with so that the algorithm can start from a reasonable estimation of O and can be any number
much smaller "burnout" term. Recall that the exponent of the exploration larger than 1. The order of the "burnout" term depends on the exponent of the w in the exploration set the more exploration there is, the smaller the "burnout" term. The exponent can be chosen depending on the situation - how ample the historical data is.
Remark 4 (Constant CL. of Tı). While constant CL.L.Do depends on h, L, Ao, the primary dependency actually on h and L. The order of xa in terms of dimensions and noise level is o dg. We do not assume the order of Do or bound it with high probubility bound in order to show its role time-averaged expected cumulative regret. If we utilize the order a / dzz then Ch, can be replaced by a constant depending on h and L only.
Remark 5 (Dependence on dimensions da, dz and rank d). When T is small, the "burnous" term (the first term) dominates. It depends on T and the dimensions but not the rank as + (log(dd d whose order depends on the exponent defining the exploration set
(da + )=)) dz)" (i.e., how frequent we explore). As T grows, the second term dominates. Recall Remark 4 No is of order dd so the second terms depends on T, d, and d but not d, at the order of (dd,, dT t).
Without the low-rank assumption, the order would be 22d -bb) instead. When T becomes even larger, the last two terms dominates, at the order (dd.T 3. However, the last case rarely happens, as it requires the order T equal to larger than 55, Therefore, taking dimensions and rank into consideration, the time-averaged expected cumulative regret is mostly the order of VdT t).
Proof sketch We outline the proof strategy for Theorem 1 There are two major steps: (1) bound- ing the estimation error for the low-rank representation matrix estimator; (2) bounding the expected cumulative regret. The detailed proof of Theorem iis in Appendix A
(1) Bounding the estimation error of θ with a high probability bound. Denote 80, = + e. We show that for large t,
oodd +I
22d
3 +
2
2
2
P |100 < Tie VTN2
+ 6λO
<1 -
h2T?
12
L1
1,333
T
7

---

wwwprieedssttoonnooneeeeeee ttpe result of the sum of squares in Under review as a conference paper at ICLR 2023
0g =10, d, =100
d. =30, d, =100
=50. d, =100
4000
Curulahe ragiee
Curulativ regres.
regnet
2000
1000
250
300
750
1000
250
5:00
1000
0
250
500
750
1000
Time
Time
Time
dg =10, dy =100, 5g =2
0g =30, d, =100, =2
dg
=50, d2 =100, Sc =2
2500
Cumulaaive regret
Cumuratvve rogret
1000
3500
500
2000
2000
04
250
300
750
1000
250
500
1000
250
750
1000
HI-CCAB
LinUCS
Lasco Bandit
NeuralJCD
EE-Met
Figure 1: Cumulative regret under the non-sparse (first row) and sparse (second row) settings.
Note that the action taken based on previous estimators and affect the accuracy of future esti- mators, leading to lots of dependencies. The classical matrix completion results can no longer apply. Through careful use conditional expectations, martingales, and empirical process separate different sources of randomness (i.e., ō1. ,6,, #1...' zt,)) to derive the bounds. Lemma establishes a the objective function. Lemma 2 establishes Lipschitz-type result of the sum of squares in the objective function. Further analysis of the nuclear- sorar-normpaa norm-penalized sum of squares with the two lemmas and low-rank properties gives the tail bound of the estimation enror.
(2) Bounding the time-aveen averagee expected regret. Let Q: {1|d0,1| I 7h ovd, 6λo v2d
VLA²
12T8
be the event such that ,0, is bounded. We know that from the first step, for large I, P(Qi) < Consider the expectation of the regret on Q, and Qi separately and
2 i + - La + L + TT
both terms vanish with at the polynomial rate.
4 SIMULATION STUDY AND
CASE STUDY
In this section, conduct simulation studies to compare the proposed Hi-CCAB with LinUCB Li et al. 2010), Lasso Bandit (Bastani & Bayati| 2020). NeuralUCB (/hou et al. 2020) and EE-Net (Ban al. 2022); then study the joint essortment-pricinn problem the e-commerce platform for one of the largest instant noodles producers in China. Details on the tuning parameters of each algorithms and additional results of the case study are provided in Appendix B
We consider the multi- linear bandit setup, a special case of our model as shown in Proposition]] | i.e., O (B1. B2. ... B..) so that each row of @ is the parameter of each arm for the multi-arm contextual bandit. Specifically, we the number of arms d, {10,30 30, 50) and the dimension of covariates d 100. For θ we consider a non-sparse and sparse case. For UDV where is diagonal matrix with (1, .9, .9, .8, 5) as the diagonal entries. All entries of U and V are first generated from i..d. (0, 1). and then applied Srum-Schmidt to make each column orthogonal. U scaled have length /d, so that the rewards are comparable across different d, 's. For the sparse 2 randomly selected elements that are drawn from (0, 1). We generate the covariate æ i.i.d N(0, Ia.) and the rewards from with o = 0.1.
Simulation study
non-sparse case, we generate
Rdd xr, V E Rd, xr (r = 5), and D
case, each row of O are set as zero except for
Figure I shows the cumulative regret (averaged over 50 simulations). For the non-sparse case, Hi-CCAB converges faster than all other methods. The advantage of Hi-CCAB is more pronounced when the dimension of the arms becomes larger. For the sparse case, which not to the advantage of Hi-CCAB, when the dimension of arms is relatively small (d,, 10), Lasso Bandit converges faster but the gap between Hi CCAB and Lasso Bandit is small. As the number arms increases, Hi-CCAB outperforms all other methods.
Assortment-pricing case study. The original data contains daily sales of 176 products across 369 cities from March 1st, 2021 to May 31st, 2022 (T 456 days). We aggregate the sales by 31 provinces. Each product is of either single assorted flavors (13 possible flavors) with different
8

---

Under review as a conference paper at ICLR 2023
Time ael--eeereeee cumutartaae Forbve regret
50000
Pct sales Increase
480%
40000
440%
30003
400%
20000
HI-CEAR:
350%
HI-CCAR: 11-100
action
100
100
Time
Time
(a) Time-averaged cumulative regret.
(b) Percentage cumulative sales gain.
Figure 2: Performance of Hi -CCAR compared with real actions over 100 simulations. The bound- aries of the shadow boundaries are the 5-th and 95-th quantiles.
counts. The assortment and price of each product changed daily. In addition, we know the dates for promotion. The assortment, prices, and promotions were the same across locations. The maximum number of products to be shown the homepage is K 30. The total possible combinations are then (106) and therefore if we consider one combination as one arm, are facing extremely high-dimensional arms, for which most multi-arm bandit algorithms are not applicable.
To apply Hi-CCAB, we specify the arms a and the covariate vectors (= at given time following the setup in Example [ The arm is represented as (PI. Pi.PI. Pr. pÝ. promo1, promoo.. 22(m+2)K+1=001 where Pt:
PK PK. PK: PK.Pl Pk.promop promok.
a vector of non-negative integers to denote the counts of m = 13 flavors, Pk is the price, promo, is the indicator of promotion of product k and P is the element-wise quadratics. The covariate æt. TR:00 for location includes dummy variables of 31 provinces, year 2021/2022, 12 months, weekdays, and an indicator of annual sales event on Jun 18 and Nov 11. More details are deferred to Appendix
(# [flaoorr.1,"
# flavork.m)
To run simulations using the dataset, we first create a pseudo-truth model. To be specific, we esti- mate & and using all data of 456 days and consider them as the pseudo ground truth. We perform a sanity check on our model assumption [ the pseudo ground truth against our data before pre- ceding to the formal analysis and further examine the structure of the representation matrix O in Appendix C We evaluate the performance of Hi-CCAB in terms of the cumulative regret (2) and the percentage gain of the cumulative sales by comparing with the original actions, since no existing bandit algorithm applicable to this problem.
Figure 2a shows the time-averaged cumulative regret (averaged over 100 simulations) and Figure 2b shows the percentage gain in cumulative sales compared to the real sales The time-averaged cumulative regret of Hi Hi-CCAB converges to zero while that of original actions remains flat. In terms of percentage gain in cumulative sales, Hi-CCAS boosts cumulative sales by more than times. On a separate note, HC-CCAB with exploration performs better in terms of both cumulative regret and percentage sales gain than Hi-CCAB without exploration.
5 CONCLUSION
With an increasing demand for online decision-making the bandit problem is receiving increasingly more attention from both thcoreticians and practitioners. Even though the volume of bandit literature has been expanding, there exists very little literature tual bandit with high-dimensional covariates. In this work, we formulate and propose a model for this problem. Our model is general as it unifies large class of bandit problems and has interpretabil- ity and predictive power. We propose an efficient algorithm Hi-CCAB by adopting the low-rank matrix estimator and provide an upper bound for its convergence rate in terms of the time-averaged expected cumulative regret. The generality and flexibility of our model allow for its application in the joint ussortment-priccng problem, where the assortment and pricing optimization problems have been studied extensively in operation research separately but not their joint optimization problem. problem
digeediimnnional continuum armed contex-
By applying our model and algorithm to the real case study on the joint assortment- for one of the largest instant noodles producers in China, we are able to boost the sales by four times and provide insights into the underlying structure the effect on the reward the arms and covariates such purchasing behaviors. Therefore, both the theoretical and the real case study in- dicate that our model and algorithm can be effective for the digh-dimensional continuum armed and nigh-dimessional contextual bandit problem faced by decision-maker in various fields. Since model new to the bandit literature, there is space for improvement in our regret analysis. This is an interesting future direction.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Yasin Abbasi-Yadkee David Pal, and Csaba Szepesvari. Online-to-comfidenc-sstt conversions and application to sparse stochastic bandits. In Artificial Intelligence and Statistics, pp. 1-9. PMLR, 2012.
Rajeev Agrawal. The continuum-armed bandit problem. SIAM journal control and optimization, 33(6):1926-199 1951, 1995.
Shipra Agrawal, Vashist Avadhanula, Vineet Goyal, and Assaf Zeevi. Mnl-bandit: A dynamic learning approach to assortment selection. Operations Research, 67(5):1453-1485, 2019.
Gah-Yi Ban and N Bora Keskin. Personalized dynamic pricing with machine learning: High-
dimensional features and heterogeneous elasticity.. Management Science, 67(9):5549-555 2021.
Yikun Ban, Yuchen Yan, Arindam Banerjee, and Jingrui He. Re-net: Exploitation-exploraiii neural networks in contextual bandits. 2022.
Hamsa Bastani and Mohsen Bayati. Online decision making with high-dimensional covariates. Operations Research, 68(1):276-29) 2020.
T Tony Cai and Anru Zhang. Rate-optimal perturbation bounds for singular subspaces with appli- cations to high- nigh-dimensional statistics. The Annals of Statistics, 46(1):60-89) 2018.
Emmanuel J Candes and Yaniv Plan. Matrix completion with noise. Proceedings the IEEE, 98 (6):925-936, 2010.
Emmanuel J Candès and Terence Tao. The power of convex relaxation: Near-optimal matrix com- pletion. IEEE Transactions on Information Theory, 56(5):2053-2088 2010.
Felipe Caro and Jérémie Gallien. Dynamic assortment with demand learning for seasonal consumer goods. Management science, 53(2):276-25 2007.
Ningyuan Chen and Guillermo Gallego. Nonparametric pricing analytics with customer covariates. Operations Research, 69(3):974-984, 2021.
Xi Chen and Yining Wang. note on a tight lower bound for mnl-bandit assortment selection models. arXiv preprint arXiv:1709.06// 2017.
Xi Chen, Chao Shi, Yining Wang, and Yuan Zhou. Dynamic assortment planning under nested logit models. Production and Operations Management, (0(11)85-102 2021.
Yang Chen, Miao Xie, Jiamou Liu, and Kaiqi Zhao. Interconnected neural linear contextual bandits with ucb exploration. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp- 169-181. Springer, 2022.
Raphaäl Féraud, Robin Allesiardo, Tanguy Urvoy, and Fabrice Clérot. Random forest for the con- textual bandit problem. In Artificial intelligence and statistics, pp. 93-101, PMLR, 2016.
Matan Gavish and David L Donoho. The optimal hard threshold for singular values is 4/v3. IEEE Transactions on Information Theory, 6008))500- 2014.
Botao Hao, Tor Lattimore, and Mengdi Wang. High-dimensional sparse linear bandits. Advances in Neural Information Processing Systems, 33: 13:10753-10763 2020.
Jiachen Hu, Xiaoyu Chen, Chi Jin, Lihong Li, and Liwei Wang. Near-optimal representation learn- ing for linear bandits and linear rl. In International Conference Machine Learning, pp. 4349- 4358. PMLR, 2021.
Henry F Kaiser. The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23 13):877-000, 1958.
Nathan Kallus and Madeleine Udell. Dynamic assortment personalization in high dimensions. Op- erations Research, 6824):1020-1033 2020.
10

---

Under review as a conference paper at ICLR 2023
Gi-Soo Kim and Myunghee Cho Paik. Doubly-robust lasso bandit. Advances in Neural Information Processing Systems, 32, 2019.
Robert Kleinberg. Nearly tight bounds for the continuum- armed bandit problem. In Proceedings of the 17th International Conference on Neural Information Processing Systems, NIPS 04, pp- 697-7044 Cambridge, MA, USA, 2004. MIT Press.
Robert Kleinberg, Aleksandrs Slivkins, and Eli Upfal. Bandits and experts in metric spaces. J. ACM, 66(4), May 2019. ISSN 0004-5411 5411. 1145/3299873
doi: 10. 10.1145/3299873, URL
org/10.
A Gürhan Kök, Marshall L Fisher, and Ramnath Vaidyanathan. Assortment planning: Review of literature and industry practice. Retail supply chain management, 122(1):99-153, 2008
Akshay Krishnamurthy, John Langford, Aleksandrs Slivkins, and Chicheng Zhang. Contextual bandits with continuous actions: Smoothing, zooming, and adapting. The Journal of Machine Learning Research, 21(1)55402-5446, 2020.
Branislav Kveton, Csaba Szepesvári, Anup Rao, Zheng Wen, Yasin Yadkori, and S Muthukr- ishnan. Stochastic low-rank bandits. arXiv preprint arXiv: 2017.
Sahin Lale, Kamyar Azizzadenesheli, Anima Anandkumar, and Babak Hassibi. Stochastic linear bandits with hidden low rank structure. arXiv preprint arXiv: 999.09999, 2019.
Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A catexttualbanddee approach to personalized news article recommendation. Proceedings of the 19th international conference on World wide web, pp. 661-670, 2010.
Tyler Lu, Dávid Pál, and Martin Pál. Contextual multi-armed bandits. In Proceedings of the Thirteenth international conference on Artificial Intelligence and Statistics, pp. 485-492, JMLR Workshop and Conference Proceedings, 2010.
Yangyi Lu, Amirhossein Meisami, and Ambuj Tewari. Low-rank generalized linear bandit problems. In International Conference on Artificial Intelligence and Statistics, pp. 460-468. PMLR, 2021.
Maryam Majzoubi, Chicheng Zhang, Rajan Chari, Akshay Krishnamurthy, John Langford, and Aleksandrs Slivkins. Efficient contextual bandits with continuous actions. Advances in Neural Information Processing Systems, 13:349-360, 2020.
Sentao Miao and Xiuli Chao. Dynamic joint assortment and pricing optimization with demand learning. Manufacturing Service Operations Management, 23(2):525-545, 2021.
Sahand Negahban and Martin J Wainwright. Estimation of (near) low-rank matrices with noise and high-dimensional scaling. The Annals of Statistics, 39(2):1069-1099 2011.
Matteo Papini, Andrea Tirinzoni, Marcello Restelli, Alessandro Lazaric, and Matteo Pirotta. Lever- aging good representations in linear contextual bandits. In International Conference on Machine Learning, pp. 8371-8380. PMLR, 2021.
Sheng Qiang and Mohsen Bayati. Dynamic pricing with demand covariates. 16X.v:1604.07463, 2016.
arXiv preprint
Herbert Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58(5):527-535, 1952.
Karl Rohe and Muzhe Zeng. Vintage factor analysis with varimax performs statistical inference. arXiv preprint arX5v:2004.05387, 2020.
Denis Sauré and Assaf Zeevi. Optimal dynamic assortment planning with demand learning. Manu- facturing Service Operations Management, 15(3):387-404, 2013.
Andrey A Shabalin and Andrew Nobel. Reconstruction of low-rank matrix in the presence of gaussian noise. Journal of Multivariate Analysis, 11::67-76, 2013.

---

Under review as a conference paper at ICLR 2023
Aleksandrs Slivkins. Contextual bandits with similarity information. In Proceedings the 24th annual Conference On Learning Theory, pp. 679-702. JMLR Workshop and Conference Pro- ceedings, 2011.
Joel A Tropp. User-friendly tail bounds for sums of random matrices. Foundations of computational mathematics, 12(4):389-433) 2012.
Eralp Turgay, Cem Bulucu, and Cem Tekin. Exploiting relevance for online decision decision-making in high-dimensions. IEEE Transactions Signal Processing, 69:1438 1433-1151,, 2020.
Hemant Tyagi. Sebastian U Stich, and Bernd Gärtner. On two continuum armed bandit problems in high dimensions. Theory of Computing Systems, 58(1):191-222, 2016.
Martin Wainwright. Iligh-dimensional statistics: non-asymptotic viewpoint, volume 48. Cam- bridge University Press, 2019.
Kan Xu and Hamsa Bastani. Learning across bandits in high dimension via robust statistics. arXiv preprint arXiv:2112.142:: 2021.
Pan Xu, Zheng Wen, Handong Zhao, and Quanquan Gu. Neural contextual bandits with deep representation and shallow exploration. 2022.
Jiaqi Yang, Wei Hu, Jason D Lee, and Simon S Du. Impact of representation learning in linear bandits. arXiv preprint arXiv:2010.06531, 2020.
Dongruo Zhou, Lihong Li, and Quanquan Gu. Neural contextual bandits with ucb-based exploration. In International Conference on Machine Learning, pp. 11492-11502. PMLR, 2020.
APPENDIX A PROOF OF THEOREM [
In this proof, we denote the true parameter as O
Let (r) == ELT Σt= Eta Ox, - r. Then we have the following lemmas that we
will prove later.
Lemma Suppose all the assumptions Theorem 7 holds. Denote ET(A) LT + +) (r(O") (VLr(("), A). Then with probability at least 1 LT
2
T
T
ET(A) 2
-47 (1 h22) (2dg + 2da + 6 T + 6 log L)² log T AlI2. (8)
2T
Lemma 2. Suppose all the assumptions in Theorem holds. With probability at least
7
2 LN
LT + t the following holds for all A
ovde +I
IVVT A ~|
T
VLT
max{d du blogdda + dx)
ShoTT-2/3lg)) 2/3logTT
L
Sho T
/ Vlgg(T (d 3log((T))) + (log()) + da) + 2logT))
||||.
(9)
Recall the definition of θ. know that
(rr)) + Ar < (r())) +r
(10)
Denote 80, = θ 4 and for notation simplicity we will drop the subscript & for J0, in the following when there is no confusion. Equation (T0) then implies that
(T( < - -VVLT(0") (0) + Ar (||*')-- =|@* + +011.).
(11)
Suppose the singular value decomposition of θ* is O USVT where S is d x diagonal matrix. Let U+ be an d, (d, - d) matrix satisfying (U.U.)(U,U, = Ia.- We define V. similarly.
12

---

Under review as a conference paper at ICLR 2023
Denote _ U Then | 50||- |
|60 801l1=
| +
| 160 80 -
| || + | v2d|
80 _0_11F.
Going back to Inequality and combing with Lemma] Tand Lemma at least 1 3
2 we have, with probability
T the following holds
LT
[
1:2
-147 -14T-3 (h+ (2dz + 2dg + 6log T + 6log L)² log |
2T
11001|r dv T
VLT
(8ho
Vlo((TL) v (d, Blog((TT)) + +3logT) + da) 2logT)+
(12)
T
max (da; dz Hogg du + do)) (160 80 || + 1
2hoT ShoT-2/3logTT 1/3
L
VT V2d 50 do 1601
VT
20
T
T
Note that | - S0lll. S -
- 50llle, divide both side with ||0@|| and multiply both
sides with 37 //2. Suppose Ti satisfies
Ti 28,
1 ) (2d, + 2d. + 6 Ti + 6log L)) log Ti. - 212 x 14(1 + h
Sho
AoTT 2 T
log (TiL) (d, + (llog
3log Ti (log(ds + do) + 2logTi)+ (13)
max da. dz } log (de + dz)
ThoT 2/3
log Ti
L
we have
3 oodez +I
22d
|0 <
Tí VLh² + 6λO
(14)
1:TT!
Note that there is a constant Cr,,,Ao depending on L, and Ag such that for
Ti 2 Cn - d=)6 (log(dz + d
Inequalities (13) holds.
Next we will proceed to bound the regret. Denote the event that
(4) holds to be Q and its comple-
ment as Q Then ((2))= <
T
+
2
1T
+ 1 and Qi, 0.) IL b Let the oracle
TE
13

---

Under review as a conference paper at ICLR 2023
optimal action at time t be aj and b, El= 21,1- Then
ai+1 O act1d a,41 O 2e41.1)
R7 - E
1=0
E=1
E
a
o x1+11
ai+ x1+1l
14
@ la
1+bb+1ll2
Õ.b,+1
(
,
E
O'
1(Q1)
+B
(10
-0bb+1 [(Q
0
b
@
|
(0)
6.)b
1
b ,,b,41,
E
O* 11Q0)
|0
2
2
2
+ 2
1
61+
b
||]
+
L1
t
33
2
l2 b 0
TV
+
8
Lt
2
2
2
||]1000
|
Ld O
f
Lt
L
t
dv
+1
V2d
3
2
2
2
2
Ldz
+ oλO
+ 2
Ld
||
O
2
+
ITS
VTN²
1121
12
Lt
L
7
(15)
Similar arguments also give
E
Σ Σ
E
(a; T O' a.,1
aj ('a...))
ST x 2 Ld, |0
(16)
t= 1=1
Therefore, for T211 Ti
R T 2 Ldg 0 1
90
(dg
1)
72
v2dd
Ld 0 22
T
+
Ao
T
13
13
h/2
112
(17)
A.1 PROOF OF LEMMA Suppose Tt, ar,, O It,1 + dEt,, then
T
L
VL,
E1,1
LT
t=1
1=1
(18)
LT
+
Σ
(_ - -.,
LT
t=
14

---

Under review as a conference paper at ICLR 2023
Now consider the terms in (18
separately. Let
S LT
11,10111111
T
Σ
(19)
T
E1,100110
LT
1=1
Elementary Calculation show that
( + 2dx)
B(1122117) S
(20)
1,TT2
Therefore,
ovd, +
1
P(|S22|r2 2
T
I
(21)
VLT
T
3
For S3, let G' be event defined as
G = max[[ex,1
: log TT.
max [x, | S L) S 2d,
6 6log LT,
(22)
max(]]6./h]] : :1 - 2d, + 6log T
Then elementary calculation shows that
2
1 1
P(G') S
L3
LT
T' +
(23)
Using Matrix Bernstein Inequality Troppp 2012 on event G, we have the operator norm of S3 on G is bounded as follows
LT P([1
-a²
Salla 2 nG) S (dn da) 203, + 2D2/3
(24)
where
0 max {
Σ E
10
00,100 T)
11:
(25)
T
Σ>>
E
11,101111
|1
[=1
and
D max
sup event G holds
1 06 1la S 61. TLh + 3log [T, + 3log T). (26)
Elementary calculation shows that taking
03, = 1 [ mmxx d
(27)
satisfies Equation 25
Taking
2hT L max d Hog(d, + +d,++
(28)
8hl MolyToggLVy (d,, +
(3logT) + da) + 2logT)
LT P(l
1
Salla 2 2t) T2
(29)
15

---

Under review as a conference paper at ICLR 2023
Therefore, have
max{da, de Hoggdd + dz)
p (11Solla s 2hoTT21/lopp
8ho
(T
(d=
+ (logT) da) + 2logl))
(30)
T
2
1
2 1
L
LT
T
T
Recalling that
(VLr(""), 1(52, A) + (Sa. A> I | - ||S5||2||A||
(31)
we get the statement of the lemma.
A.2 PROOF OF LEMMA Let b, Ei= Lt,1. Let 8, 0 for exploitation rounds. Then we know that
T L
ET(A)
Ax.,)
a.
2LT
1=
(32)
b. T
1-
8, )
2LT
b θ_11
1=
Define
T L
b, O
DA(A) = 2LT
Azt,1) + (6, Ax.,))
|6 1T
1=1 1=1
T
b. T
DI,T(A) 2LT
bb TT,1l ABt,))²
(33)
[=1 1=1
T
(2,((A) 2LT
6 Ax
Then
b T
-
A AxL.))
(34)
LT
bb OT,
Elementary calculation shows that
B(Er(A) Dr(()) 0,
(35)
and
T
E(Dar(A))
h 11
(36)
2T
Now we proceed with proving that the following two holds with high probability:
Er( - (r
inf
-77
3 h1) (2d. + 2d, + 6log T + 6 6log I.) log T
|
|
D2.T( A) EE(D,TT
inf
-77 3/ 112) (2d, 2d, 6log T log L) log T.
|>00
| |A
(37)
Note that
| ~ x2, 110//11/ x². Therefore, we have that
P(sup
| < dg + 2c1 2 ci sup (6, S d, + 2ez + 2 2222222,, e2da)
2.l
(38)
2 1 - (LT (-e1) + Txp (-62))
16

---

Under review as a conference paper at ICLR 2023
Let 61 2og LT, 62 2 T..
Denote
U1 d3 + 22g 2 E22d U2 de + 2ez + 2/ Ez
(39)
And let the event O be
0 (sup | S U1, sup | - U2}.
(40)
t
For the following, we restrict our attention to event O.
Note that
inf 5o
(r(4) Dr
inf
ET(A) Dr(A), (41)
Un/1.11AAla5500.01 10 151ST
also most [T terms in the sum of (T(A) Dr(A)
zero, and for any
in the exploration round / 6 #0 for ISIST
[1b e (0, Swai))
TT
inf [oL fo |b, TT, (6) AxT,))
2U 2U1VU
Therefore, through Functional Hoeffding theorem (Theorem 3.26 in Wainwright (219))), we have
T 7
[Tē] 16UUU
(ao(4)-Dr(C & -n10) S exp
(42)
for > O.
Similarly, for the exploration rounds (A), we have
sup U
(6, ATAAI))))
(
inf (67 ARw1)²) U.LLLSIANSS,
U1U2UR12
(43)
Again, according to Functional Hoeffding theorem, we have
T
72
[T')
P(D2.T (A) (B (D2, -721)) s exp
(44)
4U?
Take 71 =2 - 112) (2d, + 2d, + 6 T + 6log L | 12 log T.
Therefore,
P ((E(A)) { -17 -
(h (h+ (12) (2dg + 2d, + 6 T + 6log L)² | log 7) <P(ET(A) - Dr(A) S -77 3 (h+ 1:3) (2d, 2d, 6 6 6log L) A log T1O)
+ (A) S 0J0)
+ +P (D B(D2T) S -777-3 -h (h+ 023) (2d, + 2d, + 66 + 6log L) |4 log T10)
PPO0
2
< LT T T
(45)
Hence with probability at least
LT 7 1 72.
T BRA
Er(A)
147 (h + h2) (2d, + 2d, + 6log + 6 6og L) ||| lo (46)
2T
APPENDIX B DETAILS ON THE SIMULATION STUDY
In this section, detail the tuning parameters of each algorithm used for the simulation study.
17

---

Under review as a conference paper at ICLR 2023
Hi-CCAB. There are three tuning parameters for Hi-CCAB: we set the steps for initialization
100, the initialized penalization parameter Ao 21, Σ and the exploration parameter h .1.
|a O Di ( Ft, 2 [ 112-
LinUCB (Li al. 2010). We apply the LinUCB with disjoint linear models and set multiplier for the upper confidence bound a = 1++++ +l++-/n(f /n(2//)/2 with õ = .05 as suggested in the paper.
Lasso Bandit (Bastani & Bayati, 2020). There are couple of tuning parameters in the original algorithm including h for the set of "near- "near-optimal arms", 9 for the force-sample set, and λ and A2.0 as the regularization parameters for the "forced sample estimate" and "all-sample estimate". We follow the original paper and set h = 5, λ 22 = 0.05. We set q = 2 so that the size of initialized forced sample set is close to that we used for Hi Hi-CCAB.
NeuralUCB (Zhou et al., 2020). The tuning parameters of NeuralUCB include the confidence parameter as in all UCB-based algorithm, the size of neural network, as well as the step size, reg- ularization parameter for gradient descent train the neural network. We adapted the code from https://g thub. cww/ uel am /Neu ra UCB and used the default settings.
EE-Net (Ban et al. 2022). EE-Net involves tuning parameters for gradient descent to train the ex- ploitation network. exploration network, and the decksion-maker maker network. We adapted the code from https://github. wwwp:attthuu..oooaaaaeaeeeeeaaae catoom/manikkun BEE-NEE-Nee Net ICLR- 2022 and used the default settings.
APPENDIX C
MORE DETAILS ON THE CASE STUDY AND ADDITIONAL NUMERICAL RESULTS
In this section, we provides more background information on the case study and additional interpre- tations of the represetation matrix θ and numerical results.
Figure 3a shows the daily sales by product and each color represents one product (only products that appeared more than 95% of the days are colored; the rest are colored as grey). The days corresponding to the vertical dashed grey lines are days with promotion. The two red vertical lines correspond to the annual sales events. The variation between products was large and one product dominated the rest most of the time. The sales were also driven by the promotion the sales went up when there is promotion. Figure 3b) shows the median unit price across time with the 25th and 75th quantiles as the boundaries of the grey area. The median unit price was around 3.2 RMB and there were variations in unit price among products. Figure 3c shows the number of single- flavorr and multi-flavor products. three-quarters of the products were single-flavored. Note that products with the same flavor can have different package sizes. Figure Bd shows the number of products with different package sizes. The package size of about 60% of the products is larger than 20 with 30% having package sizes between 10 and 20 and the rest less than 10.
Figure compares the real sales with the simulated sales based on model (1) using the pseudo ground truth θ and that we estimated using all the data, i.e., the teal line is >)=1 Vr.j where Tt.j is generated by
Tti a. Oæ:j + Erj Et, OO,
where a; and j are from the real data. As shown in Figure 4 the real sales and the simulated sales follow quite closely across time, which indicates that both our model and estimation are reasonable.
Structure of the representation matrix O. One advantage of model is the interpretability which allows us to gain insights from the representation matrix O. Specifically, our model is able to discover the underlying factors of the effect of both arms and covariates on the reward. In the following, we will examine the pseudo ground truth θ we obtained using all the data.
The rank of @ 5 with the singular values being (2.5, 0.3, 0.2, 0.02, 0.002). The leading singular value dominates the rest and thus the leading left and right singular vectors are the most important ones in explaining the effect on the reward and we focus on the leading singular vectors in what follows.
Figure 5 shows the loadings for different covariates (i.e., the leading right singular vector) and our algorithm is able to learn interpretable patterns of the effects on the reward - for weekday, the effects are drastically different during the weekend and during the weekend; for months, the effects show different patterns during the promotion month (June and November) from other months; for location, the effects of the coastal provinces are different from the rest, which exactly corresponds to the levels of economic development of different regions in China. In sum, our model can exploit the underlying structure of the covariates and provide insights into purchasing behavior and seasonality.
18

---

Under review as a conference paper at ICLR 2023
Sales by product thousand RMBB)
الللال
Modin pripp (PAM8)
Wo 407777 Ny | الااا اور اااراو
500
250
300
2000
350
430
22.0 Time
Time
(a) Total sales by product in 1k RMB.
(b) Median price of products.
Package size
B.st
Numbor of protcca
Numbor of prouucs
yyu unu
Time
Time
(c) Number of products with various number of flavors. (d) Number of products with various package sizes.
20000
15000
Sales
10000
sales
Iimulated sales
5000
100
200
300
400
Time
Figure 4: Real sales vs simulated sales.
19

---

Under review as a conference paper at ICLR 2023
0.03
0.33
0.8
5. 0.93
0.33
01
62
13:
Jan Roy Jun
-03
(a) Weekday
(b) Month
(c) Location
Figure 5: Loadings of the leading right singular vectors for the covariates.
On the other hand, Table I explores the loadings for the arm on May 29th 2022, the last Sunday in our data (i.e., the leading left singular vectors multiplied with (v1, .) where # is the average of 2y for j = .....
, I. on May 29th 2022). Specifically, we investigate the effect of flavors on the reward given the context. We take the average of the loadings of the linear and quadratic terms for each flavor in all 30 products and compare with the total sales of each flavor across all Sundays in Mays. For ease of comparison, we further scale the sales and the loadings by their corresponding largest numbers. The loadings and sales are closely related to each other As in Table | on May 29th 2022, flavor 1 (FI) has the largest effect, followed by flavor 10, 13, 7, and 11. Therefore, our model learns the values of the flavors (per unit).
F1
F2
F3
F4
F5
F6
F7
P8
F9
F10
F11
F12
F13
Sales
1.00
0.05
0.00
0.00
0.00
0.03
0.19
0.00
0.08
0.19
0.18
0.00
0.38
(linear)
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
(quadratic)
1.00
0.12
-0.00
0.00
0.00
0.03
0.19
0.00
0.15
0.39
0.16
0.03
0.33
Table 1: Total sales and loadings of the linear and quadratic terms (scaled) of the 13 flavors.
More on simulation with additional numerical results. We first detail how we ran the simulation and then provide more simulation results. To be specific, first use f 100 for the initialization step to estimate θ : and then at each time 4 + ,,....,T T, we follow Algorithm to decide on the action a, for assortment and pricing. After determining a,, we generate the sales according to (T) using the pseudo true O and o. We further compare the performance of the cetttmment--iiiinig policy with exploration and without exploration and with different initialization time t1. Each setup is simulated 100 times.
Figures 6a 6b show cumulative regret and Figures 6d show percentage gain in cumulative sales when & 20, 50, 100 with exploration and without exploration. Hi-CCAB with exploration per- forms better then without exploration. As expected, longer initialization steps provide better initial estimation of the θ and thus helps with the performance in short time windows. As time goes by, the time-averaged cumulative regret all converge to zero and the percentage gain in cumulative sales should converge.
'The correlation of sales and the linear-term loadings is 0.91 and that of the quadratic-term loadings is 0.97.
20

---

Under review as a conference paper at ICLR 2023
Time: aueroger curuiative regret
Time averoges curulaaive regrett
40000
40000
HH-CCAB: explor explore
HH exploeeppore
HI-CCAB: no explore
CAB: -1000: explore
20002
Naive
30000
Navve
20000
20003
100
200
300
400
100
200
300
400
Time
Time
(a) Cumulative regret when 20.
(b) Cumulative regret when f 50.
Parcentage currutative sales Increase
500%
Time aavaaaggeee curruiative
40000
300%
30000
200%
explore
100%
II-CCA::
explore
20000
R-CCAB: explore
expbre f1-00 appire
expbre
0%
explore
100
200
400
100
200
400
Time
Time
(c) Cumulative regret when f 100.
Percentage cumulative sales gain.
Figure 6: Performance of Hi -CCAB with different initialization times t and with exploration and without exploration.
21


Reviews:
[{"review_id": "zxOf_Vn8Vr", "reviewer": "Reviewer_q4F5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper considers the problem of a contextual bandit with both a high-dimensional action space and a high-dimensional covariate space. The author(s) propose a new bandit model describing the aforementioned problem, and provide a learning algorithm under the model.  Theoretical results are developed for the algorithm and the effectiveness of the algorithm is demonstrated with an assortment pricing case study.", "summary_of_the_review": "In general, I think the paper considers an interesting problem and proposes an appropriate model describing the problem. The paper however can be improved with improved theoretical results and presentation.", "strength_and_weaknesses": "Strengths:\nThe paper studies an interesting and relevant problem and proposes a model that may be applied to a broader class of problems. \n\nWeaknesses:\n1. The novelty and contribution of this paper are limited, especially the theoretical guarantee. \n2. The writing of the paper can be improved.\n\nDetailed comments:\n1. The first sentences in the abstract are essentially saying \"we formulate a bandit problem as a bandit problem\".\n2. Line 5 of the abstract: \"contextual\" -> \"context\"\n3. On page 3, is the concept \"approximately low rank\" defined?\n4. In (1), are the contexts stochastic (they appear to be so from the definition of the regret)?\n5. On page 5, it is not justified why in the healthcare example one would expect a low-rank matrix.\n6. Theorem 1: \n (1) the result is not very informative in that one can not see the dependence on $d_x$, $d_a$ and $d$ (i.e., how using low-rank representation learning can address the issue of high dimensionality).\n (2) As T goes to infinity, are $d_x$ and $d_a$ also going to infinity? Since the difficulty of high-dimensionality mainly\nstems from the fact that $d\\ge T$. It seems to me from the proof that at least $d_x$ is treated as fixed as $T \\rightarrow \\infty$. Please correct me if I am wrong.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_6Gu1RcyHO0", "reviewer": "Reviewer_K6HK", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a general framework to solve the high dimensional continuum armed bandit problem and high dimensional contextual bandit problem jointly. This research problem is significant and the proposed model is novel. I agree that the proposed problem has broad scope of applications. For this problem, the paper propose an efficient algorithm called Hi-CCAB, by adopting the low-rank matrix estimator. Experimental study shows the proposed method can improve the naive approach by 4 times for the cumulative reward. ", "summary_of_the_review": "1. The problem is significant and interesting but the assumption remained to be listed clear. \n2. The experimental study should be improved. The current version is not convincing. \n3. The exploration manner should be explained and regret bound could be proved accordingly.  ", "strength_and_weaknesses": " However, I have the following concerns about the exploration manner, related works and experimental design. \n(1)\tAlthough the proposed model is based the low rank representation matrix assumption, I cannot get the reason why the epsilon term is bounded. Previous works usually assume a sub-gaussian distribution for the error term. I suggest the authors to explain why the low-rank assumption is enough to bound the epsilon. \n\n(2)\tThe paper gives two different manner (2 different gaussian distribution) for exploration in each step. Theorem 1 shows the regret bound based on the first one. What about the another one? Is it possible for us to propose a general regret bound which is independent on the distribution used in the exploration?\n\n(3)\tThere are some important related works are missing. Recently the direction of proposing deep learning based model for high dimensional contextual bandit problem is very hot, especially for the complex non-linear model assumption. For example, \n[1] Zhou D, Li L, Gu Q. Neural contextual bandits with ucb-based exploration[C]//International Conference on Machine Learning. PMLR, 2020: 11492-11502.\n[2] Chen Y, Xie M, Liu J, et al. Interconnected Neural Linear Contextual Bandits with UCB Exploration[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, Cham, 2022: 169-181.\n[3] Jacot, A., Gabriel, F., Hongler, C.: Neural tangent kernel: convergence and generalization in neural networks. In: NIPS, pp. 8580–8589 (2018)\nIf we concatenated the continuum arm space and contextual vector totally as a large feature vector, we can use these existing methods to solve it. Thus the paper should be compared to them. \n\n(4)\tThe experiment study is a toy. Only one navie method is compared to Hi-CCB with the simulation in a small dataset. Thus it is very hard to support the conclusion. I suggest the authors to justify by some public large data for real problem. In fact, we can compare with baselines in each of special case problems.\n\nMinor problem:\n(1)\tAll figures are blur. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "XbZ-0wQA01w", "reviewer": "Reviewer_t2CV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new model for contextual bandit problems with high dimensional contextual vector space and action space. Making use of singular value decomposition techniques over low-rank representation matrix, the Hi-CCAB algorithm in essence solves a regularized regression problem to find the best empirical parameter estimates till the end of horizon. Theoretical analysis provides guarantee on the Hi-CCAB algorithm, and empirical evaluation on real-time datasets corroborate with the claims.  ", "summary_of_the_review": "My rating of this paper would be a 6, although I'd be happy to adjust my scores if the author address my concerns.  ", "strength_and_weaknesses": "Strength:\n\n1. The paper is well-written, with all the sections coherently organized. The contents are largely easy to follow.\n\n2. The intuition and the generalization value of the Hi-CCAB algorithm are also well-explained.\n\n3. Empirical analysis of the algorithm is consistent with the theoretical claims.\n\nWeaknesses:\n\n1. While the overall picture/intuition of the algorithm, it is unclear how the $T^{-\\frac{1}{13}}$ bound is achieved by the descriptions of the paper alone. Since the validity of the algorithm hinges on effective SVD/PCA algorithm, it would be helpful to specify(at least on a high-level) what is meant by 'carefully leveraging the independent parts of dependent variables' and what method is actually used.\n\n2. The empirical section is a bit lacking. The only benchmark being used is the naive algorithm. While current results do suggest that the Hi-CCAB converges, it is unclear how it compares to other heuristics in bandit literature (e.g. $\\epsilon$-greedy or UCB-based algorithms), and a comparison between well-known algorithms will be helpful too.\n   ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7l4dHWtvAJF", "reviewer": "Reviewer_yX2p", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a low-rank representation matrix to address the high-dimensional continuum-armed and high-dimensional contextual bandit problem motivated by assortment pricing problems. The idea is to use an interleaved two-phases algorithm where the first step is to learn a low-rank representation matrix to capture the low-dimensional structure of both the high-dimensional arms and high-dimensional covariates, and a second step to expand the action space. On the theory side, a \"consistency\" result is given ($\\lim_{T->\\infty} \\frac{R_T}{T^{12/13}} = 0$), and on the practical side, experiments on and instant-noodle assortment pricing dataset are provided.", "summary_of_the_review": "The idea is in general interesting, in particular the policy learning phase. However, I think there could be a set of things to be done to make the paper more complete as I already mentioned in the Strength And Weaknesses section.\n\n___________________________________________\n\nI decide to increase a bit my score after rebuttal since the authors have made an effort to address some of the points made by other reviewers and myself. Those are, however, some major revisions (that are not supposed to be made during the rebuttal phase, otherwise that's like extending the deadline).", "strength_and_weaknesses": "Strength:\n- The proposed model can be reduced to some classic bandit problems (such as stochastic multi-armed bandit, linear bandit, continuum bandit without context, etc).\n- Very original dataset that corresponds to the purpose of the paper.\n- The policy learning phase which expands the action space is quite interesting to deal with the continuum arm space. And the two phases (low-rank representation learning + policy learning) although interleaved, seem to be quite modular.\n\nWeaknesses/questions/remarks:\n- Could you elaborate on the choice of $w^{3/2}$?\n- The theoretical part is not very strong in the sense that we do not really know how to assess Theorem 1. I would suggest the authors to interpret a bit Theorem 1 since it is not very common to look at the 'time-averaged regret', so some interpretation would be appreciated. In particular, could elaborate on the order $1/13$, and how is it related to the dimensions?\n- On the experiment side, as I mentioned I find the proposed two-step algorithm quite modular. That's a good point, but at the same it means that we can also try other kind of contextual bandit algorithms (or even kernel/graph/combinatorial bandit algorithms), at least as some benchmark to compare with. \n- Another point on the experiment - as the authors claim that the proposed model covers also some other more classic bandit models, a small sanity check (both experimentally and theoretically actually) that could've been done is to check the performance of the proposed algorithm under these classic models.\n- Literature review is somehow not very thorough (at least on the bandit side, contextual bandits, continuum bandits, rank-1 bandits etc), but that's a minor point.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject