Scaled Neural Multiplicative Model for Tractable Optimization
Authors: Drishya Giri
Kenneth Kuhn
Aravind Chiruvelli
Venue: Submitted to ICLR 2023 (2023)
Keywords: Input Convex Neural Networks
Shape Constrained Models
Shelf Space Optimization

Abstract: Challenging decision problems in retail and beyond are often solved using the predict-then-optimize paradigm. An initial effort to develop and parameterize a model of an uncertain environment is followed by a separate effort to identify the best possible solution of an optimization problem. Linear models are often used to ensure optimization problems are tractable. Remarkably accurate Deep Neural Network (DNN) models have recently been developed for various prediction tasks. Such models have been shown to scale to large datasets without loss of accuracy and with good computational performance. It can, however, be challenging to formulate tractable optimization problems based on DNN models. In this work we consider the problem of shelf space allocation for retail stores using DNN models. We highlight the trade-off between predictive performance and the tractability of optimization problems. We introduce a Scaled Neural Multiplicative Model (SNMM) with shape constraints for demand learning that leads to a tractable optimization formulation. Although, this work focuses on a specific application, the formulation of the models are general enough such that they can be extended to many real world applications.

Under review as a conference paper at ICLR 2023
SCALED NEURAL MULTIPLICATIVE MODEL FOR TRACTABLE OPTIMIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
Challenging decision problems in retail and beyond are often solved using the predict-hen--opimnize paradigm. An initial effort to develop and parameterize a model of an uncertain environment is followed by a separate effort to identify the best possible solution of an optimization problem. Linear models are often used to ensure optimization problems are tractable. Remarkably accurate Deep Neu- ral Network (DNN) models have recently been developed for various prediction tasks. Such models have been shown to scale to large datasets without loss accuracy and with good computational performance. It can, however, be chal- lenging to formulate tractable optimization problems based on DNN models. In this work we consider the problem of shelf space allocation for retail stores using DNN models. We highlight the trade-off between predictive performance and the tractability of optimization problems. We introduce Scaled Neural Multiplica- tive Model (SNMM) with shape constraints for demand learning that leads to a tractable optimization formulation. Although, this work focuses on specific ap- plication, the formulation of the models are general enough such that they can be extended to many real world applications.
INTRODUCTION
The predict-then-optiiie framework is ubiquitous in applied research. A predictive model is first developed to approximate the true dynamics of a system under consideration to given level of accuracy. A mathematical programming formulation based on the predictive model is then used to help researchers identify optimal policies for challenging real-world decision problems. Despite the fact that predictive models that are estimated based on the historical data need not be causal, there recent works that shed some light on the principles behind this approach (Bertsimas & Kallus 2020).
An alternative to the preopiittee--optimiie framework is integrating the two stages, predicting and optimizing at the same time, by utilizing specific loss function in prediction models (Elmachtoub Grigas 2022). For example, model-free Reinforcement learning (RL) approaches explore their environment while also exploiting it to make optimal decisions.
Big-box retailers have thousands of stores located across the country. They use data-driven decision to optimize operations; e.g, assortment planning, pricing, and supply chain optimization. They work on problems related to shelf space allocation, making the best use of limited space in stores. Space planning for apparel is particularly challenging, due to the different shapes and sizes of the merchandise and the temporal shifts in brand importance.
The problem that motivated our work in this area involves deciding how much space in terms of fixtures to assign to each of several brands within a category of products for sale. These problems typically solved the department store level. Products arranged into departments and categories; for example, 'activewear tops' a category within the women's 's apparel department. In this work, we introduce novel approach based on the thedoitttee--ppimizee framework for solving a shelf space allocation problem.
In partieular contributions as follows

---

Under review as a conference paper at ICLR 2023
We begin by identifying certain key characteristics of relevant real-world data that make modeling challenging.
We discuss predictive model selection considering the tractability of resulting optimization problem formulations and focus the convexity of the formulations.
We propose multiplicative models and establish conditions for tractability of the optimiza- tion. We also discuss why family of linear models are not suitable for the given application.
We hypothesize that it is possible to convert an intractable optimization to a tractable one without loss of prediction accuracy. We achieve this via Scaled Neural Multiplicative Model (SNMM) and demonstrate that our proposed model performs well relative to alter- native models.
RELATED WORK
The relationship between the shelf space allocated to product and that product's sales has been studied extensively (Bianchi-Aguiar et al., 2021 Hübner & Kuhn 2012; Karampatsa et al., 2017). Analyses often focus on incremental returns, estimating the space elasticity. Under the assumption of diminishing returns to scale, the relationship between shelf space and sales can be modeled using concave function (Curhan 1972, Fisend, 2014). The assumption of diminishing returns makes intu- itive sense and has been widely used in production functions (Gopalswamy & Uzsoy 2019)(Aigne) & Chu 1968); the incremental gain in sales by adding more space to showcase a product decreases as the space allocated increases. This assumption can also have side benefit of making shelf space allocation optimization problems easier to solve.
Optimization: There have been number of scientific articles focused on the assortment optimiza- tion problem; a problem related to the one we consider in this work. Retailers solve this problem to determine which products to sell. Shelf space in stores is often the most prominent constraint. Kök & Fisher (200)). (ücel et al. (2009), (2199), and numerous others focus on developing optimiza- tion frameworks that include complicated consumer choice models. This allows the authors to select the optimal product mix accounting for the fact that some consumers will substitute one product for another depending on what is and is not sold by a retailer. Our problem is somewhat different in that we are not selecting specific products to sell, but rather deciding which brands to carry and how much space to allocate these brands. These brands will offer products that are relatively unique within store.
Hübner & Kuhn (20TT) points out the relationships between space allocated, consumer demand, and inventory costs. Assuming fixed amount of space available, a retailer can choose to offer fewer facings of more diverse set of products to increase consumer interest. This will, however, also increase inventory holding and replenishment costs. There will be increasing demands placed on store labor. Ryzin & Mahajan (1999) came to a similar conclusion earlier looking specifically at apparel. Our problem is, again, somewhat different. Brand managers responsible for selecting product mix within brands sold in specific stores, managing inventory costs and the tradeoff between such costs and expected sales profits.
An integral part of our prediction model that makes the optimization tractable shape constraints. These are models that are constrained be monotonic, convex, con- cave or non-negative among others. Shape constraints provide effective regularization, reducing the chance that noisy training data or adversarial examples produce model that does not behave as ex- pected. These impose strong priors on the data and can be used effectively to produce well behaved structured prediction models. It is this property that we enforce in our prediction model yield tractable optimization. Shape constraints on neural networks have been studied in different applica- tions (Gupta al. 2018). Most of the prior works consider GAMs (Generalized Additive Models) and neural networks separately. In our work. we combine the strength of neural networks and the simplicity of GAMs and propose a Scaled Neural Multiplicative model that can model concave, convex monotone constraints effectively.
Chape-Constrainee Models:
Differentiable Optimization: Recent works have introduced classes of deep learning models where certain layers involves solving optimization problems (Donti et al.] 2017: Agrawal et al.) 2019 Wilder et al. (2019). In these differentiable optimization architectures, backpropagation is based on loss function that reflects the decision problem(s). The goal is to directly learn policy that performs
2

---

Under review as a conference paper at ICLR 2023
well the optimization problems under the true distributions of uncertain parameters. Donti al. (017)) demonstrate that this approach can outperform "both traditional modeling and purely black- box policy optimization approaches" sample stochastic optimization problems. [Agawal et al. (2019) introduce a methodology for problems involving certain optimization problems they call disciplined convex programs. [Wilder et al. (2019) introduce methods for linear and submodular maximization combinatorial optimization problems.
3 PRELIMINARIES
Given closed set X and function I's that is parameterized by 0, decision making problem is to find the solution to the optimization problem given below
arg max (o(x)
#EX
The complexity of the underlying optimization model is based on the form of the function Fo. the objective function, and the definition of the set X. the constraints. Further, the problem generally tractable only for concave functions with respect to æ (including affine).
Although, more generic functional forms can be incorporated using Mixed Integer Programming (MIP) with piecewise linear approximations (Vielma et al., (010)) (oopalswamy et al. 2019), the solution time for MIPs in general does not scale well for large problems. By tractability, we refer to the class of optimization problems that can be solved in polynomial time.
linear functional form for the objective function leads a relatively simple convex (linear) opti- mization problem if the set X is convex polytope. In this case, the prediction model must be linear with respect to the decisoptimization variable æ. The c model parameters are coefficients in linear function of while the rest of the features IP', in the prediction model do not have any restrictions. The predictive model can generic neural network, instance. Note that features other than a irrelevant to the decision problem.
Fo(x) = c' + ((P)))
A multiplicative model form can be better alternative to linear model form for two reasons: (i) it can handle heterogeneity in variance w.r. features in X and (ii) other features in IP can play role in determining the optimal solution to equation 7 A general multiplicative model is given by
Fø(x)
II ((P)).)
(2)
3 and (PP).)) E IR+ then objective function is convex. Such conditions be diffi cult to enforce on general neural network model without explicit architectural design (Amos et al.) 2017). Further, such models can be difficult interpret and communicate business stakeholders in general. To alleviate the above problems, we consider the following multiplicative form
Fo(x) = Σ 9 II
(p(w))
PEP(Z:
where wp is the feature that depends on the index i of the optimization variable. For example, variable v; could represent space for brand whereas we could be description of brand i embedded using language model. Details how constraint to enforce convexity (concavity) will be discussed in the next section.
4 PROBLEM FORMULATION
We discuss specific instance of the optimization problem based the predictive model Fg that captures the relationship between sales and features. The variable of interest in the optimization, fixture count, defines the shelf space in store. This work specifically considers the problem of finding optimal space allocation for each brand-category pair for given department in each store to maximize revenue.
3

---

Under review as a conference paper at ICLR 2023
The tractability of the above formulation depends the set X and the parametric function Fg. In this work we will consider family of functions from Generalized linear models (GLM) such as additive, multiplicative well as DNNs that extend GAMs (Agarwal et al. 202I). We will analyze the models from the perspective of flexibility and convexity with respect to the fixture count feature
We approach this problem in two stages-
Learn the functional form of (a) through demand learning.
*
Use this relationship to optimize for space across all stores.
4.1 DATA SPARSITY
A major challenge in pushing research from academia to application rests with the data quality. non trivial achieve the appropriate functional form or relationship that captures the expected model behavior without loss in performance. In the retail space in particular, we have found that the domains of the functions we would like use are not fully observed in the historical data. This is due to the challenges in experimentation in physical retail stores at the granularity needed for making decisions. For example, in apparel there are large number of items sold, while the sales traffic at each item level is very sparse. Ideally, one would be interested in estimating the space elasticity of item for time period t. This estimation problem challenging due to sparsity in data and non-dynamic space allocation in retail stores (i.e, space allocation for an item or category usually stays the same for a quarter to offset high labor costs involved). We consider models that have single coefficient for fixture count. Further, the category and brand have multiplicative effects in the models we consider. This way, we are able to estimate the space elasticity by using data across different products and also estimate total effect for any given brand and category.
4.2 GENERALIZED LINEAR MODELS
A baseline approach involves fitting a linear regression model where y. the sales amount, is linear combination of x and de the features. Here, denotes the fixture count variable and de denotes other features like brand, category and department.
Linear Fixed Effects Model:
y = So + f1x + Edd,
When used objective function with as the decision variable, the linear model has constant space elasticity across all brand-category pairs and the effect of other features other than fixture count becomes irrelevant the optimization.
Log-Log Model: A log-log model ensures that the effect of fixture count on sales is positive. More- over, unlike a simple linear model, the coefficients of log-log model have multiplicative effect on the independent variable. The multiplicative effect of coefficients here ensures that the effect of different features like brand, category and department are directly related to the fixture count vari- able. These variations across brand, category and department learned from the historical data allows the optimization model identify the right weights associated with each fixture count variable and assign space accordingly.
log y = 3o + B, log x + Σ Edd;
y == ا
y == F(
(5)
The log-log model is convex when 31 or 31 0. In motivating example, these would be the cases where allocating additional space to a brand would either reduce sales or increase sales at faster rate than before (increasing returns to scale). Neither of these cases seem realistic.
4.3 DEEP NEURAL NETWORKS
Neural Additive Models (NAM)): Deep neural networks have been successful in prediction tasks. We consider special form of neural networks that additive in nature (Agarwal et al. 2021)
4

---

Under review as a conference paper at ICLR 2023
NAMs learn a linear combination of networks, each of which attend to single input feature: each fi in ijs parameterized by a neural network. NAMs are explainable, elegant and easy to understand models.
3 + (z11 + f2(22) + +... fK(
(6)
Neural Multiplicative Models (NMM): Similar to the multiplicative linear models, we consider multiplicative form for the neural additive model using the log transformation of the dependent variable and fixture count
y e2 e (4.)
Equation 7 in general will not lead to tractable optimization problem. The functional form sidered in Agarwal et al.] (2021) involves EXU layer g(x) e (x - b) and linear layers. We consider linear functional form of f that can be modeled via linear layers. To improve learning
and expressiveness we use hidden dimension of 100, thus forming a total of 2 linear layers.
wo wi t (d))
The parameters of Equation 8 can be constrained to produce a concave function well suited for use as objective function in a subsequent optimization problem. The constraint 0 S w//w S 1, can
be modeled by considering o (ww2), (ww) instead 22, where ((x) = E. E. ;=1
exp=;
It can
be shown that 0 S ((u2)aa(u)) $1. 1.
Lemma 4.1. For any x, y E A where A == fw R Σ, Wy = 1}, we have 0 0 Os Iilk I
Proof. In appendix.
Scaled Neural Multiplicative Models (SNMM): Generally, constraining neural network models leads reduction in their capacity. The direct consequence being the inability be a universal function approximator. We propose a NMM model where the scaling of the feature æ before the log transformation be learned in end-to-end fashion.
c (t0)) ((w))))))))))))))))))))))) (ww)) +b E (dd.)
(9a)
+ max (0, S/ + 1 + 22)
(9b)
Equation 9 9is tractable with respect to the optimization when 8/81 + +12 2 0. Although, max operator cannot modeled without mixed integer program, we noticed that the above condition for tractability holds the experiments application, y represents sales dollars and x represents fixture count. Equation 9 can be written for explicit category i, brand and department k as follows
carried out with different number of stores. In our
Vi.j.k Z
(10a)
Zij.k 8 S/sii + 8/21 + la
(10b)
ai.j. = e ( by tbo fild)) ((4))
(10c)
Note that asjj.k is constant with respect to optimization variables.
4.4 OPTIMIZATION
After learning the functional form of Fg. we can formulate the problem for stage two as follows. Let the sets I and K denote the set of brand, category pairs (i.j) and set of departments, respectively. The parameter 0; J. the coefficient of category i, brand and department based on the multiplicative model in 9 Let i denote the variable fixture count for category i, brand j and department k with 4. and Us,j. as lower and upper bounds on the variable, respectively. Further, lets define parameters and Sę and 7 the total fixture count (across all departments). fixture count for department and space elasticity, respectively.
We define the optimization with explicit index (i.j) indicate the dependency of the coefficients ai.j.k on brand, category and department encoding, even though it can be combined into a single in- dex. The objective is to maximize sales revenue from equation 10 The explicit form of equation TO)
5

---

Under review as a conference paper at ICLR 2023
f2(,5,) depends the model of choice (SNMM in our case).
max
Σ Σ
Vs.j.
(11a)
MEKiJEr
8.1.
Mi.i. (I(JjA)) Vi,
(1lb)
Σ AJEL
Biiik S 8k, Ak E K
(11c)
Σ
Σ Fiijk I
(11d)
kEX
li.j S Lijik I Uigiks Vi,
(Ile)
Jo 2 0, Vi k
(1lf)
EXPERIMENTS
We demonstrate the ability of the proposed model to handle shape constraints and predictive power with two of experiments. In the first experiment, we compare SNMM with different models from (uupta et al. 2018) using real world datasets on their ability to handle constraints such as concavity with respect to the input. The models compared include: "standard unconstrained DNN" (Guptal al., 2018), 8), partial convex shape-constrained neural networks (Amos et al., (011)) Random Tiny Lattices (RTLs) (Canini et al. 2016), and calibrated linear models (Gupta et al. 2016). See Gupta al. (2018) for more details on these models. Our results regarding their performance are taken from this work. the second experiment, we provide results on a proprietary dataset from a retailer with respect to modeling space elasticity.
5.1 SHAPE-CONSTRAINED BENCHMARK
We consider three publically available datasets and test the ability of SNMM to handle convexity and concavity constraints based established benchmarks.
Table I: Synthetic experiments: Car Sales and Puzzle Review
Car Sales
Puzzle Sales
Model
Val. MSE
Test MSE
Model
Val. MSE
Test MSE
SNMM
1923
8486
SNMM
3768
8457
DNN
2035
10931
DNN
2189
5652
SCNN conv.
2262
10613
SCNN conc.
2632
7931
SCNN conv, decr.
2442
10590
SCNN conc. incr.
2437
6927
Cal Lin. decr.
2271
10727
RTL iner.
4457
8838
Cal Lin. conv. decr.
2304
10593
RTL all
3543
8315
Cal Lin. incr.
3589
8270
Cal Lin. all
3617
8189
5.1.1 CAR SALES
This dataset is I-d problem with 109 training, 14 validation, and 32 test examples (www. kaggle.
a)) for which we predict monthly car sales (in thousands) from the price (in thousands). We enforce convexity constraint on the price variable with respect to the sales variable. The results can be seen Table [ SNMM performed relatively better compared all models with the lowest Validation and Test MSE
"tsinha53/car-sall

---

Under review as a conference paper at ICLR 2023
5.1.2 PUZZLE SALES FROM REVIEWS
The data consists of 3 features, 156 training, 169 validation, and 200 non-III test examples (courtesy of Artifact Puzzles and available at www. kaggle com/ foon/dbariii mtippuuzzlss for which pre- dict the 6-month sales of different wooden jigsaw puzzles from three features based on its Amazon reviews: its average star rating, the number of reviews, and the average word count of its reviews. Here we assume a convexity constraint on the star rating, and concavity constraints on number of reviews and word count. Table I shows that for SNMM, even though the Validation MSE is highest compared to the rest of the models, the Test MSE remains within the range of the highest Test MSE. The results aren't 't impressive ing convexity and concavity are more limiting here. is important to note that these assumptions still necessary develop a tractable optimization problem formulation.
the Car Sales problem, indicating that assumptions regard-
Table 2: Synthetic experiments: Wine Quality
Wine Quality
Model
Val. MSE
Test MSE
SNMM
6.95
6.98
DNN
4.91
4.79
SCNN conc.
5.96
7.22
SCNN conc. incr.
6.13
6.21
RTL iner.
4.96
4.85
RTL conc. incr.
4.96
4,83
Cal Lin. incr.
5.25
5.10
Cal Lin. conc. incr.
5.23
5.10
5.1.3 WINE ENTHUSIAST MAGAZINE REVIEWS
The data (61 features, 84,642 training, 12,092 validation, and 24,185 test examples; www. www. kaggle com/ dwwhri/a////ea teahhr/wwwneerrainngne ragings) consists of wine's quality measured in points [80, 100], price of the wine (the most important feature), country (21 Bools), and 39 Bool features based on the wine description from Wine Enthusiast Magazine. We constrain the price feature to be concave in order predict the quality of wine. From Table [ it can be seen that while DNN has the lowest error, the Test MSE for SNMM within the range of the highest Test MSE across all models.
Table 3: Benchmark results for 5 stores
log scale MSE
linear scale
Model
R²
MAE
R²
MSE
MAE
GLM-L
0,623 + 0.01
61692 +: 2214
126 - 0.67
GLM-M
0.759 + 0.00
0.531 + 0.00
0.568 + 0.00
0.619 + 0.01
62398 + 1323
96 - 0.64
GLM-CM
0.736 0.00
0.582 + 0.00
0.603 + 0.02
0.618 + 0.01
62525
2202
97 0.87
NAM
0.684 0.01
2033
100 1.90
NMM
0.769 0.01
0,506 + 0.01
0.564 0.01
0.567 0.01
69923 1023316677
95 = 1,46
NMM-L
0.824 +
0.388 0.05
0.483 + 0.02
0.738 + 0.06
43917 + 1099
80 + 5.44
NMM-CL
0,638 1: 0.01
0.798
0.724
0.407 + 0.01
96364 2771
120 + 1.61
SNMM
0.827 1 0.01
0.382 0.02
0.476 1 0.01
0.775 1 0.02
37830
75 2.32
5.2
CASE STUDY ON REAL DATA
In this section, we perform tests on random selection of stores across the country from one retailer. We first describe the dataset and the features considered. We will then compare eight different

---

Under review as a conference paper at ICLR 2023
models for predicting sales with the metrics given below.
R = 1 y
(12)
y y 2
MSE N lyy y |
(13)
MAE N- ly 3
(14)
We consider linear and multiplicative models with variants. The variants explored in lieu of search for computationally tractable optimization model. Further, certain models offer better mod- eling capacity with respect the solution space. 'he functional relationship between the space feature (variable in optimization) and sales defines the objective function. We expect the models to capture the concave relationship from the data, but seldom happens in reality. Real-word data is noisy and unobserved confounders can affect the data. Further, retail data in general depends on exogenous uncertainty such as market dynamics, competitors, economic factors, etc. Explicit shape constraint is placed on the model such non-negativity of sales and concavity of sales with respect to space feature.
Table 4: Benchmark results for 10 stores
log scale MSE
linear scale
Model
R
MAE
R
MSE
MAE
GLM-L
0.638 + 0.00
56655
1378
121 = 0.43
GLM-M
0.753 1: 0.00
0.522
0.00
0.564 1 0.00
0.657 + 0.01
53653
1465
90 - 0.39
GLM-CM
0.727
0.00
0.577 + 0.00
0.601 + 0.00
0,616 + 0.01
60084 + 1404
92 0.46
NAM
0.668 = 0.00
+
98 1.09
NMM
0.786 + 0.01
0.452 + 0.01
0.533 0.01
0.622 0.01
86 1.52
NMM-L
0.813 + 0.01
0.396 + 0.02
0.488
0.707 + 0.01 0.01
46438
2315
2.20
NMM-CL
0.648 + 0.02
0.746
0.03
0.478 + 0.04
82035
7789
109 = 5.24
SNMM
0,817 0.01
0.387 0.00
0.479 + 0.00
0.760 + 0.01
37143 1996
72 0.51
As discussed in sections 4.2 and 4.3 we enforce the constraints and report performance metrics.
We present the metrics in both log scale and linear scale whenever applicable to offer complete comparison across all the models
The following models used for the experiment:
GLM-L: Classical linear regression model that assumes linear relationship between the dependent and independent variables, Equation 4
GLM-M: Multiplicative model that takes log transformation of dependent variable and space variable as in Equation 5
GLM-CM: Constrained GLM-M with space elasticity between and 1.
NAM: Neural Additive Model as described in Equation 6
NMM: NAM that takes log transformation of dependent variable and space variable as in Equation 7
NMM-L: NMM with linear layers for the space variable in Equation g
NMM-CL: Constrained NMM-L with space elasticity between and using lemma 4.1
SNMM: Proposed model in Equation 9
Dataset: The models consider historical space and sales data of Apparel departments from 2018 to 2022. We want to predict and optimize for space at the departmenntccteeoryy brand level, aggregate data to that level at a weekly granularity.
Model Evaluation: Numerical tests
carried out set of 10 and 20 stores. The ef- fectiveness of the models in sales prediction is evaluated using the metrics described above. The proposed model is compared with variety of different models and the results shown in Tables
8

---

Under review as a conference paper at ICLR 2023
Table 5: Data summary
Feature
Description
$ sales
Dependent variable representing the dollar sales amount
fixture count
Measure of space represented in store
store
Store number (categorical)
store area
Area of store in sq. ft.
department no
Department number
category
Category description (vector embedding)
brand
Name of the brand (vector embedding)
demographic Income data based on location of the store
Demographic data based on location of the store
income
time features
Time-based features based on the week of transaction
Table 6: Benchmark results for 20 stores
log scale MSE
linear scale
Model
R²
MAE
Ri2
MSE
MAE
GLM-L
0.583 + 0.00
71124 + 1943
126 + 0.29
GLM-M
0.742 + 0.00
0.531 + 0.00
0.568 + 0.00
0.626 0.01
63831 + 3224
91 0.56
GLM-CM
0.715 + 0.00
0.587 + 0.00
0.605
0.00
0.575
72437 = 1849
9
NAM
0.00
65838 1475
102
NMM
0.793 0.01
0.428 + 0.01
0.516
0.01
0.01
72292 841
8
NMM-L
0.807 0.01
0.400 + 0.01
0.487
0.01
0.01
0.01
55934 2198
79+
NMM-CL
0.623 0.01
0.78 + 0.03
0.713
0.02
0.352 = 0.02
113907 + 4103
113 1: 2.49
SNMM
0,806 + 0.02
0.399 + 0.03
0.486 0.02
0.716 + 0.01
48983 + 2228
74 + 2.56
and 6 The results are reported on the spst.. Models are tuned for hyper-parameters batch-size {32, 64, 128, 256, 512}, hidden-size {10, 50, 100).
SNMM model performs the best across all the scenarios; MSE and MAE for SNMM are the lowest across all the models. Comparable performance can be seen in NMM-L model where we enforce linearity of the space feature in log space. We highlight the benefit of enforcing structured straints by comparison to several other models that are unconstrained. Although constrained models usually have lower capacity, SNMM is able to recover the performance lost due constraints when compared to the constrained models and even improve on them. This clearly shows that the gener- alization ability of the shape constrained model is due regularization with respect the structure.
CONCLUSION AND FUTURE WORK
In this work, we presented a shelf space allocation problem for retail use case. A general framework of predict and optimize discussed in detail with regards to trade-off between tractability of the optimization and prediction model accuracy. We discuss the need for multiplicative models and the difficulty estimating space elasticity on sparse dataset (most often this is the case in real world problems). To that extent, we propose a Scaled Neural Multiplicative Model (SNMM) that satisfies the conditions: non-negativity of sales and concavity with respect to the space feature. The optimization problem is formulated Specifically, we use the power cone formulation provided by evxpy. The model proposed this work is general enough extend to many other applications such advertisement optimization, revenue maximization, etc. In future, we plan to explore these areas and enhance the model to handle different sets of constraints jointly.
convex problem which can solved using convex solvers.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey Hinton. Neural additive models: Interpretable machine learning with neural nets. Advances in Neural Information Processing Systems, 34:4699-4711, 2021.
Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J. Zico Kolter. Differentiable convex optimization layers. Advances in Neural Information Processing Systems, 32, 2019.
Dennis J Aigner and Shih-fan Chu. On estimating the industry production function. The American Economic Review, 58(4):826-33, 1968.
Brandon Amos, Lei Xu, and Zico Kolter. Input convex neural networks. International Confer- ence on Machine Learning, pp. 146-155. PMLR, 2017.
Dimitris Bertsimas and Nathan Kallus. From predictive to prescriptive analytics. Science, 66(3):1025-1044, 2020.
Management
Teresa Bianchi- Aguiar, Alexander Hübner, Maria Antónia Carravilla, and José Fernando Oliveira. Retail shelf space planning problems: A comprehensive review and classification framework. European Journal of Operational Research, 289(1):1-16, 2021.
K. Canini, A. Cotter, M. M. Fard, M. R. Gupta, and J. Pfeifer. Fast and flexible monotonic functions with ensembles of lattices. Advances in neural information processing systems, 2016.
Ronald C Curhan. The relationship between shelf space and unit sales in supermarkets. Journal of Marketing Research, 9(4):406-41) 412, 1972.
Priya Donti, Brandon Amos, and J. Zico Kolter. Task-based end-to-end model learning in stochastic optimization. Advances Neural Information Processing Cystems, 30, 2017.
Martin Fisend. Shelf space elasticity: A meta-analysis. Journal of Retailing, 90(2):168-181, 2014. 1.
Adam N Elmachtoub and Paul Grigas. Smart "predict, then optimize". Management Science, 68(1): I
9-26, 2022.
Karthick Gopalswamy and Reha Uzsoy. A data-driven iterative refinement approach for estimat- ing clearing functions from simulation models of production systems. International Journal of Production Research, 57(19):6013 (/11)):01136000000 2019.
Karthick Gopalswamy, Yahya Fathi, and Reha Uzsoy. Valid incqualities for concave piecewise linear regression. Operations Research Letters, 47(1):52-58, 2019.
Maya Gupta, Andrew Cotter, Jan Pfeifer, Konstantin Voevodski, Kevin Canini, Alexander Mangylov, Wojeiech Moczydlowski, and Alexander Van Esbroeck. Monotonic calibrated in- terpolated look-up tables. The Journal of Machine Learning Research, 17, 2016.
Maya Gupta, Dara Bahri, Andrew Cotter, and Kevin Canini. Diminishing returns shape constraints for interpretability and regularization. Advances in neural information processing systems, 31, 2018.
Alexander H Hübner and Heinrich Kuhn. Shelf and inventory management with space-elastic de-
mand. In Operations research proceedings 2010, pp. 405-410. Springer, 2011.
Alexander H Hübner and Heinrich Kuhn. Retail category management: State-of-the-a review of quantitative research and software applications in assortment and shelf space management. Omega, 40(2):199-209, 2012.
Marina Karampatsa, Evangelos Grigoroudis, and Nikolaos Matsatsinis, Retail category man- agement: A review on assortment and shelf-space planning models. Operational Research in Business and Economics, pp. 67, 2017.
A Gürhan Kök and Marshall L Fisher. Demand estimation and assortment optimization under sub- stitution: Methodology and application. Operations Research, 1506)1001 2007.
10

---

Under review as a conference paper at ICLR 2023
Venus Hiu Ling Lo. Capturing Product Complementarity in Assortment Optimization. PhD thesis, Corelll University, 2019.
Garrett van Ryzin and Siddharth Mahajan. On the relationship between inventory costs and variety benefits in retail assortments. Management Science, 15(11):1496-1500 1999.
Juan Pablo Vielma, Shabbir Ahmed, and George Nemhauser. Mixed-integer models for nonsepara- ble piecewise-linear optimization: Unifying framework and extensions. Operations research, 58 (2):0033115, 2010.
Bryan Wilder, Bistra Dilkina, and Milind Tambe. Melding the data-decisions pipeline: Decision- focused learning for combinatorial optimization. Proceedings of the AAAI Conference Artifi- cial Intelligence, 33, 2019.
Eda Yücel, Fikri Karaesmen, F Sibel Salman, and Metin Türkay, Optimizing product assortment under customer-driven demand substitution. European Journal of Operational Research, 199(3): 759-768, 2009.
A APPENDIX
Proof. Lemma 4.1
Σ rilli
1
Σ
2 2
Σ 3/
By Cauchy Shwartz inequality
Σ
1/1
V. 1,
x,
S x., vi S y
Σ Jcli 1
Σ
Σ
yi
Trivially, 2.34 since T₁ y4 0 Vi


Reviews:
[{"review_id": "dUwHrU35GGN", "reviewer": "Reviewer_fHjV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a neural network architecture, called Scaled Neural Multiplicative Model (SNMM). This architecture was designed with two main objectives in mind: first, the model can be fitted properly to reflect actual sales data, even though this data tends to be scarce and sparse. Second, the model is concave, which ensures it can be used by an off-the-shelf optimizer such as cvxpy to find the value of a subset of the input features that maximizes the outcome. Based on this model, the paper defines the problem of finding a shelf space assignment that maximizes sales. The paper evaluates the accuracy of SNMM on 3 public datasets, car sales, puzzle sales, and wine quality, as well as a private datasets of sales for an unspecified retailer.", "summary_of_the_review": "The contribution of this paper is fairly small: it modifies the neural multiplicative model to ensure its concavity by using the sigmoid trick. Furthermore, the evaluation of the new architecture is limited, and does not convincingly demonstrate that the model works well in practice. ", "strength_and_weaknesses": "Strengths:\n* The paper proposes a model that can be used directly by a convex optimizer, such as cvxpy.\n\nWeaknesses:\n* It's not clear that the model actually performs well. On the puzzle dataset, its accuracy was second to last, and on the wine quality dataset the model was the least accurate. That said, the model does well on the car sales dataset and the private dataset.\n* Since the paper proposes an end-to-end solution, it would be useful to validate how well the model works with the optimizer to find an optimal assignment of shelf space.\n* Other approaches have been proposed to solve similar problems under the umbrella of differentiable optimization. The related work section should mention a few of them, and it would be useful to add one of these to the baselines.\n* Two of the datasets used in the evaluation are unrealistically small, with less than 200 datapoints. One of the dataset is not public, which will prevent other researchers from replicating the results and comparing new approaches against SNMM. Is there really nothing better available ?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "CU5A_XgufQ5", "reviewer": "Reviewer_1jyf", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers a real-world problem of ​​shelf space allocation for retail stores using DNN models. It focuses on the selection of predictive model families considering the particular convexity constraint for the specific problem formulation. It then proposes a Scaled Neural Multiplicative Model (SNMM) which aims to improve model expressivity under the constraints and is empirically demonstrated on both synthetic and real-world datasets.", "summary_of_the_review": "In summary, the paper has some crucial flaws: 1) lack of novelty and significance;  2) unclear presentation and lack of soundness 3) reproducibility concerns and potential violation of academic integrity. Thus I recommend a ‘strong reject’ for the paper. \n", "strength_and_weaknesses": "**Strength**\n* The paper considers an optimization problem that is practically relevant. The collected real-world dataset could be a contribution to the community if open-sourced.\n\n\n**Weakness**\n* ***The novelty and the significance of the paper seem quite limited***. In particular, the paper targets a very specific optimization problem (i.e., shelf space allocation) and the proposed SNMM method seems to be a very straightforward generalization of the NMM method (by adding a scaler parameter) which is not convincingly justified with empirical evidence. It is crucial to discuss how the proposed method generalizes beyond the specific problem formulation and how significant it is. See “Novelty & Significance” in the next section for details. \n* ***The presentation of the paper is not very clear or easy to follow***. In particular, the specific optimization problem considered, the key issue that the proposed SNMM aims to address and the significance of SNMM need to be more clearly presented. There are also some unjustified claims and missing mathematical definitions that need to be clarified. See “Clarity & Quality” in the next section for details. \n* ***The experimental evaluation is not reproducible or convincing***. There are a lot of missing experimental details (even missing introduction of baselines) in the paper and no further experimental details are included for reproducibility. See “Reproducibility” in the next section for details.\n* ***Potential risk of plagiarism***. In Sec 5.1.1-5.1.3, the starting sentences for introducing the datasets are all **copied** from [1], which may be considered a violation of academic integrity.\n\n[1] Diminishing Returns Shape Constraints for Interpretability and Regularization.  Gupta et al. NeurIPS 2018.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "6I8IcPv2D6d", "reviewer": "Reviewer_AqEP", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the problem of optimizing shelf space in retail stores (i.e., how many fixtures to assign to each of several brands within a category of products for sale) using DNNs. Finding a tractable optimization formulation for such a problem is non-trivial, but the authors propose the scaled neural multiplicative model (SNMM) with shape constraints that leads to a tractable optimization problem. The authors evaluate this formulation empirically across a variety of synthetic/smaller datasets, then perform a case study on real-world data. Although results for SNMM are inconsistent on smaller/synthetic datasets, it seems to provide a performance benefit on real-world data relative to several baseline techniques.\n", "summary_of_the_review": "To begin, I want to emphasize that this review reflects my initial impression of the work, and that I am completely open to further discussion with other reviewers/authors. My final score will mostly be based upon the subsequent discussion.\n\nCurrently, my main concern with the work are:\n* Benefits are only observed on real-world datasets, while on the smaller/synthetic data the proposed method is actually quite a bit worse.\n* No substantial discussion is provided regarding why the method performs so poorly compared to baselines on some datasets.\n* Baselines on real-world data (where we see the benefit) are different from the smaller/synthetic experiments. The methods that outperform SNMM actually are not considered in the real-world case.\n\nI think the work could be greatly improved by performing further analysis on the cases where SNMM performs poorly and providing a better understanding of why the real-world case is so different (plus explaining the problem with the selected baselines). Overall, the results on the real-world dataset are quite promising, and I believe this paper can be valuable/useful given further discussion/modifications. ", "strength_and_weaknesses": "Strength:\n* The paper studies an interesting a relevant applied problem. The problem is well explained and familiar to any reader, which (in my opinion) makes the paper interesting/fun to read. The proposed method is explained well and is novel to the best of my knowledge.\n* The authors study a novel combination of neural networks and GAMs for effectively imposing shape constrained that are needed to model retail space allocation. \n* The proposed technique performs well on the real-world datasets consistently. \n\nWeakness:\n* SNMM does not provide a consistent performance benefit, though it seems to improve performance in some cases. The validation MSE on the puzzle sales dataset is especially concerning, as it seems to be the worst of the considered methods. A similar result is observed on the wine dataset. It seems like a deeper discussion of why SNMM performs so poorly is needed here, instead of claiming that the error is within the range of other methods. \n* Most of the baselines that outperform SNMM on the smaller datasets (e.g., DNN, RTL, SCNN) are not actually considered in the real-world experiments where SNMM performs the best. \n\nSmall Comments:\n* In related work, you use the acronym GAM without stating what this means. \n* The discussion on data sparsity and the difficulty of modeling/recording retail behavior is useful and provides good context for solving the proposed problem. \n\nQuestions:\n* What are a few examples of real world applications the authors think this approach could be extended to? It would be nice to maybe give a few examples in the text to give readers an idea. \n* Is (3) seen as a formulation that is easier to explain/interpret in comparison to (2)?\n* Do the changes in (9a)/(9b) solve the problem of reduced capacity mentioned at the beginning of the section\n* Why are different baselines explored on real data (Sec 5.2) vs. synthetic/small problems (Sec 5.1)? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject