Original: Hierarchical Lattice Layer for Partially Monotone Neural Networks
Revised: Hierarchical Lattice Layer for Partially Monotone Neural Networks

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Partially monotone regression is a regression analysis in which the target valuesare monotonically increasing with respect to a subset of input features.
Revised: Partially monotone regression is a regression analysis in which the target valuesare monotonically increasing with respect to a subset of input features.

Original: The
Revised: The

Original: TensorFlow Lattice library is one of the standard machine learning libraries forpartially monotone regression.
Revised: TensorFlow Lattice library is one of the standard machine learning libraries forpartially monotone regression.

Original: It consists of several neural network layers, and itscore component is the lattice layer.
Revised: It consists of several neural network layers, and itscore component is the lattice layer.

Original: One of the problems of the lattice layer is itsrequirement for a special training algorithm to satisfy monotonicity constraints.
Revised: One of the problems of the lattice layer is itsrequirement for a special training algorithm to satisfy monotonicity constraints.

Original: Another problem is that it cannot receive a high-dimensional input vector due tothe resultant memory consumption.
Revised: Another problem is that it cannot receive a high-dimensional input vector due tothe resultant memory consumption.

Original: We propose a novel neural network layer,the hierarchical lattice layer (HLL), as an extension of the lattice layer so thatwe can use a standard neural network algorithm to train HLL while satisfyingmonotonicity constraints and so that it can receive a high-dimensional input vector.
Revised: We propose a novel neural network layer,the hierarchical lattice layer (HLL), as an extension of the lattice layer so thatwe can use a standard neural network algorithm to train HLL while satisfyingmonotonicity constraints and so that it can receive a high-dimensional input vector.

Original: Our experiments demonstrate that HLL did not sacriﬁce its prediction performanceon real datasets compared with the lattice layer
Revised: Our experiments demonstrate that HLL did not sacriﬁce its prediction performanceon real datasets compared with the lattice layer

Original:  Introduction
Revised:  Introduction

Original: In regression analysis, we often have prior knowledge that the target values are monotonicallyincreasing (or decreasing) with respect to a subset of input features.
Revised: In regression analysis, we often have prior knowledge that the target values are monotonicallyincreasing (or decreasing) with respect to a subset of input features.

Original: For example, when we estimatehouse prices from house data (e.g., the area size, the number of rooms, and the distance to the nearestsupermarket), we know that the target value (i.e., the house price) is monotonically increasing withrespect to some of the input features (e.g., the area size) if the other features are equal Potharst and
Revised: For example, when we estimatehouse prices from house data (e.g., the area size, the number of rooms, and the distance to the nearestsupermarket), we know that the target value (i.e., the house price) is monotonically increasing withrespect to some of the input features (e.g., the area size) if the other features are equal [Potharst and

Original: Feelders [2002].
Revised: Feelders, 2002].

Original: Partially monotone regression is a regression analysis that exploits such prior knowledge.
Revised: Partially monotone regression is a regression analysis that exploits such prior knowledge.

Original: The inputof this regression is a feature vector x ∈ R d , which can be partitioned into x = ( p, q )
Revised: The inputof this regression is a feature vector x ∈ R d , which can be partitioned into x = ( p, q )

Original: such thatp ∈ R d − m is non-monotone features and q ∈ R m is monotone features.
Revised: such thatp ∈ R d − m is non-monotone features and q ∈ R m is monotone features.

Original: The goal of this regressionis to ﬁnd a function f such that f ( x ) =
Revised: The goal of this regressionis to ﬁnd a function f such that f ( x ) =

Original: f ( p, q ) ∈ R is monotonically increasing on q .
Revised: f ( p, q ) ∈ R is monotonically increasing on q .

Original: Partiallymonotone regression models have advantages over the standard regression models.
Revised: Partiallymonotone regression models have advantages over the standard regression models.

Original: For example, theyhave a better regularization capability Dugas et al.
Revised: For example, theyhave a better regularization capability Dugas et al.

Original: [2000]; Fard et al.
Revised: [2000]; Fard et al.

Original: [2016]; You et al.
Revised: [2016]; You et al.

Original: [2017] andbetter interpretability Gupta et al.
Revised: [2017] andbetter interpretability Gupta et al.

Original: [2016], and they can be used for fair machine learning Wang and
Revised: [2016], and they can be used for fair machine learning Wang and

Original: Gupta [2020].
Revised: Gupta [2020].

Original: Many machine learning models have been developed for partially monotone regression.
Revised: Many machine learning models have been developed for partially monotone regression.

Original: For example,there are decision-tree based models Potharst and Feelders [2002], SVM models Lauer and Bloch[2008], boosting models Bartley et al.
Revised: For example,there are decision-tree based models Potharst and Feelders [2002], SVM models Lauer and Bloch[2008], boosting models Bartley et al.

Original: [2018], and random forest models Bartley et al.
Revised: [2018], and random forest models Bartley et al.

Original: Manypopular open source libraries for gradient boosting can handle partial monotonicity (e.g., CatBoost,
Revised: Manypopular open source libraries for gradient boosting can handle partial monotonicity (e.g., CatBoost,

Original: LightGBM, and XGBoost).
Revised: LightGBM, and XGBoost).

Original: See a survey paper Cano et al.
Revised: See a survey paper Cano et al.

Original: [2019] and the references in Gupta et al.[2016] for comprehensive surveys.
Revised: [2019] and the references in Gupta et al.[2016] for comprehensive surveys.

Original: As for neural network models, the TensorFlow Lattice library Canini et al.
Revised: As for neural network models, the TensorFlow Lattice library Canini et al.

Original: [2016]; Gupta et al.
Revised: [2016]; Gupta et al.

Original: You et al.
Revised: You et al.

Original: [2017] is a standard model for partially monotone regression, and it is publicly available as an open source library.
Revised: [2017] is a standard model for partially monotone regression, and it is publicly available as an open source library.

Original: It consists of several neural network layers, and its core component isthe lattice layer Gupta et al.
Revised: It consists of several neural network layers, and its core component isthe lattice layer Gupta et al.

Original: Hereafter, we call this layer TL Lattice.
Revised: Hereafter, we call this layer TL Lattice.

Original: TL Lattice constructsa hypercube with k d vertices as an internal structure, where k ≥ 2 is an integer hyperparameterfor specifying the granularity of the hypercube, and d is the dimension of the input vector x ∈ R d .
Revised: TL Lattice constructsa hypercube with k d vertices as an internal structure, where k ≥ 2 is an integer hyperparameterfor specifying the granularity of the hypercube, and d is the dimension of the input vector x ∈ R d .

Original: One of its problems is its memory consumption in storing k d parameters, and therefore, it cannotreceive a high-dimensional input vector with a large d .
Revised: One of its problems is its memory consumption in storing k d parameters, and therefore, it cannotreceive a high-dimensional input vector with a large d .

Original: This means that we need to combine multipleneural network layers to handle an input vector with a large d (e.g., by using small lattices Caniniet al.
Revised: This means that we need to combine multipleneural network layers to handle an input vector with a large d (e.g., by using small lattices Caniniet al.

Original: [2016]; You et al.
Revised: [2016]; You et al.

Original: Another problem is its requirement for a special algorithm to trainit to handle monotonicity constraints, which means that we cannot use a standard neural networkalgorithm to train it.
Revised: Another problem is its requirement for a special algorithm to trainit to handle monotonicity constraints, which means that we cannot use a standard neural networkalgorithm to train it.

Original: The details of the special training algorithms used for the TensorFlow Latticelibrary are described in Gupta et al.
Revised: The details of the special training algorithms used for the TensorFlow Latticelibrary are described in Gupta et al.

Original: [2016]; You et al.
Revised: [2016]; You et al.

Original: In this paper, we propose a novel neural network layer, the hierarchical lattice layer (HLL), asan extension of TL Lattice.
Revised: In this paper, we propose a novel neural network layer, the hierarchical lattice layer (HLL), asan extension of TL Lattice.

Original: The internal structure of HLL is similar to that of TL Lattice inthat both layers construct hypercube structures.
Revised: The internal structure of HLL is similar to that of TL Lattice inthat both layers construct hypercube structures.

Original: However, they are different in several aspects.
Revised: However, they are different in several aspects.

Original: For example, HLL constructs an m -dimensional hypercube, whereas TL Lattice constructs a ddimensional hypercube, where d and m are the dimensions of the input vector x ∈ R d and itsmonotone feature q ∈ R m , respectively.
Revised: For example, HLL constructs an m -dimensional hypercube, whereas TL Lattice constructs a ddimensional hypercube, where d and m are the dimensions of the input vector x ∈ R d and itsmonotone feature q ∈ R m , respectively.

Original: Since m is usually a small integer in practice, the numberof vertices k m in HLL is signiﬁcantly smaller than that of TL Lattice (i.e., k d ).
Revised: Since m is usually a small integer in practice, the numberof vertices k m in HLL is signiﬁcantly smaller than that of TL Lattice (i.e., k d ).

Original: Therefore, HLLcan receive a high-dimensional input vector as long as m is small.
Revised: Therefore, HLLcan receive a high-dimensional input vector as long as m is small.

Original: For another example, theparameterization of HLL is designed to represent a class of partially monotone functions (i.e., HLLcannot represent a function f ( p, q ) that is not monotonic on q ) by using the hierarchical relationshipbetween the vertices of the hypercube.
Revised: For another example, theparameterization of HLL is designed to represent a class of partially monotone functions (i.e., HLLcannot represent a function f ( p, q ) that is not monotonic on q ) by using the hierarchical relationshipbetween the vertices of the hypercube.

Original: Therefore, the internal structure of HLL ensures partialmonotonicity regardless of the training algorithm used.
Revised: Therefore, the internal structure of HLL ensures partialmonotonicity regardless of the training algorithm used.

Original: In contrast, the internal structure of TL
Revised: In contrast, the internal structure of TL

Original: Lattice is a lookup table, and the table itself can represent any function f
Revised: Lattice is a lookup table, and the table itself can represent any function f

Original: ( p, q ) , which means that theparameterization of TL Lattice is not designed to satisfy monotonicity constraints.
Revised: ( p, q ) , which means that theparameterization of TL Lattice is not designed to satisfy monotonicity constraints.

Original: Therefore, TL
Revised: Therefore, TL

Original: Lattice requires a special algorithm to train its parameters to ensure these constraints.
Revised: Lattice requires a special algorithm to train its parameters to ensure these constraints.

Original: In our experiments, we demonstrate that HLL did not sacriﬁce its prediction performance comparedwith TL Lattice.
Revised: In our experiments, we demonstrate that HLL did not sacriﬁce its prediction performance comparedwith TL Lattice.

Original: Note that HLL is designed to satisfy monotonicity constraints and is not designedto improve the prediction performance compared with TL Lattice.
Revised: Note that HLL is designed to satisfy monotonicity constraints and is not designedto improve the prediction performance compared with TL Lattice.

Original: Therefore, the purpose of ourexperiments was not to show improvements in prediction performance compared with TL Lattice butto show that HLL did not sacriﬁce the prediction performance to satisfy the constraints.
Revised: Therefore, the purpose of ourexperiments was not to show improvements in prediction performance compared with TL Lattice butto show that HLL did not sacriﬁce the prediction performance to satisfy the constraints.

Original: In addition,we also demonstrate that we could use HLL for high-dimensional input vectors with a small m .
Revised: In addition,we also demonstrate that we could use HLL for high-dimensional input vectors with a small m .

Original:  Partially Monotone Regression
Revised:  Partially Monotone Regression

Original: Regression analysis is a method for ﬁnding a function f : R d → R such that f ( x i ) approximatesy i for a given dataset D
Revised: Regression analysis is a method for ﬁnding a function f : R d → R such that f ( x i ) approximatesy i for a given dataset D

Original: = { ( x i , y i ) } ni
Revised: = { ( x i , y i ) } ni

Original: =1 , where x i ∈ R
Revised: =1 , where x i ∈ R

Original: d is a feature vector, and y i ∈ R is a targetvalue.
Revised: d is a feature vector, and y i ∈ R is a targetvalue.

Original: In partially monotone regression, we assume that we can partition a feature vector x ∈ R d intox =
Revised: In partially monotone regression, we assume that we can partition a feature vector x ∈ R d intox =

Original: ( p, q ) ∈ R
Revised: ( p, q ) ∈ R

Original: d − m × R m , where 

Revised: d − m × R m , where

Original: < m < d , such that f ( x ) ∈ R is monotonic on q .
Revised: < m < d , such that f ( x ) ∈ R is monotonic on q .

Original: A functionf ( x ) for x = ( p, q ) is monotonic on q if this inequality holds:
Revised: A functionf ( x ) for x = ( p, q ) is monotonic on q if this inequality holds:

Original: [Equation]
Revised: [Equation]

Original: where q ≤ q (cid:48) denotes the inequality for all the elements (i.e., q [ i ] ≤ q (cid:48)
Revised: where q ≤ q (cid:48) denotes the inequality for all the elements (i.e., q [ i ] ≤ q (cid:48)

Original: [ i ] for all 1 ≤ i ≤ m , whereq [ i ] denotes the i -th element of q ).
Revised: [ i ] for all 1 ≤ i ≤ m , whereq [ i ] denotes the i -th element of q ).

Original: Partially monotone regression has many applications in various ﬁelds, and it has important applicationseven if m = 1 .
Revised: Partially monotone regression has many applications in various ﬁelds, and it has important applicationseven if m = 1 .

Original: An example of such application with m = 1 is quantile regression Koenker and
Revised: An example of such application with m = 1 is quantile regression Koenker and

Original: Bassett [1978]; Zhou et al.
Revised: Bassett [1978]; Zhou et al.

Original: [2020], which is used to estimate the conditional quantiles of a targetvariable distribution.
Revised: [2020], which is used to estimate the conditional quantiles of a targetvariable distribution.

Original: We show that the output q ( x ) =
Revised: We show that the output q ( x ) =

Original: q ( x (cid:48) , τ ) of the quantile regression for an inputx =
Revised: q ( x (cid:48) , τ ) of the quantile regression for an inputx =

Original: ( x (cid:48) , τ ) is monotonic on τ , where x (cid:48) ∈
Revised: ( x (cid:48) , τ ) is monotonic on τ , where x (cid:48) ∈

Original: R d − 1 is a feature vector, and τ ∈ [0 , 1] is a quantile level.
Revised: R d − 1 is a feature vector, and τ ∈ [0 , 1] is a quantile level.

Original: In quantile regression, we estimate the quantile value q ( x (cid:48) , τ ) deﬁned byq ( x (cid:48) , τ ) = inf { y | F ( y | X = x (cid:48) )
Revised: In quantile regression, we estimate the quantile value q ( x (cid:48) , τ ) deﬁned byq ( x (cid:48) , τ ) = inf { y | F ( y | X = x (cid:48) )

Original: ≥ τ } , where F ( y | X
Revised: ≥ τ } , where F ( y | X

Original: = x (cid:48) ) is the conditional cumulative distribution function of Y given X = x (cid:48) :
Revised: = x (cid:48) ) is the conditional cumulative distribution function of Y given X = x (cid:48) :

Original: F ( y | X = x (cid:48) )
Revised: F ( y | X = x (cid:48) )

Original: =
Revised: =

Original: Pr ( Y ≤ y | X =
Revised: Pr ( Y ≤ y | X =

Original: x (cid:48) ) .
Revised: x (cid:48) ) .

Original: Since the function q ( x (cid:48) , τ ) is monotonically increasing on τ , this problem can be modeled as partiallymonotone regression with m = 1 .
Revised: Since the function q ( x (cid:48) , τ ) is monotonically increasing on τ , this problem can be modeled as partiallymonotone regression with m = 1 .

Original: Another example of partially monotone regression with m = 1 is survival analysis, which is alsoknown as time-to-event analysis Wang et al.
Revised: Another example of partially monotone regression with m = 1 is survival analysis, which is alsoknown as time-to-event analysis Wang et al.

Original: [2019]; Rindt et al.
Revised: [2019]; Rindt et al.

Original: We show that the outputf ( x ) =
Revised: We show that the outputf ( x ) =

Original: f
Revised: f

Original: ( x (cid:48) , t ) of the survival analysis for an input x = ( x (cid:48) , t ) is monotonic on t , where x (cid:48) ∈
Revised: ( x (cid:48) , t ) of the survival analysis for an input x = ( x (cid:48) , t ) is monotonic on t , where x (cid:48) ∈

Original: R d − 1is a feature vector of a patient, and t ∈ R is the time of an event of interest of this patient.
Revised: R d − 1is a feature vector of a patient, and t ∈ R is the time of an event of interest of this patient.

Original: In survivalanalysis, the event of interest typically corresponds to the death of a patient.
Revised: In survivalanalysis, the event of interest typically corresponds to the death of a patient.

Original: The goal of the analysisis to estimate a survival function S ( x (cid:48) , t ) for patient x (cid:48) , which represents the probability that theevent does not occur until time t for patient x (cid:48) .
Revised: The goal of the analysisis to estimate a survival function S ( x (cid:48) , t ) for patient x (cid:48) , which represents the probability that theevent does not occur until time t for patient x (cid:48) .

Original: By setting f ( x (cid:48) , t ) = 1 − S ( x (cid:48) , t ) , we can model thesurvival analysis as a partially monotone regression with m = 1 , where the value f
Revised: By setting f ( x (cid:48) , t ) = 1 − S ( x (cid:48) , t ) , we can model thesurvival analysis as a partially monotone regression with m = 1 , where the value f

Original: ( x (cid:48) , t ) shows themortality rate of patient x (cid:48) at time t , and f ( x (cid:48) , t ) is monotonically increasing on t .
Revised: ( x (cid:48) , t ) shows themortality rate of patient x (cid:48) at time t , and f ( x (cid:48) , t ) is monotonically increasing on t .

Original:  Hierarchical Lattice Layer
Revised:  Hierarchical Lattice Layer

Original: In this section, we explain our new hierarchical lattice layer (HLL) for neural network models forpartially monotone regression.
Revised: In this section, we explain our new hierarchical lattice layer (HLL) for neural network models forpartially monotone regression.

Original: It can be seen as an extension of TL Lattice Gupta et al.
Revised: It can be seen as an extension of TL Lattice Gupta et al.

Original: Notethat the input of HLL is a d -dimensional vector x ∈ [0 , 1] d , and its output f ( x ) is in [0 , 1] , where theinput vector x can be partitioned into two subvectors p ∈ [0 , 1] d − m and q ∈ [0 , 1] m such that f ( x ) ismonotonic on q .
Revised: Notethat the input of HLL is a d -dimensional vector x ∈ [0 , 1] d , and its output f ( x ) is in [0 , 1] , where theinput vector x can be partitioned into two subvectors p ∈ [0 , 1] d − m and q ∈ [0 , 1] m such that f ( x ) ismonotonic on q .

Original: 3.1 Hypercube Structure
Revised: 3.1 Hypercube Structure

Original: In HLL, we create the m -dimensional unit hypercube [0 , 1] m , and we decompose it into ( k 1 − 1) ×
Revised: In HLL, we create the m -dimensional unit hypercube [0 , 1] m , and we decompose it into ( k 1 − 1) ×

Original: ( k 2 − 1) × · · · × ( k m − 1) subhypercubes by dividing the interval [0 , 1] of the i -th dimension intok i − 1 subintervals, where each k i ≥ 2 is an integer hyperparameter that speciﬁes the granularity ofthe hypercube.
Revised: ( k 2 − 1) × · · · × ( k m − 1) subhypercubes by dividing the interval [0 , 1] of the i -th dimension intok i − 1 subintervals, where each k i ≥ 2 is an integer hyperparameter that speciﬁes the granularity ofthe hypercube.

Original: Unless otherwise stated, we assume that k 1 = k 2 = · · · = k m =
Revised: Unless otherwise stated, we assume that k 1 = k 2 = · · · = k m =

Original: k , and the interval[0 , 1] is divided into k − 1 equal-length subintervals just for simplicity.
Revised: k , and the interval[0 , 1] is divided into k − 1 equal-length subintervals just for simplicity.

Original: This means that the length ofan edge of a subhypercube is 1 / ( k − 1) , and we have the k m vertices in the unit hypercube.
Revised: This means that the length ofan edge of a subhypercube is 1 / ( k − 1) , and we have the k m vertices in the unit hypercube.

Original: We denotethe set of the vertices in the unit hypercube by V m .
Revised: We denotethe set of the vertices in the unit hypercube by V m .

Original: In this paper, we do not distinguish between avertex v ∈ V m and its coordinate q v ∈ R m .
Revised: In this paper, we do not distinguish between avertex v ∈ V m and its coordinate q v ∈ R m .

Original: This means that the value f ( p, q v ) for coordinate q v iswritten as f ( p, v ) .
Revised: This means that the value f ( p, q v ) for coordinate q v iswritten as f ( p, v ) .

Original: To construct HLL, we assume that there is a total order σ = ( v 0 , v 1 , . .
Revised: To construct HLL, we assume that there is a total order σ = ( v 0 , v 1 , . .

Original: . , v | V m |− 1 ) of vertices in
Revised: . , v | V m |− 1 ) of vertices in

Original: V m .
Revised: V m .

Original: We can use any total order σ to construct HLL, but unless otherwise stated, we assume thatσ is determined by the increasing order of the Manhattan distance (i.e., L 1 distance) from theorigin vertex v 

Revised: We can use any total order σ to construct HLL, but unless otherwise stated, we assume thatσ is determined by the increasing order of the Manhattan distance (i.e., L 1 distance) from theorigin vertex v

Original: = (0 , 0 , . .
Revised: = (0 , 0 , . .

Original: . , 0) ∈
Revised: . , 0) ∈

Original: [0 , 1] m of the unit hypercube with an arbitrary tie-breaker.
Revised: [0 , 1] m of the unit hypercube with an arbitrary tie-breaker.

Original: See
Revised: See

Original: Figure 1 for an example of a decomposed unit hypercube with m = 2 and k = 4 and the total orderσ
Revised: Figure 1 for an example of a decomposed unit hypercube with m = 2 and k = 4 and the total orderσ

Original: = ( v 0 , v 1 , . .
Revised: = ( v 0 , v 1 , . .

Original: , v 15 ) determined by the Manhattan distance.
Revised: , v 15 ) determined by the Manhattan distance.

Original: We deﬁne the domination relationship between vertices in V m , and we deﬁne the minimal dominatedand dominating sets based on σ .
Revised: We deﬁne the domination relationship between vertices in V m , and we deﬁne the minimal dominatedand dominating sets based on σ .

Original: Deﬁnition 3.1 (Domination.)
Revised: Deﬁnition 3.1 (Domination.)

Original: A vertex v ∈ R m is said to dominate another vertex u ∈ R m if andonly if u [ i ] ≤ v [ i ] holds for all i ∈ { 1 , 2 , . . .
Revised: A vertex v ∈ R m is said to dominate another vertex u ∈ R m if andonly if u [ i ] ≤ v [ i ] holds for all i ∈ { 1 , 2 , . . .

Original: , m } and there exists an index i (cid:48) ∈ { 1 , 2 , . .
Revised: , m } and there exists an index i (cid:48) ∈ { 1 , 2 , . .

Original: , m }
Revised: , m }

Original: suchthat u
Revised: suchthat u

Original: [ i (cid:48) ] <
Revised: [ i (cid:48) ] <

Original: v [ i (cid:48) ] .
Revised: v [ i (cid:48) ] .

Original: Deﬁnition 3.2 (Minimal dominated set.)
Revised: Deﬁnition 3.2 (Minimal dominated set.)

Original: The set of vertices L v,σ denotes the minimal set of verticesthat satisfy three conditions (i) every vertex u ∈ L v,σ is dominated by v , (ii) vertex u precedes v in σ ,and (iii) no other vertex w ∈ L v,σ dominates u .
Revised: The set of vertices L v,σ denotes the minimal set of verticesthat satisfy three conditions (i) every vertex u ∈ L v,σ is dominated by v , (ii) vertex u precedes v in σ ,and (iii) no other vertex w ∈ L v,σ dominates u .

Original: Deﬁnition 3.3 (Minimal dominating set.)
Revised: Deﬁnition 3.3 (Minimal dominating set.)

Original: The set of vertices U v,σ denotes the minimal set of verticesthat satisfy three conditions (i) every vertex u ∈ U v,σ dominates v , (ii) vertex u precedes v in σ , and(iii) no other vertex w ∈ U v,σ is dominated by u .
Revised: The set of vertices U v,σ denotes the minimal set of verticesthat satisfy three conditions (i) every vertex u ∈ U v,σ dominates v , (ii) vertex u precedes v in σ , and(iii) no other vertex w ∈ U v,σ is dominated by u .

Original: Note that the domination relation means that f ( p, u ) ≤ f ( p, v ) holds for any p ∈ R d − m if vertex vdominates vertex u .
Revised: Note that the domination relation means that f ( p, u ) ≤ f ( p, v ) holds for any p ∈ R d − m if vertex vdominates vertex u .

Original: Regarding the minimal dominated and dominating sets, there can be multipleminimal sets that satisfy the conditions.
Revised: Regarding the minimal dominated and dominating sets, there can be multipleminimal sets that satisfy the conditions.

Original: Throughout this paper, we ﬁx an (arbitrary) minimaldominated set L v,σ and an (arbitrary) minimal dominating set U v,σ for each v ∈ σ .
Revised: Throughout this paper, we ﬁx an (arbitrary) minimaldominated set L v,σ and an (arbitrary) minimal dominating set U v,σ for each v ∈ σ .

Original: Note also that wehave | L v,σ | ≤ m and | U v,σ | = 0 if σ is determined by the Manhattan distance.
Revised: Note also that wehave | L v,σ | ≤ m and | U v,σ | = 0 if σ is determined by the Manhattan distance.

Original: This is why we usethe Manhattan distance to determine σ , because we have weaker upper bounds | L v,σ | ≤ | V m | and| U v,σ | ≤ | V m | if σ is an arbitrary ordering.
Revised: This is why we usethe Manhattan distance to determine σ , because we have weaker upper bounds | L v,σ | ≤ | V m | and| U v,σ | ≤ | V m | if σ is an arbitrary ordering.

Original: vvvvvvvvvvvvvvvvv 

Revised: vvvvvvvvvvvvvvvvv

Original: = (0 , 0) v 9 = (0 , 1) v 6 = (1 , 0) v 15 = (1 , 1) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108)
Revised: = (0 , 0) v 9 = (0 , 1) v 6 = (1 , 0) v 15 = (1 , 1) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108) (cid:108)

Original: Example.
Revised: Example.

Original: In the hypercube shown in Figure 1, vertex v 8 dominates the vertices in{ v 0 , v 1 , v 2 , v 4 , v 5 } and is dominated by the vertices in { v 11 , v 12 , v 13 , v 14 , v 15 } .
Revised: In the hypercube shown in Figure 1, vertex v 8 dominates the vertices in{ v 0 , v 1 , v 2 , v 4 , v 5 } and is dominated by the vertices in { v 11 , v 12 , v 13 , v 14 , v 15 } .

Original: By the deﬁnition ofthe minimal dominated set, we have L v 8 ,σ = { v 4 , v 5 } because vertices v 0 , v 1 , and v 2 are dominatedby v 4 or v 5 .
Revised: By the deﬁnition ofthe minimal dominated set, we have L v 8 ,σ = { v 4 , v 5 } because vertices v 0 , v 1 , and v 2 are dominatedby v 4 or v 5 .

Original: By the deﬁnition of the minimal dominating set, we have U v 8 ,σ = ∅ because all thevertices that dominate v 8 appear after v 8 in ordering σ .
Revised: By the deﬁnition of the minimal dominating set, we have U v 8 ,σ = ∅ because all thevertices that dominate v 8 appear after v 8 in ordering σ .

Original: 3.2 Auxiliary Neural Network
Revised: 3.2 Auxiliary Neural Network

Original: For HLL, we construct an auxiliary neural network g θ ( p, v ) .
Revised: For HLL, we construct an auxiliary neural network g θ ( p, v ) .

Original: To explain its construction, we deﬁnetwo functions l ( p, v ) and u ( p, v ) for p ∈ R d − m and a vertex v ∈ V m as
Revised: To explain its construction, we deﬁnetwo functions l ( p, v ) and u ( p, v ) for p ∈ R d − m and a vertex v ∈ V m as

Original: [Equation]
Revised: [Equation]

Original: where L v,σ and U v,σ are the minimal dominated and dominating sets, respectively.
Revised: where L v,σ and U v,σ are the minimal dominated and dominating sets, respectively.

Original: Note that l ( p, v )and u ( p, v ) are deﬁned so as to satisfy the inequality l ( p, v ) ≤ f ( p, v ) ≤ u ( p, v ) .
Revised: Note that l ( p, v )and u ( p, v ) are deﬁned so as to satisfy the inequality l ( p, v ) ≤ f ( p, v ) ≤ u ( p, v ) .

Original: By using thisinequality, we deﬁne a third function g ( p, v ) as a function that satisﬁes
Revised: By using thisinequality, we deﬁne a third function g ( p, v ) as a function that satisﬁes

Original: [Equation]
Revised: [Equation]

Original: Since we have l ( p, v ) ≤ f ( p, v ) ≤ u ( p, v ) , there always exists 0 ≤ g ( p, v ) ≤ 1 that satisﬁes Eq.
Revised: Since we have l ( p, v ) ≤ f ( p, v ) ≤ u ( p, v ) , there always exists 0 ≤ g ( p, v ) ≤ 1 that satisﬁes Eq.

Original: We show an example of the deﬁnitions of the three functions for a simple case with m = 1 and k = 3in Appendix A.
Revised: We show an example of the deﬁnitions of the three functions for a simple case with m = 1 and k = 3in Appendix A.

Original: The auxiliary neural network g θ ( p, v ) in HLL is trained to learn the function g ( p, v ) .
Revised: The auxiliary neural network g θ ( p, v ) in HLL is trained to learn the function g ( p, v ) .

Original: To satisfy theconstraint 0 ≤ g ( p, v ) ≤ 1 , we use the sigmoid function in the last layer of g θ ( p, v ) as the activationfunction.
Revised: To satisfy theconstraint 0 ≤ g ( p, v ) ≤ 1 , we use the sigmoid function in the last layer of g θ ( p, v ) as the activationfunction.

Original: Note that, by the deﬁnitions of the three functions l ( p, v ) , u ( p, v ) , and g ( p, v ) , we cancompute f ( p, v ) for any p and v by using g θ ( p, v ) .
Revised: Note that, by the deﬁnitions of the three functions l ( p, v ) , u ( p, v ) , and g ( p, v ) , we cancompute f ( p, v ) for any p and v by using g θ ( p, v ) .

Original: 3.3 Inference and Training Algorithms
Revised: 3.3 Inference and Training Algorithms

Original: The inference algorithm of HLL is similar to TL Lattice, and Figure 2 illustrates it.
Revised: The inference algorithm of HLL is similar to TL Lattice, and Figure 2 illustrates it.

Original: Given an inputx =
Revised: Given an inputx =

Original: ( p, q ) ,
Revised: ( p, q ) ,

Original: we ﬁrst calculate f ( p, v ) for all v ∈ V q by using g θ ( p, v ) , where V q ⊆
Revised: we ﬁrst calculate f ( p, v ) for all v ∈ V q by using g θ ( p, v ) , where V q ⊆

Original: V m is the set ofthe vertices that surround q in the hypercube.
Revised: V m is the set ofthe vertices that surround q in the hypercube.

Original: Then, we estimate f ( p, q ) as an interpolation of thevalues f
Revised: Then, we estimate f ( p, q ) as an interpolation of thevalues f

Original: ( p, v ) for v ∈ V q .
Revised: ( p, v ) for v ∈ V q .

Original: As for the interpolation algorithm, we can use multilinear interpolationor simplex interpolation Gupta et al.
Revised: As for the interpolation algorithm, we can use multilinear interpolationor simplex interpolation Gupta et al.

Original: Note that the size of V q depends on the interpolationalgorithm (i.e., | V q | = 2 m if we use multilinear interpolation, and | V q | =
Revised: Note that the size of V q depends on the interpolationalgorithm (i.e., | V q | = 2 m if we use multilinear interpolation, and | V q | =

Original: O ( m log m ) if we usesimplex interpolation).
Revised: O ( m log m ) if we usesimplex interpolation).

Original: The training algorithm of HLL is the same as that of the standard neural network layer.
Revised: The training algorithm of HLL is the same as that of the standard neural network layer.

Original: For example,if we train a neural network model that consists of a single HLL, the parameters of HLL (i.e. theparameters θ of the auxiliary neural network g θ ) are trained to minimize the empirical loss
Revised: For example,if we train a neural network model that consists of a single HLL, the parameters of HLL (i.e. theparameters θ of the auxiliary neural network g θ ) are trained to minimize the empirical loss

Original: [Equation]
Revised: [Equation]

Original: x = ( p, q ) (cid:45)
Revised: x = ( p, q ) (cid:45)

Original: Compute f ( p, v ) by using g θ f ( p, v 1 ) f ( p, v 5 ) f ( p, v 7 ) (cid:113) q f ( p, v 2 ) f
Revised: Compute f ( p, v ) by using g θ f ( p, v 1 ) f ( p, v 5 ) f ( p, v 7 ) (cid:113) q f ( p, v 2 ) f

Original: ( p, v 4 ) f ( p, v 8 ) (cid:0)(cid:0) (cid:0)(cid:0) (cid:0)(cid:0) (cid:0)(cid:0) (cid:45)
Revised: ( p, v 4 ) f ( p, v 8 ) (cid:0)(cid:0) (cid:0)(cid:0) (cid:0)(cid:0) (cid:0)(cid:0) (cid:45)

Original: Interpolate f ( p, q ) where { ( x i , y i ) } ni =1 is a dataset, f ( x i ) is the output of the inference algorithm of HLL for input x i ,and (cid:96) is a loss function.
Revised: Interpolate f ( p, q ) where { ( x i , y i ) } ni =1 is a dataset, f ( x i ) is the output of the inference algorithm of HLL for input x i ,and (cid:96) is a loss function.

Original: 3.4 Partial Monotonicity
Revised: 3.4 Partial Monotonicity

Original: As already proven in Gupta et al.
Revised: As already proven in Gupta et al.

Original: [2016], the function f ( p, q ) computed by an interpolation algorithmsatisﬁes monotonicity constraints if the values corresponding to the vertices of the hypercube satisfythe constraints.
Revised: [2016], the function f ( p, q ) computed by an interpolation algorithmsatisﬁes monotonicity constraints if the values corresponding to the vertices of the hypercube satisfythe constraints.

Original: Therefore, we show that we have f ( p, u ) ≤ f ( p, v ) for any pair of vertices u, v ∈ V msuch that u is dominated by v .
Revised: Therefore, we show that we have f ( p, u ) ≤ f ( p, v ) for any pair of vertices u, v ∈ V msuch that u is dominated by v .

Original: Note that this lemma holds for any vertex ordering σ and is notrestricted to the vertex ordering determined by the Manhattan distance.
Revised: Note that this lemma holds for any vertex ordering σ and is notrestricted to the vertex ordering determined by the Manhattan distance.

Original: Lemma 3.4 In HLL, f ( p, u ) ≤ f ( p, v ) holds for any p ∈ R d − m and any pair of vertices u, v ∈ V msuch that u is dominated by v
Revised: Lemma 3.4 In HLL, f ( p, u ) ≤ f ( p, v ) holds for any p ∈ R d − m and any pair of vertices u, v ∈ V msuch that u is dominated by v

Original: Proof.
Revised: Proof.

Original: Suppose that u precedes v in the total ordering σ .
Revised: Suppose that u precedes v in the total ordering σ .

Original: By the deﬁnition of the minimal dominatedset, there exists a sequence of vertices u 0 (= u ) , u 1 , u 2 , . .
Revised: By the deﬁnition of the minimal dominatedset, there exists a sequence of vertices u 0 (= u ) , u 1 , u 2 , . .

Original: , u J (= v )
Revised: , u J (= v )

Original: such that u j − 1 ∈
Revised: such that u j − 1 ∈

Original: L u j ,σ holdsfor j = 1 , 2 , . . .
Revised: L u j ,σ holdsfor j = 1 , 2 , . . .

Original: , J .
Revised: , J .

Original: Since u j − 1 ∈
Revised: Since u j − 1 ∈

Original: L u j ,σ , we have f ( p, u j − 1 ) ≤ l ( p, u j ) ≤ f ( p, u j ) by Eq.
Revised: L u j ,σ , we have f ( p, u j − 1 ) ≤ l ( p, u j ) ≤ f ( p, u j ) by Eq.

Original: Hence, we have f ( p, u ) =
Revised: Hence, we have f ( p, u ) =

Original: f ( p, u 0 ) ≤ f ( p, u 1 ) ≤ · · · ≤ f ( p, u J ) =
Revised: f ( p, u 0 ) ≤ f ( p, u 1 ) ≤ · · · ≤ f ( p, u J ) =

Original: f ( p, v ) .
Revised: f ( p, v ) .

Original: Similarly, suppose that v precedes u in the total ordering σ .
Revised: Similarly, suppose that v precedes u in the total ordering σ .

Original: By the deﬁnition of the minimaldominating set, there exists a sequence of vertices v 0 (= v ) , v 1 , v 2 , . .
Revised: By the deﬁnition of the minimaldominating set, there exists a sequence of vertices v 0 (= v ) , v 1 , v 2 , . .

Original: . , v J (= u ) such that v j − 1 ∈
Revised: . , v J (= u ) such that v j − 1 ∈

Original: U v j ,σ holds for j = 1 , 2 , . . .
Revised: U v j ,σ holds for j = 1 , 2 , . . .

Original: , J .
Revised: , J .

Original: Since v j − 1 ∈ U v j ,σ , we have f ( p, v j − 1 ) ≥ u ( p, v j ) ≥ f ( p, v j ) by
Revised: Since v j − 1 ∈ U v j ,σ , we have f ( p, v j − 1 ) ≥ u ( p, v j ) ≥ f ( p, v j ) by

Original: Eq.
Revised: Eq.

Original: Hence, we have f ( p, v ) =
Revised: Hence, we have f ( p, v ) =

Original: f ( p, v 0 ) ≥ f
Revised: f ( p, v 0 ) ≥ f

Original: ( p, v 1 ) ≥ · · · ≥ f ( p, v J ) =
Revised: ( p, v 1 ) ≥ · · · ≥ f ( p, v J ) =

Original: f ( p, u ) .
Revised: f ( p, u ) .

Original: 3.5 Complexity Analysis
Revised: 3.5 Complexity Analysis

Original: We show that the time complexity of the inference algorithm of HLL is O (( T ( g θ ) + m ) | V m | ) , where
Revised: We show that the time complexity of the inference algorithm of HLL is O (( T ( g θ ) + m ) | V m | ) , where

Original: T ( g θ ) is the time complexity of computing g θ ( p, v ) for an input ( p, v ) ∈ R d − m × R m .
Revised: T ( g θ ) is the time complexity of computing g θ ( p, v ) for an input ( p, v ) ∈ R d − m × R m .

Original: First, thetime complexity to compute f ( p, v ) for all vertices v ∈ V m is O
Revised: First, thetime complexity to compute f ( p, v ) for all vertices v ∈ V m is O

Original: (( T ( g θ ) + m ) | V m | ) since it takes
Revised: (( T ( g θ ) + m ) | V m | ) since it takes

Original: O ( T ( g θ ) + | L v,σ | + | U v,σ | ) time to compute f ( p, v ) for a single vertex v ∈ V m , and we have| L v,σ | + | U v,σ | ≤ m .
Revised: O ( T ( g θ ) + | L v,σ | + | U v,σ | ) time to compute f ( p, v ) for a single vertex v ∈ V m , and we have| L v,σ | + | U v,σ | ≤ m .

Original: Then, the time complexity for the interpolation algorithm is O (2 m ) if we usemultilinear interpolation and O ( m log m ) if we use simplex interpolation Gupta et al.
Revised: Then, the time complexity for the interpolation algorithm is O (2 m ) if we usemultilinear interpolation and O ( m log m ) if we use simplex interpolation Gupta et al.

Original: Sincewe have 2 m ≤ | V m | , the time complexity for the interpolation algorithm is smaller than that forthe computation of f ( p, v ) .
Revised: Sincewe have 2 m ≤ | V m | , the time complexity for the interpolation algorithm is smaller than that forthe computation of f ( p, v ) .

Original: Hence, the overall time complexity remains O (( T ( g θ ) + m ) | V m | ) .
Revised: Hence, the overall time complexity remains O (( T ( g θ ) + m ) | V m | ) .

Original: Ifwe can assume that m is a small integer (i.e., m is a constant), the computation time for inferencecan be written simply as O ( T ( g θ )) .
Revised: Ifwe can assume that m is a small integer (i.e., m is a constant), the computation time for inferencecan be written simply as O ( T ( g θ )) .

Original: Note that, if the time complexity T ( g θ ) is a linear functionon d , the time complexity of the inference algorithm for HLL is smaller than that for TL Latticebecause the inference algorithm of TL Lattice takes O (2 d ) and O ( d log d ) times if we use multilinearinterpolation and simplex interpolation, respectively Gupta et al.
Revised: Note that, if the time complexity T ( g θ ) is a linear functionon d , the time complexity of the inference algorithm for HLL is smaller than that for TL Latticebecause the inference algorithm of TL Lattice takes O (2 d ) and O ( d log d ) times if we use multilinearinterpolation and simplex interpolation, respectively Gupta et al.

Original: The space complexity of HLL is O ( | g θ | + m | V m | ) if the vertex ordering σ is determined by the
Revised: The space complexity of HLL is O ( | g θ | + m | V m | ) if the vertex ordering σ is determined by the

Original: Manhattan distance, where we denote the size of the auxiliary neural network g θ ( p, v ) by | g θ | , and the
Revised: Manhattan distance, where we denote the size of the auxiliary neural network g θ ( p, v ) by | g θ | , and the

Original: O ( m | V m | ) term is the space required to store L v,σ and U v,σ for all v ∈ V m .
Revised: O ( m | V m | ) term is the space required to store L v,σ and U v,σ for all v ∈ V m .

Original: Note that, if we use anarbitrary vertex ordering σ , we require O ( | V m | 2 ) space to store L v,σ and U v,σ for all v ∈ V m becausewe know only that | L v,σ | ≤ | V m | and | U v,σ | ≤ | V m | .
Revised: Note that, if we use anarbitrary vertex ordering σ , we require O ( | V m | 2 ) space to store L v,σ and U v,σ for all v ∈ V m becausewe know only that | L v,σ | ≤ | V m | and | U v,σ | ≤ | V m | .

Original: Since | V m | 2 is much larger than m | V m | , werecommend using the Manhattan distance to determine σ in HLL.
Revised: Since | V m | 2 is much larger than m | V m | , werecommend using the Manhattan distance to determine σ in HLL.

Original:  Experiments
Revised:  Experiments

Original: In our experiments, we demonstrate that HLL did not sacriﬁce its prediction performance comparedwith TL Lattice.
Revised: In our experiments, we demonstrate that HLL did not sacriﬁce its prediction performance comparedwith TL Lattice.

Original: Note that HLL is designed to satisfy monotonicity constraints and is not designedto improve the prediction performance compared with TL Lattice.
Revised: Note that HLL is designed to satisfy monotonicity constraints and is not designedto improve the prediction performance compared with TL Lattice.

Original: Therefore, the purpose of ourexperiments was not to show such improvements but to show that HLL did not sacriﬁce the predictionperformance to satisfy the constraints.
Revised: Therefore, the purpose of ourexperiments was not to show such improvements but to show that HLL did not sacriﬁce the predictionperformance to satisfy the constraints.

Original: In addition, we also demonstrate that we could use HLL forhigh-dimensional input vectors with a small m .
Revised: In addition, we also demonstrate that we could use HLL forhigh-dimensional input vectors with a small m .

Original: All our experiments were conducted on a virtualmachine with an Intel Xeon CPU (3.30 GHz) processor without any GPU and 64 GB of memoryrunning Red Hat Enterprise Linux Server 7.6.
Revised: All our experiments were conducted on a virtualmachine with an Intel Xeon CPU (3.30 GHz) processor without any GPU and 64 GB of memoryrunning Red Hat Enterprise Linux Server 7.6.

Original: 4.1 Neural Network Models
Revised: 4.1 Neural Network Models

Original: In our experiments, we compared these four neural network models.
Revised: In our experiments, we compared these four neural network models.

Original: • MLP .
Revised: • MLP .

Original: A multi-layer perceptron model with three hidden layers.
Revised: A multi-layer perceptron model with three hidden layers.

Original: We used the ReLU acti-
Revised: We used the ReLU acti-

Original: vation layer for the hidden layers and the sigmoid function for the output layer.
Revised: vation layer for the hidden layers and the sigmoid function for the output layer.

Original: Note thatthis neural network is not designed for partially monotone regression and cannot handlemonotonicity constraints.• HLL .
Revised: Note thatthis neural network is not designed for partially monotone regression and cannot handlemonotonicity constraints.• HLL .

Original: A neural network model consisting of a single hierarchical lattice layer.
Revised: A neural network model consisting of a single hierarchical lattice layer.

Original: We used anMLP model with three hidden layers for the auxiliary neural network g θ ( p, v ) , and we usedmultilinear interpolation in the inference algorithm.• TL Lattice .
Revised: We used anMLP model with three hidden layers for the auxiliary neural network g θ ( p, v ) , and we usedmultilinear interpolation in the inference algorithm.• TL Lattice .

Original: A neural network model consisting of a single TL Lattice layer Gupta et al.[2016].
Revised: A neural network model consisting of a single TL Lattice layer Gupta et al.[2016].

Original: This layer uses O ( k d ) space, and therefore, it can be used only for low-dimensionalinputs (i.e., d is a small integer).
Revised: This layer uses O ( k d ) space, and therefore, it can be used only for low-dimensionalinputs (i.e., d is a small integer).

Original: We used multilinear interpolation in the inference algorithm.• TL RTL.
Revised: We used multilinear interpolation in the inference algorithm.• TL RTL.

Original: A neural network model consisting of a single RTL layer Canini et al.
Revised: A neural network model consisting of a single RTL layer Canini et al.

Original: [2016] fromthe TensorFlow Lattice library, which is a random ensemble of tiny lattices (i.e., TL Lattices).This layer consists of (cid:100) d/r (cid:101) tiny lattices, and each tiny lattice receives an r -dimensionalinput vector, where r is an integer hyperparameter.
Revised: [2016] fromthe TensorFlow Lattice library, which is a random ensemble of tiny lattices (i.e., TL Lattices).This layer consists of (cid:100) d/r (cid:101) tiny lattices, and each tiny lattice receives an r -dimensionalinput vector, where r is an integer hyperparameter.

Original: Hence, this layer uses O
Revised: Hence, this layer uses O

Original: (( d/r ) k r ) space,which is signiﬁcantly smaller than that of TL Lattice when r (cid:28) d .
Revised: (( d/r ) k r ) space,which is signiﬁcantly smaller than that of TL Lattice when r (cid:28) d .

Original: Note that a theoreticalanalysis in Canini et al.
Revised: Note that a theoreticalanalysis in Canini et al.

Original: [2016] shows that TL RTL with d/r tiny lattices is equivalent to thesingle d -dimensional TL Lattice.
Revised: [2016] shows that TL RTL with d/r tiny lattices is equivalent to thesingle d -dimensional TL Lattice.

Original: We used this model for datasets with high-dimensionalinputs (i.e., d is a large integer).
Revised: We used this model for datasets with high-dimensionalinputs (i.e., d is a large integer).

Original: We used Python 3.8.12 and PyTorch 1.8.1 to implement MLP and HLL.
Revised: We used Python 3.8.12 and PyTorch 1.8.1 to implement MLP and HLL.

Original: As for TL Lattice and TL
Revised: As for TL Lattice and TL

Original: RTL, we used TensorFlow 2.3.0 and TensorFlow Lattice 2.0.10, which is the implementation providedby the authors Canini et al.
Revised: RTL, we used TensorFlow 2.3.0 and TensorFlow Lattice 2.0.10, which is the implementation providedby the authors Canini et al.

Original: [2016]; Gupta et al.
Revised: [2016]; Gupta et al.

Original: [2016] and available as an open source library under
Revised: [2016] and available as an open source library under

Original: Apache License 2.0 at https://www.tensorflow.org/lattice .
Revised: Apache License 2.0 at https://www.tensorflow.org/lattice .

Original: The TensorFlow Lattice libraryalso provides an implementation of the Kronecker-Factored lattice layer Morioka et al.
Revised: The TensorFlow Lattice libraryalso provides an implementation of the Kronecker-Factored lattice layer Morioka et al.

Original: [2021], whichuses a smaller amount of memory than TL Lattice.
Revised: [2021], whichuses a smaller amount of memory than TL Lattice.

Original: However, we did not include the experimentalresults for this layer because TL RTL signiﬁcantly outperformed it for the datasets we used in termsof prediction performance.
Revised: However, we did not include the experimentalresults for this layer because TL RTL signiﬁcantly outperformed it for the datasets we used in termsof prediction performance.

Original: Throughout our experiments, all problems were solved as regression problems, and we used meansquared error (MSE) for the evaluation metric.
Revised: Throughout our experiments, all problems were solved as regression problems, and we used meansquared error (MSE) for the evaluation metric.

Original: For the training of the neural network models, weused the Adam optimizer Kingma and Ba [2015].
Revised: For the training of the neural network models, weused the Adam optimizer Kingma and Ba [2015].

Original: In HLL, TL Lattice, and TL RTL, the number ofvertices in each dimension of the hypercube was set to k by default.
Revised: In HLL, TL Lattice, and TL RTL, the number ofvertices in each dimension of the hypercube was set to k by default.

Original: However, the numbers of thevertices in the dimensions that correspond to small categorical values were set to the numbers of thecategories.
Revised: However, the numbers of thevertices in the dimensions that correspond to small categorical values were set to the numbers of thecategories.

Original: For example, if the i -th dimension corresponds to a binary feature, then we set k i = 2(instead of k i = k ).
Revised: For example, if the i -th dimension corresponds to a binary feature, then we set k i = 2(instead of k i = k ).

Original: 4.2 Real Datasets
Revised: 4.2 Real Datasets

Original: We used 12 real datasets taken from the UCI Machine Learning Repository Dua and Graff [2017] inour experiments.
Revised: We used 12 real datasets taken from the UCI Machine Learning Repository Dua and Graff [2017] inour experiments.

Original: We used six small datasets
Revised: We used six small datasets

Original: [Table]
Revised: [Table]

Original: [2015], Bank Marketing Moro et al.
Revised: [2015], Bank Marketing Moro et al.

Original: [2014], and Blog Feedback Buza [2014].
Revised: [2014], and Blog Feedback Buza [2014].

Original: We conﬁrmed thatnone of these datasets contained any personally identiﬁable data points nor any offensive content.
Revised: We conﬁrmed thatnone of these datasets contained any personally identiﬁable data points nor any offensive content.

Original: Table 1 shows a summary of these 12 datasets.
Revised: Table 1 shows a summary of these 12 datasets.

Original: Some of the datasets were already split into trainingand test data, and the others were unsplit (i.e., a single dataset was given).
Revised: Some of the datasets were already split into trainingand test data, and the others were unsplit (i.e., a single dataset was given).

Original: Column n of this tableshows the number of data points in the training dataset for the split datasets and the number of datapoints for the unsplit datasets.
Revised: Column n of this tableshows the number of data points in the training dataset for the split datasets and the number of datapoints for the unsplit datasets.

Original: In our experiments, we removed all the data points that had missingfeature values.
Revised: In our experiments, we removed all the data points that had missingfeature values.

Original: Column d of Table 1 shows the number of features after our feature encoding.
Revised: Column d of Table 1 shows the number of features after our feature encoding.

Original: We brieﬂy explainour feature encoding here.
Revised: We brieﬂy explainour feature encoding here.

Original: We normalized all the continuous values (including the target values) to arange between 0 and 1 by using the min-max scaling algorithm (i.e. a simple linear transformation).
Revised: We normalized all the continuous values (including the target values) to arange between 0 and 1 by using the min-max scaling algorithm (i.e. a simple linear transformation).

Original: We used label encoding to encode the ordinal categorical values.
Revised: We used label encoding to encode the ordinal categorical values.

Original: For example, we replaced the ordinalcategorical values { low , med , high } with low = 0 , med = 0 .
Revised: For example, we replaced the ordinalcategorical values { low , med , high } with low = 0 , med = 0 .

Original: 5 , and high = 1 so that the replacedvalues were in the range between 0 and 1.
Revised: 5 , and high = 1 so that the replacedvalues were in the range between 0 and 1.

Original: We used one-hot encoding for the other categorical values.
Revised: We used one-hot encoding for the other categorical values.

Original: When we used TL Lattice and TL RTL, we renormalized the input feature vectors from [0 , 1] d to[0 , k − 1] d , where k is a hyperparameter, to conform to the input speciﬁcations of these two models.
Revised: When we used TL Lattice and TL RTL, we renormalized the input feature vectors from [0 , 1] d to[0 , k − 1] d , where k is a hyperparameter, to conform to the input speciﬁcations of these two models.

Original: The last column of Table 1 shows the monotone features we used in our experiments.
Revised: The last column of Table 1 shows the monotone features we used in our experiments.

Original: The italicfeature names represent monotonically decreasing features, and the others represent monotonicallyincreasing features.
Revised: The italicfeature names represent monotonically decreasing features, and the others represent monotonicallyincreasing features.

Original: We should choose monotone features by using domain knowledge for eachdataset, but we used the SHAP values Lundberg and Lee [2017] to ﬁnd candidates of monotonefeatures for the datasets for which we had little domain knowledge.
Revised: We should choose monotone features by using domain knowledge for eachdataset, but we used the SHAP values Lundberg and Lee [2017] to ﬁnd candidates of monotonefeatures for the datasets for which we had little domain knowledge.

Original: We expected that there would bean almost monotonic relationship between the SHAP value and the i -th feature value (i.e., x [ i ] ) if thetarget value was monotonic on the i -th feature.
Revised: We expected that there would bean almost monotonic relationship between the SHAP value and the i -th feature value (i.e., x [ i ] ) if thetarget value was monotonic on the i -th feature.

Original: In our experiments, we constructed an MLP model foreach dataset and computed its SHAP values.
Revised: In our experiments, we constructed an MLP model foreach dataset and computed its SHAP values.

Original: Then, we checked for each feature if the SHAP valueswere monotonically increasing (or decreasing) with respect to the feature values by using Spearman’srank correlation coefﬁcients.
Revised: Then, we checked for each feature if the SHAP valueswere monotonically increasing (or decreasing) with respect to the feature values by using Spearman’srank correlation coefﬁcients.

Original: Note that such correlation between SHAP values and feature valuesdoes not mean a monotone relationship, but we found this approach helpful to ﬁnd candidates ofmonotone features for datasets with little domain knowledge.
Revised: Note that such correlation between SHAP values and feature valuesdoes not mean a monotone relationship, but we found this approach helpful to ﬁnd candidates ofmonotone features for datasets with little domain knowledge.

Original: 4.3 Experimental Results
Revised: 4.3 Experimental Results

Original: In our experiments, we split the data points into training, validation, and test data points.
Revised: In our experiments, we split the data points into training, validation, and test data points.

Original: For theunsplit datasets, we divided the data points into training (60%), validation (20%), and test (20%).
Revised: For theunsplit datasets, we divided the data points into training (60%), validation (20%), and test (20%).

Original: Weused a single random split to search for the best hyperparameters and ﬁve random splits to obtain theaverage prediction performance.
Revised: Weused a single random split to search for the best hyperparameters and ﬁve random splits to obtain theaverage prediction performance.

Original: For the datasets that were already split into training and test datasets,we further divided the data points in the training dataset into training (80%) and validation (20%) andkept the test dataset unchanged.
Revised: For the datasets that were already split into training and test datasets,we further divided the data points in the training dataset into training (80%) and validation (20%) andkept the test dataset unchanged.

Original: We used a single random split to search for the best hyperparametersand ﬁve random splits to obtain the average prediction performance without changing the test dataset.
Revised: We used a single random split to search for the best hyperparametersand ﬁve random splits to obtain the average prediction performance without changing the test dataset.

Original: Regarding the hyperparameters, we chose the best hyperparameters for each combination of a neuralnetwork and a dataset; the learning rate was chosen from { 1 , 10 − 1 , 10 − 2 , 10 − 3 , 10 − 4 , 10 − 5 } , the
Revised: Regarding the hyperparameters, we chose the best hyperparameters for each combination of a neuralnetwork and a dataset; the learning rate was chosen from { 1 , 10 − 1 , 10 − 2 , 10 − 3 , 10 − 4 , 10 − 5 } , the

Original: [Figure]
Revised: [Figure]

Original: Results for Small Datasets.
Revised: Results for Small Datasets.

Original: We compared the prediction performances of MLP, HLL, and TL
Revised: We compared the prediction performances of MLP, HLL, and TL

Original: Lattice by using the six small datasets (i.e., d was small).
Revised: Lattice by using the six small datasets (i.e., d was small).

Original: We trained these neural network models for1000 epochs with certain early stopping criteria, and Figure 3 shows the results for various k .
Revised: We trained these neural network models for1000 epochs with certain early stopping criteria, and Figure 3 shows the results for various k .

Original: The redlines show the average of the MSEs for ﬁve runs of MLP, and we regard these values as baselines ofthe prediction performance.
Revised: The redlines show the average of the MSEs for ﬁve runs of MLP, and we regard these values as baselines ofthe prediction performance.

Original: The blue and orange lines show the average MSEs together with theirstandard deviations for ﬁve runs of HLL and TL Lattice, respectively.
Revised: The blue and orange lines show the average MSEs together with theirstandard deviations for ﬁve runs of HLL and TL Lattice, respectively.

Original: These results showed that our
Revised: These results showed that our

Original: HLL performed comparably to MLP for all six datasets.
Revised: HLL performed comparably to MLP for all six datasets.

Original: Somewhat surprisingly, we note that using aﬁner-grained lattice (i.e., increasing k ) did not necessarily improve the MSEs.
Revised: Somewhat surprisingly, we note that using aﬁner-grained lattice (i.e., increasing k ) did not necessarily improve the MSEs.

Original: This phenomenon wasalready mentioned in Gupta et al.
Revised: This phenomenon wasalready mentioned in Gupta et al.

Original: [2016] for TL Lattice, and we conﬁrmed this phenomenon both for
Revised: [2016] for TL Lattice, and we conﬁrmed this phenomenon both for

Original: TL Lattice and HLL in our experiments.
Revised: TL Lattice and HLL in our experiments.

Original: Figure 3 also shows that the prediction performance of HLLwere also comparable with TL Lattice.
Revised: Figure 3 also shows that the prediction performance of HLLwere also comparable with TL Lattice.

Original: Although TL Lattice showed worse MSEs than MLP and
Revised: Although TL Lattice showed worse MSEs than MLP and

Original: HLL for some datasets, these differences in prediction performance were supposed to come from twofactors.
Revised: HLL for some datasets, these differences in prediction performance were supposed to come from twofactors.

Original: One was the difference of the parameterization; HLL used a hypercube of size k m , whereas
Revised: One was the difference of the parameterization; HLL used a hypercube of size k m , whereas

Original: TL Lattice used a hypercube of size k d .
Revised: TL Lattice used a hypercube of size k d .

Original: The other one was the difference of the neural networktraining algorithm; MLP and HLL used the standard training algorithm, whereas TL Lattice used itsown special training algorithm.
Revised: The other one was the difference of the neural networktraining algorithm; MLP and HLL used the standard training algorithm, whereas TL Lattice used itsown special training algorithm.

Original: Results for Large Datasets.
Revised: Results for Large Datasets.

Original: For the datasets with a large d , we could not use TL Lattice due to itsspace complexity O ( k d ) , so we used TL RTL instead.
Revised: For the datasets with a large d , we could not use TL Lattice due to itsspace complexity O ( k d ) , so we used TL RTL instead.

Original: TL Lattice requires that at least 2 52 parametersbe stored even if k = 2 since d ≥ 52 , but we cannot store such a large number of parameters inmemory.
Revised: TL Lattice requires that at least 2 52 parametersbe stored even if k = 2 since d ≥ 52 , but we cannot store such a large number of parameters inmemory.

Original: In contrast, we could use HLL since m is small in the datasets we used.
Revised: In contrast, we could use HLL since m is small in the datasets we used.

Original: We compared the prediction performance of MLP, HLL, and TL RTL by using the six large datasets.
Revised: We compared the prediction performance of MLP, HLL, and TL RTL by using the six large datasets.

Original: We trained these neural network models for 100 epochs with certain early stopping criteria, and
Revised: We trained these neural network models for 100 epochs with certain early stopping criteria, and

Original: Figure 4 shows the results for various k .
Revised: Figure 4 shows the results for various k .

Original: The red lines show the average MSEs for ﬁve runs of
Revised: The red lines show the average MSEs for ﬁve runs of

Original: MLP, and we regard these values as baselines of the prediction performance.
Revised: MLP, and we regard these values as baselines of the prediction performance.

Original: The blue and greenlines show the average MSEs together with their standard deviations for ﬁve runs of HLL and TL
Revised: The blue and greenlines show the average MSEs together with their standard deviations for ﬁve runs of HLL and TL

Original: RTL, respectively.
Revised: RTL, respectively.

Original: Note that the result for TL RTL with k = 5 on the Blog Feedback dataset wasmissing due to a memory limitation in our computer environment.
Revised: Note that the result for TL RTL with k = 5 on the Blog Feedback dataset wasmissing due to a memory limitation in our computer environment.

Original: These results showed that our
Revised: These results showed that our

Original: HLL performed comparably to MLP and TL RTL regardless of k for the six large datasets.
Revised: HLL performed comparably to MLP and TL RTL regardless of k for the six large datasets.

Original: We think
Revised: We think

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: RTL
Revised: RTL

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: e
Revised: e

Original: [Figure]
Revised: [Figure]

Original: RTL
Revised: RTL

Original: [Figure]
Revised: [Figure]

Original: that HLL performed better than for some thanks
Revised: that HLL performed better than for some thanks

Original:  Related Work
Revised:  Related Work

Original: There are some neural networks for partially monotone regression other than the TensorFlow Latticelibrary.
Revised: There are some neural networks for partially monotone regression other than the TensorFlow Latticelibrary.

Original: One of the simplest approaches is to enforce non-negativity constraints on neural networkweights that are related to monotone inputs (e.g., Daniels and Velikova [2010]; You et al.
Revised: One of the simplest approaches is to enforce non-negativity constraints on neural networkweights that are related to monotone inputs (e.g., Daniels and Velikova [2010]; You et al.

Original: Another simple approach is to use a regularization term to enforce monotonicity Gupta et al.
Revised: Another simple approach is to use a regularization term to enforce monotonicity Gupta et al.

Original: Monteiro et al.
Revised: Monteiro et al.

Original: [2021], but a prediction model that uses this approach may not always satisfy themonotonicity constraints.
Revised: [2021], but a prediction model that uses this approach may not always satisfy themonotonicity constraints.

Original: More complex approaches are proposed in Liu et al.
Revised: More complex approaches are proposed in Liu et al.

Original: [2020]; Sivaramanet al.
Revised: [2020]; Sivaramanet al.

Original: These approaches use a mixed integer linear programming solver or a satisﬁabilitymodulo theories (SMT) solver to ensure the monotonicity of the neural network.
Revised: These approaches use a mixed integer linear programming solver or a satisﬁabilitymodulo theories (SMT) solver to ensure the monotonicity of the neural network.

Original: The disadvantagesof these approaches are their computational complexity and the costs of using the solvers.
Revised: The disadvantagesof these approaches are their computational complexity and the costs of using the solvers.

Original:  Conclusion
Revised:  Conclusion

Original: We proposed the hierarchical lattice layer (HLL) for partially monotone neural networks as anextension of TL Lattice Gupta et al.
Revised: We proposed the hierarchical lattice layer (HLL) for partially monotone neural networks as anextension of TL Lattice Gupta et al.

Original: We demonstrated that HLL resolved two problems with
Revised: We demonstrated that HLL resolved two problems with

Original: TL Lattice: the requirement for a special training algorithm and the memory consumption.
Revised: TL Lattice: the requirement for a special training algorithm and the memory consumption.

Original: In ourexperiments, we showed that HLL did not sacriﬁce prediction performances to resolve these problems.
Revised: In ourexperiments, we showed that HLL did not sacriﬁce prediction performances to resolve these problems.

Original: Note that our experiments were limited to a comparison between single layers (i.e., HLL and TL
Revised: Note that our experiments were limited to a comparison between single layers (i.e., HLL and TL

Original: Lattice).
Revised: Lattice).

Original: The prediction performance of these models might be able to be improved by combiningmultiple layers (e.g., You et al.
Revised: The prediction performance of these models might be able to be improved by combiningmultiple layers (e.g., You et al.

Original: [2017]) or any other techniques (e.g., changing feature encoding andincreasing the number of layers in MLP) and the memory consumption might also be reduced byusing smaller lattices, but note that the requirement for a special training algorithm remains for TL
Revised: [2017]) or any other techniques (e.g., changing feature encoding andincreasing the number of layers in MLP) and the memory consumption might also be reduced byusing smaller lattices, but note that the requirement for a special training algorithm remains for TL

Original: Lattice.
Revised: Lattice.

Original: A limitation of our HLL is that it cannot be used for the applications with a large m .
Revised: A limitation of our HLL is that it cannot be used for the applications with a large m .

Original: If m is large, weneed to combine small HLL layers to construct a large neural network as demonstrated in Canini etal.
Revised: If m is large, weneed to combine small HLL layers to construct a large neural network as demonstrated in Canini etal.

Original: [2016]; You et al.
Revised: [2016]; You et al.

Original: [2017] for TL Lattice.
Revised: [2017] for TL Lattice.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes]
Revised: For each question, change the default [TODO] to [Yes]

Original: [No] or[N/A] .
Revised: [No] or[N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .•
Revised: [Yes] See Section ?? .•

Original: Did you include the license to the code and datasets?
Revised: Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.•
Revised: [No] The code and the data areproprietary.•

Original: Did you include the license to the code and datasets?
Revised: Did you include the license to the code and datasets?

Original: [N/A]Please do not modify the questions
Revised: [N/A]Please do not modify the questions

Original: and
Revised: and

Original: only
Revised: only

Original: use
Revised: use

Original: the
Revised: the

Original: provided macros
Revised: provided macros

Original: for
Revised: for

Original: your answers.
Revised: your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count
Revised: Checklist section does not count

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] We discussed the limitation ofour work in Sec.
Revised: [Yes] We discussed the limitation ofour work in Sec.

Original: 6.(c)
Revised: 6.(c)

Original: Did you discuss any potential negative societal impacts of your work?
Revised: Did you discuss any potential negative societal impacts of your work?

Original: [No] We believe
Revised: [No] We believe

Original: this paper does not have any negative societal impacts, because we propose a general
Revised: this paper does not have any negative societal impacts, because we propose a general

Original: machine learning technique applicable to any application domain.
Revised: machine learning technique applicable to any application domain.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] We included
Revised: [Yes] We included

Original: the codes used in our experiments in the supplemental material.
Revised: the codes used in our experiments in the supplemental material.

Original: (b) Did you specify all the training details (e
Revised: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how theywere chosen)?
Revised: g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] We described all the training details in Sec. 4.3.
Revised: [Yes] We described all the training details in Sec. 4.3.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: g.,
Revised: g.,

Original: type of GPUs, internal cluster, or cloud provider)?
Revised: type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] We described our computing
Revised: [Yes] We described our computing

Original: environment in Sec. 4.
Revised: environment in Sec. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] We described the
Revised: [Yes] We described the

Original: details of the asset TensorFlow Lattice in Sec.
Revised: details of the asset TensorFlow Lattice in Sec.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] We described the details of the asset
Revised: [Yes] We described the details of the asset

Original: TensorFlow Lattice in Sec.
Revised: TensorFlow Lattice in Sec.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]
Revised: [Yes]

Original: We included the codes used in our experiments in the supplemental material.
Revised: We included the codes used in our experiments in the supplemental material.

Original: (d) Did you discuss whether and how consent was obtained from people whose data
Revised: (d) Did you discuss whether and how consent was obtained from people whose data

Original: you’re using/curating?
Revised: you’re using/curating?

Original: [Yes] We obtained all datasets from the UCI Machine Learning
Revised: [Yes] We obtained all datasets from the UCI Machine Learning

Original: Repository Dua and Graff [2017], and we follow the citation requests by each datasetprovider in Sec. 4.2.
Revised: Repository Dua and Graff [2017], and we follow the citation requests by each datasetprovider in Sec. 4.2.

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [Yes] We conﬁrmed that the datasets we used do not
Revised: [Yes] We conﬁrmed that the datasets we used do not

Original: contain such information in Sec.
Revised: contain such information in Sec.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: g ( p, v 1 ) : 1 − g ( p, v 1 ) g ( p, v 2 ) : 1 − g ( p, v 2 ) g
Revised: g ( p, v 1 ) : 1 − g ( p, v 1 ) g ( p, v 2 ) : 1 − g ( p, v 2 ) g

Original: ( p, v 3 ) : 1 − g ( p, v 3 ) f
Revised: ( p, v 3 ) : 1 − g ( p, v 3 ) f

Original: ( p, v 0 )
Revised: ( p, v 0 )

Original: f ( p, v 1 ) f ( p, v 2 )
Revised: f ( p, v 1 ) f ( p, v 2 )

Original: A Parameterization of HLL for Simple Case
Revised: A Parameterization of HLL for Simple Case

Original: We show an example of the functions l ( p, v ) , u ( p, v ) , and g ( p, v ) for a simple case with m =
Revised: We show an example of the functions l ( p, v ) , u ( p, v ) , and g ( p, v ) for a simple case with m =

Original: 1and k = 3 .
Revised: 1and k = 3 .

Original: In this case, the hypercube has three vertices V m = { v 0 , v 1 , v 2 } , and we use the totalordering σ = ( v 0 , v 1 , v 2 ) here.
Revised: In this case, the hypercube has three vertices V m = { v 0 , v 1 , v 2 } , and we use the totalordering σ = ( v 0 , v 1 , v 2 ) here.

Original: Recall that HLL is constructed to satisfy the monotonicity constraint0 ≤ f ( p, v 0 ) ≤ f ( p, v 1 ) ≤ f ( p, v 2 ) ≤ 1 for any p ∈ [0 , 1] d − 1 .
Revised: Recall that HLL is constructed to satisfy the monotonicity constraint0 ≤ f ( p, v 0 ) ≤ f ( p, v 1 ) ≤ f ( p, v 2 ) ≤ 1 for any p ∈ [0 , 1] d − 1 .

Original: See Figure 5 for an illustration of thedeﬁnitions of g ( p, v 0 ) , g ( p, v 1 ) , and g ( p, v 2 ) .
Revised: See Figure 5 for an illustration of thedeﬁnitions of g ( p, v 0 ) , g ( p, v 1 ) , and g ( p, v 2 ) .

Original: The three functions l ( p, v ) , u ( p, v ) , and g ( p, v ) are deﬁned to satisfy l ( p, v ) ≤ f ( p, v ) ≤ u ( p, v )and 0 ≤ g ( p, v ) ≤ 1 for any p ∈ [0 , 1] d − 1 and any v ∈ V m .
Revised: The three functions l ( p, v ) , u ( p, v ) , and g ( p, v ) are deﬁned to satisfy l ( p, v ) ≤ f ( p, v ) ≤ u ( p, v )and 0 ≤ g ( p, v ) ≤ 1 for any p ∈ [0 , 1] d − 1 and any v ∈ V m .

Original: For the ﬁrst element v 0 of σ , we havel ( p, v 0 )
Revised: For the ﬁrst element v 0 of σ , we havel ( p, v 0 )

Original: = 

Revised: =

Original: and u ( p, v 0 )
Revised: and u ( p, v 0 )

Original: = 

Revised: =

Original: because L v 0 ,σ = U v 0 ,σ = ∅ and g ( p, v 0 )
Revised: because L v 0 ,σ = U v 0 ,σ = ∅ and g ( p, v 0 )

Original: =
Revised: =

Original: f ( p, v 0 ) , which satisﬁesl ( p, v 0 ) ≤ f ( p, v 0 ) ≤ u ( p, v 0 ) and 0 ≤ g ( p, v 0 ) ≤ 1 .
Revised: f ( p, v 0 ) , which satisﬁesl ( p, v 0 ) ≤ f ( p, v 0 ) ≤ u ( p, v 0 ) and 0 ≤ g ( p, v 0 ) ≤ 1 .

Original: For the second element v 1 of σ , we havel ( p, v 1 ) =
Revised: For the second element v 1 of σ , we havel ( p, v 1 ) =

Original: f ( p, v 0 ) and u ( p, v 1 )
Revised: f ( p, v 0 ) and u ( p, v 1 )

Original: = 1 because L v 1 ,σ = { v 0 } and U v 1 ,σ = ∅ .
Revised: = 1 because L v 1 ,σ = { v 0 } and U v 1 ,σ = ∅ .

Original: The function g ( p, v 1 ) isdeﬁned as a function that satisﬁes
Revised: The function g ( p, v 1 ) isdeﬁned as a function that satisﬁes

Original: [Equation]
Revised: [Equation]

Original: Note that this deﬁnition satisﬁes l ( p, v 1 ) ≤ f ( p, v 1 ) ≤ u ( p, v 1 ) and 0 ≤ g ( p, v 1 ) ≤ 1 , which meansthat the monotonicity constraint f ( p, v 0 ) ≤ f ( p, v 1 ) ≤ 1 is satisﬁed.
Revised: Note that this deﬁnition satisﬁes l ( p, v 1 ) ≤ f ( p, v 1 ) ≤ u ( p, v 1 ) and 0 ≤ g ( p, v 1 ) ≤ 1 , which meansthat the monotonicity constraint f ( p, v 0 ) ≤ f ( p, v 1 ) ≤ 1 is satisﬁed.

Original: Similarly, for the third elementv 2 in σ , we have l ( p, v 2 ) =
Revised: Similarly, for the third elementv 2 in σ , we have l ( p, v 2 ) =

Original: f ( p, v 1 ) and u ( p, v 2 )
Revised: f ( p, v 1 ) and u ( p, v 2 )

Original: = 

Revised: =

Original: because L v 2 ,σ = { v 1 } and U v 2 ,σ = ∅ .
Revised: because L v 2 ,σ = { v 1 } and U v 2 ,σ = ∅ .

Original: Thefunction g ( p, v 2 ) is deﬁned as a function that satisﬁes
Revised: Thefunction g ( p, v 2 ) is deﬁned as a function that satisﬁes

Original: [Equation]
Revised: [Equation]

Original: Note that this deﬁnition satisﬁes l ( p, v 2 ) ≤ f ( p, v 2 ) ≤ u ( p, v 2 ) and 0 ≤ g ( p, v 2 ) ≤ 1 , which meansthat the monotonicity constraint f ( p, v 1 ) ≤ f ( p, v 2 ) ≤ 1 is satisﬁed.
Revised: Note that this deﬁnition satisﬁes l ( p, v 2 ) ≤ f ( p, v 2 ) ≤ u ( p, v 2 ) and 0 ≤ g ( p, v 2 ) ≤ 1 , which meansthat the monotonicity constraint f ( p, v 1 ) ≤ f ( p, v 2 ) ≤ 1 is satisﬁed.

Original: B Multilinear Interpolation
Revised: B Multilinear Interpolation

Original: For the completeness of this paper, we brieﬂy describe multilinear interpolation algorithm Gupta et al.[2016] here.
Revised: For the completeness of this paper, we brieﬂy describe multilinear interpolation algorithm Gupta et al.[2016] here.

Original: For simplicity, we assume that we are given a vector q ∈ [0 , 1] m , and the set of verticesthat surround q is the set of vertices of the (undivided) unit hypercube V q
Revised: For simplicity, we assume that we are given a vector q ∈ [0 , 1] m , and the set of verticesthat surround q is the set of vertices of the (undivided) unit hypercube V q

Original: = { v | v ∈ { 0 , 1 } m } .
Revised: = { v | v ∈ { 0 , 1 } m } .

Original: Assuming that we know the value f
Revised: Assuming that we know the value f

Original: ( p, v ) for any v ∈ V q and p ∈ R d − m , we can compute themultilinear interpolation by
Revised: ( p, v ) for any v ∈ V q and p ∈ R d − m , we can compute themultilinear interpolation by

Original: [Equation]
Revised: [Equation]

Original: where q [ i ] and v [ i ] represent the i -th dimension of vectors q and v , respectively.
Revised: where q [ i ] and v [ i ] represent the i -th dimension of vectors q and v , respectively.

Original: Example.
Revised: Example.

Original: We can compute f ( p, q ) for q = (0 . 3 , 0 . 6) by
Revised: We can compute f ( p, q ) for q = (0 . 3 , 0 . 6) by

Original: [Equation]
Revised: [Equation]

Original: when m = 2 and V q
Revised: when m = 2 and V q

Original: = { v (0 , 0) , v (0 , 1) , v (1 , 0) , v (1 , 1) } .
Revised: = { v (0 , 0) , v (0 , 1) , v (1 , 0) , v (1 , 1) } .

Original: C Experiments on Synthetic Datasets
Revised: C Experiments on Synthetic Datasets

Original: [Figure]
Revised: [Figure]

Original: 1D Synthetic Datasets.
Revised: 1D Synthetic Datasets.

Original: We show our experimental results on one-dimensional synthetic datasets
Revised: We show our experimental results on one-dimensional synthetic datasets

Original: D square

Revised: D square

Original: =
Revised: =

Original: { ( x i , y i ) } ni =

Revised: { ( x i , y i ) } ni =

Original: such that
Revised: such that

Original: y
Revised: y

Original: i
Revised: i

Original: =
Revised: =

Original: x 2 i and D sqrt1 = { ( x i , y i ) } ni =1 such that y i = √ x i .
Revised: x 2 i and D sqrt1 = { ( x i , y i ) } ni =1 such that y i = √ x i .

Original: Figures 6 and 7 show the prediction results with various k for these two datasets.
Revised: Figures 6 and 7 show the prediction results with various k for these two datasets.

Original: The blue lines showthe predictions by HLL, and the black lines show the ground truth.
Revised: The blue lines showthe predictions by HLL, and the black lines show the ground truth.

Original: Both ﬁgures show that increasingk yields better prediction results on these synthetic datasets.
Revised: Both ﬁgures show that increasingk yields better prediction results on these synthetic datasets.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 2D Synthetic Datasets.
Revised: 2D Synthetic Datasets.

Original: We show our experimental results on two-dimensional synthetic datasets
Revised: We show our experimental results on two-dimensional synthetic datasets

Original: D square

Revised: D square

Original: = { ( x i , y i ) } ni =1 such that
Revised: = { ( x i , y i ) } ni =1 such that

Original: y i
Revised: y i

Original: =
Revised: =

Original: ( x i [1] 2 + x i [2] 2 ) / 2 and D sqrt

Revised: ( x i [1] 2 + x i [2] 2 ) / 2 and D sqrt

Original: = { ( x i , y i ) } ni =1 such thaty i
Revised: = { ( x i , y i ) } ni =1 such thaty i

Original: =
Revised: =

Original: ( (cid:112) x
Revised: ( (cid:112) x

Original: i [1] + (cid:112) x
Revised: i [1] + (cid:112) x

Original: i [2]) / 2 (recall that the i -th element of vector x is denoted as x [ i ] ).
Revised: i [2]) / 2 (recall that the i -th element of vector x is denoted as x [ i ] ).

Original: Figures 8 and9 show the prediction results of HLL, TL Lattice, and TL RTL with k = 4 against the ground truth.
Revised: Figures 8 and9 show the prediction results of HLL, TL Lattice, and TL RTL with k = 4 against the ground truth.

Original: These prediction models show similar prediction results on these synthetic datasets.
Revised: These prediction models show similar prediction results on these synthetic datasets.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: D Experiments on Training Time
Revised: D Experiments on Training Time

Original: We compared the training times of HLL and TL Lattice on the small datasets.
Revised: We compared the training times of HLL and TL Lattice on the small datasets.

Original: In these experiments,we ﬁxed the hyperparameters; k was set to 2 , the learning rate was set to 0 .
Revised: In these experiments,we ﬁxed the hyperparameters; k was set to 2 , the learning rate was set to 0 .

Original: 001 , and the batch size wasset to 128 .
Revised: 001 , and the batch size wasset to 128 .

Original: The number of epochs was set to 100, and we did not use any early stopping.
Revised: The number of epochs was set to 100, and we did not use any early stopping.

Original: Figure 10shows the results, where we varied the number of neurons in the auxiliary neural network g θ ( p, v )from 16 to 512 for HLL
Revised: Figure 10shows the results, where we varied the number of neurons in the auxiliary neural network g θ ( p, v )from 16 to 512 for HLL

Original: [Figure]
Revised: [Figure]

Original: and the execution times were the averages of ﬁve runs.
Revised: and the execution times were the averages of ﬁve runs.

Original: These results showthat the training times of HLL were shorter than those of TL Lattice for these six small datasets
Revised: These results showthat the training times of HLL were shorter than those of TL Lattice for these six small datasets

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Revised: E Experiments with Calibration Layers

Revised: We conducted additional experiments by using the calibration layers [You et al. , 2017].

Revised: Morespeciﬁcally, we added the calibration layer for each dimension of the input x before the input of thelattice layer.

Revised: The number of the end points in each calibration layer is set to 100.

Revised: We used both themultilinear interpolation and the simplex interpolation for TL Lattice.

Revised: Figure 11 shows the results.

Revised: These results show that the prediction performance of TL Lattice with thecalibration layers are improved for the three datasets (a) Energy Efﬁciency, (c) Concrete Compressive

Revised: Strength, and (f) Shill Bidding compared with the results shown in Figure 3.

Revised: However, they are stillnot better than HL Lattice.

Revised: We also note that there are no clear advantage of the simplex interpolationcompared to the multilinear interpolation with respect to MSEs.

Revised: [Figure]

Revised: the neural network models with various k on small datasets.