Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors propose a simple scheme based on the convex combination of a function and its point reflection in order to produce a monotonic building block from which input monotonic (or partially input monotonic) neural networks can be built.  The construction is validated both theoretically and experimentally.
strengths_and_weaknesses: Originality:  There is quite a bit of work on monotonicity in neural networks, and at least to my knowledge, the authors seem to do a good job of positioning their work.  While point reflections have been used in the past, their application here does appear to be novel.

Quality:  I didn't find any serious errors with the technical content.  However, I think that the experiments fall a bit on the short end:  the number of data sets is limited and they are somewhat smallish and the performance improvements in many cases are either not statistically significant or too small to get really excited about.  As a result it feels like a more robust set of experiments, including on problems where monotonicity isn't even required, would significantly bolster the case for acceptance.  Additionally a careful critique of the competing methods in the experiments should be done to emphasize the strengths and weaknesses of the present approach.

Clarity:  I found the article easy to read for the most part.  Some of the definitions could be written more clearly with less fluff, e.g., in Definition 1 use R^n, etc. to simplify the presentation, and there are a few minor typos here and there, e.g., line 268.  I also feel like way too much of the main text is devoted to background work (some of which is repeated way too much -- I got the point the first time).  I'd prefer more experiments to supplant it.

Significance:  The construction is simple and the resulting networks are seemingly easy to train.  As a result, I could see this work being of practical significance.  
questions: See strengths + weaknesses.
limitations: Seem to be adequately addressed.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a differentiable neural network layer which is monotonic in its input by construction. It is defined as a convex combination of a convex and a concave monotonic functions on the input. The layer is simple, both conceptually and to implement, and may be useful to reduce parameter count when the modeler has prior knowledge that the function to be learned is monotonic. This layer is also computationally efficient: it has the same amount of computation as a typical feed forward layer followed by a nonlinearity, up to a constant factor.
After describing the layer, the authors provide empirical validation on some tasks, showing that models using the proposed layer achieve better accuracy with smaller parameter count vs monotonic layers in the literature.
strengths_and_weaknesses: Strengths:
Constraining a model to be input monotonic is an important problem in ML. Monotonicity can stem from either domain knowledge (price of a house should be increasing wrt surface all else fixed), or functional constraints (estimating a cdf).
The proposed method is simple, which is a strength. Although the specific shape of the layer seems to have been explored by previous works, it has not been used in the specific context of monotonicity. 
The paper is well written and clear, although perhaps more space could be used for empirical results, and discussing settings in ML where monotonicity is important.
The context of the work is clear.

Main criticism:
The paper does not provide an approximation theoretical result. Are there increasing functions that cannot be approximated by the proposed method?
questions: - The authors discuss convex monotonic networks [1]. On the other hand, they do not discuss Input convex neural networks [2]. Taking the derivative of an input convex neural network wrt input would also yield an nondecreasing function. How would this compare to the proposed model?

[1] Xingchao Liu, Xing Han, Na Zhang, and Qiang Liu. Certified monotonic neural networks. NeurIPS 2020

[2] Amos, B., Xu, L., & Kolter, J.Z. (2017). Input Convex Neural Networks. ArXiv, abs/1609.07152.

limitations: The paper proposes a class of parametric functions to approximate monotonic functions. It does not provide a guarantee that this class can approximate any monotonic function.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: In the current paper, the authors propose a deep neural network that produces monotonic functions. It is easy to produce a convex function (constrain the weight to non-negative values) but it is not trivial to get monotonic function.   To that end, the author introduce a so called Monotonic Dense Unit that combine convex and concave activation functions in order to output a provably monotonic function.                                                                   The authors then justify their approach on 3 benchmarks.
strengths_and_weaknesses: # Strengths

Modeling functions with specific constraints such as monotonicity or convexity is an interesting and broad topic of ML with many application (despite the thoughts of the authors l139). The proposed method is sounds and provably outputs monotonic function in selected components.

# Weaknesses

The paper lacks proper definition about what are increasing and decreasing features and how do they interact with each other.                                                                                                               The propose method is very incremental and the paper lacks of a proper develop theory. The author conjecture l189 the universality of their approach but to not provide elements. Also, the PMU seems a bit over-complex and tend to look like an engineer approach.       
                                                                                                                                                                                                                                                                                                                                                                                               * Global style issues

The paper is written in a somehow bad style. Nothing is properly defined (the core of the paper is "defined" in the caption of figure 2), the text is full of pleonasms such as "monotonic increasing functions" and often contradicts one paragraph with another (e.g. l36 claims it is not trivial, l49 asserts it is).                                                                                                                                                               A lot of points are repeated over and over, sometimes with useless information (e.g. the name of bozillions activations function from the literature) and it often feels like the author just wanted to fill 9 pages. This gives room to properly introduce what is the code of the paper and all the needed notions.                                                                                                                                                                   Whether an equation is a definition or a proposition is often unclear (e.g. l237 seems to be a proposition (because of 'then') but is in fact the definition).                                                                              Two names are proposed for the global algorithm "Monotonic Dense Unit" or "Constrained Monotone Fully Connected Layer" (figure 1, l.219-220) none of those being really used in the later. Definition 3 is yet another "Monotone Constrained Fully Connected Layer".

* Ethics

And seriously what the actual heck is the following (l29-33):                                                                                                                                                                               "All other things being equal, a larger house should be deemed more valuable, a bank's client with higher income should be eligible for a larger loan, an offender with a longer crime history should be predicted as more likely to commit another crime, etc. A model without such a monotonic property would not, and certainly should not, be trusted by the society to provide a basis for such important decisions."                                                              This has NOTHING to do in an ML paper.                                                                                                                                                                                                                                                                                                                                                                                                                                                  The use of COMPAS dataset should be ethically discussed.    

* More precisely among much more

How to "constrain" the weights is never properly said (only in the introduction, I guess the author mean that the weights are >=0 elsewhere too).

Definition 1: h is defined twice in an inconsistent manner (let h be [...], h=...).

Section 2.4: the question of universality is indeed interesting but this section is somehow very weak if the authors have nothing to say on this topic.

Figure 2: why are W "*kernel* weight"

I do not understand Figure 4 and the explanation (l280-288) is really confusing.     
                                                                                                                                                                                                                                                                                                                                                                                                   Eq (1): the case = 0 is ill-defined

* Global remark

A major limitation of the proposed approach is that the assumed monotonic components of the input are dealt in an independent manner.

* Experiments:
    - the experimental protocol is insufficiently described;                                                                                                                                                                                    
    - the networks used are never properly described;                                                                                                                                                                                                   
    - dataset are not described well enough, and the hypothesis (monotonicity) never discussed;                                                                                                                                                 
    - not enough experiments, I would also have expected some ablation studies and hand-crafted problems;                                                                                                                                       
    - the use of COMPAS dataset must trigger some ethical concerns that are ignored in the paper;                                                                                                                                               
    - the choice of datasets and the chosen "monotonic" target features should be accurately discussed;                                                                                                                                         
    - the proposed model should at least be compared with "constrained NN" in order to assert the improvement of the proposed algorithm over naivety.

* Typos

Punctuation is missing in many if not most equations.

References: many if not most references are incomplete (arxiv instead of peer reviewed journal/conferences)                                                                                                                                                  
questions: How do you compute the vector t (monotonicity indicator) in practice?

What is the advantage of your method over constrained networks? Do you have examples that highlights the benefits of Monotonic Dense Unit?

As mentioned, the question of universality is particularly interesting, could you describe the reach of your algorithm in term of descriptive power?

What would prevent to use any other concave decreasing activation function instead of the point symmetric?
limitations: The author must discuss their usage of COMPAS dataset, as well as the comment l29-l33 as already stated in the main review.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors proposed a constrained monotonic dense layer architecture that is compositable to form a monotonic neural network which can approximate non-convex function and has strong empirical results.
strengths_and_weaknesses: Strengths:

1. The proposed monotonic dense layer employs simple, off-shelf components in today's deep learning frameworks, making it very easy to use;
2. The design of the monotonic layer allows composition, making it extensible to deep neural networks;
3. Strong empirical results (models with fewer parameters and better quality).


Weakness:

1. Several definitions are missing from the writing: in section 3.1., definition of a partially monotonic function and a compact segment should be given for clarity.

2. If I understand correctly, in the composed monotonic model with both monotonic and non-monotonic inputs, the layers after the concatenation layer can only be monotonic dense layers. This seems to pose a limitation on stacking multiple layers, such as a stack of transformer blocks, or a stack of convolutional layers.
questions: 
1. On the limitation of the stacking multiple layers, would it be possible to adapt this method to build monotonic layers other than dense layers (such as self-attention or convolutional layers)?

2. In the experiment section, grid search is performed to find the best hyper-parameters for the current method. Would it be possible to perform a similar search for one of the other methods in Table 1 to verify the advantage of the new architecture?
limitations: NA
review_title: null

Meta-review: metareview: There are some ethical issues about using COMPAS dataset in the paper. The ethics reviewers recommend that 
1) Given the limitations of the COMPAS dataset, the authors could just use another dataset with the monotonicity constraint property. There are a lot of datasets out there with this property. But the authors should discuss the issues with the COMPAS data in greater detail if they want to keep the results in there.
2) The changes below could be implemented in the current version of the paper:
- The quoted sentence from the introduction should be removed and replaced with specific, evidence backed statements about why monotonicity is a goal without making value judgements and stating them as fact. 
- The checklist answers could be updated as described above. 
- How monotonic vs not features are determined could be added. 

The reviewer also suggests the authors do more benchmark and revise the mathematical writing since it makes the readers hard to follow. Although the paper is in borderline, most of the reviewers do not really support the publication of the paper. Therefore, I encourage the authors to take the comments and suggestions from the reviewers and improve it in order to resubmit to the next conference. 

confidence: Less certain
Decision: Reject