Paper:
Artificial neural networks are built on the basic operation of linear combination and non-linear activation function. Theoretically this structure can approximate any continuous function with three layer architecture. But in practice learning the parameters of such network can be hard. Also the choice of activation function can greatly impact the performance of the network. In this paper we are proposing to replace the basic linear combination operation with non-linear operations that do away with the need of additional non-linear activation function. To this end we are proposing the use of elementary morphological operations (dilation and erosion) as the basic operation in neurons. We show that these networks (Denoted as DenMo-Net) with morphological operations can approximate any smooth function requiring less number of parameters than what is necessary for normal neural networks. The results show that our network perform favorably when compared with similar structured network.

Review:
The authors introduce Morph-Net, a single layer neural network where
the mapping is performed using morphological dilation and erosion.
I was expecting something applied to convolutional networks as such operators
are very popular in image processing, so the naming is a bit misleading.
It is shown that the proposed network can approximate any smooth function, 
assuming a sufficiently large number of hidden neurons, that is a nice result.
Clarity should be improved, for example it is mentioned that the structuring
element is learned but never clearly explained how and what difficulties it poses.
In the main text it is written that alpha is {-1, 1}, which would result in a
combinatorial search, but never explained how it is learned in practice.
This is shown only in the appendix but it is not clear to me that using a binarization
with the weights is not prone to degenerate solutions and/or to learn at all
if proper initialization is not used.
Did the authors experiment with smooth versions or other form of binarization with
straight-through estimator or sampling?
In the proof for theorem 1 it is not clear if the convergence of the proposed
network is faster or slower than that of a classic single layer network.
The main result of the paper is that the structuring element can be learned,
but there is no discussion on what it is learned. Also, there is no comparison
on related approaches that try to learn the structuring element in an end-to-end
fashion such as [1].
Experiments lack a more thorough comparison with state-of-the-art and at least
an ablation study to show that the proposed approach is effective and has merit.
For example, what is the relative contribution of using dilation and erosion
jointly versus either one of them.
What is the comparison with a winner-take-all unit over groups of neurons
such as max-pooling?
It seems that extending the work to multiple layers should be trivial but it is
not reported and is left to future investigations. This hints at issues with
the optimization and should be discussed, is it related to the binarization
mentioned above?
Overall the idea is interesting but the way the structuring element is learned
should be discussed in more details and exemplified visually. Experiments need
to be improved and overall applicability is uncertain at this stage.
=======
[1] Masci et al., A Learning Framework for Morphological Operators Using Counter--Harmonic Mean.