## 3 Algorithm Overview
ReLU is fast, non-saturating and shows very impressive results. However, in some situations, where it is not possible for ReLUs to saturate, it will turn units \"dead\" consequently. Moreover, they will never be activated because of the negative pre-activation value. In this situation, no gradient will flow through the network. Furthermore, ReLU is always non-negative; this means activation is always positive. The solution is ELU [49]; for positive values, it acts like ReLU, but in the case of negative values, it is a function bounded by a fixed value \(-1\). In this way, it helps to push the neuron's mean-activation closer to zero. It is quite beneficial for learning, and it facilitates learning for representations, which are more robust to noise. As a result, our proposed network will attain better speed-up learning via avoiding a bias shift. Furthermore, [PERSON] et al. [48] proposed dropout units that will offer us a reduced chance of overfitting to the training data in comparison to SegNet. In fact, the proposed network will become less sensitive to the specific neuron's weights. Consequently, the network will be able to better generalize and less likely to be overfitted. The proposed final version for segmentation network is initiated with a class-specific activation map \(\mathbf{g}_{i}^{l}\) with a given input image \(\mathbf{x}_{i}\). The input image is extracted from input layers and processed by convolution and pooling layers. Finally, from the decoder unit, it is passed to the class-specific segmentation map \(M(\mathbf{g}_{i}^{l};\theta_{s})\) after applying the softmax function (where \(\theta_{s}\) is the model parameter of the segmentation network). Here, \(M(\mathbf{g}_{i}^{l};\theta_{s})\) has a background and foreground channels, which are denoted by \(M_{b}(\mathbf{g}_{i}^{l};\theta_{s})\) and \(M_{f}(\mathbf{g}_{i}^{l};\theta_{s})\), respectively. Later, the segmentation job is formulated as the per-pixel regression to the ground-truth segmentation, which minimizes:

\[\min_{\theta_{s}}\sum_{i}e_{s}(\mathbf{z}_{i}^{l},M(\mathbf{g}_{i}^{l};\theta_{ s})), \tag{3}\]

where \(\mathbf{z}_{i}^{l}\) is taken as the binary ground-truth segmentation mask for category \(l\) of the \(i\)-th image \(\mathbf{x}_{i}\) and \(e_{s}(\mathbf{z}_{i},M(\mathbf{g}_{i}^{l};\theta_{s})\) is the segmentation loss of \(M_{f}(\mathbf{g}_{i}^{l};\theta_{s})\), or equivalently, the segmentation loss of \(M_{b}(\mathbf{g}_{i}^{l};\theta_{s})\), with respect to \(\mathbf{z}_{i}^{l}\). A channel-shared variant is considered as \(f(yi)=\max(0,yi)+a\min(0,yi)\), where the coefficient is shared by all processing channels of the specific layer. In this way, this variant simply introduces a single extra parameter into every layer [56]. ### Training

The Caffe Berkeley Vision library [57] is used to develop and train the whole network. Caffe offers a great deal of freedom to write network layers and train the network according to the proposed requirements. Therefore, the network is trained once it converges, and no significant reduction in training loss is observed. Finally, the overall results are assessed, analyzed and later compared to the established benchmark results. #### 5.2.1 Datasets

We have used two well-known segmentation datasets (CamVid, PASCAL VOC-12) for overall experimentation. The aforementioned CamVid [37] dataset is used for overall training and testing of the proposed CNN architecture. It is composed of road scenes (day and dusk scenes) RGB images of a \(360\times 480\) resolution. Our proposed network is designed to classify and segment 11 classes (e.g., cars, people, roads, buildings, poles, signs, trees, etc.). Local contrast normalization [58] is performed for each input image prior to training the network. There are testing, training and validation datasets for the overall CNN assessment. The PASCAL VOC-12 segmentation dataset [7] is also employed for overall CNN training and testing. Pascal VOC-12 is an RGB dataset for segmentation. It contains training and validation images of indoor and outdoor scenes.