## 3 Experimental Results
### Parameter Settings  
The parameters for the model are listed as the overlapped patch size 32 \(\times\) 32 pixel. The PaDBN model has two hidden layers. The numbers of nodes in each layer are 100 and 100, respectively. Two nodes in the output layer enable the model to select the two most relevant features to represent each patch. Thus, the deep learning features have two dimensions. To select a deep network structure, we used a random search [53] to select the structure for best performance. First, we tried to identify the range of the hyperparameters and after that, we selected the values randomly. Then, we trained our model with these selected values and we repeated this procedure until getting the best performance. The L2 regularization was set to 0.00001. The learning rate for pretraining was 0.05 and that for fine-tuning was 0.008. The momentum for pretraining was 0.5 and that for fine-tuning was 0.9. Same mini batch size 128 is used to achieve efficiency during pretraining and fine-tuning. We used 50 and 3000 epochs in pretraining and fine-tuning, respectively. The parameter combination scheme that achieves the top performance of our proposed method is shown in Table 1.  
The learning curve of the PaDBN model is shown in Figure 6. A normalized training MSE of 0.026 and a normalized validation MSE of 0.032 were obtained after training. The learning curve shows high divergence before 500 epochs and becomes stable after 1700 epochs in the left side of Figure 6 (top left), but the (top right) graph shows the overfitting problem because the validation curve considerably deviates from the training curve after 500 epochs. The correct training curve, as shown in the (top left) graph, is obtained using the heuristic approach. Figure 6 (bottom) shows the poor training MSE graph with high learning rate (bottom left) and low learning rate (bottom right). Thus, weight initialization is important in deep learning. Figure 7 illustrates a typical feature representation of first and second hidden layers learned by two-RBM layer PaDBN model. Each square shows the weight between one hidden node and the pixel of the original image at the same position. The gray pixel indicates zero in the weight matrix, and white pixels denote positive values. The left side of Figure 7 shows the 100 node RBM1, amplified images show its weight visualization, and the right side of Figure 7 shows the weight visualization of 100 node RBM2. The segmentation results are shown in Figures 8-10 on different test axial images. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Pretraining** & \multicolumn{3}{c}{**Fine-Tuning**} \\ \hline Hidden Layers & 2 & Scaling learning rate & 1 \\ No. of neurons in each layer & 100-100 & No. of epochs & 3000 \\ No. of epochs & 50 & Learning rate & 0.008 \\ Learning rate & 0.05 & Momentum & 0.9 \\ Momentum & 0.5 & Mini batch size & 128 \\ Mini batch size & 128 & weightPenaltyL2 & 0.00001 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Parameters combination scheme. Figure 8: Segmentation results from randomly selected axial slices of test CT image 1: (**a**) raw image; (**b**) initial segmentation; (**c**) final segmentation; (**d**) ground truth; and (**e**) segmentation superimposed on CT image. Figure 7: **Left side**: The visualization of 100 weights in the first layer of the deep belief network (DBN), restricted Boltzmann machines 1 (RBM1), and amplified images are some example of weights. **Right side**: The visualization of 100 weights in the second layer of DBN, RBM2. Figure 10: Segmentation results from randomly selected axial slices of test CT image 3: (**a**) raw image; (**b**) initial segmentation; (**c**) finial segmentation; (**d**) ground truth; and (**e**) segmentation superimposed on CT image. Figure 9: Segmentation results from randomly selected axial slices of test CT image 2: (**a**) raw image; (**b**) initial segmentation; (**c**) finial segmentation; (**d**) ground truth; and (**e**) segmentation superimposed on CT image.