## 3 Implementation and Experimentation
### Parameters' Selection  
It is not intuitive what the most efficient network parameters would be for using ConvNet. It requires extensive testing and artistry to design the optimum layers' structure. There is no pre-defined solution to select ideal values of parameters in different layers. Thus, multiple combinations of settings were attempted with different parameters for tuning and adjusting the settings according to each dataset.  
Table 5 lists the different layers and their parameters, the corresponding filters and their sizes, as well as the division of data for our proposed architecture.  
The output of ConvNet is highly sensitive to fine tuning of the parameters. The parameters like batch size, number of iterations, splitting of data and filter size dramatically affect the final results. A brief explanation of a few of the parameters is provided below:  
* Epochs: The epoch refers to passing of an entire set of data through the complete architecture. At each iteration, there is random shuffling of training data. When it classifies the image wrongly, it is counted as a loss, and through back-propagation, the weights of different layers are updated, which eventually decreases the loss and error rate. The epochs are set to 40 in our case.
* Learning rate: This is considered to be a very significant and critical parameter of the network and is modified whenever the input or layer structure is modified. The learning rate is used to  
Figure 5: Down-sampling and normalization of MR images.  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline
**Layer** & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 & 12 & 13 \\
**Type** & conv & ReLU & model & conv & ReLU & model & conv & conv & conv & model & fc & fc & softmax \\
**Support** & 20 & 1 & 3 & 5 & 1 & 3 & 3 & 3 & 3 & 3 & 1 & 1 & 1 \\
**Filter dim** & 1 & n/a & n/a & 48 & n/a & n/a & 192 & 192 & 128 & n/a & n/a & n/a & n/a \\
**No. of filters** & 96 & n/a & n/a & 256 & n/a & n/a & 384 & 384 & 256 & n/a & 4096 & 4096 & 3 \\
**Stride** & 1 & n/a & 3 & 1 & n/a & 3 & 1 & 1 & 3 & 1 & 1 & 1 & 1 \\
**Data Size** & 160 & 55 & 55 & 27 & 27 & 13 & 13 & 13 & 1 & 1 & 1 \\
**Data Depth** & 48 & 48 & 48 & 128 & 128 & 128 & 192 & 192 & 128 & 4096 & 4096 & 1 \\ \hline \hline \end{tabular}
\end{table}
Table 5: The parameter design of our Convolutional Neural Network (ConvNet).  
determine the size of the gradient taken by the network weights. If your learning rate is small, it means you need more epochs to acquire the best trained model. In our case, the learning rate was fixed to 0.01 since this value was optimal for our dataset.
* Weight decay: This is a parameter required for the weight update rule, which is useful when there is no scheduled update, so it causes the weight to decay exponentially. The weight decay value is set to 0.0005 in our proposed network.
* Batch size: This characterizes the number of samples that will be propagated along the ConvNet. In our case, the batch size was fixed to 50.
* Dropout: This is a regularization technique where randomly-selected neurons drop out during training. It results in multiple independent internal representations being learned by the network. Hence, the network prevents over-fitting on the training data and is capable of improved generalization. In the proposed network, the dropout layer is added after each of the fully-connected layers with the dropout rate set to 25%. This rate means 1 in 4 inputs will be randomly excluded from each update cycle.