# Spinal Cord Segmentation in Ultrasound Medical Imagery
## 3 Proposed Method
### CNN: Convolutional Neural Network  
If we have data that contains N training samples \(\{(x_{i},y_{i})\}_{i=1}^{N}\), where \(x\) represents the annotated input, \(y\) represents the label, Convolutional neural network (CNN) is able to construct a model \(F\) that maps the relationship between the input data \(x\) and the output data \(y\). The CNN is built by stacking a series of layers that perform operations like convolution using a kernel function, non-linear activation, and max pooling. A training process of the model is used to give this built CNN model the set of parameters that best fits this mapping relationship with the minimal error. The training process includes five steps. The first step makes the initialization of the parameters and weights of CNN with random values. The second step is the forward propagation phase. During this step, a training sample \((x_{i},y_{i})\) is passed to the network, \(x_{i}\) is transferred from the input layer to the output layer. Finally, we get the output \(o_{i}\), which is formulated as:  
\[o_{i}=F_{L}(...)F_{2}(F_{1}(x_{i}w_{1})w_{2})...)w_{L}), \tag{1}\]  
\(L\) is the number of layers, \(w_{L}\) is the weight vector of the \(L\)th layer \(F_{L}\).  
The third step consists of estimating the loss function or the cost function, which is the function that calculates the error margin between the resulting output \(o_{i}\) and the correct output value \(y_{i}\). The fourth step will make the correction of the weight vector \(w_{1}\), \(w_{2}\), \(w_{3}\), \(w_{L}\) to minimize this loss function following this optimization problem:  
\[argmin_{w_{1},w_{2},...,w_{L}}\frac{1}{N}\sum_{i=1}^{N}l(o_{i},y_{i}), \tag{2}\]  
where \(l\) is the loss function. Usually, like here in our work, the cross-entropy loss function is used, as we made in the training phase of the model we adopted in this paper. In fact, we used a weighted version of cross-entropy to alleviate the imbalance between the class Spinal Cord and the class Other. To solve the numerical optimization problem, we use back-propagation and stochastic gradient descent methods. After this step, a more adequate set of parameters and weights is given to our model. In the fifth step, we will repeat the second, third, and fourth steps through all of the training data: \(\{(x_{i},y_{i})\}_{i=1}^{N}\). Usually, this training will end by converging the loss function into a small value. This convergence is assured especially when we use a state of the art architecture like the architectures that we tested in this paper. Passing the full training data one time through the CNN network is named one epoch. Training CNNs usually involves running multiple epochs. Many techniques are used to make the cost function decrease faster. First, the Batch normalization technique [24] introduced in 2015 is becoming a state of the art method used, like we used here in our work, for normalizing the output of convolutional layers and fully connected layer before applying the non-linear activation function. This has a significant effect on making the loss function converge faster. Also, optimizers are also used to make the loss function converge faster. We used in our experiments the ADAM (Adaptive Moment Estimation) [25] introduced in 2015 as it is becoming a state of the art gradient descent optimizer.