# Deep Machine Learning for Path Length Characterization Using Acoustic Diffraction
## 2 Materials and Methods
### Convolutional Neural Networks  
#### 2.3.1 Background  
The Convolutional Neural Network (CNN) architecture builds on its predecessor, Artificial Neural Network (ANN), to maintain spatial relationships which are typically seen in image data. As with ANNs, CNNs are made up of layers of neurons that use non-linear functions to optimize their weights, thus learning relationships within the data [24]. Unlike ANNs, CNNs are able to take in images in a three-dimensional representation, which allows the network to maintain the relationship between rows and columns of data. Because of this property of CNNs, they are typically used in image processing and pattern recognition  
Figure 8: Pairs of SDFTs are shown for several samples from the database representing changes in the transfer function amplitude (**left**) and transfer function phase (**right**). Figure 9: Reduced CSV of LSTM data shows how phase and amplitude are arranged for LSTM to easily consider each lateral step as a subsequent sample. problems, although they may be applicable to any problem which has a clear relationship between two dimensions of data [24; 25]. For instance, in 2020 [PERSON] and [PERSON] applied a CNN to non-image-based breast cancer data. They did this by transforming the data into bar graphs, distance matrices, and normalized numeric values and feeding these images or 2-D arrays into the CNN as the input data. The authors achieved 94-100% accuracy on the breast cancer dataset, proving even non-image-based data can be successfully analyzed using CNNs [25]. The general components of a CNN model are convolution layers, pooling layers, and fully connected layers [26]. The two-dimensional data are passed to the CNN's input layer where it goes through these components and finally outputs the predicted values using features extracted along the way. The first layer through which the data travel is the convolution layer. This layer determines the output through the activation of regions of the input data using the scalar product between the weights of the neurons and the region [24]. The weight vector is commonly referred to as a filter or kernel. It is applied in a sliding fashion so that the scalar product is calculated between the weight vector and all input components [26]. The typical activation function used at this layer is the rectified linear units (ReLU) function. This one is commonly selected as it corrects negative values to a value of 0, which leads to the removal of these weights. This is evident from the ReLU formula in Equation (8). This operation extracts a specified number of features from the input data [26]. \[ReLU(x)= \begin{cases}0&\text{if }x<0\\ x&\text{if }x>=0\end{cases}  \tag{8}\]  
After leaving a convolution layer, the extracted features are typically sent to a pooling layer. At this point in the model, the location of the extracted features is no longer significant [26]. As such, down-sampling can be performed to reduce the number of features the model must keep up with [24]. Additionally, performing max-pooling helps to increase generalizations in the learned features by providing an abstract representation of the features. Few pooling techniques exist, but the most common are average pooling and max pooling. These differ in the way the resulting value is calculated to represent the contributing data components. Max-pooling represents the data in the pooling window with the maximum of all values while average pooling uses the average of the contributing values [26]. Finally, after some number of convolution and pooling layers, the resulting matrices are flattened into vectors and passed to fully connected layers. These layers perform the same duties as ANNs to make decisions about the data [24]. In the fully connected layers, the dot product of the weight vector and input vector, flattened features, is computed. This operation can be optimized using various batch sizes and optimizers [26]. One powerful optimizer is the Adaptive Moment Estimation (ADAM) optimizer [27].