# In-Motion, Non-Contact Detection of Ties and Ballasts on Railroad Tracks
## 5 Analysis
### Machine Learning Approaches  
Another approach for detecting and learning data patterns and better differentiating between ties and ballasts is machine learning (ML). ML models can generally handle highly uncertain tasks, making them well-suited for this task. Due to the high capability of ML models in handling large datasets and automating decision-making processes, the newly developed models are expected to address the challenges previously associated with using the moving standard deviation. With the extensive amount of data collected through our experiments, these models can learn complex patterns and adapt to various conditions, improving the accuracy and reliability of tie and ballast differentiation. The desired ML model will receive the LiDAR data as input and return a classification of tie or ballast as output (prediction). Like the moving standard deviation method, a moving window is used to process the data at each location. However, unlike the moving standard deviation, which uses a spatial window, this method defines the window size based on the number of surrounding data points required to classify an individual point as tie or ballast. The window size is a tunable parameter that needs to be optimized to achieve the best results. To select the required window size for our model, we evaluated the accuracy that can be achieved for each method for various window sizes, ranging from small to large. This task considers two standard classification \"algorithms\": Logistic Regression and Decision Tree. Through a series of initial tests, these two algorithms were selected from a larger pool of classification algorithms, including Support Vector Machines (SVMs), k-Nearest Neighbors (k-NN), and neural networks. Logistic Regression and Decision Tree were chosen because they belong to two distinct families--linear and non-linear, respectively--and demonstrated higher accuracy [19]. Logistic Regression uses a linear function to model the relationship between features (i.e., inputs) and a binary outcome. It then applies a sigmoid function to convert this linear output into a 0 or 1 probability, indicating the likelihood of belonging to a specific class [20]. A \"0\" probability suggests a high likelihood of ballast, and \"1\" indicates a tie. Equation (3) shows the formula associated with Logistic Regression, where \(b_{0}+b_{1}x\) is the linear function for separating the classes (i.e., ties and ballasts). \(b_{0}\) and \(b_{1}\) are the parameters that will be learned through the training process. \[p=\frac{1}{1+e^{-(b_{0}+b_{1}x)}} \tag{3}\]  
In contrast, as [PERSON] et al. (1984) described, the Decision Tree creates a tree-like structure, where each internal node splits the data based on a chosen feature value. This process continues recursively until the data at each leaf node belong predominantly to a single class [21]. These algorithms differ in their interpretability. Logistic Regression provides a single equation representing the entire model, making it relatively straightforward to understand the pattern of each input. On the other hand, a Decision Tree offers a series of branching rules that often provide a more intuitive understanding of how decisions are made, as the tree visually represents the decision process. Both Logistic Regression and Decision Tree are supervised learning algorithms, meaning they require training with input data and their respective output labels. In other words, the models need ground-truth labeled data to learn the patterns associated with each label (tie or ballast) and effectively differentiate ties from ballasts. Once trained, these models can predict future unseen data. A small portion of each dataset (approximately 20 percent) is manually labeled as either tie or ballast (i.e., 1 or 0) to construct a ground-truth \"labeled dataset\" to facilitate this. A portion of the labeled dataset (approximately 60 percent), called here the \"training dataset\", is used to train the models.