# A Machine Learning Approach to Determine Oyster Vessel Behavior
## 4 Materials and Methods
_Support Vector Machine Classifier_: We employ a radial basis function (RBF) kernel support vector machine (SVM) as our classification technique. An SVM uses maximum-margin affine hyperplanes to separate data into distinct groupings, allowing it to predict the behavior of new data based on which grouping that data falls into. It penalizes the instances on the wrong side of the decision boundary using a penalty term or, cost parameter, C. In addition to the cost parameter C, RBF kernel SVM possess an attribute, gamma, which affects the area of influence of each training instance. SVM has been found to perform better for binary classification problems. However, in cases where more than two classes are being considered, an SVM can divide a problem into a series of two-class problems. There are several approaches to this binary classification system, including one vs. all and one vs. one comparison. In our implementation, we applied one-vs-one classification technique for multi-class classification which is the default implementation for the SVC library [16]. In one vs. one case, a classifier is made for each combination of two classes. Each classifier casts a vote as to which class the new input belongs and the class with the most votes wins [7]. _Grid Search to Find the Optimal Parameters_: The RBF kernel parameter, \(\gamma\) and the cost parameter, C were optimized to achieve best accuracy using a method called grid searching. In grid search, an array of parameter values is explored to determine those that produce the best accuracy on a validation dataset. There can be multiple parameters, such as when using an RBF SVM, the parameters are grouped together in every possible combination. For example, if there are values C = (_a,b_) and \(\gamma\) = (_x,y_), a grid search would produce the sets (_a,x_), (_a,y_), (_b,x_) and (_b,y_). These sets would be tested with three-fold cross-validation (CV) on a subset of the available data [16]. Because each set is independent of the others, this process is easy to run in parallel, which is especially efficient as the exhaustive searching of values is time-consuming. After testing all combinations, the algorithm returns the set that produced the greatest accuracy. The best values of the parameters found are, C = 10 K and \(\gamma\) = 1000 and used as the representative parameter values for the 10-fold CV of RBF kernel SVM for the entire dataset. _Cross-Validation (CV)_: Estimating the prediction error of the classifier was done by using 10-fold CV, a testing method that is both simple and widely used [7, 8, 9, 15]. CV is performed in folds, where the data is segmented into \(K\) parts, which are each of about equal size. When a fold is set aside for validation, the other (\(K\)\(-\)1) folds are used to train the classifier. This process repeats until each fold has been set aside once for validation and then the \(K\) estimates of error are combined to find the average [7]. In our data set, there are five distinct behaviors to consider: unknown, docked, poling, fishing and traveling. Points with unknown behavior are not inputted into our validator at this stage, since we do not want the validator to produce unknown points. Of the other four behaviors, we have an abundance of fishing and traveling points, a smaller number of poling points and comparably few docked points. There were concerns that due to the comparably large number of poling points in comparison to docked points, our classifier would discriminate against docked points and label them as poling, since the ranges of their behaviors were so similar. To counteract this unwanted behavior, when the folds are prepared, only paths that have at least a single docked point are gathered. Docked points are still a minority, but now they make up a larger portion of the data and are more likely to be correctly classified. However, if the folds happened to be grouped in such a way that all the docked points ended up in a single fold, it could greatly impact the classifier's ability to correctly predict docked behavior.