# A Framework for the Detection of Search and Rescue Patterns Using Shapelet Classification
## 4 Experimental Evaluation & Results  
The unsupervised approaches that already exist in the literature, show that using some simplistic rules [2] that seek for repetitive changes in speed and orientation or using a spatial clustering of important trajectory points [13] is enough for distinguishing between a normal movement and a potential SAR maneuver. However, when we need to detect the type of maneuver and decide whether it is a Search one (e.g., a step-up pattern) or a Rescue one (e.g., a spiral) it is necessary to classify the maneuver to the nearest document pattern. Since there is no previous work that focuses on this level of detail and there is no manually annotated dataset that contains information about the type of maneuvers performed be the vessels, we evaluated our approach mainly on synthetic data and we demonstrate its use of a real dataset that comprises trajectories of vessels that have been engaged in SAR missions.  
In our experiments, we worked on three different scenarios:  
* Optimization scenarios on synthetic data of a single pattern. At this type of scenarios, the best accuracy score it's evaluated for the cross validation of test data with different settings of trajectory generator and GENDIS module. There are 8 different scenarios.
* Binary classification scenarios on synthetic data: For these scenarios the best values from the above optimization have been used and different binary classification tasks (one per SAR pattern) have been evaluated. The classes are balanced in all synthetic data scenarios both in training and test datasets. The scenarios just give accuracy scores between the different kind of movement. We run two binary classification scenarios for eacg pattern. In the first type of scenario the classifier had to decide between the pattern and the random movement, whereas in the second, had to decide between the pattern and the step up pattern. The aim of the second type of experiments was to evaluate the ability of the classifier, which uses the shapelet feature representation, to distinguish between two SAR patterns. The step up pattern was selected as the basis for this type of experiments, because it is the most general and simple pattern of all.
* Multi-class classification scenarios on real data: For these experiments we evaluated all the pre-processing techniques described in Section 3.3 and manually examined the identified SAR patterns.  
The invariable setting of the optimization scenarios are:  
* Kind of movement: with a Steps vs Random movement.
* Time-series length: this setting was always 25 due to hardware restriction and due to the complexity of the GENDIS algorithm.
* Sampling frequency: using 1 sample per 3 min
* Trajectory feature: Bearing was the only feature used.
* Population size: It was selected to be 20 because of hardware limitations.
* add_noise_prob, add_shapelet_prob, crossover_prob, remove_shapelet_prob (of GENDIS module) with values 20, 0.3, 0.3, 0.66, 0.3 respectively, which are the proposed default settings of the GENDIS algorithm.  
The varying parameters in the optimization scenarios are:* The number of time series to produce (using the trajectory generator), with values of 100 and 150
* The number of GENDIS iterations, with values of 5, 10, 15 and 20.