## 4 Classification Results  
In order to classify the patterns, the 15 strongest features according to equation (2) were selected from the local discriminant basis. The \(C\)-SVM results were obtained using a toolbox from [28]; custom code in Matlab was written to obtain the \(\
u\)-SVM results. For both SVM approaches a total number of 21 support vector machines was used to classify the seven classes; every support vector machine was trained with 100 patterns. Each machine has two parameters that can be tuned, the width and either the parameter \(C\) or \(\
u\). To simplify the training stage, their values were kept identical over all machines. In order to select suitable parameters, the machines were trained while varying one parameter and keeping the other one fixed. At each value the classifier was trained 10 times with noise added to the data. The noise was drawn from a zero-mean normal distribution with the standard deviation taken from the standard deviation of the features. Parameter selection for C-SVM is shown in Figure 5. The values were based on the second set which proved more problematic than the others. On the left side the error penalty \(C\) is varied between 0.1 to 2. The top left graph shows the true positive and false positive rates. In order to create these values, all other classes were combined into the negative class. It has to be kept in mind that the classifier was not especially trained to improve performance on this set, which means that for some parameters the classifier as a whole might have had better performance, while set 2 performed worse. Bottom left shows the total number of support vector machines that were used for each \(C\). As can be seen, a low penalty allowed many patterns to reside inside the margin, leading to many support vectors, while increasing the penalty discouraged this. On the right side variation of the kernel width is shown. In this case, a narrow width did not cover the feature space very well, and many support vectors were used (the maximum total number of support vectors that could be used were \(21\times 100\) for 21 machines with 100 training patterns each), increasing the width and coverage in feature space required less vectors. A high true positive rate was often combined with a high number of support vectors. A reasonable value for \(C\) would be around 1.0 and for the width around 0.30. In the case of \(\
u\)-SVM similar figures were made to decide on reasonable parameter values in Figure 6. The affect of varying \(\
u\) is seen on the left side, in this case increasing \(\
u\) (and thus increasing the width of the margin) will also increase the lower bound given in Equation (8) leading to a higher number of support vectors. Here, a narrow margin was preferred with \(\
u\) around 0.13 and \(\sigma\) around 0.25. The left side of Table 2 shows the complete classification results using \(C\)-SVM with \(\sigma=0.30\) and \(C=1.1\). The machines used 337 support vectors in total, 141 of which were unique (training patterns from one class can be used by multiple machines). The classification values are the percentages of correctly classified clicks. The undecided row contains the percentages of patterns that could not be classified by the voting mechanism of the SVM classifiers. The generalisation to the validation set is  
Figure 5: Performance of the \(C\)-svm classifier on problem set 6. The top row images plot the false positive rate (FPR) versus the true positive rate (TPR). On the left the regularisation parameter \(C\) is varied between the values on the colour bar with fixed kernel width \(\sigma=0.30\). The right image varies the kernel width between the values on the colour bar with fixed \(C=1.1\). quite good except for the second set. In particular, the eighth set classifies very well. Analysing the errors in this set showed that there was one particular noisy time period during which many clicks were misclassified. Removing this period left 993 clicks, 82% of which were correctly classified.