## 4 Results and Discussion
### Comparison with Other P300 Detectors
#### 4.3.3 Gradient Boosting with OLS  
We briefly review this algorithm and its iterative learning procedure (see [27] for detailed derivations). Given a training set \(\left\{\mathbf{x}_{i}\in R^{d},i=1\ldots L\right\}\) and class labels \(y_{i}\in\left\{0,1\right\}\) (if the training data \(\mathbf{x}_{i}\) belongs to P300 class, \(y_{i}=~{}1;y_{i}=0\) otherwise), where \(d=N_{c}\times N_{s}\) is the number of features, \(N_{c}\) is the number of EEG channels (\(N_{c}=2\) because Pz and Cz are recording sites for P300 detection), \(N_{s}\) is the number of samples in one 500 ms epoch (\(N_{s}=125\) because of 250-Hz sampling rate), the boosting algorithm finds an ensemble of weak classifiers in an iterative manner. The chosen weak classifier is the linear combination of the training data, _i.e._, \(f\left(\mathbf{x}\right)=\mathbf{w}^{\prime}\mathbf{x}\), which is the same as the one in [27]. Let \(F_{m}\) denote the ensemble of classifiers after step \(m\). Pseudocode for the gradient boosting with OLS provided in [27] is as follows:  
1. Step 1: Set \(p_{0}(y_{i}=1|\mathbf{x}_{i})=0.5,\forall i=1,\ldots,L\)
2. Step 2: Initialize \(F_{0}\left(\mathbf{x}_{i}\right)=0,\forall i=1,\ldots,L\)
3. Step 3: For \(m=1\) to \(M\) do 1. \(\widetilde{y}_{i}=2\left(y_{i}-p_{m-1}(y_{i}-1|\mathbf{x}_{i})\right),\forall i =1,\ldots,L\) 2. \(f_{m}=\operatorname*{argmin}_{f}\sum_{i=1}^{N}\left(\widetilde{y}_{i}-f\left( \mathbf{x}_{i}\right)\right)^{2}\) 3. \(\gamma=\operatorname*{argmax}_{\gamma}Q\left(F_{m-1}+\gamma f_{m}\right)\) 4. \(F_{m}=F_{m-1}+\epsilon\gamma f_{m}\) 5.  Compute \(p_{m}(y_{i}=1|\mathbf{x}_{i})\) Endfor In step 3(a) and (e), \(p_{m}(y_{i}=1|\mathbf{x}_{i})\) is computed by the logistic regression model expressed as  
\[p_{m}\left(y_{i}=1|\mathbf{x}_{i}\right)=\frac{e^{F_{m}\left(\mathbf{x}_{i} \right)}}{e^{F_{m}\left(\mathbf{x}_{i}\right)}+e^{-F_{m}\left(\mathbf{x}_{i} \right)}} \tag{25}\]  
In step 3(b), the optimal parameters of the weak classifier is obtained by least squares fitting the \(\mathbf{x}_{i}\) to \(\widetilde{y}_{i}\). In step 3(c), \(\gamma\) represents the relative importance of the weak classifier in the ensemble, and \(Q\) denotes the Bernoulli log-likelihood of \(F_{m}\):  
\[Q\left(F_{m}\right)=\log\left(\prod_{i=1}^{L}p_{m}\left(y_{i}=1|\mathbf{x}_{i} \right)^{y_{i}}p_{m}\left(y_{i}=0|\mathbf{x}_{i}\right)^{1-y_{i}}\right) \tag{26}\]  
During the iteration, this likelihood is maximized by first setting \(F_{0}\left(\mathbf{x}_{i}\right)=~{}0\) in step 1 and successively adding weak classifiers to the ensemble as expressed in step 3(d). To find the best \(\gamma\), we need to solve the optimization problem in step 3(c). This optimization problem is easy to solve, since the likelihood is a concave function [41]. Therefore, in our experiment, we use a greedy method to find the best \(\gamma\) value in step 3(c). In step 3(d), \(\gamma\) is further shrunk to a smaller value through multiplication with a small \(\epsilon\). The shrinking strategy can make the gradient boosting procedure less greedy. Following the same setting in [27], we set \(\epsilon\) to 0.05. The sub-steps in step 3 is repeated until the number of iterations \(M\) is reached. A large value of \(M\) results in overfitting, while a too-small value of \(M\) may lead to underfitting. We find the optimal \(M\) with a grid search method (within the interval of 1-100). The optimal \(M\) gives the highest average classification rate of P300 and non-P300 over 10-run two-fold cross validations.