Schreuder et al. Listen, you are writing! length (1–3 iterations) was added. That the visual target presenta- tion was not necessary was proven in a session with a single, blind subject. Then, subjects were asked to write word for word one of two sentences: “FRANZ JAGT IM TAXI DURCH BERLIN” or “SYLVIA WAGT QUICK DEN JUX BEI PFORZHEIM.” Sentence assign- ment was random. Wrong letters had to be corrected by the user by performing a backspace action, requiring two selections. No prequel was included. The stimulation sequence and the overhead of label, and result presentation for a single trial took around 34 s. With a two-step letter selection process, the maximum theoretical speed is thus 0.89 char/min. SESSION 2 Subjects who were able to successfully write with the system in Session 1 came back for Session 2. This session consisted of a cali- bration phase – equal to Session 1 – and the online writing of two German sentences. For Session 2 the protocol was adjusted in sev- eral ways. First, a dynamic stopping method was introduced which could stop a trial after 4–15 iterations when enough evidence for a correct selection was found (see below). Second, subjects were asked to study the selections that are needed for each character at home. They were no longer provided with auditory labels prior to a trial, which reduced the trial time by about 6.2 s. The maximum theoretical speed with 15 iterations was thus 1.10 char/min. During the ﬁrst sentence, subjects got visual information on the labels to familiarize them again with the interface. During the second sentence this was turned off, thus relying exclusively on the auditory sense. During both sentences, subjects still received audi- tory information after a selection to know their current location in the spelling tree. The ﬁrst sentence was the remaining sentence from Session 1, whereas the second sentence could be chosen by the subject but should be at least ﬁve words long. FEATURES AND CLASSIFIER Classiﬁcation was done on each subtrial, thus reducing the prob- lem to a binary task, i.e., to classify each subtrial as a target or non- target. Spatio-temporal features for classiﬁcation were extracted according to the following procedure. First the data were baselined, using the 150-ms pre stimulus data as reference. Then, the sample- wise r 2 coefﬁcient was calculated for targets vs. non-targets. Based on this, a set of two to four intervals with high discriminative information content were hand-picked, such that both early and late components were represented. All samples within each inter- val were then averaged, so that one interval was represented by one feature per channel. This resulted in an overall dimensionality of 112–224 features. The beneﬁt of this method over conventional sub sampling (Krusienski et al., 2006; Schreuder et al., 2010) is a relatively small number of features per channel which capture most of the dis- criminative information. Furthermore, by combining consecutive samples instead of picking individual samples, as is the case in step- wise linear discriminant analysis (Farwell and Donchin, 1988), the interpretation of the chosen features is straight-forward and a pri- ori knowledge can be included during the selection. Using this feature vector, a linear, binary classiﬁer was trained. In order to prevent overﬁtting, the classiﬁer was conditioned using shrinkage regularization (Ledoit and Wolf, 2004; Blankertz et al., 2011). DECISION MAKING The classiﬁer was trained to assign negative scores to target sub- trials and positive scores to non-target subtrials. Let X ∈ RC × J be a matrix of classiﬁer scores of a trial, where C is the number of classes (six in our case), and J the number of performed iterations. If c = {1,. . .,C }, then let x˜ be a row vector, where x˜c denotes the median value of classiﬁer scores for class c. The winning class c ∗ can be described as c ∗ = argminc x˜c , i.e., the class with the lowest median value. For Session 1 J was ﬁxed to 15. In order to further optimize the spelling speed, a dynamic stop- ping method was introduced in Session 2. Such a method allows for trials with less than 15 iterations, when the obtained data sup- port an early stop. Different methods for tackling this have been introduced to BCI before (Serby et al., 2005; Lenhardt et al., 2008; Zhang et al., 2008; Liu et al., 2010), and were mainly tested in the context of visual ERP based BCI and one-step interfaces. It can be assumed that at a low number of iterations the qual- ity of a decision will be susceptible to sporadic outlying classiﬁer scores, and that the inﬂuence of these will decrease with an increas- ing number of iterations. In line with this assumption, an approach which uses iteration-speciﬁc thresholds was designed. Thresholds are based on the calibration data, and are thus subject speciﬁc. The variable p is deﬁned as the distance from x˜c∗ to the second lowest value in x˜ . It represents a conﬁdence measure of the win- ning class. Note that, as this is a relative value, it is robust to linear biases that might arise when moving from the calibration phase to the online phase. In the online setting, the BCI checks after each iteration j ∈ {1,. . .,J } if pj – based on the data gathered thus far – exceeds threshold Tj (see below). If so, the trial is stopped and class c ∗ is selected. Heuristic for decision threshold determination For ﬁnding the decision thresholds T, the trained classiﬁer is re- applied to the complete calibration data. Although this can be considered overﬁtting and might lead to an overestimation of the class distances, it is not consider as a caveat when ﬁnding a conservative threshold. For calculating Tj , we consider all calibration data collected within a trial, up to iteration j. First, the winning class for this iter- ation cj∗ and the conﬁdence pj are calculated and pj is assigned a label; true (pj+) if cj∗ corresponds to the target direction, and false (pj−) otherwise. This is repeated for each trial n ∈ {1,. . ., N } and RN×J iteration j, and results are collected in the matrices P + ∈ and P − ∈ RN×J . In order to approximate a smooth upper error bound, a third order polynomial F is ﬁtted to the upper limit of P −. Then, if fj refers to the value of F at iteration j, the iteration-speciﬁc threshold Tj is deﬁned as the maximum of fj and the product of the median of column j in P + and R. The hyperparameter R is used to control the number of false negative stops. In this study, R was set to 1, which means that maximally only 50% of possible correct early stops is recognized. The above is visualized in Figure 2. The threshold is rather conservative, as it is biased away from false positives (incorrect early stops). The rational behind such Frontiers in Neuroscience | Neuroprosthetics October 2011 | Volume 5 | Article 112 | 4