Real-Time Prediction of Brain States Using FMRI

S. M. LaConte, S. Peltier, X. Hu Biomedical Engineering, Emory University/Georgia Tech, Atlanta, GA, United States INTRODUCTION In this study we explore the application of real-time predictive modeling during FMRI data acquisition. Specifically we have incorporated the support vector machine (SVM) classifier into the latter stages of the EPI image reconstruction chain (Fig. 1). The development of online predictive measures provides for an entirely new experimental approach, namely feedback of the stimulus paradigm to the subject based on predicted brain state. This work is fundamentally different from recent FMRI feedback studies, which display feedback to the volunteer in the form of activation maps [1,2]. Distinguishing characteristics of this work are that i) we are using multi-slice data, ii) feature selection relies only on threshold-based brain masking, iii) classifier output is related to predicted brain state rather than detected activation and is obtained at each individual image time point, and, iv) for the data acquisitions reported here, the computational burden was just barely perceptible during model training, causing only a slight (second or so) delay after image acquisition. Here we illustrate prediction performance with and without feedback. In addition we report algorithmic performance. Continuation of this work will include investigation into the tradeoffs involved with experimental design, training strategies, and variation in volunteer performance. THEORY AND METHODS Here we briefly describe temporal classification of FMRI data using the SVM. In general, training data, consisting of input vectors, xi, and their corresponding scalar class labels, yi, are used to obtain an SVM model. For FMRI, each xi and yi are the measured brain voxels and the class label for the time point, i, respectively. For simplicity, we examine the binary classification problem (yi ± = 1), corresponding to e.g. left hand vs. right hand. The input vectors are mapped to high dimensional feature space via a non-linear mapping function z=g(x). The SVM algorithm attempts to find linear decision boundaries (separating hyperplanes) in the feature space, formalized by the decision function D(z)=(w•z) +wo, where w defines the linear decision boundaries. Once the SVM model is determined, new testing scans can be classified with the decision function. The C-based SVMlight software [3] was incorporated into the Siemens Image Calculation Environment (ICE) running on an NT box with an Intel XEON processor and 2GB ram as follows. For a training run, pre-assigned class labels corresponding to the upcoming paradigm are read during scan preparation. Then reconstructed image data is used to obtain a threshold-based mask of brain pixels and the masked data is incorporated into SVMlight's data structures on an ongoing basis as each slice is acquired. After scanning is complete, the SVM model is calculated and saved to disk. For a test run, the SVM model is read during scan preparation. At each TR, the imaged brain volume is submitted for classification, and if desired, the result is sent out on an I/O port (e.g. the parallel port). FMRI data were collected on a 3T Siemens Trio, with 8 axial EPI slices covering the top of the head to the top of the corpus collosum (TR/TE = 2000/31 msec, voxel=3.4 X 3.4 X 5 mm). The FMRI scan protocol consisted of a single subject performing four experimental runs. In each task the volunteer was presented with a target on either side of a back projected display along with an arrow in the center of the screen, pointing toward the target. The display software used was Presentation® (Version 0.52, www.neurobs.com). The subject was instructed to rapidly press a button on a fiber optic button box (Current Designs, www.curdes.com) with the index finger of the hand corresponding to the target position and to actively visualize moving the arrow toward the target. Run 1 was a training task consisting of 30 seconds left, 30 seconds right button presses, repeated 8 times (for a total of 8 min.). Runs 2 and 3 were identical to Run 1, except that Run 2 was acquired with the reconstruction software running in testing mode (Run 3 was a second training session). Finally, in Run 4 feedback was presented to the subject by updating the arrow orientation and position based on the direction predicted by the classifier. Run 4 lasted 4 minutes, and at an unexpected time, the target position switched from right to left. Note that although button box responses were recorded in Presentation, the visual display was only updated by input from the parallel port sent from the image reconstruction computer based on the SVM class decision of left or right. Gross motion was not detected as screened with a cine replay of the EPI data. RESULTS AND DISCUSSION The SVM models from Run 1 and Run 3 were qualitatively similar, requiring 31.52 and 19.54 cpu-seconds and using 180 and 179 support vectors, respectively. Figs. 2 and 3 show the classifier output for Runs 2 and 4, respectively. Prediction performance seemed best in the case of no feedback. This result has many possible explanations such as the fact that the paradigm was only loosely matched to the training paradigm, possible attention shift or fatigue, or acquisition related artifacts. We are currently working to see if this result holds over different training strategies and subjects. We have demonstrated online classification, which allows for adaptive feedback based on classified brain state, providing for a new level of sophisticated exploration of brain function that goes beyond linear systems input-output relationships. Indeed, feedback provides the underpinning of all living systems, and, in the future, adaptive fMRI and other complementary techniques may provide insights unattainable through traditional stimulus response experiments. REFERENCES [1] Yoo, S.S. , Jolesz, F.A. , Neuroreport 13:1377-1381, 2002. [2] Weiskopf, N. , et al. NeuroImage 19:577-586, 2003. [3] Joachims, T. Advances in Kernel Methods Support Vector Learning, B. Scholkopf and C. Burges and A. Smola (ed.), MIT-Press, 1999. ACKNOWLEDGEMENT Work supported by the Whitaker Foundation, the Georgia Research Alliance, and NIH grant RO1EB002009.