LEVERAGING BRAIN COMPUTER INTERACTION TECHNOLOGIES FOR MILITARY APPLICATIONS

Recent advances in neuroscience, signal processing, machine learning, and related technologies have made it possible to reliably detect brain signatures specific to visual target recognition in real time. Utilizing these technologies together has shown an increase in the speed and accuracy of visual target identification over traditional visual scanning techniques. Images containing a target of interest elicit a unique neural signature in the brain (e.g. P300 event-related potential) when detected by the human observer. Computer vision exploits the P300based signal to identify specific features in the target image that are different from other non-target images. Coupling the brain and computer in this way along with using rapid serial visual presentation (RSVP) of the images enables large image datasets to be accurately interrogated in a short amount of time. Together this technology allows for potential military applications ranging from image triaging for the image analyst to target geo-tagging for ground troops. INTRODUCTION One area of research that may potentially lead to disruptive innovations is that of Brain-Computer Interaction Technologies (BCIT). BCITs use non-invasively measured neural data, such as electroencephalography (EEG), in combination with other physiological and behavioral measures to enhance joint human-system performance for healthy individuals [1]. Recent advances in neuroscience, signal processing and machine learning have made it possible to exploit and integrate visual, motor, and cognitive processing of the brain with advanced computational algorithms of the computer [2]. There are many potential Army-relevant tasks that may benefit from these types of BCITs, including processing large quantities of image data. Advancements in sensor technology and digital storage continue to worsen the mismatch between our ability to collect and store data on the one hand and process and analyze it on the other. Large amounts of data present high computational demands on the limited processing capacities of both humans and computers, resulting in information overload and slow and often inaccurate performance. As the quantity of information increases, so does the challenge of sifting through large amounts of data in order to quickly and accurately detect potential targets of interest. Throughput for image analysis is confined by processing limits of both the human brain and computer algorithms. Manually searching through large images or datasets is time consuming and impractical, especially when decisions about potential targets must be made quickly, whether searching for targets in satellite imagery or looking for enemy vehicles in video taken from an unmanned aerial vehicle (UAV). Given the critical nature of identifying targets in military operations, automated capabilities for processing image data are still limited. The human and computer each have their limitations in processing visual imagery; however, their strengths may be combined to overcome these shortcomings. Humans are capable of rapidly making semantic distinctions based on visual input, while computers are capable of quickly searching large image databases. In this paper we describe a method for developing mutual human-machine systems that utilize the capabilities of both the human and computer systems to improve capabilities for analyzing image data by using brain-based signals related to target recognition such as the P300 event-related potential (ERP) and techniques Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS) Leveraging brain computer interaction technologies for military applications Unclassified: Distribution A. Approved for public release . Page 2 of 6 such as displaying image data in rapid serial visual presentation. NEURAL SIGNATURES OF TARGET DETECTION A neural signal commonly used in EEG-based BCIT applications to identify when an observer detects a target of interest is the P300, or P3 event-related potential (ERP); e.g. [3-5]. An ERP reflects averaged EEG epochs time-locked to a specific stimulus and is used to improve the signal to noise ratio of the EEG signal [6]. The P3 is a large positive deflection in the ERP waveform occurring roughly 300ms after stimulus onset and is maximal over frontal/central and central/parietal electrodes. Typically both P3 amplitude and latency are measured, with amplitude assessed by comparing the pre-stimulus baseline amplitude to the largest positive peak of the ERP within a variable time window post-stimulus, and latency measured as the point of maximum positive peak amplitude within a time window beginning at the onset of stimulus presentation [7]. While the P3 can be clearly seen in an averaged ERP, it can also be measured in single trials, (as shown by Makeig et al. [8]; and for simultaneous EEG/fMRI by Goldman et al. [9]). The P3 has most commonly been evaluated using the ‘oddball’ paradigm where an infrequent target stimulus is presented within a series of frequently occurring non-target distractor stimuli [10]. In a two-stimulus oddball paradigm an infrequent “oddball” target stimulus is presented with many frequent distractor stimuli where only the infrequent oddball stimulus requires a response. The time between stimulus presentation in a series ranges between 1-3 seconds and the probability of an oddball stimulus appearing is generally between 5-20%. The infrequent target evokes a P3 component, specifically the P3b, which reflects the detection of a task-relevant stimulus [7], [11]. The P3 is also obtained when an observer views a highly familiar image or one that is currently held in memory [12], [13]. Overall, P3-related activity reflects neural process associated with attentional orienting toward a task-relevant stimulus in the sensory