Auditory-visual aversive stimuli modulate the conscious experience of fear.

In a natural environment, affective information is perceived via multiple senses, mostly audition and vision. However, the impact of multisensory information on affect remains relatively undiscovered. In this study, we investigated whether the auditory-visual presentation of aversive stimuli influences the experience of fear. We used the advantages of virtual reality to manipulate multisensory presentation and to display potentially fearful dog stimuli embedded in a natural context. We manipulated the affective reactions evoked by the dog stimuli by recruiting two groups of participants: dog-fearful and non-fearful participants. The sensitivity to dog fear was assessed psychometrically by a questionnaire and also at behavioral and subjective levels using a Behavioral Avoidance Test (BAT). Participants navigated in virtual environments, in which they encountered virtual dog stimuli presented through the auditory channel, the visual channel or both. They were asked to report their fear using Subjective Units of Distress. We compared the fear for unimodal (visual or auditory) and bimodal (auditory-visual) dog stimuli. Dog-fearful participants as well as non-fearful participants reported more fear in response to bimodal audiovisual compared to unimodal presentation of dog stimuli. These results suggest that fear is more intense when the affective information is processed via multiple sensory pathways, which might be due to a cross-modal potentiation. Our findings have implications for the field of virtual reality-based therapy of phobias. Therapies could be refined and improved by implicating and manipulating the multisensory presentation of the feared situations.

Related papers:

title: Neuromagnetic oscillations to emotional faces and prosody
abstract: Higher association cortices as well as unisensory areas can support multisensory integration [D. Senkowski et al. (2008) Trends Neurosci., 31, 401–409]. The present study investigated whether audiovisual integration of emotional information emerges early at unisensory or later at higher association cortices. Emotional stimuli were presented in three blocks: audiovisual (AV), auditory (A) and visual (V). Eighteen participants performed a delayed emotional recognition task (happy, angry or neutral prosody and/or facial expression) while whole‐brain magnetoencephalography (MEG) data were obtained. Time–frequency evoked and total power analyses were performed on the sensor data, and source localization of the frequencies of interest performed via a synthetic aperture magnetometry beamformer. To examine crossmodal integration between bimodal and unimodal conditions, two contrasts were specified: AV > A and AV > V. In the AV > A contrast, early effects were observed on both the temporal and the occipital evoked responses. However, at the source level, early alpha suppression was limited to the occipital sources without changes in temporal cortices. In the AV > V contrast, sensor and source findings revealed increased alpha suppression only in temporal cortices, with no changes in visual cortex. Thus, no crossmodal effect in unisensory areas emerged. Instead, increased frontal alpha activity in both the AV > A and AV > V contrasts supports the view that affective information from face and prosody converges at higher association cortices.

title: Audio-visual integration of emotion expression
abstract: Regardless of the fact that emotions are usually recognized by combining facial and vocal expressions, the multisensory nature of affect perception has scarcely been investigated. In the present study, we show results of three experiments on multisensory perception of emotions using newly validated sets of dynamic visual and non-linguistic vocal clips of affect expressions. In Experiment 1, participants were required to categorize fear and disgust expressions displayed auditorily, visually, or using congruent or incongruent audio-visual stimuli. Results showed faster and more accurate categorisation in the bimodal congruent situation than in the unimodal conditions. In the incongruent situation, participant preferentially categorized the affective expression based on the visual modality, demonstrating a visual dominance in emotional processing. However, when the reliability of the visual stimuli was diminished, participants categorized incongruent bimodal stimuli preferentially via the auditory modality. These results demonstrate that visual dominance in affect perception does not occur in a rigid manner, but follows flexible situation-dependent rules. In Experiment 2, we requested the participants to pay attention to only one sensory modality at a time in order to test the putative mandatory nature of multisensory affective interactions. We observed that even if they were asked to ignore concurrent sensory information, the irrelevant information significantly affected the processing of the target. This observation was especially true when the target modality was less reliable. Altogether, these findings indicate that the perception of emotion expressions is a robust multisensory situation which follows rules that have been previously observed in other perceptual domains.

title: The combined perception of emotion from voice and face: early interaction revealed by human electric brain responses
abstract: Judgement of the emotional tone of a spoken utterance is influenced by a simultaneously presented face expression. The time course of this integration was investigated by measuring the mismatch negativity (MMN). In one condition, the standard stimulus was an angry voice fragment combined with a (congruous) angry face expression. In the deviant pair, the voice expression was kept the same and only the face expression changed to an (incongruous) sad face. The pairs with a deviant visual item evoked a negative electric brain response showing the characteristics of the MMN, which is usually evoked only by auditory deviations. Similar results were obtained by employing incongruous standard and congruous deviant pairs. These findings provide compelling evidence of an early integration of face with voice information in the processing of affect.

title: Crossmodal binding of fear in voice and face
abstract: In social environments, multiple sensory channels are simultaneously engaged in the service of communication. In this experiment, we were concerned with defining the neuronal mechanisms for a perceptual bias in processing simultaneously presented emotional voices and faces. Specifically, we were interested in how bimodal presentation of a fearful voice facilitates recognition of fearful facial expression. By using event-related functional MRI, that crossed sensory modality (visual or auditory) with emotional expression (fearful or happy), we show that perceptual facilitation during face fear processing is expressed through modulation of neuronal responses in the amygdala and the fusiform cortex. These data suggest that the amygdala is important for emotional crossmodal sensory convergence with the associated perceptual bias during fear processing, being mediated by task-related modulation of face-processing regions of fusiform cortex.

title: Preattentive processing of audio-visual emotional signals.
abstract: Previous research has shown that redundant information in faces and voices leads to faster emotional categorization compared to incongruent emotional information even when attending to only one modality. The aim of the present study was to test whether these crossmodal effects are predominantly due to a response conflict rather than interference at earlier, e.g. perceptual processing stages. In Experiment 1, participants had to categorize the valence and rate the intensity of happy, sad, angry and neutral unimodal or bimodal face-voice stimuli. They were asked to rate either the facial or vocal expression and ignore the emotion expressed in the other modality. Participants responded faster and more precisely to emotionally congruent compared to incongruent face-voice pairs in both the Attend Face and in the Attend Voice condition. Moreover, when attending to faces, emotionally congruent bimodal stimuli were more efficiently processed than unimodal visual stimuli. To study the role of a possible response conflict, Experiment 2 used a modified paradigm in which emotional and response conflicts were disentangled. Incongruency effects were significant even in the absence of response conflicts. The results suggest that emotional signals available through different sensory channels are automatically combined prior to response selection.

title: MEG demonstrates a supra-additive response to facial and vocal emotion in the right superior temporal sulcus
abstract: An influential neural model of face perception suggests that the posterior superior temporal sulcus (STS) is sensitive to those aspects of faces that produce transient visual changes, including facial expression. Other researchers note that recognition of expression involves multiple sensory modalities and suggest that the STS also may respond to crossmodal facial signals that change transiently. Indeed, many studies of audiovisual (AV) speech perception show STS involvement in AV speech integration. Here we examine whether these findings extend to AV emotion. We used magnetoencephalography to measure the neural responses of participants as they viewed and heard emotionally congruent fear and minimally congruent neutral face and voice stimuli. We demonstrate significant supra-additive responses (i.e., where AV > [unimodal auditory + unimodal visual]) in the posterior STS within the first 250 ms for emotionally congruent AV stimuli. These findings show a role for the STS in processing crossmodal emotive signals.

title: The temporal dynamics of processing emotions from vocal, facial, and bodily expressions
abstract: Face-to-face communication works multimodally. Not only do we employ vocal and facial expressions; body language provides valuable information as well. Here we focused on multimodal perception of emotion expressions, monitoring the temporal unfolding of the interaction of different modalities in the electroencephalogram (EEG). In the auditory condition, participants listened to emotional interjections such as "ah", while they saw mute video clips containing emotional body language in the visual condition. In the audiovisual condition participants saw video clips with matching interjections. In all three conditions, the emotions "anger" and "fear", as well as non-emotional stimuli were used. The N100 amplitude was strongly reduced in the audiovisual compared to the auditory condition, suggesting a significant impact of visual information on early auditory processing. Furthermore, anger and fear expressions were distinct in the auditory but not the audiovisual condition. Complementing these event-related potential (ERP) findings, we report strong similarities in the alpha- and beta-band in the visual and the audiovisual conditions, suggesting a strong visual processing component in the perception of audiovisual stimuli. Overall, our results show an early interaction of modalities in emotional face-to-face communication using complex and highly natural stimuli.

title: The effects of anxiety on the interpretation of emotion in the face–voice pairs
abstract: Anxious individuals have been shown to interpret others’ emotional states negatively. Since most studies have used facial expressions as emotional cues, we examined whether trait anxiety affects the recognition of emotion in a dynamic face and voice that were presented in synchrony. The face and voice cues conveyed either matched (e.g., happy face and voice) or mismatched emotions (e.g., happy face and angry voice). Participants with high or low trait anxiety were to indicate the perceived emotion using one of the cues while ignoring the other. The results showed that individuals with high trait anxiety were more likely to interpret others’ emotions in a negative manner, putting more weight on the to-be-ignored angry cues. This interpretation bias was found regardless of the cue modality (i.e., face or voice). Since trait anxiety did not affect recognition of the face or voice cues presented in isolation, this interpretation bias appears to reflect an altered integration of the face and voice cues among anxious individuals.

title: Audiovisual integration of emotional signals in voice and face: An event-related fMRI study
abstract: In a natural environment, non-verbal emotional communication is multimodal (i.e. speech melody, facial expression) and multifaceted concerning the variety of expressed emotions. Understanding these communicative signals and integrating them into a common percept is paramount to successful social behaviour. While many previous studies have focused on the neurobiology of emotional communication in the auditory or visual modality alone, far less is known about multimodal integration of auditory and visual non-verbal emotional information. The present study investigated this process using event-related fMRI. Behavioural data revealed that audiovisual presentation of non-verbal emotional information resulted in a significant increase in correctly classified stimuli when compared with visual and auditory stimulation. This behavioural gain was paralleled by enhanced activation in bilateral posterior superior temporal gyrus (pSTG) and right thalamus, when contrasting audiovisual to auditory and visual conditions. Further, a characteristic of these brain regions, substantiating their role in the emotional integration process, is a linear relationship between the gain in classification accuracy and the strength of the BOLD response during the bimodal condition. Additionally, enhanced effective connectivity between audiovisual integration areas and associative auditory and visual cortices was observed during audiovisual stimulation, offering further insight into the neural process accomplishing multimodal integration. Finally, we were able to document an enhanced sensitivity of the putative integration sites to stimuli with emotional non-verbal content as compared to neutral stimuli.

title: Perceiving affect from the voice and the face
abstract: This experiment examines how emotion is perceived by using facial and vocal cues of a speaker. Three levels of facial affect were presented using a computer-generated face. Three levels of vocal affect were obtained by recording the voice of a male amateur actor who spoke a semantically neutral word in different simulated emotional states. These two independent variables were presented to subjects in all possible permutations—visual cues alone, vocal cues alone, and visual and vocal cues together—which gave a total set of 15 stimuli. The subjects were asked to judge the emotion of the stimuli in a two-alternative forced choice task (either HAPPY or ANGRY). The results indicate that subjects evaluate and integrate information from both modalities to perceive emotion. The influence of one modality was greater to the extent that the other was ambiguous (neutral). The fuzzy logical model of perception (FLMP) fit the judgments significantly better than an additive model, which weakens theories based on an additive combination of modalities, categorical perception, and influence from only a single modality.

title: Crossmodal interactions in audiovisual emotion processing
abstract: Emotion in daily life is often expressed in a multimodal fashion. Consequently emotional information from one modality can influence processing in another. In a previous fMRI study we assessed the neural correlates of audio-visual integration and found that activity in the left amygdala is significantly attenuated when a neutral stimulus is paired with an emotional one compared to conditions where emotional stimuli were present in both channels. Here we used dynamic causal modelling to investigate the effective connectivity in the neuronal network underlying this emotion presence congruence effect. Our results provided strong evidence in favor of a model family, differing only in the interhemispheric interactions. All winning models share a connection from the bilateral fusiform gyrus (FFG) into the left amygdala and a non-linear modulatory influence of bilateral posterior superior temporal sulcus (pSTS) on these connections. This result indicates that the pSTS not only integrates multi-modal information from visual and auditory regions (as reflected in our model by significant feed-forward connections) but also gates the influence of the sensory information on the left amygdala, leading to attenuation of amygdala activity when a neutral stimulus is integrated. Moreover, we found a significant lateralization of the FFG due to stronger driving input by the stimuli (faces) into the right hemisphere, whereas such lateralization was not present for sound-driven input into the superior temporal gyrus. In summary, our data provides further evidence for a rightward lateralization of the FFG and in particular for a key role of the pSTS in the integration and gating of audio-visual emotional information.

title: The time‐course of intermodal binding between seeing and hearing affective information
abstract: Intermodal binding between affective information that is seen as well as heard triggers a mandatory process of audiovisual integration. In order to track the time course of this audiovisual binding, event related brain potentials were recorded while subjects saw facial expression and concurrently heard auditory fragment. The results suggest that the combination of the two inputs is early in time (110 ms post-stimulus) and translates as a specific enhancement in amplitude of the auditory N1 component. These findings are compatible with previous functional neuroimaging results of audiovisual speech showing strong audiovisual interactions in auditory cortex in the form of magnetic response amplifications, as well as with electrophysiological studies demonstrating early audiovisual interactions (before 200 ms post-stimulus). Moreover, our results show that the informational content present in the two modalities plays a crucial role in triggering the intermodal binding process.

title: Perception of Facial Expressions and Voices and of their Combination in the Human Brain
abstract: Using positron emission tomography we explored brain regions activated during the perception of face expressions, emotional voices and combined audio-visual pairs. A convergence region situated in the left lateral temporal cortex was more activated by bimodal stimuli than by either visual only or auditory only stimuli. Separate analyses for the emotions happiness and fear revealed supplementary convergence areas situated mainly anteriorly in the left hemisphere for happy pairings and in the right hemisphere for fear pairings indicating different neuro-anatomical substrates for multisensory integration of positive versus negative emotions. Activation in the right extended amygdala was obtained for fearful faces and fearful audio-visual pairs but not for fearful voices only. These results suggest that during the multisensory perception of emotion, affective information from face and voice converge in heteromodal regions of the human brain.

title: Superior temporal activation in response to dynamic audio-visual emotional cues
abstract: Perception of emotion is critical for successful social interaction, yet the neural mechanisms underlying the perception of dynamic, audio-visual emotional cues are poorly understood. Evidence from language and sensory paradigms suggests that the superior temporal sulcus and gyrus (STS/STG) play a key role in the integration of auditory and visual cues. Emotion perception research has focused on static facial cues; however, dynamic audio-visual (AV) cues mimic real-world social cues more accurately than static and/or unimodal stimuli. Novel dynamic AV stimuli were presented using a block design in two fMRI studies, comparing bimodal stimuli to unimodal conditions, and emotional to neutral stimuli. Results suggest that the bilateral superior temporal region plays distinct roles in the perception of emotion and in the integration of auditory and visual cues. Given the greater ecological validity of the stimuli developed for this study, this paradigm may be helpful in elucidating the deficits in emotion perception experienced by clinical populations.

title: I Feel Your Voice
abstract: Cultural differences in emotion perception have been reported mainly for facial expressions and to a lesser extent for vocal expressions. However, the way in which the perceiver combines auditory and visual cues may itself be subject to cultural variability. Our study investigated cultural differences between Japanese and Dutch participants in the multisensory perception of emotion. A face and a voice, expressing either congruent or incongruent emotions, were presented on each trial. Participants were instructed to judge the emotion expressed in one of the two sources. The effect of to-be-ignored voice information on facial judgments was larger in Japanese than in Dutch participants, whereas the effect of to-be-ignored face information on vocal judgments was smaller in Japanese than in Dutch participants. This result indicates that Japanese people are more attuned than Dutch people to vocal processing in the multisensory perception of emotion. Our findings provide the first evidence that multisensory integration of affective information is modulated by perceivers’ cultural background.

title: Is cross-modal integration of emotional expressions independent of attentional resources?
abstract: In this study, we examined whether integration of visual and auditory information about emotions requires limited attentional resources. Subjects judged whether a voice expressed happiness or fear, while trying to ignore a concurrently presented static facial expression. As an additional task, the subjects had to add two numbers together rapidly (Experiment 1), count the occurrences of a target digit in a rapid serial visual presentation (Experiment 2), or judge the pitch of a tone as high or low (Experiment 3). The visible face had an impact on judgments of the emotion of the heard voice in all the experiments. This cross-modal effect was independent of whether or not the subjects performed a demanding additional task. This suggests that integration of visual and auditory information about emotions may be a mandatory process, unconstrained by attentional resources.

title: Audiovisual emotion recognition in schizophrenia: Reduced integration of facial and vocal affect
abstract: Since Kraepelin called dementia praecox what we nowadays call schizophrenia, cognitive dysfunction has been regarded as central to its psychopathological profile. Disturbed experience and integration of emotions are, both intuitively and experimentally, likely to be intermediates between basic, non-social cognitive disturbances and functional outcome in schizophrenia. While a number of studies have consistently proven that, as part of social cognition, recognition of emotional faces and voices is disturbed in schizophrenics, studies on multisensory integration of facial and vocal affect are rare. We investigated audiovisual integration of emotional faces and voices in three groups: schizophrenic patients, non-schizophrenic psychosis patients and mentally healthy controls, all diagnosed by means of the Schedules of Clinical Assessment in Neuropsychiatry (SCAN 2.1). We found diminished crossmodal influence of emotional faces on emotional voice categorization in schizophrenics, but not in non-schizophrenia psychosis patients. Results are discussed in the perspective of recent theories on multisensory integration.

title: Multisensory Integration and Attention in Autism Spectrum Disorder: Evidence from Event-Related Potentials
abstract: Successful integration of various simultaneously perceived perceptual signals is crucial for social behavior. Recent findings indicate that this multisensory integration (MSI) can be modulated by attention. Theories of Autism Spectrum Disorders (ASDs) suggest that MSI is affected in this population while it remains unclear to what extent this is related to impairments in attentional capacity. In the present study Event-related potentials (ERPs) following emotionally congruent and incongruent face-voice pairs were measured in 23 high-functioning, adult ASD individuals and 24 age- and IQ-matched controls. MSI was studied while the attention of the participants was manipulated. ERPs were measured at typical auditory and visual processing peaks, namely, P2 and N170. While controls showed MSI during divided attention and easy selective attention tasks, individuals with ASD showed MSI during easy selective attention tasks only. It was concluded that individuals with ASD are able to process multisensory emotional stimuli, but this is differently modulated by attention mechanisms in these participants, especially those associated with divided attention. This atypical interaction between attention and MSI is also relevant to treatment strategies, with training of multisensory attentional control possibly being more beneficial than conventional sensory integration therapy.

title: Facial electromyographic responses to emotional information from faces and voices in individuals with pervasive developmental disorder.
abstract: Despite extensive research, it is still debated whether impairments in social skills of individuals with pervasive developmental disorder (PDD) are related to specific deficits in the early processing of emotional information. We aimed to test both automatic processing of facial affect as well as the integration of auditory and visual emotion cues in individuals with PDD.



In a group of high-functioning adult individuals with PDD and an age- and IQ-matched control group, we measured facial electromyography (EMG) following presentation of visual emotion stimuli (facial expressions) as well as the presentation of audiovisual emotion pairs (faces plus voices). This emotionally driven EMG activity is considered to be a direct correlate of automatic affect processing that is not under intentional control.



Our data clearly indicate that among individuals with PDD facial EMG activity is heightened in response to happy and fearful faces, and intact in response to audiovisual affective information.



This study provides evidence for enhanced sensitivity to facial cues at the level of reflex-like emotional responses in individuals with PDD. Furthermore, the findings argue against impairments in crossmodal affect processing at this level of perception. However, given how little comparative work has been done in the area of multisensory perception, there is certainly need for further exploration.

title: Atypical processing of fearful face–voice pairs in Pervasive Developmental Disorder: An ERP study
abstract: An important premise for successful social-affective communication is rapid perception of visual and auditory emotional cues, as well as their multisensory integration (MSI). We investigated to what extent a deficit in recognition of emotions in individuals with Pervasive Developmental Disorder (PDD) may have its roots in abnormal MSI of emotional cues provided by the sight of a facial expression and an emotional tone of voice.  In twelve high-functioning, adult PDD individuals and thirteen age- and IQ-matched controls, (1) the processing of fearful faces was compared with that of happy faces; (2) MSI was assessed by characterizing the interaction effects of crossmodal presentation, using EEG.  Increased P1 and N170 amplitudes were seen in response to fearful faces compared with happy faces in both groups. However, PDD individuals differed from healthy controls in MSI of fearful information from visual and auditory cues.  Both groups show a similar pattern as concerns the early components of visual emotion processing, but there are anomalies in processing of fearful face–voice combinations in the PDD group.  Because of the importance of rapid MSI for social competence, MSI anomalies in PDD may be linked to the observed deficits in their emotional behavior.

title: The auditory-visual integration of anger is impaired in alcoholism: an event-related potentials study.
abstract: Chronic alcoholism leads to impaired visual and auditory processing of emotions, but the cross-modal (auditory-visual) processing of emotional stimuli has not yet been explored. Our objectives were to describe the electrophysiological correlates of unimodal (visual and auditory) impairments in emotion processing in people suffering from alcoholism, to determine whether this deficit is general or emotion-specific, and to explore potential deterioration in the specific cross-modal integration processes in alcoholism.



We used an emotion-detection task, with recording of event-related potentials (ERPs), in which 15 patients suffering from alcoholism and 15 matched healthy control subjects were asked to detect the emotion (angry, happy or neutral) displayed by auditory, visual or auditory-visual stimuli. Behavioural performance and ERP data recorded between June 2005 and April 2006 were analyzed.



ERPs demonstrated that the deficit in alcoholism originates earlier in the cognitive stream than has previously been described (mainly P300), namely, at the level of specific face (N170) and voice (N2) perceptive processing. Moreover, while patients with alcoholism did not show impaired processing of happy and neutral audio-visual stimuli, they did have a specific impairment in the cross-modal processing of anger. A source location analysis was used to confirm and illustrate the results.



These results suggest that the specific deficit that people with alcoholism demonstrate in processing anger stimuli, widely described in clinical situations but not clearly identified in earlier studies (using unimodal stimuli), is particularly obvious during cross-modal processing, which is more common than unimodal processing in everyday life.

title: Emotion in the perspective of an integrated nervous system
 1
 Published on the World Wide Web on 27 January 1998.
 
 
 1
 

abstract: It would not be possible to discuss the integrative aspects of brain function without considering the operations that arise in large-scale neural systems; and it would be unreasonable not to single-out emotion among the critical integrative operations arising in that level. Yet, throughout the twentieth century, the integrated brain and mind have often been discussed with hardly any acknowledgment that emotion does exist, let alone that it is an important function and that understanding its neural underpinnings is of great advantage. There are numerous reasons behind the benign neglect of emotion and I cannot mention them all and much less discuss them. In general, however, it seems fair to say that emotion must have seemed both too elusive and too subjective to attract the interest of neuroscientists or cognitive scientists concerned with researching that which appeared most concrete and objective. It seems clear now that there is nothing more elusive about emotion than about, say, perception or memory —in fact less so, in my opinion— and it is equally clear that emotion is also no less objective. In the event, however, it did seem more elusive and less objective than other cognitive and behavioral phenomena. Perhaps this was so be-

title: Neurobiology of emotion perception I: the neural basis of normal emotion perception
abstract: There is at present limited understanding of the neurobiological basis of the different processes underlying emotion perception. We have aimed to identify potential neural correlates of three processes suggested by appraisalist theories as important for emotion perception: 1) the identification of the emotional significance of a stimulus; 2) the production of an affective state in response to 1; and 3) the regulation of the affective state. In a critical review, we have examined findings from recent animal, human lesion, and functional neuroimaging studies. Findings from these studies indicate that these processes may be dependent upon the functioning of two neural systems: a ventral system, including the amygdala, insula, ventral striatum, and ventral regions of the anterior cingulate gyrus and prefrontal cortex, predominantly important for processes 1 and 2 and automatic regulation of emotional responses; and a dorsal system, including the hippocampus and dorsal regions of anterior cingulate gyrus and prefrontal cortex, predominantly important for process 3. We suggest that the extent to which a stimulus is identified as emotive and is associated with the production of an affective state may be dependent upon levels of activity within these two neural systems.

title: Enter feelings: somatosensory responses following early stages of visual induction of emotion.
abstract: In a MEG experiment, we imaged the early dynamics of the human cerebral cortex during the induction of emotion by visual stimuli. We tested the hypothesis that early cortical responses would correlate with the emotional competence of visual stimuli and subsequent subjective ratings of feeling in a set of specific target regions important for somatosensory, attentional, and motivational functions, just after initial visual and appraisal related cortical responses to picture presentation. Relative to the neutral condition, cortical responses, during the 350-500 ms phase of the MEG evoked response, were stronger for both pleasant and unpleasant stimuli in the orbitofrontal cortex, ventromedial prefrontal cortex, anterior cingulate and somatosensory cortices. These responses, which correlated with subjective ratings of arousal, emerged after an initial spreading of cortical activity from early visual cortex (70-200 ms) to the ventral visual stream, temporopolar and orbitofrontal regions (200-350 ms), higher for emotionally competent stimuli than for neutral in the 200-350 ms window, in a manner compatible with an appraisal function. Heart beats occurring during the first 500 ms post stimulus showed longer intervals for unpleasant than for neutral stimuli relative to the preceding beat. The results support the hypothesis of a sequence of processing regarding the emergence of feelings and suggest that, even in the early phase of feeling induction, actual body responses to the inducing stimuli could be represented in the brain.

title: Cross-modal interactions in the perception of musical performance
abstract: We investigate the dynamics of sensory integration for perceiving musical performance, a complex natural behavior. Thirty musically trained participants saw, heard, or both saw and heard, performances by two clarinetists. All participants used a sliding potentiometer to make continuous judgments of tension (a measure correlated with emotional response) and continuous judgments of phrasing (a measure correlated with perceived musical structure) as performances were presented. The data analysis sought to reveal relations between the sensory modalities (vision and audition) and to quantify the effect of seeing the performances on participants' overall subjective experience of the music. In addition to traditional statistics, functional data analysis techniques were employed to analyze time-varying aspects of the data. The auditory and visual channels were found to convey similar experiences of phrasing but different experiences of tension through much of the performances. We found that visual information served both to augment and to reduce the experience of tension at different points in the musical piece (as revealed by functional linear modeling and functional significance testing). In addition, the musicians' movements served to extend the sense of phrasing, to cue the beginning of new phrases, to indicate musical interpretation, and to anticipate changes in emotional content. Evidence for an interaction effect suggests that there may exist an emergent quality when musical performances are both seen and heard. The investigation augments knowledge of human communicative processes spanning language and music, and involving multiple modalities of emotion and information transfer.

title: Music to my eyes: Cross-modal interactions in the perception of emotions in musical performance
abstract: We investigate non-verbal communication through expressive body movement and musical sound, to reveal higher cognitive processes involved in the integration of emotion from multiple sensory modalities. Participants heard, saw, or both heard and saw recordings of a Stravinsky solo clarinet piece, performed with three distinct expressive styles: restrained, standard, and exaggerated intention. Participants used a 5-point Likert scale to rate each performance on 19 different emotional qualities. The data analysis revealed that variations in expressive intention had their greatest impact when the performances could be seen; the ratings from participants who could only hear the performances were the same across the three expressive styles. Evidence was also found for an interaction effect leading to an emergent property, intensity of positive emotion, when participants both heard and saw the musical performances. An exploratory factor analysis revealed orthogonal dimensions for positive and negative emotions, which may account for the subjective experience that many listeners report of having multi-valent or complex reactions to music, such as "bittersweet."

title: The emotional power of music: How music enhances the feeling of affective pictures
abstract: Music is an intriguing stimulus widely used in movies to increase the emotional experience. However, no brain imaging study has to date examined this enhancement effect using emotional pictures (the modality mostly used in emotion research) and musical excerpts. Therefore, we designed this functional magnetic resonance imaging study to explore how musical stimuli enhance the feeling of affective pictures. In a classical block design carefully controlling for habituation and order effects, we presented fearful and sad pictures (mostly taken from the IAPS) either alone or combined with congruent emotional musical excerpts (classical pieces). Subjective ratings clearly indicated that the emotional experience was markedly increased in the combined relative to the picture condition. Furthermore, using a second-level analysis and regions of interest approach, we observed a clear functional and structural dissociation between the combined and the picture condition. Besides increased activation in brain areas known to be involved in auditory as well as in neutral and emotional visual-auditory integration processes, the combined condition showed increased activation in many structures known to be involved in emotion processing (including for example amygdala, hippocampus, parahippocampus, insula, striatum, medial ventral frontal cortex, cerebellum, fusiform gyrus). In contrast, the picture condition only showed an activation increase in the cognitive part of the prefrontal cortex, mainly in the right dorsolateral prefrontal cortex. Based on these findings, we suggest that emotional pictures evoke a more cognitive mode of emotion perception, whereas congruent presentations of emotional visual and musical stimuli rather automatically evoke strong emotional feelings and experiences.

title: Virtual reality in neuroscience research and therapy
abstract: Virtual reality (VR) environments are increasingly being used by neuroscientists to simulate natural events and social interactions. VR creates interactive, multimodal sensory stimuli that offer unique advantages over other approaches to neuroscientific research and applications. VR's compatibility with imaging technologies such as functional MRI allows researchers to present multimodal stimuli with a high degree of ecological validity and control while recording changes in brain activity. Therapists, too, stand to gain from progress in VR technology, which provides a high degree of control over the therapeutic experience. Here we review the latest advances in VR technology and its applications in neuroscience research.

title: Anxiety Induced by Flooding Therapy for Phobias Does Not Elicit Prolactin Secretory Response*
abstract: &NA; Eight subjects with severe phobias to insects and small animals had blood samples taken for measurement of prolactin concentration at regular intervals during five sessions of 3 hr duration each. Severe anxiety was induced by treating the phobia with in vivo flooding during the middle hour of the third and fourth sessions. Despite intense anxiety experienced by the subjects, they showed no change in plasma prolactin levels.

title: Emotional processing and outcome of imaginal flooding therapy in Vietnam veterans with chronic posttraumatic stress disorder.
abstract: This study examined emotional processing and outcome in 20 Vietnam veterans with chronic posttraumatic stress disorder (PTSD) who underwent imaginal flooding therapy. Results supported the occurrence of emotional processing, as manifest in significant activation, within-session habituation, and partial across-session habituation of physiologic and self-reported process variables. The flooding therapy produced only modest overall improvement, which was statistically significant for avoidance symptomatology measured by the impact of Events Scale (IOES) and number of intrusions per day recorded by the subject in a log. Symptomatic improvement appeared to generalize from a treated to an untreated experience. Heart rate activation during the first flooding session predicted a decrease in daily number of intrusive combat memories across the therapy. Otherwise, there was little association between extent of emotional processing and therapeutic outcome. The results provide limited support for the notion that mobilization of psychophysiologic arousal during exposure therapy predicts improvement.

title: Time course of the subjective emotional response to aversive pictures: relevance to fMRI studies
abstract: Using functional magnetic resonance imaging (fMRI) to study brain activity related to the experience of emotion presents unique challenges to neuroscientists. One important consideration arises when an experimentally induced subjective emotional response persists after the end of the emotional stimulation epoch. In this case, brain activity related to the emotional response may continue during the subsequent control or comparison epoch. The comparison epoch of the experiment may then contain a lingering emotional response. This study was conducted to better understand the time course of the subjective emotional response to intensely aversive pictures, with the goal of applying this knowledge to the design and analysis of fMRI studies of emotion. A total of 18 women in two separate experiments were shown a series of aversive, neutral and scrambled pictures presented in alternating block designs. Subjects rated the intensity of their negative feelings every 4 s while viewing the pictures. Results indicate that the subjective emotional response persists well after the end of the emotional stimulation epoch. Following a 16-s block of aversive pictures, an average of an additional 16 s elapsed before self-reported negative feelings showed a 74-80% decline. These data suggest that fMRI studies of emotion should consider the time course of the subjective response to emotionally laden stimuli.

title: Emotion and motivation I: defensive and appetitive reactions in picture processing.
abstract: Emotional reactions are organized by underlying motivational states--defensive and appetitive--that have evolved to promote the survival of individuals and species. Affective responses were measured while participants viewed pictures with varied emotional and neutral content. Consistent with the motivational hypothesis, reports of the strongest emotional arousal, largest skin conductance responses, most pronounced cardiac deceleration, and greatest modulation of the startle reflex occurred when participants viewed pictures depicting threat, violent death, and erotica. Moreover, reflex modulation and conductance change varied with arousal, whereas facial patterns were content specific. The findings suggest that affective responses serve different functions-mobilization for action, attention, and social communication-and reflect the motivational system that is engaged, its intensity of activation, and the specific emotional context.

title: Progressive perceptual audio rendering of complex scenes
abstract: Despite recent advances, including sound source clustering and perceptual auditory masking, high quality rendering of complex virtual scenes with thousands of sound sources remains a challenge. Two major bottlenecks appear as the scene complexity increases: the cost of clustering itself, and the cost of pre-mixing source signals within each cluster. In this paper, we first propose an improved hierarchical clustering algorithm that remains efficient for large numbers of sources and clusters while providing progressive refinement capabilities. We then present a lossy pre-mixing method based on a progressive representation of the input audio signals and the perceptual importance of each sound source. Our quality evaluation user tests indicate that the recently introduced audio saliency map is inappropriate for this task. Consequently we propose a "pinnacle", loudness-based metric, which gives the best results for a variety of target computing budgets. We also performed a perceptual pilot study which indicates that in audio-visual environments, it is better to allocate more clusters to visible sound sources. We propose a new clustering metric using this result. As a result of these three solutions, our system can provide high quality rendering of thousands of 3D-sound sources on a "gamer-style" PC.

title: Ventriloquism aftereffects occur in the rear hemisphere
abstract: After exposure to a consistent spatial disparity of auditory and visual stimuli, subjective localization of sound sources is usually shifted in the direction of the visual stimuli. This study investigates whether such aftereffects can be observed in humans after exposure to a conflicting bimodal stimulation in virtual reality and whether these aftereffects are confined to the trained locations. Fourteen subjects participated in an adaptation experiment, in which auditory stimuli were convolved with non-individual head-related transfer functions, delivered via headphones. First, we assessed the auditory localization of subjects in darkness. They indicated the perceived direction of a sound using an angular pointer. We then immersed the subjects in a virtual environment by means of a head-mounted display. They were asked to reproduce sequences of movements of virtual objects with a mouse click on the objects. However, we introduced a spatial disparity of 15 degrees between the visual event and the concurrent auditory stimulation. After 20 min of exposure, we tested the subjects again in total darkness to determine whether their auditory localization system had been modified by the conflicting visual signals. We observed a shift of subjective localization towards the left in both dorsal and frontal hemifields of the subject, mainly for auditory stimuli located in the right hemispace. This result suggests that interaural difference cues and monaural spectral cues were not equally adapted, and that visual stimuli mainly influence the processing of binaural directional cues of sound localization.

title: Manual for the State-Trait Anxiety Inventory
abstract: The STAI serves as an indicator of two types of anxiety, the state and trait anxiety, and measure the severity of the overall anxiety level.The STAI, which is appropriate for those who have at least a sixth grade reading level, contains four-point Likert items. The instrument is divided into two sections, each having twenty questions. Approximately 15 minutes are required for adults to complete the both STAI. The number on the scale is positively correlated to the anxiety related to in the question.

title: The Experience of Presence: Factor Analytic Insights
abstract: Within an embodied cognition framework, it is argued that presence in a virtual environment (VE) develops from the construction of a spatial-functional mental model of the VE. Two cognitive processes lead to this model: the representation of bodily actions as possible actions in the VE, and the suppression of incompatible sensory input. It is hypothesized that the conscious sense of presence reflects these two components as spatial presence and involvement. This prediction was confirmed in two studies (N = 246 and N = 296) assessing self-reports of presence and immersion experiences. Additionally, judgments of realness were observed as a third presence component. A second-order factor analysis showed a distinction between presence, immersion, and interaction factors. Building on these results, a thirteen-item presence scale consisting of three independent components was developed and verified using confirmatory factor analyses across the two studies.

title: The practice of behavior therapy
abstract: Well, someone can decide by themselves what they want to do and need to do but sometimes, that kind of person will need some practice of behavior therapy references. People with open minded will always try to seek for the new things and information from many sources. On the contrary, people with closed mind will always think that they can do it by their principals. So, what kind of person are you?

title: Virtual reality treatment of claustrophobia: a case report.
abstract: The efficacy of a treatment for claustrophobia using only Virtual Reality (VR) exposure was examined. The subject was a 43-year-old female who suffered from clinically significant distress and impairment and sought psychological therapy. Eight individual VR graded exposure sessions were conducted. All self-report measures were reduced following VR exposure and were maintained at one month follow-up. The necessity of a theoretical framework for this new medium for exposure therapy is discussed.

title: Virtual Reality Treatment in Acrophobia: A Comparison with Exposure in Vivo
abstract: The aim of the present study was to evaluate the effectiveness of low-budget virtual reality exposure versus exposure in vivo in a within-group design in 10 individuals suffering from acrophobia. Virtual reality exposure was found to be at least as effective as exposure in vivo on anxiety and avoidance as measured with the Acrophobia Questionnaire (AQ), and even more effective on the Attitude towards Heights Questionnaire (AHQ). The present study shows that virtual reality exposure can be effective with relatively cheap hardware and software on stand-alone computers currently on the market. Further studies are recommended, in which virtual reality exposure is compared with in vivo exposure in a between-group design, thus enabling investigation of the long-term effects of virtual reality treatment.

title: Virtual Reality Graded Exposure in the Treatment of Acrophobia: A Case Report
abstract: This is the first case report to test the efficacy of computer-generated virtual reality (VR) for the treatment of acrophobia (fear of heights). The subject was a 19-year-old undergraduate student with a fear of heights, particularly of elevators. Twice weekly, sessions were conducted for 3 weeks, for a total of 5 sessions. Outcome was assessed on measures of anxiety, avoidance, attitude, distress, and included a behavioral avoidance test. VR graded exposure was successful in reducing fears of heights. VR graded exposure is proposed as a new medium for exposure therapy.

title: Autonomic correlates of the subjective anxiety scale.
abstract: The subjective anxiety scale is an assessment tool commonly used in behavioral research and therapy to quantify verbal report of private events, usually states of fear. An investigation was undertaken to determine the extent of relationship between the subjective anxiety scale and two concurrent measures of autonomic arousal, peripheral vasoconstriction and heart rate. Measures were recorded during baseline, surgical film and second baseline phases for 20 subjects. The subjective and autonomic measures indicated increased anxiety during the film phase and significant correlations were found between subjective report of anxiety and the two physiological indices. The results support the continued use of the subjective anxiety scale as a clinical assessment tool. Practice and research implications are discussed.

title: Auditory-Visual Virtual Reality as a Diagnostic and Therapeutic Tool for Cynophobia
abstract: Traditionally, virtual reality (VR) exposure-based treatment concentrates primarily on the presentation of a high-fidelity visual experience. However, adequately combining the visual and the auditory experience provides a powerful tool to enhance sensory processing and modulate attention. We present the design and usability testing of an auditory-visual interactive environment for investigating VR exposure-based treatment for cynophobia. The specificity of our application involves 3D sound, allowing the presentation and spatial manipulations of a fearful stimulus in the auditory modality and in the visual modality. We conducted an evaluation test with 10 participants who fear dogs to assess the capacity of our auditory-visual virtual environment (VE) to generate fear reactions. The specific perceptual characteristics of the dog model that were implemented in the VE were highly arousing, suggesting that VR is a promising tool to treat cynophobia.

title: A continuous measure of phasic electrodermal activity
abstract: Electrodermal activity is characterized by the superposition of what appear to be single distinct skin conductance responses (SCRs). Classic trough-to-peak analysis of these responses is impeded by their apparent superposition. A deconvolution approach is proposed, which separates SC data into continuous signals of tonic and phasic activity. The resulting phasic activity shows a zero baseline, and overlapping SCRs are represented by predominantly distinct, compact impulses showing an average duration of less than 2 s. A time integration of the continuous measure of phasic activity is proposed as a straightforward indicator of event-related sympathetic activity. The quality and benefit of the proposed measure is demonstrated in an experiment with short interstimulus intervals as well as by means of a simulation study. The advances compared to previous decomposition methods are discussed.

title: Correction for multiple testing: is there a resolution?
abstract: In most studies, many statistical tests are performed. They can be run to compare the groups at baseline, look at relationships among the various measures, and, for intervention trials, examine more than one end point. As the number of tests increases, so does the probability of finding at least one of them to be statistically significant just by chance (the problem of multiplicity). A number of procedures have been developed to deal with multiplicity, such as the Bonferroni correction, but there is continuing controversy regarding if and when these procedures should be used. In this article, we offer recommendations about when they should and should not be brought into play.

title: Imagery and fear influence height perception.
abstract: The current study tested whether height overestimation is related to height fear and influenced by images of falling. To assess perceptual biases, participants high (n=65) versus low (n=64) in height fear estimated the vertical extents of two balconies using a visual matching task. On one of the balconies, participants engaged in an imagery exercise designed to enhance the subjective sense that they were acting in a dangerous environment by picturing themselves falling. As expected, we found that individuals overestimated the balcony's height more after they imagined themselves falling, particularly if they were already afraid of heights. These findings suggest that height fear may serve as a vulnerability factor that leads to perceptual biases when triggered by a stressor (in this case, images of falling).

title: Near space and its relation to claustrophobic fear
abstract: It is well established that the near space immediately surrounding the body (also known as peripersonal space) is represented differently than the space farther away. When bisecting horizontal lines, for example, neurologically-healthy adults show a slight leftward bias (known as pseudoneglect) in near space; this attentional bias, however, transitions rightward in far space. Recent research has used the rate at which this shift occurs to quantify the extent (i.e., size) of near space, showing consistent individual differences that relate to arm length. Here we examined whether the size of near space relates to individual differences in claustrophobic fear, as measured by reported anxiety of enclosed spaces and physically restrictive situations. Trait feelings of claustrophobic fear predicted the size of near space, even after accounting for the relation to arm length. Specifically, people with larger near spaces reported higher rates of claustrophobic fear than people with smaller near spaces. These results are consistent with a defensive function of near space representation and suggest that an over-projection of near space may play an important role in the etiology of claustrophobia.

title: Threat modulates perception of looming visual stimuli
abstract: Objects on a collision course with an observer produce a specific pattern of optical expansion on the retina known as looming, which in theory exactly specifies time-to-collision. Looming stimuli produce stereotyped defensive responses in monkeys and human infants, indicating that the primate visual system is intrinsically tuned to interpret this stimulus as threatening. We investigated how emotional reactions to the semantic content of a looming stimulus affects perceived time-to-collision. We presented either threatening (snakes, spiders) or non-threatening (butterflies and rabbits) stimuli which expanded in size at a rate indicating one of five different times to contact (3.0, 3.5, 4.0, 4.5 or 5.0 s). After 1 s, the image disappeared and participants judged when it would have made contact with them continuing on the same trajectory. Consistent with previous findings, time-to-collision judgments systematically underestimated true arrival time. More importantly, our results showed that the magnitude of this underestimating significantly increased for threatening, compared to non-threatening, stimuli. Further, the magnitude of this increase was correlated with participants’ self-reported fear of snakes and spiders. Traditionally, looming has been interpreted as a purely optical cue to collision. Against that view, our results demonstrate that the semantic content of visual stimuli modulates perceived time-to-contact. Our results suggest that perceived threat led participants to use a larger margin of safety: underestimating time-to-collision errs on the side of allowing more time to prepare defensive responses (or flee). More generally, these results demonstrate that emotion has widespread effects on even very basic aspects of perception.

title: Perceiving Control Over Aversive and Fearful Events Can Alter How We Experience Those Events: An Investigation of Time Perception in Spider-Fearful Individuals
abstract: We used a time perception task to study the effects of the subjective experience of control on emotion and cognitive processing. This task is uniquely sensitive to the emotionality of the stimuli: high-arousing negative stimuli are perceived as lasting longer than high-arousing positive events, while the opposite pattern is observed for low-arousing stimuli. We evaluated the temporal distortions of emotionally charged events in non-anxious (Experiments 1 and 5) and spider-fearful individuals (Experiments 2–4). Participants were shown images of varying durations between 400 and 1600 ms and were asked to report if the perceived duration of the image seemed closer to a short (400 ms) or to a long (1600 ms) standard duration. Our results replicate previous findings showing that the emotional content of the image modulated the perceived duration of that image. More importantly, we studied whether giving participants the illusion that they have some control over the emotional content of the images could eliminate this temporal distortion. Results confirmed this hypothesis, even though our participant population was composed of highly reactive emotional individuals (spider-fearful) facing fear-related images (spiders). Further, we also showed that under conditions of little-to-no control, spider-fearful individuals perceive temporal distortions in a distinct manner from non-anxious participants: the duration of events was entirely determined by the valence of the events, rather than by the typical valence × arousal interaction. That is, spider-fearful participants perceived negative events as lasting longer than positive events, regardless of their level of arousal. Finally, we also showed that under conditions of cognitive dissonance, control can eliminate temporal distortions of low arousal events, but not of high-arousing events, providing an important boundary condition to the otherwise positive effects of control on time estimation.

title: Born to choose: the origins and value of the need for control
abstract: Belief in one's ability to exert control over the environment and to produce desired results is essential for an individual's wellbeing. It has repeatedly been argued that perception of control is not only desirable, but is also probably a psychological and biological necessity. In this article, we review the literature supporting this claim and present evidence of a biological basis for the need for control and for choice-that is, the means by which we exercise control over the environment. Converging evidence from animal research, clinical studies and neuroimaging suggests that the need for control is a biological imperative for survival, and a corticostriatal network is implicated as the neural substrate of this adaptive behavior.

title: Virtual reality in the treatment of spider phobia: a controlled study.
abstract: This study explored whether virtual reality (VR) exposure therapy was effective in the treatment of spider phobia. We compared a treatment condition vs. a waiting list condition in a between group design with 23 participants. Participants in the VR treatment group received an average of four one-hour exposure therapy sessions. VR exposure was effective in treating spider phobia compared to a control condition as measured with a Fear of Spiders questionnaire, a Behavioural Avoidance Test (BAT), and severity ratings made by the clinician and an independent assessor. Eighty-three percent of patients in the VR treatment group showed clinically significant improvement compared with 0% in the waiting list group, and no patients dropped out. This study shows that VR exposure can be effective in the treatment of phobias.

title: Virtual Reality in Psychotherapy: Review
abstract: Virtual reality (VR) has recently emerged as a potentially effective way to provide general and specialty health care services, and appears poised to enter mainstream psychotherapy delivery. Because VR could be part of the future of clinical psychology, it is critical to all psychotherapists that it be defined broadly. To ensure appropriate development of VR applications, clinicians must have a clear understanding of the opportunities and challenges it will provide in professional practice. This review outlines the current state of clinical research relevant to the development of virtual environments for use in psychotherapy. In particular, the paper focuses its analysis on both actual applications of VR in clinical psychology and how different clinical perspectives can use this approach to improve the process of therapeutic change.

title: Efficacy of virtual reality exposure therapy to treat driving phobia: a case report.
abstract: An AB case design was used to examine the efficacy of virtual reality exposure therapy (VRET) in treating driving phobia. After a one week baseline, the patient received three treatment sessions over a ten day period. Treatment included practice of four VR driving scenarios. Peak anxiety decreased within and across sessions. Ratings of anxiety and avoidance declined from pre-treatment and post-treatment, with gains maintained at seven month followup. Phobia-related interference in daily functioning similarly decreased. The results suggest that it would be useful to further evaluate the efficacy of VRET for driving phobia in controlled clinical trials.

title: Cross-modal interactions in the experience of musical performances: Physiological correlates
abstract: This experiment was conducted to investigate cross-modal interactions in the emotional experience of music listeners. Previous research showed that visual information present in a musical performance is rich in expressive content, and moderates the subjective emotional experience of a participant listening and/or observing musical stimuli [Vines, B. W., Krumhansl, C. L., Wanderley, M. M., & Levitin, D. J. (2006). Cross-modal interactions in the perception of musical performance. Cognition, 101, 80--113.]. The goal of this follow-up experiment was to replicate this cross-modal interaction by investigating the objective, physiological aspect of emotional response to music measuring electrodermal activity. The scaled average of electrodermal amplitude for visual-auditory presentation was found to be significantly higher than the sum of the reactions when the music was presented in visual only (VO) and auditory only (AO) conditions, suggesting the presence of an emergent property created by bimodal interaction. Functional data analysis revealed that electrodermal activity generally followed the same contour across modalities of presentation, except during rests (silent parts of the performance) when the visual information took on particular salience. Finally, electrodermal activity and subjective tension judgments were found to be most highly correlated in the audio-visual (AV) condition than in the unimodal conditions. The present study provides converging evidence for the importance of seeing musical performances, and preliminary evidence for the utility of electrodermal activity as an objective measure in studies of continuous music-elicited emotions.

title: The Merging of the Senses
abstract: Bringing together neural, perceptual, and behavioral studies, The Merging of the Senses provides the first detailed review of how the brain assembles information from different sensory systems in order to produce a coherent view of the external world. Stein and Meredith marshall evidence from a broad array of species to show that interactions among senses are the most ancient scheme of sensory organization, an integrative system reflecting a general plan that supersedes structure and species. Most importantly, they explore what is known about the neural processes by which interactions among the senses take place at the level of the single cell.The authors draw on their own experiments to illustrate how sensory inputs converge (from visual, auditory, and somatosensory modalities, for instance) on individual neurons in different areas of the brain, how these neurons integrate their inputs, the principles by which this integration occurs, and what this may mean for perception and behavior. Neurons in the superior colliculus and cortex are emphasized as models of multiple sensory integrators.Barry E. Stein is Professor of Physiology and M. Alex Meredith is Associate Professor of Anatomy, both at the Medical College of Virginia, Virginia Commonwealth University.

title: Multisensory integration: current issues from the perspective of the single neuron
abstract: For thousands of years science philosophers have been impressed by how effectively the senses work together to enhance the salience of biologically meaningful events. However, they really had no idea how this was accomplished. Recent insights into the underlying physiological mechanisms reveal that, in at least one circuit, this ability depends on an intimate dialogue among neurons at multiple levels of the neuraxis; this dialogue cannot take place until long after birth and might require a specific kind of experience. Understanding the acquisition and usage of multisensory integration in the midbrain and cerebral cortex of mammals has been aided by a multiplicity of approaches. Here we examine some of the fundamental advances that have been made and some of the challenging questions that remain.

title: Behavioral models for anxiety and multisensory integration in animals and humans
abstract: Complaints related to dizziness, balance problems and spatial disorientation in psychiatry have seldom been considered as a possible manifestation of a distorted multisensory integrative ability. Several kinds of mismatches among simultaneous sensory information are encountered in everyday life but despite these, the central nervous system usually manages to update the internal representation of the body in the surrounding space. In some cases, a sensory mismatch may elicit an erroneous perception of the body in space, resulting in anxiety, dizziness and balance problems. As vestibular system dysfunction leads to dizziness and disorientation, it has been hypothesized that a peripheral vestibular abnormality could explain the presence of certain symptoms related to sensory mismatches in anxiety disorders. Several studies tried to find a link between panic disorder with or without agoraphobia and vestibular system dysfunction. Yet, even though some vestibular abnormalities have been demonstrated in these patients, it is difficult to demonstrate a cause-and-effect relationship between panic disorder and vestibular dysfunction. However, this does not rule out a possible influence of anxiety on normal vestibular function. The study of the relation between vestibular system and anxiety has to take into account that the vestibular system has three main functions: to maintain equilibrium through the vestibular spinal reflexes; to stabilize the visualization of the world through the vestibular-ocular reflex; to contribute to perception and orientation in space. We will review different studies in humans, which have particularly paid attention to the third function and its relation to anxiety. Animal experiments offer possibilities to more precisely analyze the different parameters underlying the behavioral results, as well as possible pharmacological actions on them. Two attempts have been made by our group to model, in mice, the preceding human data on integrated functional sensory relations of the body to space in anxiety disorders: the rotating beam and the rotating tunnel. We summarize here the main results obtained.