The Influence of Emotional Content on Event-Related Brain Potentials during Spoken Word Processing

In our everyday lives, language is an indispensable tool for communication and for the establishment and preservation of social interactions. Language can be divided in two different modalities, namely the auditory and visual modality. The auditory modality comprises spoken language, whereas the visual modality is composed of the written part of language. Even though a day without speaking is probably inconceivable for most of us, previous research has neglected the investigation of effects of emotional meaning on spoken word processing, as opposed to processing of written words. For written word processing, several studies elaborately investigated effects of emotional meaning on event-related brain potentials (ERPs). In contrast to this, emotional content in spoken word processing was investigated only very occasionally and mostly either in its interaction with emotional prosody or focused merely on the existence of a specific component. Therefore, it remains an open question how and at which stages emotional content of spoken words affects event-related brain potentials regardless of emotional prosody and whether it shows similarities to the processing of written emotional words. In this thesis, I investigate the processing of single spoken words with positive, neutral and negative content, with the objective of understanding whether emotional content of spoken words leads to emotion effects in ERPs and if those are comparable to those shown for written words. In the first study of this dissertation, spoken words of emotional and neutral content were presented to participants at two different volume levels to elucidate possible interactions of emotion effects with bottom-up attention effects driven by stimulus size. For visual stimuli of emotional content as pictures and written words, stimulus size has been shown to increase emotion-related ERP effects, for example at level of the early posterior negativity (EPN). It was investigated whether this augmented relevance of larger visual stimuli might be transferred to the auditory modality. Negative emotional content leads to an increased frontal positivity and parieto-occipital negativity between 370 and 530 ms. This component reveals resemblance to the visual EPN, however, the negativity expands further towards central scalp areas. Therefore, the question arises whether this component might reflect an auditory counterpart of the visual EPN. Importantly, no interaction of this emotion-related ERP component with volume level is revealed. The following aspects, if comparing them to the visual modality, point towards a broader difference between visual and auditory language processing: The missing interaction of stimulus size and emotion effects, differences in topographies of the emotion effects and the different latencies compared to the visual EPN. The second part of this thesis aims at a more direct comparison of emotion effects in the visual and auditory modality. For this purpose, a second study was conducted, in which the same words were presented visually and auditorily to the participants. Spoken words were either produced by a computer-voice (Experiment 1) or a human speaker (Experiment 2). This study was designed in order to investigate the existence of an “auditory EPN” and its boundary conditions. In addition, it was investigated whether the higher social relevance of a human voice augments the emotion effects. In both experiments, emotion effects are evident. For written words, effects are evident between 230 and 400 ms, in the early posterior negativity (EPN), for spoken words between 460 and 510 ms. Interestingly, when considering the scalp distribution of the ERP differences between emotional and neutral auditory words, the effect shows even higher similarity to the visual EPN than in the first part of this thesis. Source localization revealed comparable neural generators in the superior parietal lobule (SPL) and inferior parietal lobule (IPL) in both the visual and auditory EPN time window. The findings indicate similarities in the processing of emotional content across modalities that – at least partly – rely on the same neural system. However, these similarities are surprising since the visual EPN is assumed to reflect enhanced sensory encoding in visual areas. The emotion effects revealed in the studies described above differed in terms of latencies, topographies and the valence that elicits the effect (positive or negative). Therefore, in the last part of this thesis, I investigate potential causes for these differences. Sex differences at scalp topography level are revealed, however, they can not explain the reported differences between the studies. It is hypothesized that both studies reveal the same auditory emotion-related component (AEC) in a comparable time frame (Study 1: 477 530; Study 2: 464 515 ms), which was preceded by an earlier emotion effect (371-477 ms) with a N400-like scalp distribution in Study 1. Even though no interactions of emotional content and volume level are revealed, presumably volume level manipulation in the first study changed the context of the experiment, which caused the additional effect. Even though no verifiable cause for the described differences in emotion effects could be revealed, I was able to show the existence of an auditory emotion-related component that is elicited by emotional (compared to neutral) content during spoken word processing. This component is reflected in an anterior positivity and posterior negativity around 460 to 520 ms after word onset. It is invariantly occurring, unaffected by the social significance of the speaker’s voice or by a volume level manipulation. Concerning a comparison of the underlying neural network during the processing of content in spoken and written words, it can be concluded that the processing activates brain areas which are at least partly shared in the SPL and IPL. Even though the scalp distribution of the AEC reveals high similarity to the visual EPN, it is not assumed that this effect reflects an auditory counterpart. This conclusion is drawn first on the fact that the typical EPN-distribution is only revealed when calculating the difference waves of emotional and neutral stimuli. The resulting posterior negativity reflects enhanced activation in visual areas to emotional stimuli. The analysis of the underlying neural generators for the difference between auditory emotional and neutral stimuli do not show significant results. However, underlying topographies of the separated emotion categories reveal that the similarity at the level of difference waves resulted from entirely different scalp distributions. Future research has to control stimulus material more strictly in terms of word length or recognition point in order to reduce the temporal jitter in the data and determine the neural generators of the auditory emotion-related component.