Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognition (2023)

Abstract: Previous work has established that a person’s demographics and speech style affect how well speech processing models perform for them. But where does this bias come from? In this work, we present the Speech Embedding Association Test (SpEAT), a method for detecting bias in one type of model used for many speech tasks: pre-trained models. The SpEAT is inspired by word embedding association tests in natural language processing, which quantify intrinsic bias in a model’s representations of different concepts, such as race or valence—something’s pleasantness or unpleasantness—and capture the extent to which a model trained on large-scale socio-cultural data has learned human-like biases. Using the SpEAT, we test for six types of bias in 16 English speech models (including 4 models also trained on multilingual data), which come from the wav2vec 2.0, HuBERT, WavLM, and Whisper model families. We find that 14 or more models reveal positive valence (pleasantness) associations with abled people over disabled people, with European-Americans over African-Americans, with females over males, with U.S. accented speakers over non-U.S. accented speakers, and with younger people over older people. Beyond establishing that pre-trained speech models contain these biases, we also show that they can have real world effects. We compare biases found in pre-trained models to biases in downstream models adapted to the task of Speech Emotion Recognition (SER) and find that in 66 of the 96 tests performed (69%), the group that is more associated with positive valence as indicated by the SpEAT also tends to be predicted as speaking with higher valence by the downstream model. Our work provides evidence that, like text and image-based models, pre-trained speech based-models frequently learn human-like biases when trained on large-scale socio-cultural datasets. Our work also shows that bias found in pre-trained models can propagate to the downstream task of SER.

## introduction recent approaches to many speech tasks rely on pre-trained models: large models trained to learn speech representations (multi-dimensional matrices referred to as embeddings) from large-scale corpora, which can be adapted to a variety of tasks @xcite @xcite . in computer vision and natural language processing, methods called embedding association tests (eats) have been used to evaluate biases in the ways pre-trained models represent social groups, allowing researchers to identify bias early in the machine learning pipeline, before it propagates to downstream tasks @xcite @xcite @xcite . in this paper, we present the first intrinsic association and bias evaluation method for pre-trained models in speech processing: the speech embedding association test @xcite . a speat measures bias related to two social groups in a speech model, and produces an effect size d, which when positive indicates that the speech model favors the social group that humans also tend to favor. we evaluate the speat first by testing whether it reveals positive effect sizes, (congruent with human stereotypes), for biases related to twelve social groups. the types of bias that we study have been documented in large populations, and all involve associations with valence, a term often used in psychology literature to describe emotions. valence is frequently equated with "pleasantness" or "pleasure," where positive valence indicates something pleasant and negative valence indicates something unpleasant @xcite @xcite . we study associations with valence due to its role as a primary dimension of affect, strong signal in speech and language, and determinant of how people form attitudes @xcite . using the speat, we find that 15 of 16 models we test show bias for u.s. accented speakers over non-u.s. accented speakers (where relative to non-u.s. accented speakers, u.s. accented speakers are more associated with positive valence than negative valence ), 15 of 16 models show bias for young speakers over old speakers, 14 of 16 models show bias for female speakers over male speakers, 14 of 16 models show bias for european-american speakers over african-american speakers, and 14 of 16 models show bias for abled speakers over disabled speakers. our results indicate that, as with models trained in other modalities, models trained on large corpora of speech data also learn humanlike biases. to understand the potential impact of these biases, we evaluate whether results found with the speat are indicative of downstream effects. we do so by considering the task of speech emotion recognition (ser), which predicts emotions based on speech @xcite . we compare speat scores for two social groups to disparities in a downstream model's predictions of valence for speech from the groups. 1 we find that social groups that have positive associations with valence in pretrained models also tend to be predicted by downstream ser models as more positive in valence: for 66 of 96 (69%) speats performed, a downstream ser model tends to predict speech from the social group favored in the speat as more positively valenced than speech from the other group considered. as negative valence is associated with anger and sadness @xcite , this could potentially result in speech from one group of people being translated as more frequently angry than speech from another in a speech translation system that considered emotion when performing translations, or speech from one group being treated as more frequently sad than speech from another in a diagnostic model used in a mental health setting, two suggested applications for ser models @xcite . beyond showing that biases found in pre-trained models can propagate, our work adds to the growing body of evidence showing bias in 1 previous work studying propagation between intrinsic bias in models and bias in how they perform downstream has suggested that upstream bias is more likely to be indicative of downstream bias if the biases are conceptually similar @xcite . because the speat relates to associations with valence, we choose ser as a related downstream task. automated emotion recognition (aer), one of many issues that have been raised concerning this area of research @xcite . finally, we provide an approach for studying how the number of stimuli (sample size) used in an eat can change its results. eats performed in other modalities have used fixed numbers of stimuli to represent social groups and concepts when measuring bias. to evaluate how an eat score changes when differing sample sizes are used, we calculate bootstrap estimates of the standard error (se) at different sample sizes @xcite . the se measures how much a statistic would change if it were calculated repeatedly based on new data, and lower se values indicate that there is less uncertainty associated with the statistic. we find that the se of the speat decreases sharply as the number of stimuli used to represent social groups increases, for example that increasing the number of stimuli from 2 to 10 can lead to an increase in precision by a factor of two. our results show that the number of stimuli used in an eat can have a large effect on the uncertainty of its effect size d. 2 2 background and related work we now survey related work on pre-trained models in speech processing, as well as work measuring bias in speech models and models in other modalities. pre-trained models in speech processing pretrained models initially learn from large quantities of unstructured data, for example corpora containing hundreds of thousands of hours of speech, and can then be adapted using smaller structured datasets to a variety of specific tasks @xcite . pre-trained models used for speech processing often train on speech samples that are of variable lengths @xcite . after initial pretraining, a downstream model for a specific task may then be created based on the pre-trained model, using transfer learning. this is often done using fine-tuning, whereby slight architectural changes are made to the model before it is retrained on a smaller dataset, or feature extraction, which treats the pre-trained model as a pre-processing step for a downstream task-specific model @xcite . pre-trained models are often transformer based, and with feature extraction a user extracts numerical representations of input data, (also called 2 code for this work is available at https://github.com/isaaconline/speat embeddings), after each layer in the neural network. the user then employs these embeddings as input data when training a downstream model for a new task @xcite . ## methodology building on eats, the speat measures how a model's embeddings of stimuli representing two target concepts, (for example female and male), relate to its embeddings of stimuli representing two attribute concepts (for example positive and negative valence). after extracting embeddings for stimuli corresponding to the four different concepts from a speech model, the speat is carried out by measuring the relative cosine similarities between the speech models' embeddings of samples representing the four concepts. let x and y be the sets of embeddings representing the target concepts, for example embeddings derived from speech samples from female (x) and male (y ) speakers respectively, and a and b be the set of embeddings representing the attribute concepts, such as embeddings derived from speech samples rated as being positive (a) and negative (b) in valence, respectively. similar to other eats, we present an effect size metric, the speat d, which shows which of the target sets, x or y , is relatively more similar to the first attribute set, a, than to the second, b. as eats were originally based on iats, the speat d is based on the iat d, an adaptation of cohen's d that uses unified standard deviation rather than pooled standard deviation @xcite . values of cohen's d of 0.20, 0.50 and 0.80 were originally introduced as being small, medium, and large @xcite . to calculate the speat d, as for past eats, first the mean cosine similarity between a target embedding w and each embedding @xmath0, followed by the mean cosine similarity between w and each @xmath1. the difference between these means is the relative association between each embedding @xmath2: for example, if w were the embedding of a speech sample from a male speaker, this difference in means s would measure how much more associated the sample was with positive valence than negative valence. these values for each of the target stimuli are then aggregated to construct the effect size measurement, d. the speat d is given below, and shows how much closer the embeddings in x are to a than b, relative to the embeddings in y . the effect size is the same as that used by @xcite and @xcite : before cosine distances can be compared, however, embeddings of target and attribute stimuli need to be extracted. pre-trained speech models, like some of the sentence encoders studied by @xcite with the sentence embedding association test (seat), create dynamically sized embeddings depending on the length of the sequence fed to the model as input @xcite @xcite . because embeddings are compared using cosine similarity in an eat, which requires operands to have the same dimensions, this means that raw embeddings cannot be used directly. another relevant feature of the pre-trained speech processing models we consider, also similar to many models tested with the seat, is that these models are multi-layered, with different layers providing information that is relevant to different downstream tasks @xcite @xcite . recent approaches for adapting pre-trained models to specific tasks based on feature extraction rely on embeddings from all layers in the pre-trained model, rather than just a single layer @xcite @xcite . intuitively, embeddings from internal layers may contain information not relevant to the task the model was pre-trained for, but important for the task the model is being adapted to. in speech, for example, embeddings from later layers in hubert and wavlm models have been shown to be more useful for granular tasks like asr than embeddings from earlier layers, while for more speaker-centric tasks such as sid, embeddings from earlier in the models have been shown to be useful as well @xcite . because of the similarities in the models they study, as well as in the way that embeddings from the models are used, the speat follows the seat in how it handles variable-sized, multi-layered embeddings @xcite . while the seat uses differing methods for aggregating embeddings for different sentence encoders, however, we elect to apply the aggregation method they propose for elmo @xcite to all models we test. for each stimuli, this method involves taking the mean embedding within a layer, then summing these averaged embeddings together. we elect to use this aggregation method because the methods that @xcite propose for other models only use embeddings from a single layer in a model, which discards a large amount of information that is sometimes used when adapting pre-trained speech models to downstream tasks and can potentially contain bias @xcite . we do test alternative strategies for extracting embeddings in appendix d, however. ## data we now describe the pre-trained models we focus on, their training data, and relevant features of the stimuli that we use for evaluating these models. pre-trained models and their training data to study pre-trained models, we chose model families that are either widely used or achieve state-of-theart performance on relevant speech benchmarks. we use multiple models from each family, rather than merely the largest model, as it is not clear that models from the same family encode the same biases, and multiple models from a family are often used by practitioners-not necessarily just the largest one. we use 16 models from these families: three from wav2vec 2.0, three from hubert, three from wavlm, and seven from whisper. further details concerning the models we use appear in appendix b. of these models, wav2vec 2.0, hubert, and wavlm use self-supervised learning (ssl) and train exclusively on speech, while models from the whisper family, which use weakly supervised learning (wsl), train on speech paired with transcripts. the wav2vec 2.0, hubert, and wavlm models we use are trained on either librispeech @xcite or libri-light @xcite , speech datasets derived from readings of audio books recorded by volunteers as part of the lib-rivox project. in addition to the libri-light training set, some wavlm models were also trained on additional data: 10,000 hours of audio from the gigaspeech corpus @xcite , which is derived from podcasts, youtube, and audiobooks, as well as 24,000 hours of audio from the voxpopuli corpus @xcite , which is derived from european parliament events. the whisper models we use are either trained on a multilingual corpus, which consists of 680,000 hours of audio paired with transcripts in 97 distinct languages, or on the subset of the multilingual dataset which is in english. 3 weber ( 2021 ) performed an analysis of users who both had contributed to librivox and had catalog names that could be associated with gender, and identified 45.3% as female and 54.7% as male. beyond this we are not aware of any information documenting the extent to which different speech styles are represented in the dataset, such as information on speaking styles. we note however that similar speech datasets have relied on u.s. speech. for example, of speech in the common voice english (7.0) dataset @xcite labeled by accent, about 47% was identified as u.s. english, compared to the next most frequent accent, england english, which made up about 16% of labeled speech @xcite . 4 the people's speech dataset @xcite , as another example, is identified by its authors as being made up primarily of american english. we hypothesize that u.s. english is also more prominent in the datasets that were used for training the models we consider, and following best practices for iats that stimuli should be familiar to the entity being tested @xcite , we choose to center our work on u.s. english speech. we include results for u.k.-based tests in appendix g however. target and attribute stimuli following previous 3 transcripts in the multilingual dataset are either in the same language as the speech or in english. 4 common voice has been used for training similar speech models such as xlsr @xcite and unispeech @xcite . eats, we evaluate the speat by testing for biases that have been found in humans using iats and comparing the results in speech models to those found in humans ( because @xcite originally represented concepts visually, and because audio iats from @xcite and @xcite used short samples which may not fully capture variations in accents (for example recordings of speakers saying individual words), we adapt these tests by using alternate stimuli to represent the intended concepts. 5 to do so we follow guidmean across all models, and blue triangle refers to mean iat d for bias measured in humans. a positive speat d indicates that in the model's embeddings, speech samples from the first target group (e.g. abled speakers or european-american (ea) speakers) are relatively more associated with positive valence over negative valence than samples from the second target group are (e.g. disabled speakers or african-american (aa) speakers). a speat d with the same sign as the iat d indicates that the speech model tends to favor the group that humans also tend to favor. 14 or more (out of 16) models show bias results that are congruent with biases found in humans regarding ability, race, gender, age, and accent. to ensure that differences between the stimuli representing the target concepts in a test are only due to differences in these concepts, and not due to another factor, we ensure that speech samples between the two target groups are matched. stimuli are matched on gender as well as approximate age, with the exception of the female and male test, which is not matched on gender, the young and old test, which is not matched on age, the human and synthesized test, which is not matched on age, 6 and the abled and disabled test (for which age information was not made available for speakers). all tests, other than the test comparing speech from european-american speakers and from african-american speakers (which uses extemporaneous speech) are also matched in speech content, and involve phrases, sentences, or paragraphs that are repeated across conditions. further details on the target stimuli chosen, as well as on the matching procedures used, can be found in appendix c. 7 represents the target concepts using short phrases. 6 synthesized speech is generated with microsoft sapi 4.0, which contains the synthetic voices originally employed for evaluating bias in humans. we use this generator in order to ensure that the biases we measure in speech models relate to the same concepts as the biases we compare to in humans. each voice is microsoft sapi 4.0 is labeled with a gender or a gendered name that the voice is intended to represent, and we use these labels to ensure that the set of speech samples we use is balanced on gender. as of submission, code for the generator is available at https://github.com/tetyys/sapi4 . 7 to ensure that differences in associations with valence in the european-american and african-american test are not due to differences in semantics, we test whether the speech content from one social group tends to show more positive valence for attribute stimuli, we use speech from the morgan emotional speech set (mess) @xcite , a corpus of 1,800 semantically neutral recordings made by six caucasian actors, aged 19-21. the morgan emotional speech set is the only publicly-available corpus of which we are aware that contains sae speech rated on valence. voice actors were native speakers of american english, evenly split on gender, and were asked to read sentences while portraying one of four categorical emotions-happy, sad, angry, or calm. the recordings were then rated on valence by 10 listeners who were aged 19-28 and evenly split on gender. to ensure that differences in association with positive and negative valence are not due to differences in the speakers whose speech is used to represent the valenced concepts, we ensure that each speaker is represented equally in each valence pole. we select the 10 recordings from each speaker that have the highest average valence ratings to represent positively valenced speech, and the 10 recordings from each speaker that have the lowest average valence ratings to represent negatively valenced than speech content from the other. we pass transcripts for the speech from european-american and african-american speakers through five commonly used sentiment analysis tools @xcite @xcite @xcite , then use a welch's unequal variances t-test to evaluate whether the sentiment scores differ by race on average @xcite . each sentiment analysis tool gives three scores (negative sentiment, neutral sentiment, and positive sentiment); vader also gives a compound score. using the typical @xmath3.05 threshold, across all 16 scores output by these models, only one significant difference is found: the positive sentiment score given by the model from @xcite (@xmath4.02). in this case, however, the social group that is scored higher on positive sentiment for semantics is not the one that tends to be favored by the speech processing models. ## evaluation and results we provide three evaluations of the speat. first, we test whether it captures human-like biases in speech models by comparing results found in pretrained models using the speat to results found in humans using iats. second, as the speat is designed for pre-trained models that are often adapted to other tasks, we also evaluate the method by testing whether these biases propagate, and whether biases found in models using the speat are indicative of biases in downstream models. third, we measure how much uncertainty is associated with the speat effect size by studying how its se varies when different numbers of stimuli are used to represent target concepts. comparison to human biases similar to other eats, our first evaluation of the speat is to test whether it detects biases in speech models using ground truth data from human biases @xcite . we compare the direction of the speat d to the direction of the mean iat d among humans for a number of well documented biases to check for stereotypecongruence. for each speech model discussed in section 4, we calculate the speat d for each bias in if the speat were able to find human-like biases in pre-trained models in speech, we would expect speat d values to be identical in sign to the mean iat d values given in ## discussion the results in figure 1 show that pre-trained speech processing models trained on large corpora of speech data, like those trained on large corpora of image or text data, can easily learn human-like biases related to gender, race, ability, age, and accent. work in computer vision has suggested that biases detected by eats may arise from co-occurrence in the training data @xcite , and we hypothesize that co-occurrence may contribute here as well, for example if speakers who use one speech style tend to more frequently express positive valence than speakers who use other speech styles in the training set. we also hypothesize that the wide dispersion in the test comparing human and synthesized speech may be related to frequency as it is not clear that synthesized speech is contained in any of the pre-training corpora that the models we consider were trained on. beyond just identifying that biases can be learned from data in a new modality, however, this work also identifies a new means of representing target concepts. in using speaking styles to represent concepts, rather than names or words as in text-based models, we've shown that not only do models demonstrate bias based on language content, but also based on how, and by whom, it is delivered. the data used for training models in computer vision, natural language processing, and speech processing is inherently multi-faceted-it does not connect to social groups in just one way. this data is produced by people who belong to certain social groups; it may be about people of certain social groups; it may be recorded by people of certain social groups; and it may also have meaning to people of certain social groups. our results suggest that, at least in speech processing, models are able to learn associations beyond just content, and raises the question of whether there are differential associations related to the provenance of text and image data as well. would text of similar meaning, but written by people of different social groups, have different associations with valence? beyond adding further evidence for bias in aer @xcite , our experiment with ser models also adds evidence that biases in pre-trained models can propagate downstream. the biases that we test for in upstream and downstream models are connected in that they both involve valence-in the upstream model the speat tests for associations with valence, while in the downstream model we measure bias using standardized differences in predicted valence between speech samples from people of differing social groups (which have been matched using the procedure in section 4). studying the relationship between intrinsic bias in embeddings measured using an eat and performance bias in downstream tasks using a variety of performance metrics, tasks, and models, @xcite find that the relationship between the two biases is mostly uncorrelated. they do, however, find that one of two models they test shows a slightly positive relationship between intrinsic bias and bias in a conceptually related downstream task. work by @xcite similarly finds a slight connection between upstream and downstream bias in one of the two tasks they test, and @xcite find that when considering name-nationality bias, abstractive upstream models can pass bias on to a down-stream task. considering these works together, our results add more evidence that there can be a relationship between upstream and downstream bias when the two biases are related in concept. our work calls attention to the need for detailed study of bias mitigation techniques in pre-trained speech processing models. while we are not aware of any technical solutions for mitigating implicit bias in pre-trained speech models, work by @xcite has tested the effect of the makeup of the training set on performance biases observed in downstream tasks. their results suggest that altering dataset composition is likely not a complete solution to bias in pre-trained models, however. it is possible that methods that have previously been proposed for mitigating bias in other modalities may be useful for managing the bias that has been shown in speech. strategies that have been explored in computer vision and natural language processing include oversampling the training set to make it more representative of social groups; adversarial training, whereby in learning a task the model also learns to explicitly ignore demographic information; and embedding modification, whereby the geometry of the embeddings is altered in an attempt to remove information related to demographics @xcite . ## conclusion in this paper we contributed the speat, a novel method for measuring bias in pre-trained speech processing models. we used the speat to study state-of-the-art pre-trained speech models which are likely to have a wide impact on a variety of speech processing tasks. we tested for six categories of bias in 16 pre-trained models, and found consistent biases in how speech samples from speakers of different ages, races, genders, and abilities were associated with speech samples rated on valence by human listeners. we also showed that biases detected with the speat propagated to the downstream task of ser, as a pre-trained model displaying bias toward a group was correlated with bias in downstream models adapted to ser. these findings imply, first, that association-based biases are present in a larger number of modalities than had previously been known, and second, that upstream bias may propagate to downstream bias, when the two biases are connected.