# Crosslinguistic Intelligibility of Russian and German Speech in Noisy Environment

Authors: Rodmonga Potapova; Maria Grigorieva
Journal: Journal of Electrical and Computer Engineering
 (2017)

---

## Abstract

This paper discusses the results of the pilot experimental research dedicated to speech recognition and perception of the semantic content of the utterances in noisy environment. The experiment included perceptual-auditory analysis of words and phrases in Russian and German (in comparison) in the same noisy environment: various (pink and white) types of noise with various levels of signal-to-noise ratio. The statistical analysis showed that intelligibility and perception of the speech in noisy environment are influenced not only by noise type and its signal-to-noise ratio, but also by some linguistic and extralinguistic factors, such as the existing redundancy of a particular language at various levels of linguistic structure, changes in the acoustic characteristics of the speaker while switching from one language to another one, the level of speaker and listener’s proficiency in a specific language, and acoustic characteristics of the speaker’s voice.


---

## Body

## 1. Introduction

Speech intelligibility and speech recognition are important and trending topics of research in various fields of science: Linguistics, Medicine, Electrical Engineering, and Information Technology. Speech recognition process is investigated from different sides, as only an integrated approach could lead to a better understanding of this process. One of research directions is study of biological and neurologic mechanisms of speech perception [1–5]. Another research area is intelligibility of synthesized speech in noise [6, 7]. Much attention is paid to development of algorithms improving speech intelligibility in noise [8–10].Study of listener’s specifics on the process of speech recognition showed that music training affects positively speech-in-noise perception [11, 12]. Another important and rather controversial topic is perception of accented speech in noise by native and nonnative listeners; thus the research [13] revealed that native listeners were able to percept the test material at the same level regardless of the accent of the speaker while previous studies [14, 15] showed that speech of native speakers was generally found by native listeners more intelligible than speech of nonnative ones; nonnative listeners showed a trend of better perception of speech produced by speakers from the same language environment as themselves, that is, those having familiar accent [16, 17].In Russia studies of speech characteristics and speech intelligibility and recognition in noise environment started in the middle of 20th century ([18–20], etc.). Experiments [19, 20] on word intelligibility for Russian speech against white and pink noise with levels of signal-to-noise ratio of 0 dB and lower showed different results: according to [19] the results for these two types of noise were very similar, while according to [20] word intelligibility for speech against white noise was higher. At the end of 20th century a major study of speech perception was carried out in white noise with various levels of signal-to-noise ratio [21], which investigated influence of different factors on Russian speech recognition: natural versus synthesized speech, different parts of speech, number of syllables in the word, place of stress, different types of phonemes, and so forth, which resulted in the range of the factors, which actually have influence on speech perception and recognition in noise environment and the level of influence of each of these factors. Another study [22] investigated changes of speaker characteristics (such as voice pitch, tempo of speech, voice strength) in the conditions of switching from a native (Russian) language to a foreign one (English). The latest researches were focused on cognitive mechanisms of semantic content decoding of Russian speech in noise [23, 24], which demonstrated that dialogs have better intelligibility than monologs and reading, and such factors as background knowledge of listeners about the topic of the conversation and their general interest, as well as emotional level of the speaker, also influenced the process of speech perception.The current research studies perception of native (Russian) and nonnative (German) speech in noisy environment (pink and white types of noise were chosen for the experiment) focuses on the following aims:(1)
To identify the effect of the tested types of noise with various levels of signal-to-noise ratio (in comparison) on speech perception;(2)
To identify effects of linguistic and extralinguistic factors on speech perception in noisy environment.


## 2. Method and Experiment

Our pilot research included perceptual-auditory analysis at various levels of linguistic structure of speech utterances in Russian and German (in comparison) in the same noisy environment: various (pink and white) types of noise with various levels of signal-to-noise ratio as well as effects of linguistic and extralinguistic factors on speech perception in noisy environment.The research material of the study was a specially composed (according to the method of Potapova [25, 26]) ad hoc corpus of words and phrases in Russian and German in realizations of Russian and German native speakers, which were mixed with various (pink and white) types of noise with various levels of signal-to-noise ratio (0 dB, −3 dB, −6 dB, −9 dB, and −12 dB). The material allowed analyzing of protection and intelligibility degrees at acoustic, phonetic, syntactic, and lexical levels of linguistic structure.The following requirements for development of the ad hoc research material were stated:(1)
Test phrases should be grammatically and semantically linked and consist of words which exist in both languages;(2)
Various types of consonants and vowels should be represented in the test phrases;(3)
Acoustic realization of the chosen types of consonants and vowels should be similar and comparable in both languages;(4)
Comparable (by place and manner of articulation) consonants and vowels should be in identical positions in a syllable (for all test words and phrases in Russian and German);(5)
The rhythmic scheme of test words and phrases should be identical for both languages;(6)
Combinations with various types of vowels in stressed position in the first syllable (with regard to unilateral distribution) should be tested for each type of consonants.According to the requirements of the research material the following test phrases were formulated (see Table1: analyzed syllables are bold in the table; hereafter the system of IPA was used for transcriptions).Table 1
Test phrases for experiment.
N
Analyzed languages
Russian
German
1
Б
аба  била  Борю ＇babə  ＇bilə  ＇borju  (Eng.: Woman beat Borya)
Barby bittet Boris  ＇barbi  ＇bitət  ＇bo:ris  (Eng.: Barby asks Boris)
2
П
апа  пишет  Поле  ＇papə  ＇piʃət  ＇polje  (Eng.: Father writes to Polya)
Papa spielte Poker  ＇pa:pa ＇ʃpi:ltə  ＇po:kɐ  (Eng.: Father played poker)
3
З
анят  Зинин  зонтик  ＇zanjat  ＇zinin  ＇zontik  (Eng.: Zina’s umbrella is booked)
Sara sitzt am Sofa  ＇za:ra  ＇zitst am  ＇zo:fa  (Eng.: Sara is sitting on the sofa)
4
С
аша  сито  сyшит  ＇saʃə  ＇sitə  ＇suʃɨt  (Eng.: Sasha dries sieve)
Sascha schickte Schuhe  ＇sa:ʃa ＇ʃi:ktə  ＇ʃu:ə  (Eng.: Sasha sent shoes)
5
М
ама  Милу  моет  ＇mamə  ＇milu  ＇mojet  (Eng.: Mother washes Mila)
Mama mietet Molle  ＇ma:ma  ＇mi:tət  ＇mɔlə  (Eng.: Mother helps Molle)Each phrase consists of 3 words, having in the first stressed syllable a combination of the tested type of consonant with one of the tested types of vowels. Since pronunciation norms of the German language require voicing the voiceless fricative consonant “s” preceding a vowel, speakers were instructed to pronounce this sound as a voiceless one in the word “Sascha.”Speakers, who took part in the study, were native speakers of the literary Russian language without prominent dialectal features of pronunciation, speaking also German (50%) and being native speakers of the literary German language without prominent dialectal features of pronunciation, speaking also Russian (50%). The level of knowledge of foreign language of all speakers was the same, B2-C1, which was tested according to the system developed by the Council of Europe [27]. 50% of speakers were women (native Russian speakers and native German speakers) and 50% were men (native Russian speakers and native German speakers).Each speaker read aloud test phrases and isolated words from phrases three times each. Thus, the total number of obtained realizations of the test words and phrases totaled 480; the total number of realizations for a single speaker was 120 (60 in Russian and 60 in German).All test words and phrases were combined into 2 tables (in Russian and in German). The order of words and phrases was random and differed for different speakers. All test words and phrases were read with the intonation of the completed narrative, followed by a pause.Audio recording of the research material was conducted in a specially equipped room, preventing foreign interference and noise: an anechoic chamber of the Institute of Applied and Mathematical Linguistics of Moscow State Linguistic University.Two samples of noise (white and pink) were generated using the program Cool Edit Pro 2.0. for mixing them with audio records of the test words and phrases realizations.Each speech segment was mixed with white and pink noise with various levels of signal-to-noise ratio: 0 dB, −3 dB, −6 dB, −9 dB, and −12 dB. Thus, for each spoken realization of the test material 10 variants of mixed signals were obtained, 4800 samples in total, plus 150 phonograms containing only noise (75 with white noise and 75 with pink noise). The total number of phonograms for the experiment was 4950.The number of listeners was 21: 6 males and 15 females, 19–21 y.o., native speakers of the Russian language without prominent dialectal features of pronunciation with normal hearing, who have a command of English at level B2-C1 (which was tested on the system developed by the Council of Europe [27]). Some of them (12 listeners) are proficient in German at level B1-B2 and some (9 listeners) do not know German at all.All phonograms were numbered randomly for presentation to each listener: total number of rotation variants was 11.Listeners have to listen to phonograms according to their sequence numbers in the proposed rotation variant and to write down the answers for each of them in the table (see example of the answer table in the Table2).Table 2
Answer table for listeners.
# of phonogram
Do you hear a speech signal? (mark the corresponding cell with«x»)
What is the utterance language?(mark the corresponding cell with «x»)
Write down everything, you have heard (in Russian)
Yes
No
Russian
German
Do not know
1
2
⋮
nThey could listen to each phonogram as many times they wanted. Each half an hour there were short breaks. Work time per day did not exceed 4 hours. The perception test run during 2 days: total work time for each listener was 8 hours.The total number of played phonograms was 23085. The total number of played phonograms, which contained only noise, was 727.The total number of played phonograms, containing speech signal (test words and phrases mixed with noise), was 22358 (the share of phonograms with white and pink noise types made up 50% each).These calculations indicate that the size of the base of played phonograms is sufficient to ensure reliable and stable quality of the data.A summary table with quantitative description of the experiment is presented in Table3.Table 3
Quantitative description of the experiment.
N
Characteristics
Number
1
Research material: number of words and phrases in Russian
20
2
Research material: number of words and phrases in German
20
3
Research material: number of words and phrases in Russian and German
40
4
Number of speakers
4
5
Number of realizations of test words and phrases in Russian and German per one speaker
120
6
Total number of realizations (phonograms) of test words and phrases in Russian and German
480
7
Number of tested types of noise
2
8
Number of tested levels of signal-to-noise ratio for each type of noise
5
9
Number of phonograms for perceptual-auditory analysis, which contain speech signal (mixes of test words and phrases with noise)
4800
10
Number of phonograms for perceptual-auditory analysis, which contain only noise
150
11
Total number of phonograms for perceptual-auditory analysis
4950
12
Number of listeners
21
13
Number of auditions, containing only noise
727
14
Total number of auditions, containing speech signal (mixes of test words and phrases with noise)
22358
15
Total number of auditions
23085We calculated statistical sampling error for the findings to prove the observed tendencies statistically. The statistical sampling error was calculated using the following formula [28]:(1)E=Zα/2pqn,where E is the statistical sampling error, Zα/2 is a z-value with a given confidence probability: in our case Zα/2=1.96 or a confidence level of 95%, pq is dispersion of an alternative characteristic (dispersion of the sample share): in our case, as the sample share is unknown, the maximum value pq = 0,25 is taken [29], and n is the sample size.


## 3. Discussion

The working hypothesis of the study was as follows: speech recognition (detection of speech in noise and identification of the utterance language) and perception of the semantic content of the utterance in the variable noisy environment are influenced by the type of noise and the signal-to-noise ratio, as well as by some of linguistic and extralinguistic factors. These factors are the existing redundancy of a particular language at various levels of linguistic structure, changes in the acoustic characteristics of the speaker while switching from one language to another one, the speaker and listener’s level of proficiency in a specific language, and acoustic characteristics of the speaker’s voice.The experiment showed that within the corpus of the research material pink noise provides better protection of the utterance than white noise at equal integral level of signal-to-noise ratio (for all tested levels) in terms of the following indicators: detection of speech signal in noise (see Figure1), correct identification of the utterance language (see Figure 2), and correct perception of the semantic content of the utterance. On Figures 1 and 2 statistically significantly higher (at the level of 95%) values in relation to the previous higher level of signal-to-noise ratio are marked with a frame.Figure 1
Influence of the signal-to-noise ratio on detection of speech signal in noise.Figure 2
Recognition of the utterance language: comparison of Russian and German in noisy environment (with various types of noise and various levels of signal-to-noise ratio).The lower the level of signal-to-noise ratio (the higher the level of noise over the level of the desired signal), the higher the difference of efficiency degree between pink and white types of noise, reaching its maximum at the lowest tested signal-to-noise ratio (−12 dB): while assessing detection of speech signal in noise, the efficiency of white noise masking is ~4.7 times lower as compared to pink noise. This result corresponds to findings observed in [20].Detection of the utterance in noise also depends on level of speaker and listener’s proficiency in a specific language, as well as on utterance language. Thus, for listeners, who are native Russian speakers, a higher score of detection of utterance in noise was shown for utterances in German, pronounced by native German speakers, than for utterances in Russian, pronounced by native Russian speakers (see Figure3: a statistically significantly higher (at the level of 95%) value of utterance miss (false negative error) for German (depending on the characteristics of the speaker) is marked with a frame), which demonstrates the effect of the language (and its acoustic characteristics) of the utterance on detection and recognition of the utterance in noise.Figure 3
Utterance miss (false negative error) for listeners, who are native Russian speakers, depending on the characteristics of the speaker.Listeners, who are native speakers of the language of the utterance, are able to detect native speech in noisy environment regardless the speaker’s level of proficiency in this language of (see Figure3); however a foreign accent of the speaker significantly reduces the score of recognition of the utterance language (see Figure 4: a statistically significantly higher (at the level of 95%) value of recognition (depending on the characteristics of the speaker) of the word «Полe» is marked with a frame), and recognition of the semantic content of the utterance, which confirms findings of experiments [14, 15].Figure 4
Recognition of the word«Полe» ([＇papə ＇piʃət ＇polje], Eng.: «Papa writes to Polya»), depending on the characteristics of the speaker.Besides, for a number of phonograms with low scores of recognition, the list of these substituting words also differed for speech of native and nonnative speakers: Tables4 and 5 show most frequent substitutes for the word «била» ([＇bilə＇],Eng.:«beat») for phonograms of native and nonnative speakers.Table 4
Main substitutes (4% and more) for phonogram«била» ([＇bilə], Eng.:  «beat»): native Russian speakers at signal-to-noise ratios = −9 and −12 dB ([Base: n=90]).
NN
Substitutes
English transcription
Answer share (in %)
1
била
[＇bilə]
42
2
Мила
[＇milə]
6
3
мыла
[＇mɨlə]
4
4
хиар
[＇hiar]
4
5
26 substitutes with share 1%–3%
43Table 5
Main substitutes (4% and more) for phonogram«била» ([＇bilə], Eng.:«beat»): native German speakers at signal-to-noise ratios = −9 and −12 dB ([Base: n=107]).
NN
Substitutes
English transcription
Answer share (in %)
1
била
[＇bilə]
34
2
Зина
[＇zinə]
13
3
Мила
[＇milə]
13
4
Дима
[＇dimə]
6
5
28 substitutes with share 1%–3%
35From Tables4 and 5 we can also see that quite a very wide range of substituting words with low answer shares (1%–3%) was observed for these phonograms: 26 substitutes for phonograms of Russian native speakers and 28 substitutes for phonograms German native speakers and these lists were very different. These varieties of words with low answer shares are visible on the «tag clouds» for these two phonograms (e.g., on Figures 5 and 6). The «tag cloud» or «word cloud» technique was used for analysis of the substituting word frequencies, in which the font size for a particular word was used depending on frequency of its occurrence: the higher the frequency, the greater the font size for a specific word. All the “word clouds” for this experiment were drawn using the online program Wordle™ [30]. Thus on Figures 5 and 6 we can see lots of small words which are different for both phonograms.Figure 5
«Word cloud» for phonogram «била»: native Russian speakers at signal-to-noise ratios = -9 and −12 dB [Base: n=90].Figure 6
«Word cloud» for phonogram «била»: native German speakers at signal-to-noise ratios = -9 and −12 dB [Base: n=107].At the acoustic level the degree of the utterance protection also depends on the fundamental frequency of the speaker’s voice: within the corpus of the research words and phrases in Russian and German utterances in realizations of males are more concealed from detection against noise (i.e., they demonstrated higher score of utterance miss (false negative error) against noise) than utterances in realizations of females against both tested types of noise with all tested levels of signal-to-noise ratio (see Figures7 and 8).Figure 7
Utterance miss (false negative error) for utterances in Russian against pink noise: comparison of male and female voices.Figure 8
Utterance miss (false negative error) for utterances in Russian against white noise: comparison of male and female voices.At the phonetic level various sounds have various degrees of intelligibility depending on their acoustic nature. Thus, the most resistant (among those tested within the research material of the experiment) to recognition in the stressed syllable are consonants [s] and [m], as well as vowel [a], while the most masked are consonants [b] and [p] and vowel [i] (see Figures9–12).Figure 9
Correct recognition of consonants in first stressed syllable against pink noise.Figure 10
Correct recognition of consonants in first stressed syllable against white noise.Figure 11
Correct recognition of vowels in first stressed syllable against pink noise.Figure 12
Correct recognition of vowels in first stressed syllable against white noise.At the syntactic level correct recognition of words depends on the context: the scores of correct recognition of words, which functioned as a part of a phrase, were higher than the scores for the same words in an isolated position against both tested (pink and white) types of noise at all tested levels of signal-to-noise ratios (see Figures13 and 14: the data in Figure 13 is sorted in descending order and formatted in MS Excel from the dark colour to the light one).Figure 13
Comparative analysis of correct recognition of words in Russian against various types of noise in various context (isolated words or as a part of a phrase), in %.Figure 14
Number of tested words (from 15), the score of recognition of which did not exceed 50% at each level of signal-to-noise ratio.In the phrases the best recognized parts were subjects (which always came first in the phonogram corpus), with predicates showing the second highest score (see Figures15 and 16).Figure 15
Recognition of isolated words in the phrase«Папа  пишeт  Полe»(Eng.: Father writes to Polya) against white noise [Base: n=252].Figure 16
Recognition of isolated words in the phrase«Папа  пишeт  Полe»(Eng.: Father writes to Polya) against pink noise [Base: n=247].At the lexical level recognition of utterances in Russian is influenced by the occurrence of words in the language: within the research corpus of Russian words the highest score of recognition was shown by words«мама», «Саша», and «папа», which have the highest frequency of occurrence in the Russian language among all the tested words, while the lowest scores of recognition were shown by words «Полe», «Зинин», «Борю», and «Милу», which are proper names and less common than «Саша» (see Table 6 index; ipm represents number of occurrences of lemma per one million words in the corpus; here the score is presented according to [31]).Table 6
Frequency of occurrence in the Russian language of the tested words with the highest and the lowest scores of recognition (according to the results of the current experiment).
N
Word
ipm(number of occurrences of lemma per one million word)
1
мама(Eng.:«mother»)
322,6
2
Саша(Eng.:«Sasha»)
93,6
3
папа(Eng.:«father»)
143,4
4
Милу(Eng.:«Milu»)
10
5
Борю(Eng.:«Boryu»)
19,6
6
Зинин(Eng.:«Zina’s»)
20,8
7
Поле(Eng.:«Polye»)
8,1


## 4. Conclusion

According to the results of the experiment the following factors influence intelligibility and perception of the speech in noisy environment:(1)
Type of noise and signal-to-noise ratio (pink noise provides better protection of the utterance than white noise at equal integral level of signal-to-noise ratio (for all tested levels) in terms of the following indicators: detection of speech signal in noise, correct identification of the utterance language, and correct perception of the semantic content of the utterance);(2)
Utterance language and speaker and listener’s proficiency in a specific language;(3)
Fundamental frequency of the speaker’s voice (within the corpus of the research material in Russian and German utterances read by males was detected by listeners statistically rarely than utterances read by females against both tested types of noise with all levels of signal-to-noise ratio);(4)
Context: isolated word or as a part of the phrase (within the corpus of the research material in Russian intelligibility of words within the phrase was better against both tested types of noise with all levels of signal-to-noise ratio compared to isolated words);(5)
Frequency of word occurrence in the language (according to the results of the experiment, words with higher frequency of occurrence in the Russian language showed better intelligibility);(6)
Phonetic composition of the word (within the corpus of the research material in Russian the voiceless sibilant fricative alveolar [s] and sonorant bilabial [m] among consonants and central open [a] among vowels showed the best intelligibility (i.e., the worst ability of masking using noise) within the tested set of sounds, while among consonants stop bilabial ones: voiced [b] and voiceless [p] and front close [i] among vowels showed the worst intelligibility, that is, the best ability of masking using noise).Among the further possible directions of analysis the following can be mentioned:(1)
Increase of volume of bilingual research material;(2)
Expansion of the inventory of acoustic parameters for the analysis of the language sounds recognition;(3)
Increase of the number of speakers and listeners taking into account such factors as age, gender, degree of experience in listening, and proficiency in the utterance language, in relation to the studied languages;(4)
Study of the influence of linguistic and extralinguistic factors on the recognition in noisy environment for long connected texts;(5)
Organization of the database, including units of the sound composition and intonation system of various languages.



---

*Source: 1831856-2017-07-13.xml*